So, my name is Jason Clark as introduced. I work for New Relic and specifically work, have worked on the Ruby agents. So that's the gem portion of New Relic that you might have installed in some of your applications. It's not gonna play too deeply into what we're gonna talk about here today, but it did give me some of the spring word for things that I've learned. That URL there provides some examples of what I'm gonna be talking about today. And specifically, I'm actually really glad that Laura gave her presentation about Docker because all of the examples that I have are set up as Docker files that you can run with Docker. And that's great because what we're gonna talk about is a set of tools that you can use to look at Ruby's behavior kind of from the outside. Specifically, we're gonna look at S Trace, RB Trace, and TCP dump. And these are tools that you may have heard of in some context, but may not have had the chance to dig into deeper.  I would highly encourage everyone to start downloading all of those images off of the conference wifi right now. That would be a really great idea. It's probably not a good idea to go downloading and installing all of those at once, but you can take that away later and play with these. So this talk is set up as kind of a series of mysteries, mystery stories for us to investigate. And like any good story, each of these stories has a moral. And the overarching moral that I want you to take away from this talk is to dig deeper. There are always layers in your software below the layer where you work day to day, and you can dig into and find out what those layers are doing. And that's what these tools are gonna help us do. So let's get started. The first case that we're gonna look at, I call the case of the missing config file. So this case comes out of a startup. It was dealing with this really very secret data stream. I can't even really talk to you about what it was or what's happening, but it was very important that we'd be able to process it really quickly. And so there was a member of the team guy by the name of Carl, a venerable old C hacker who wrote up a library super fast that was able to parse this data stream for us and do the processing. Now, Carl was a nice guy, and so he wrapped it up in some ruby. We had a gem that wraps this native library that he had written to make it a little more palatable. And there was a server out there that we had just one kind of custom hand built thing that did this processing. And of course, as always happens when you have one server, one thing that's responsible for something critical in your infrastructure, well, it imploded. And the unfortunate thing is we had copies of the library, but when we went to install it and run this on a different machine, put it on a different server, we got this. So the first parts just are kind of logging and then a message from Carl that that's not how that's supposed to be configured. We remembered that there was some sort of config file, some sort of setup that was on that box, but none of us knew what it was. None of us knew where these things belonged. And of course, when we went looking for Carl or looking for Carl's source code to be able to figure out what this library was doing under the covers, we came up rather empty-handed. So there's a lot of ways that you could approach this sort of problem, pulling your hair out, crying, screaming, all very appropriate reactions to this situation. But somebody suggested that we try to use Strace. So we spin that up and go execute our data streamer program. You can start Strace and hand it a command and it will go run that. And it looked something like this. Yeah, okay. This is really, really helping, really helping us solve the problem a lot. I mean, this is, if you've ever run Strace or some of these other tools that we're gonna talk about, they can generate a lot of output. But what I want us to look at is I want us to take a step back, take a breath and look closer at what this output is and try to understand and see how it can help us solve our problem. So we're gonna take, first start, one line out of this. So we're gonna take this line out of the middle and let's break apart what that is and understand how it's working and then see how that can lead us to a solution to our problem. So format it a little for size. It says open, it has some parameters, kind of looks like a function call, equal sign and something on the other side. So the first thing is what we refer to as a syscall. And we'll talk a little bit more about that. This is open is the name of the command that's being given. Parameters that are handed to that are shown next. So this looks like a file path and then some sort of bit flag with some descriptors about how we're trying to open that file. And this actually, the more that we look at one of these in isolation, the more it makes sense, right? And when it's just all of the screen full of output streaming past, it's easy to get overwhelmed. But each little bit of it makes a little bit more sense. In this case, open has a return value and that return value is the number of the file descriptor within that process that was opened. And so here this turned out to be file descriptor three when this process ran. So what are these syscalls? Like what is actually going on here? In your computer, there are two sort of major divisions between code that runs. There's the kernel, which has kind of direct access to your hardware. It's the piece that is able to talk to your memory, your disk, it can do everything that it wants to do and it just has all those resources available. The kernel is code that, you know, some very smart people write, it's mostly written in C, C and assembly, it's very low level, very tightly controlled, very carefully written. And the other side of the world is user land. That's where your services, where your applications run. And, you know, the kernel feels a little bit like this about your user code. It doesn't really want you to be doing whatever you want with the system. It doesn't want you to be able to hurt other programs by interactions that you might have. And so the interaction between you and the kernel is carefully mediated by a series of what we call syscalls. So a syscall, in user land, we make a call to the operating system and ask it to do something on our behalf and there's a very well-defined interface for what it does and then it will take that action for us. So there's a lot of things that are syscalls, a lot of process management functionality, a lot of file reading and writing, network writing. All of those are mediated by syscalls at the kernel level for us to be able to get into. You can find documentation on each of these syscalls that you see in the strace output by the man command and giving it section two is where those functions are defined. So like if we said man open two, we can see here the open command possibly creates a file, shows us some of the C headers for what those functions look like and gives us a bunch of information. So we can dig into what this syscall is that we're seeing there. Strace is a tool that on Linux systems allows us to spy on those syscalls as they're happening while the program's running. So that's what all of that output was. That was every call that was coming from our Ruby application and ending up resulting in a syscall to the underlying OS. So how does this help us, right? I mean this gives us all of this output, gives us all these things. I think the important thing to do with it is to look for landmarks, look for the things that we can recognize in the output and what we're gonna be able to find so that we can find the locations in it for us to solve our problem. So specifically there's a couple of key outputs that we get here. So we see our loading messages and these get written out and that writing is something that's going to show up. The operating system is taking action on our behalf and so those are gonna show up in our output. And then at some point after that, we get our message from Carl and that's also written. So if we can look in our output and find the two locations where that writing is happening, we can look at the things that happened between those points in time and maybe get a hint as to where things went wrong. So we dig into our Strace output, use some sort of text tool to go search through it and eventually we land on these lines. So we see a write, so that's the right syscall. It's going to one, that's a file descriptor number that happens to be the descriptor for standard outputs. So that's why it's outputting to the terminal for us. And then we see the data that we're actually writing out those messages that were being written. So we can be pretty confident that our problem in this output is after this point in time. Doing the same trick from the bottom, we look up and go until we find Carl's nice message. That's not how I would configure that. And so now we've got our two points that we can bisect between. And it greatly reduces the surface area that we need to search through to try to figure out what's going wrong. And in fact, looking between those two points, we ended up finding this line. Open user local slurp.confee. And then we can see that it returned with a value that says no such file or directory. So Strace is telling us that the application tried to open this location and didn't find it. That seems simple enough. Carl certainly could have made the error message a little kinder. And frankly, that's a really odd name to be giving a configuration file. Don't name your config files like this. But Strace has given us the ability to watch what's going on at this level and find something that we wouldn't have been able to find otherwise. Strace is useful for a whole lot of other things and there's tons of resources around it. It's great for watching things with process invocation, writing to any sort of file descriptor, any sort of thing where you need to watch what your interactions with the operating system are doing. A couple of great resources. Julia Evans has been doing a lot of blogging and even has a zine that's out there about Strace that will show you a bunch more flags, a lot more things that it can do. And James Gallick also had a wonderful presentation at Gorucco a couple of years back about using this to debug things in a language that he didn't even know and how he solved a real problem for one of his friends when no one else could on the server live. So Strace is a powerful tool and you should definitely take a look at it. But Strace doesn't solve everything. I mean, it looks at the syscalls that are happening. It can tell us about the operating system interactions but sometimes we need other things. So we bring ourselves to the case of the unused method or was it? In this particular case, there was an application and this application allowed for our external users to write some scripts that interact with our system. So this is kind of a little mini DSL that we give them access to and that they can run. And in particular, there was a bit of code in there that was really becoming problematic for us to maintain. But these scripts are dynamic, people are passing them in and so there's no way for us to really easily see whether things were being used or not. You know, there was a theory running around that like this was kind of a deserted part of the system that nobody was really using these objects and we should just get rid of them since they were causing us maintenance pain. But the business and product side didn't want us to get rid of them without some proof that they were not being used. Now, there are a lot of ways that you could skin that you could deal with this that you could work around it, find out what's going on. So I've worked for New Relic, you could put a method tracer on the methods that you care about and watch to see whether those end up getting anything. You could do some sort of logging. But unfortunately, we were under the constraint that shipping new versions of our software was actually really hard. Like you'd have to wait a long time and go through approval processes. It was very tough for us to do that. And this brings us to another moral and that is make proper preparations for the things that you want to do later on. So in this case, we were fortunate that somebody had installed an application, a Ruby gem actually for us, called RB Trace. RB Trace allows you to get visibility into method calls as they're being made into the Ruby VM. So much like our S Trace diagram that we saw earlier, when somebody makes a method call in Ruby, either on our own objects or on core objects, RB Trace is able to spy on that and tell us what's happening. In the simplest case, you can give RB Trace a list of methods and then have it execute a script and then it will give you an output every time that one of those methods is called. There's a variety of other flags, you can have it time how long they are, apply timestamps and some other formatting as well if you need. But at the basis form, this can just tell us when these methods are happening. So this seems like something really powerful that we would be able to use for detecting whether this code that we suspect is unused is actually unused in practice. Now, starting up a whole new application from the command line and altering how we're calling it to put RB Trace on the front of it was not really palatable to our operations folks. But thankfully, RB Trace and S Trace has some similar modes, actually allows you to attach to a process after it's already running. Now, in your application, you have to have required RB Trace, that's the one code tweak that's actually necessary for this particular technique to work. But once you've done that, you can say RB Trace minus P, give it the PID and then the other parameters that you pass to it are set up the same way as in the other invocations that we showed. So with this, we were able to set up this Trace, run it on the process and just log that output to a file. We watched for a long time, watched for a period of time that we had arranged with our product people. Unfortunately, we found that there was no output in the file. Nobody during that period of time had actually called these Ruby methods that we cared about. And so we were able to make the changes that we wanted. This sort of method level tracing is really useful and RB Trace also provides some built-in sets of methods. So there are tracers that you can apply for IO methods, active record methods, and even you can make your own arbitrary lists of methods that you can pass in for it to do. So this is really powerful if you need to find out what Ruby code is actually being called dynamically at runtime. But that's not the only thing that RB Trace can help us with. There are some other things that it gives us visibility to, like in the case of the occasional lag. We had a system, and this system on 100 millisecond timer would make a call out kind of a heartbeat ping across the network to another service. And we noticed that we would consistently get the number of heartbeats that we were supposed to. But every once in a while, there was a lag. We would see one of the heartbeats would get delayed by a certain amount and then pick back up and things were very flat. Now, this was not super alarming, but we kind of wanted to understand why this wasn't happening as consistently as we thought because it's so rock solid the rest of the time. Why at this particular moment would we end up getting the sort of hang? And the key insight that we eventually came to was that there were some background tasks that would run in this process, and they would happen at about the 10 minute mark here. And right after we would see these background processes happen, we would get the bump in the heartbeat. Now, this was a good hint because of the timing, but it didn't make a lot of sense, right? The background tasks that we had were running off on a totally separate thread. They were completely independent of this heartbeat thread that was supposed to be running and doing this operation here. And neither the heartbeat nor the background really had anything to do with each other apart from coexisting in the same process. So why was this slow? And how could these tools tell us why that was? RB Trace comes to the rescue here with a flag that it has for being able to trace garbage collection. So if you pass minus minus GC, in here I'm using the minus T flag to apply timestamps to the output which allows us to see a little better. We had some output that was telling us when we were running our background job and we could see that immediately after that background job ran, we would get multiple garbage collections occurring. And at the steady state of our application, the rest of the time, garbage collections were far fewer and more further between and did not happen in these sorts of clusters. Now, if you dig into Ruby's garbage collection, when a major GC happens, the VM is paused. There is a portion of the time that although they've done a lot of work in recent versions to make that smaller, there is still a point in time when everything pauses for these GCs.  Once we found this out, we were able to look at some other independent sources. And so like New Relic has a graph that shows object allocations and a variety of other Ruby VM things. And we could see that on the 10 minute mark, sure enough, there was a very large spike in our average object allocations. And that corroborated the theory that this was the thing that was causing our heartbeats to stagger. So taking that information, we were able to clean up, allocate fewer objects during those background tasks and got things back onto a steadier footing. But our retrace was able to give us the hint of where that problem was and show us the garbage collections that were happening silently underneath us. If you're interested in RB Trace, probably the best source to find out about it is its GitHub repo to begin with. Amon Gupta from GitHub is the primary author of it. And RB Trace is positioned well as something that you can use in production. It's designed to be low overhead and to be only enabled when it's actually doing some tracing. So definitely take a look at it. Justin Weiss had an excellent blog that kind of goes through a lot of the basic setup that we've covered here today as well. This brings us to our final case. That was the case of the delicate negotiations. In this particular case, I was working on an application. It's actually the Ruby agent that I formerly referred to. And what it does is every 60 seconds, it gathers up a bunch of data or it gathers the data across a minute and then transmits it to a service across the internet. Talks to our collector, we called it. And what we were doing at that point in time, naively, we would package all that data up and make a new net HTTP request. And this would be a completely fresh request from scratch. Kind of looks a little bit like this, different URL. But basically every single time we would make a full new connection to New Relic, we would go send the data and when we were finished, we would shut that down. And then when the next minute came along, we would do all of that again. And there's actually some costs that comes with this because these are connections over HTTPS. And so there's SSL negotiation that has to go on. There's additional packets that go across the network to communicate the certs back and forth. There's computation on both the client and the server that has to happen for this. And so each of these new connections that we start up has a cost associated with it. And when you're running something that the scale like New Relic does, saving that sort of connection time on the server is a big positive. And it's a small positive for the client as well to be able to hold on to those connections for a longer time. In sort of rough Ruby code, that looks a little bit more like this. So we set up a connection, configure it properly, and then we tell that connection to start. And that will go make the connection out to the service that you're talking to. And then subsequently we can make multiple requests across that connection. And those don't have to reestablish and do that initial work. They can just send the data across the socket that we're intending to. So we plug this code in to reuse these connections and the question comes up of how should we test it? So we do the functional testing. We see that the data is still being transmitted, but it's really hard to pin down our staging environment, the one request and whether we're reusing it or not. And anything that we would do to log stuff locally could be lying, right? When you put in logging code for something, if you think that it's doing the thing that it's supposed to do, your logging is gonna tell you what you already think. So what we needed was we needed a way to look at the layer below us and understand what this was really actually doing. To that, we applied TCP dump. Now there are some other tools like Wireshark and things that play well with us or that can be better replacements sometimes. But TCP dump is a really simple tool that's available on Linux and other Unix systems for spying on the network packets that are being sent back and forth from your machine. In our code, it looks at the communication that happens from the Ruby level over to our network stack. And so the moral here, and this is a moral that I've kind of been beating on with all of these, is to find the boundaries and realize that you can spy on them. Realize that when there is a system that you hand off to, that there is probably a tool that lets you see that traffic. And then interestingly as well, as you learn more about the systems that are underneath you, you'll realize that there's layers within layers here. So we talked previously about our kernel and the syscalls that we're going back and forth between our code. In fact, when we make the call to the kernel or we make our HTTP call, that's being a right to a socket and the kernel itself is actually making the network calls that we're tracing with TCP dump. So there's layers everywhere in your system and you can find out what those layers are doing. Let's take a look at what it looks like when we actually run TCP dump. So as opposed to the other processes where we would attach it to a particular process or run a particular script using it, TCP dump sort of runs a little more ambiently. You run it on the same machine and it spies on all of the traffic that's going on. Actually, it's fun to spin it up on your Mac or your laptop machine and see how much stuff is actually flying around that you had no idea that your computer was doing. It's actually pretty baffling how much output that can give. But it looks a little like this. Some new lines inserted to help. But let's break down one of those first things because this is, again, will give you just a huge stream of output. So the first option here, we'll take a look and dig in too deeper. The first item in it is a timestamp. So this tells you when the thing happened. It can be useful if you're trying to correlate things and logging over a period of time. The second piece tells us the addresses that are involved in this packet communication. And in particular, throughout the rest of the slides that I'll show we're abbreviating those addresses. So the one on the left was me, my particular address here and the other side was I was making these requests to Google and the angle bracket there tells us which direction that communication's going. So this was us sending a packet to Google. There are a variety of flags that get applied to these packets and in this case this has a flag of S. S is the SIN flag. So this is part of us starting a communication with Google and setting up that socket communication. Well later on we'll see dots and P's as data is sent back and forth and there's an F flag that gets sent when we're done with the connection and we're closing it.  This may be useful for different cases depending on what you're looking at. And you can actually pass verbosity flags to TCP dump that will do even more information out or even output the entire contents of the packet for you to be able to spy on. There are tools that will allow you to do that but for our purposes here, what we care about is just inspecting that communication that happens so that we can confirm that our connection reuse is working. So as a reminder, this is what the naive form of the code looks like when we establish a connection every single time. So let's take a look at what this looks like when we actually run TCP dump on it.  So that's us talking to Google saying, I wanna talk to you, Google saying back, yep, sure, that's great, we have a connection established now, let's go. Then there's a variety of packets that gets sent back and forth. I've excluded about 24 of them in this particular case for a very basic request. And part of those packets that are being sent back and forth is the actual communication of Google's SSL certificate and then us responding back that we see that and making sure that this connection is secure for us to be able to talk across. And then part of it is the actual packets of the response, the request that we're making to Google and its response back to us. So there's a big pile of data that gets sent here and then once we're done, because NET HTTP closes out this connection when we use the naive form, we have a finish flag, we say we're done, Google confirms that we're done, the connection's closed and we're finished. So how does this look different if we share the connection and recall on that same thing more than once? Well, the first part doesn't really change. So the first time that we make a connection across that shared object, we're still gonna see our send flags, we're still gonna see the same amount of data sent in that first pass, but the big difference is we're not gonna see the finish on the connection here. And so this tells us that we kept that connection open to Google and we can potentially send more data across it as long as both sides are still happy. There are timeouts that come into play, eventually the client or the server can decide that they're gonna close the connection and we'll have to re-establish, but as long as we're within a window when we're both still happy that it's open, you can send more data. And in fact, when we make that second request, this is all of the output that ends up getting shown for those packets. So you remember there were almost 24 packets excluded, plenty of stuff to scroll off the screen for this all fits on one screen now for just the request that we're making because we're able to share that connection. This gets us down to just about 15 packets for that basic request. And we don't have to do all of that negotiation. We don't pay on the client or the server for doing it. And this gives me a warm fuzzy feeling. This lets me feel confident that the code that I've written is actually behaving at the lower level the way that I expect that it should. There's some resources out there for TCP dump. Depending on the exact operating system that you're running, there may be other flags or other things that you need to pass into it to be able to get it configured right. And like I said, if you run this on directly on your local laptop, you will have a lot of noise of other things that are going on to work through. Benjamin Cain's quick start reference was very useful for me in laying out some of the most common options. And former coworker of mine, Joan and Sheffler, he had a talk at PDXRB about the life cycle of a web request. So if you're interested in learning more about what those packets actually are that we were watching streaming back and forth between us and Google as we started up, this will fill you in on all the gory details. Like any good story, this one needs to come to a close. I hope that you found it interesting learning about these tools, Strace, RB Trace and TCP dump. These are really just the tip of the iceberg. There are so many tools out there that you can use to dig into the lower levels of your system and understand the behavior of what's going on. Always dig deeper. Don't stick in Ruby. Don't think, oh, I develop at this level and I can't understand what's going on below me. You can always get to a deeper level. It was code that other people wrote. If other people wrote it, you can understand it. When you do dig deeper, look for landmarks. Don't panic when you get a wall of output that you don't understand. Look for the things that you do recognize and start there. Be prepared. These sorts of tools often do require some extra privileges on systems or may need to be installed. So think ahead about the sorts of things that you would want to watch on a production system before you need it in the middle of the night when things are broken and not working properly. And lastly, look for the boundaries. Figure out where the things are, where your code crosses over to another system. And I bet you'll find a tool that can spy on it. I hope that this has been useful to you and I hope that some of you will take a peek at that repo, take these cases and go run them yourselves and play with the tools. There's also a bonus case in the repo that I didn't talk about here for you to explore. Thank you.


All right, welcome to the bookmap platform details webinar. This is Bruce at bookmap risk disclaimer trading futures equities and digital currencies involves substantial risk of loss And it's not suitable for all investors past performance is not necessarily indicative of future results If you're new here to bookmap, you can find more information at bookmap.com and note that when you Purchase the product it comes along with education. Okay, there's an educational course it's four parts and It goes through just the very basics of bookmap and then on up Not not so much about actually the program itself, but much more about how to use it how to identify a very specific behavior that is is really challenging to view using other software So as the advantage that you're going to get The educational course continues on it gets more advanced in that fourth part And then every day we have the advanced order flow webinar series. Okay, so during those webinars and they start in about 27 minutes or so every day after this webinar And we look at the live market and we go through the same content That is covered in the educational course. So we You know kind of talk the talk in the educational course and then walk the walk in the Live markets and see it right there in front of you. Okay All right Let's jump in to the website here and take a look So if you're not familiar with bookmap.com just scroll down here There's an intro video here information about bookmap a bit further down you'll see one of the connectivity measures that we have here or Offers that we have here for a NASDAQ total view. So all US equities. It's a great Data feed a bit further down connectivity, okay, so this is important because bookmap is a visualization software trading platform Okay, you'll see some other platforms in here as well like Ninja Trader Interactive Brokers Traders workstation in TT X Trader Pro Okay, we connect via the API of these three platforms But we are a platform just like they are so you can you can connect directly and we recommend that it is better To connect directly through One of these supported data feeds CQG rhythmic gain IQ feed transact or dev experts for that NASDAQ total view All right a bit further down here the different packages available so you can Subscribe monthly or yearly and you get the discount here There's a free version. It is limited. You own no credit card required or anything What you get though is only access to one digital currency Okay, that's it and the connectivity is through G DAX and it's free as well. So it's real time You get the you know full version of bookmap here. It's not the with without the add-on indicators and you You the one-click trading is here also, but it's only in simulation. Okay, not in the live market You only get basic education and you also get just limited support. Okay Digital plus here is for all digital currencies through the G DAX exchange. You get up to 20 Digital currencies And you get the advanced education you get fold up the market, you know You can you can use the one-click trading and click and trade right from the chart in bookmap So that's what you get here global is the Everything in digital, but you also get support for Futures and us equities data feeds. Okay, we don't provide the data, but we you have this the ability to connect And you can see that's $49 per month that too comes with the advanced education and full support and the global plus what you get is everything in global, but you also get the Add-on indicators. Okay, so the ability to trade from the chart And these proprietary indicators we put together that start to identify larger players out there in the in the market. Okay, so it's a very specific Proprietary indicators for order flow. Okay, or understanding bigger players, okay, like the iceberg detector and the Large lot tracker here. So we're looking for not only like imbalances in the book But looking at specific players Okay All right, follow us on Twitter here. You'll see all sorts of good stuff. Here's one from futures trader 71 a bookmap user And click and see what he's looking at. We can go over this strategy here that he's talking about nice Follow through through pretty high liquidity retest And you can see they kind of flip to the other side here And there's a liquidity on the other side and it kind of exhausts out here as well And you get that continuation to the downside really nice strategy but And we'll take a look at that probably maybe today in the advanced order flow webinar. All right So that's a Twitter you can also subscribe to our YouTube channel All sorts of videos here. So if you're new and you and you do that free version This is the limit of support that you get. Okay, so the our education Hey, it would be here on the YouTube page intro videos are here. There's a playlist for features and components And then these order flow video snippets. Okay going through very specific things in the Order flow that book map can visualize and this is where you're gonna you're gonna get your advantage Okay, these concepts here in these two to three minute videos. We go over in detail in that advanced order flow webinar All right, well, let's take a look at book map We're gonna look at the ES because things are moving pretty nicely these days now And and here we go. Okay, so what is it? We're looking at here in book map. All right, so For those of you new here and I see a few new new traders here in the in the room So welcome and Just go through it Rather quickly because there's some other things I want to get to today Let's close up the CBD indicator sub-panel And then let's take a look here at book map. Okay, so There's basically just three elements on this chart Okay, there's historical best bid and offer the volume dots This is the volume that traded on that historical best bid and offer and then you can see this liquidity heat map Okay, the colored heat map here. Okay, what that is is a Recording of the of the dome. Okay, so it's just not a recording of price and volume It's a recording of the total depth of market Okay, so when you see this like really high liquidity here around this 65 and a quarter area Well, you can see it's it's blue here. Okay, well, it's high liquidity And it's recorded here in this window here and then projected onto the chart and here's that recording. That's what it looks like Okay, so we're gaining all sorts of information here just with these three elements Okay, starting to understand where larger players are positioned down here And you can see that they're staying in the book and it looks like they're absorbing all of this selling pressure into them Okay, and and and they absorbed yet again down here a little bit lower Okay, and they're also down here. Okay at around 57 and a quarter Okay, well, that's going to start to absorb the selling pressure and buyers are going to step in on the other side These guys are long down here. These larger players with their limit orders Hey, all sorts of information and just really really good insight here Just with these three elements so Let's Let's take a look at a candlestick chart so you can get reference to what I'm talking about and let's take away all the rest of the those those three items those that three data Items there and just look at this candlestick chart. Okay, so five minute candlestick chart There's only four data points per candlestick Open high low and close. That's it Okay, and that's the problem Okay candlesticks are pretty ingenious And you can start to read the wicks and the bodies etc But it's just too opaque We demand today more insight and clarity To understand what's going on and it's just not here in this chart Okay, so for example, what happened within this candlestick? Okay, what happened over here? Okay, was it buying pressure was it selling pressure? I mean this looks like it could be you know either or we're not really sure And we're gonna look at this area here and you know, you would think well. Yeah, I should be I should be buying down here, right? Or I you know Maybe if we can get up just a little bit above that candle you'd be you'd be a buyer, right?   It's not in here All right So let's turn on that microstructure and that's just very simply here. It is beautiful it very simply displayed with the Historical best bid and offer and that's it. Okay, because it records everything It's just the best bid and offer very very simple Simpler in concept than a candlestick. Okay, there's no aggregation of data here. It's just the market scrolling by It's like a horizontal tape And we're gonna read it like a like a tape So Now we want to answer that next question. All right, here's so here's our breakdown See see how we broke down below. Here's some structure here We broke down below it and we came back up and retested where we broke from and then we see that continuation Okay, that's what happened within this five-minute period Okay, and it's it's it's really not helpful by just looking at this at this candle here and understanding what what really occurred All right, so a point made with the microstructure now. What about the volume? Okay? Well, who's in control here within this five-minute period? Okay. Well sellers are gonna be in control All right, we're gonna see that they're gonna they're gonna be hitting the bid here pretty hard We may see some volume up here on the on the buy side But then we're gonna see sellers again start to hit the bid and take control here. All right, so let's turn on that volume and Let's see if we get the insight here Okay, and let's zoom in here All right Okay, so here's our five-minute period. Okay, and let's bring up the volume dot size Okay, actually a little bit on both sides here you know, so We if I zoom in a little bit more here, we're gonna see a little bit more clarity Okay Actually kind of surprised here I mean we do see the selling coming in here as you can see the red dots pulling that market down There's actually quite a bit of buying and they they mount an attempt here to You know lift the the offer they just can't make it up above this swing though Okay, and then that's where the sellers start to come in on the other side and you can see them. All right and starting to Hit the bid nice cluster of selling down here. Okay? nice cluster and And that's why we get this kind of retest in these areas here as well due to Noticing all of the selling pressure and Now the So that's what we're looking at in terms of two just two elements on that on this chart so far Okay, historical best bid and offer and and volume. Okay, so let's zoom in and just show you exactly what I mean here Okay Right. So here is our historical best bid and offer best offers the red line best bid is the green line This is one tick wide right here. Okay? And S&P's pretty amazing market. It's basically almost always one tick wide and Let's zoom in a little bit more Okay, so now you can see these dots here. These are transactions. Let's bring it up a little bit These are transactions on that historical offer Okay, we can we can know know exactly what traded here. Okay volume of 10 here. We get the date the time And what was the liquidity here on the ask at this price level and then the volume it traded here? Okay, here. I just get what was on the ask You know, so you only get four Well, no, you do get four lines of text here But if I if I hover over this then I also get that volume Okay So Now there's more volume that actually traded Within that one dot there and we're going to see everything here Let's go back to the bigger one because we're just gonna be more trades within there and there we go Okay, now look at the timeline down here. Okay, we're looking at Milliseconds, okay, these are millions of seconds I'm sorry. We're looking at microseconds and these are millions of seconds and We can we can continue on and you know, we can go into nanosecond levels, right? I'll spare you you guys get the idea, but we're I just want to demonstrate the power of Bookmap here. We're recording every single market event And then it's being plotted here. So there is no aggregation. It's just the market And and how it really it truly unfolded here. It's like a footprint chart is going to aggregate this Okay, and you won't be able to understand all that detail now. We don't trade at those levels, right? So look how bookmap will take that volume and then put it into a bigger dot Okay, so this is aggregated now Right, so volume of 10 overall. Okay, but we're every single event is still there All right when I zoom in Because and why is that because we don't trade at those levels and we just we need to know the overall What occurred at these areas here and then you'll start to see as I zoom out more and Compress that timeline together. You'll see this pie display start to show up Okay, and what this pie display is showing is The overall okay, because there's so many transactions here. There's 46 volume of 46, but there could be a minimum at 46 Events trade events. Okay And there was buying and selling and it happens so quickly here Okay, so if we zoom in I start to pull it apart Okay, and we see exactly what traded here, but as they zoom out Okay, because we want that aggregate view and we want to understand that within Context of this microstructure and we can see who won the battle here. There's more buying Okay, there's about three quarters more buying and then there's about one quarter selling So we have that kind of insight here. We can see that up at this little microstructural area This is actually one of the things we look for Is the buyers to start to step in up at this area and lift the offer and that's exactly what they did here Okay, there was more buying pressure in this little area right there Okay, so those are just two elements that we're showing here Right now let's zoom out and let's get this third element onto the chart because this is where you're really going to get a lot of insight Let's bring this dot size down Okay Okay, so we noted that The large selling transactions down here Okay, down in this area here All right Yeah, it's in it is it is fantastic. I mean it's just amazing what? to really Take a look in Inside what's really going on in these markets and to be honest so that everyone knows you know why we why we display things this way and I This came about well, it's from our history. I mean actually This software well Basically that the team was developing algorithms, okay for for trading and We we needed to develop our own software and basically quants develop their own Visualization tools because they need to test and know and understand Exactly how their algorithms are behaving and that's where this this software came from. Okay, that's why we record such, you know Small, you know low low time frames because we needed to know the details of exactly what unfolded and and that's why it looks this way as well, so That third element, okay, we're you know, we can see this cluster here, and then we see this rebound Well, that doesn't make too much sense, right? I mean, why would there be all this aggressive selling and then you see a really nice rebound here? Okay, that's that third element. We're gonna see and we're gonna see a cluster of limit orders here Most of us never see this The closest that we would see it would be in the dome Okay, we would see that there's liquidity here in the dome on the on the on the bid compared to the offer and And you can see that we're as we're driving lower here I mean, this is where traders are lined up in the dome to trade They're providing the quity here on the offer and then below current prices on the bid they want to be buyers down here okay, so Even a footprint chart, I mean it's not displaying. It's just displaying and and aggregating. It's not only is it Not displaying the limit orders It's it's aggregating that data within a rotation bar or a candlestick or a time or whatever And you don't get to see the microstructure So it that that footprint chart and I like footprint charts quite a bit, but for those two reasons This is superior Because we're getting so much more insight We're gonna see that we traded right into lots of Limit orders down here. Okay, and we'll just turn this on Okay, and there we go. Okay, so actually it occurred over here and then there were some down here as well Okay, and let's take the candlestick off Okay, and here's that absorption okay into this area here a little bit more And then and then we see the nice rebound here. Okay, here. They are again down here it looks like they kind of just gonna be probably a combination of absorption and Pulling but this was key to understand let's zoom in here Okay, and then this is really what unfolded. Okay high levels of liquidity and and and the market is trading right into them Okay, and we can we can see what what traded down here over a 1300 contracts traded here at 61 All right. Okay, so we're getting very objective view of What occurred in the market with these three elements? Okay, that historical best bid and offer the volume and the Limit orders. Okay, so let's let me let me go to the current market and I want to show you how this is working All right So that we really get it get an understanding here of the heat map. Okay, so to the right of this white line What we're looking at here is the best bid and offer Okay, and then we see this heat map. Okay, this heat map is is just like the dome over here Okay, the current order book. This is our dome depth of market You can see the price ladder here, of course And then here's your best bid and offer All right, so these levels here you look at the numbers Okay, and you can see down here while there's 800 contracts is by far the biggest in the book Okay, it's at this half figure of 2650 as well. So there's no surprise there All right so That that's we just take the numbers here And they're painted in the heat map. Okay, look at a little bit of spoofing here trying to get it down into this 2650 looks like And skewing that auction. Okay, we're just witnessing that right now Okay, great stuff to see And or you know, maybe this guy really really does want to be a seller here And he's waiting for Market to maybe pull back and and get involved in the market here Well, we'll see it's when the market comes up here and tests him But to me it looked more like a little bit of a push or nudge to get it below this 2650 And into higher areas of liquidity. We're probably gonna see some Iceberg orders down here. This is our iceberg detector Okay, yeah, here's 157 And not as many as I thought In some of these areas here. All right, well, that's fine Let's take that back off for now. I Yeah, they're getting being really aggressive on the on the offer. Look at this new Phenomenon just showing up here now. Okay, very very aggressive so We take the numbers we Paint it in this window here and you'll see that when the numbers change the heat map changes Now that's what the change looks like here when it's plotted on to the chart historically Okay, so for for example in this little area here We can see them Adding some liquidity in and it gets blue becomes blue. It gets lighter blue So there's more liquidity. It started with about 500 or 462 Went up to about 550 Okay, 575 then here at 639 or 35 You get the idea. Okay, so That's what we're looking at here. Okay, and we're able to gain a lot of insight By by looking at this data. Okay. In fact, what we're starting to see here is You know, I was calling this a spoof here that it could be need to kind of take a look at some bigger picture stuff here, but Potentially a flip of the book that you know Larger traders are coming in and saying, you know We believe that anything that comes up to 2650 is a deal and they're willing to let's see if they stay in the book Here or not. Okay, and they're willing to grab as much as they can Okay, now here's our answer Okay looks like this guy is He's he's kind of staying in the book It's kind of trading but look up look up above here 51. He pulled so and this guy pulled as well So this looks more like a spoof now to me. Okay, because we're and we can determine this. This is fake liquidity Okay, this isn't fake here. I mean we're seeing that it's transacting into this area here Some of it is polling though too. Okay, but look at this guy. So High high liquidity and polls up here as well. They don't have the intent to trade They had the intent to press the market into these lower areas And let's see if our iceberg order says something now You know and see some some large icebergs in some of these areas Not as many as I I was Assuming there Okay Now now that these guys are gonna pull their liquidity You know the market The buyers are starting to step in on the other side here Okay, and if the if the buyers if they want to I mean you can see they're lifting the offer here Okay, so We're looking for targets on that and I'm not really seeing much here. Maybe up at this 55. They'll start to come back in But the higher liquidity is down here at 45 Okay All right guys Well some interesting stuff to see this little kind of flush through this 2650 area here and Look at the buyers starting to step in on the other side. Okay, can you see the order flow? Can you see how clearly like that? You know, they're they're buying here. Okay at a higher area Okay, we need to get up above actually 54 and see a little more significant volume trade up here and You know, we'd be looking for You know to see how it behaves up here if it returns back into the range or not but that's the potential of This this being a reversal. Okay has that potential Okay, because there's a lot of buying here. We just need to see it up at higher levels Anyway, let's wrap it up for those of you signed up for the advanced webinar. We'll See you right over there Okay, take care. We'll see you tomorrow


What's up everybody? I'm evil rabbit. It's Tuesday. So it's time for another torque drift Tuesday episode here on the channel Make sure you follow me Instagram and Twitter all that you found in the description box below. Let's get to it So if you guys remember last episode, we were working on the FC So we're continuing the work on the FC. We did put a turbo in it recently Where do we look we're looking right here? We did put a turbo in there and we did bump it up to 9 psi so a little bit more boost but problem is For us right now with that FC is That this little message right here.  We're gonna try and complete this one here today and hopefully get some crates And then we're gonna see what races we can get into with that FC. We can get the mad bull title and Yes, that looks amazing. I love the look of that mad bull. So let's go hit the track going up against looks like Pretty nasty FC Let's do it Those in second gear Okay, so I probably need to go into Okay, now now we got into it. There we go little wall tap Probably need to go into force gear, but we're we're kind of hanging on there The initiation kind of messes me a little bit So force gear were okay. I've never done this full track Section before so this is all very different to me and we lost it So I've never actually run this full tracks. We're trying to scan apparently we got to go into force gear Let's uh on the fourth. It's a bit better a little bit on the handbrake Kind of sucking up the door We should have ran that a much wider, but we're still we're still in the drift section quick transition It's up car definitely has a snappy transition kind of a Smacking into them really quick on that. There we go Try to stay as close as we can and I don't remember what the challenge score was for the chase 85 okay That is a high high challenge to get here From me on a controller. We may have to try this on the mobile platform to try to get that But we're gonna give it another shot Trying to get that mad bull Sponsorship we'll start in third this time She ain't in Kind of need to get up there with him and get the close calls. I think Now we're kind of far back Yeah, this one's not gonna do it. Yes. We could find a way to Just lay on this man's door Still very twitchy on a controller We didn't even get as good as last time not even a 60 for the five torture crates Definitely want to get that 60 for the torture crates for sure because those are definitely coming clutch with three things so Give us another go Looks like fourth gear was my option So let's see if we can't oh that was kind of better. We were Much closer on the close calls there. There we go I stay within as much of his door as we can a little left for breaking And we might be able to get at least the 60 We can at least get the 60 we least got the 60 we did get our five torture case So we're able to get something for today That 85 I don't know if I'm gonna be able to do it here on PC with a controller We're definitely gonna give it one more shot and we're gonna go take the FC out This is a tough challenge for me Let me know if you guys completed it and what you guys did Go I think that is where I definitely fault out I can't seem to bring it out as far as that van does on his line we're just Door diving and door bashing this FC. Oh We were close to 75 though. I Might be able to actually get this we're gonna keep trying this because It's 72 we're getting there. We're close. I Think we might be able to maybe get this before the end of the episode. I hope I Know I said one more time, but that was close feeling that itch. I just need to Not do that a little bit better gas feather there.  I kind of got a little bit a little bit You know washed out in that You get there you get so close and you just want to keep going Wolf Diving that man's door Okay, too much on the too much on the sticks there Too much on the stick going one way It's a tough challenge for me. Like I said, I may have to try and take this on in the mobile device Since I'm not super acclimated here with the controller Seem like I can stay there and then I lose him there. It will come back though wait Why did I miss all those points? Oh? I think I got I think I got shafted on that one. I got shafted on that I probably would have had way more points on that if I didn't go ghost for some reason Only 66 for that time. I don't know why I went ghost Guys can explain to me why I went ghost on that one. I would be great Let me know down in the comments when you think I went ghost because I'm unsure in the forest I'll try and stay there sort of a hard hit. We didn't go ghosts though this time We're definitely behind the curve on this one compared to the other runs And too much of a wall tap there And got all wonky on the sticks there Tough challenge for me. I really would like to believe this challenge though Because I always tend to not complete the challenges and I go off from recording and I complete it like the next run You know, this is much better As long as we can keep this No Come back. Ah, that was that was the run That was definitely the run right there. Oh We had the beginning section too. I Felt like that felt so good and then we lost it a little overconfident I think I got a not hit handbrake and break at the same time One more shot here for today's episode and then I'm gonna call it a quits for today If you guys have some tips and tricks, let me know down in the comments Be very interested to see how many times it took you guys to complete this challenge It's not good overzealous with the sticks here. It's not where I need to be Now we need it to be oh go sending it over Think I'm gonna take that one as a loss for today If you guys like this episode you guys want to see more fc continuing to build on next episode when that motor comes in So make sure you guys follow me on social media. And as always, I thank you guys for watching. I'm evil rabbit I'll see you guys on the track See I leave I stopped basically thinking I'm recording and I get to 75. It's not even close to 85


Okay, so today I would like to present a method that I started working on for my dissertation and have continued to work on since then, on and off, a lot with my colleague Kevin Ryan who is at Harvard in Linguistics and more recently with a colleague Taylor Arnold who was at my current institution, University of Richmond in the math department and the method has just been developed specifically to sort of guide reconstruction of the big beta but I would like to think that it has some broader applications and so I will be as always interested in your particular method but I'm at least as interested in your ideas of where we might potentially, I say elsewhere, should I stand more at this point? Thank you. Okay. All right, so just a brief sort of intro to the rig beta. This is the oldest text of the Indic branch of Indo-European, also called Indo-Aryan. The language is likewise called Vedic and unlike classical Sanskrit it's a living language and we know that because we watch changes, observe changes in every part of the grammar between the oldest Vedic text which is the Rig Veda and the sort of through middle Vedic and younger Vedic until it sort of fits this classical Sanskrit and what the Rig Veda contains are a bunch of praise poems mostly praising the deities of the Vedic Pantheon and sort of trying to motivate them to attend the ritual where the poem is being performed and sort of benefit the ritualists. The entire text is in poetic meter and there are, and all of the meters have in common that they regulate syllable count and also syllable weight distribution and some of them also have Cesare that is to say points in the verse line where a word or phrase break is required and it's important to know about Vedic that there's a two-way syllable weight distinction between light on the one hand and heavy on the other and if a syllable ends in a short vowel only it counts as light and all other syllables are heavy that is to say long vowel syllables, short vowel plus consonant rhymes, long vowel plus consonant rhymes and so forth. So here's a sort of rough representation of the eight syllable verse type and so I'm using the sort of X or I should start with the breath. So I'm using that to show positions that are usually implemented with the light syllables about a third of the time only heavy about a third of the time then the X just shows relatively free positions that are sort of only bit or heavy anywhere from one third to two thirds of the time and the macron or the long mark shows you preferentially heavy positions and you can see that there's essentially like an iambic rhythm to the verse that goes you know the done, the done, the done, the done and there's sort of two additional and well-known principles at work here. One is so-called, I think this is on the next line, yes, one is final strictness and this applies to the verse line and it essentially says that the later in the line the more strictly syllable weight is regulated, I guess there are good typological parallels for this as well and then the other one is final indifference and this just applies to the last position or the last syllable of the verse and that's relatively indifferent to weight so even though you know analyzing this as an underlyingly iambic meter you would expect you know to find heavy syllables in the last position, final indifference sort of permits you to implement that with either like or heavy syllable and the 11th syllable verse line either has a caesura after the fourth position or after the fifth and you can see it's sort of there's an iambic opening like done, done, the done and then the done, then the done, then or you could go the done, the done, then the done, the done, and so I have to stress the preferentially heavy positions to sort of be able to hear them because I'm an English speaker. And the 12 is really just like the 11 except that you sort of descriptively insert a preferentially light position in the pen ultimate position of the verse so it opens the same way the done, the done and then you have the same sort of the done and then instead of having done, the done, then you have done, the done, the done. Okay and so I'm almost over with the description of the meter so bear with me so okay and so you basically you just you just put these verse lines together in threes or fours usually sometimes other combinations to make up a stanza and you can show that there's more structure than just the line it's not a purely stickic verse type there's evidence for a couplet like structure and for stanza structure and so forth too but for us it's really just the verse line that's important. So most of the Rig Veda 83% is composed in verse lines of the three types that I just described and so for the rest of this talk I'm just going to be excluding the rest of them as sort of unusual meters to be able to do some comparisons that I'll talk about in just a second and so you can tally the number of verse lines which are called padas by sans-critus usually and then you know you can sort of calculate the corpus size this way by multiplying the number of padas by the number of syllables in the padas to get a sense of how big the corpus is. Okay so the text we think was maybe composed around 1200 or Professor Hill suggested 12th century I don't think it really matters for our purposes it's we don't really know exactly. It must have been composed over a relatively long period of time because we do see differences in the language of the kind of older parts of the text and the younger parts of the text and then it was transmitted orally for many centuries in an incredibly accurate way and that's you know thanks to these feats of memory and those were of course more motivated by the importance sort of given to correct recitation in this particular culture or subculture but nevertheless there were some changes made to the text and some of those involve the replacement of linguistically older more okay forms with with younger ones and sometimes the younger ones are the ones that are familiar to us from classical sans-crit so you have some basically some old Vedic stuff getting replaced with some younger classical sans-crit stuff okay so one of the one of the examples that is usually given because it's almost completely exceptionless here involves a sequence of a consonant in the transmitted text the consonant followed by glide either a yah or a wah which is written with the V followed by a vowel that has like a grove accent mark on it and that grove is originally representing falling pitch and this is also essentially the only place where you find a grove accent in the text so it's a pitch accent language as a rule with with some systematic exceptions lexical words have one syllable that receives a high pitch and the rest of you know the rise to that high in the fall from that high back is is sort of fanatic right so this is a little a little exception to that oh it's and I should also say that the the accent is this almost if not purely morphologically determined the placement of that okay so wherever you find this sequence that is to say consonant glide and then vowel with falling pitch the verse line is one syllable short so this so this motivate sort of paying attention to the meter motivates the reconstruction of an extra syllable so just as an example I took the word for son which is transmitted as swar and I just took some eight syllable verses with the phrase swar darshi which means to see the son and and you can see Prathyan Vishwan swar darshi is seven syllables but if you reconstruct Suvar darshi then you get eight syllables and so this is you I think they're like maybe they're hundreds if not thousands of examples of this kind of a sequence it's almost perfectly regular it's a super clear case of you know basically paying attention to the meter and then restoring a slightly older phonological sequence and so the change that happened obviously was the gliding of the high vowel and this seems to have introduced a new accent contrast to the language because the pitch fall was then phonologized apparently as falling pitch so Suvar you know had that was at a stage where there was just one high pitch accent and syllable and then and then you say swar and there's falling pitch still on what's now the only syllable and that gets phonologized as the accent that we write or transliterate with the graph I hope that was clear so that was an example where where we we know and have known for a long time that that we need to reconstruct an extra syllable and now comes an example where we need to reconstruct something like that in just about syllable weight not count and so this is another just you know classic textbook example so there's there's a root mirr which means something like be compassionate take pity and wherever the root is followed by a vowel that is to say the syllable as spelled the first syllable consists of M and a syllabic R the rig data the meeting you know paying attention to the meter meter the rig data shows that the first syllable must have been heavy right and so normally mirr would be a light syllable that consists of just a short vowel counts as a short vowel but it appears to either have been a long vowel or a vowel closed by a consonant and you can tell this again by looking at forms such as the imperative murder daya which means take pity and the poets regularly you know place this at the end of eight syllable verses where we actually expect then the done so we expect something like murder daya or and and but you know so and so we in this case so this is just you know sort of again like alerting us to the need for some reconstruction and then we'll actually use the comparative net well internal reconstruction and the comparative method as always to insert the right form back into the text and here the fact that the D slash L is retroflex plus the Iranian evidence where you find you know murder with a consonant between the syllabic R and the D and a best and is going to motivate us to reconstruct something between murder and murdered for for the rig data whatever that was certainly it was it still counted as a heavy syllable then okay and then so those are pretty exceptionless cases and then there are cases where you actually have very good evidence for variation so and and and this seems to have been sort up to the poet to use whichever form you know he wanted to and I don't think that this variation has been very closely studied so there may be more factors than just metrical factors going on here but it it looks pretty pretty metrically determined to me and so a well-known case is the the date of ablative plural suffix yes which can be realized either be us or yes wherever it's that it comes after a heavy syllable and that's just up to you as the poet and then there's the older form of the genitive plural ending either um or um that's very hard to tell and the younger form um so it's only transmitted as um but we see that essentially the poet got to choose whether he wanted to use monosyllabic um or disyllabic older um when we're diversifying stuff and so for example if you want to close an eight syllable verse or 12 syllable verse and remember those have a diambic cadence they go but then but then then you will say to mean of the people and if you are composing an 11 syllable verse then you're going to say John on as it gives you the right rhythm so and so I guess many into your penis would reconstruct this genitive ending is all home or something like that and I think pretty much everybody agrees that it sort of survives into into both Vedic and Avestan as um and then there's the younger form with with the very unsurprising looking contraction and another reason to assume that it was originally disyllabic is the the accent of the Greek genitive ending own which is circumflex and should probably be derived from a two vowel sequence where you have all own so I think that's the standard line on that okay so so everyone agrees on on the examples that I've that I've shown you so far I think and they people have studied this very closely including a lot of people in the mid and late 19th century and early 20th century Hermann Odenbach and Yvonne and Arnold being two of the most prominent and many of those were then adopted into a metrically restored that is to say reconstructed text that was published in 1994 and so so is there is there more to do and I would say yes I think we can we can do more and also should want to do more because we are now in a position I think to be able to say relatively decisive things about forms that are much less frequently attested or and or occur in parts of the verse that are not as strictly regulated so basically we can you know we thanks to some advances and and statistics and and and stuff like that we can sort of be a little bit more sure about things than say well then then 1920 early 20th century scholars who just didn't have the same sort of tools that we do now so here's here's an example a verb form ishia that's that means something like I I could be Lord I could be master and so that has the up so-called optative suffix e so that's sort of an irrealist type form followed by the ending for the first person singular in the middle the middle voice which is just here and so the form is only attested three times in the rake data and here you see at the end of the of an eight-silver line where you expect ishia not she so this is a but but if the other if you look at the other two attestations there in the first half of an eight-silver verse where the meter is much less strictly regulated and so they you know they're not people would not normally feel these to contribute much or any evidence one way or the other so so what do we so what do we do right we say well it is in the cadence that one time so I don't know how often do you expect that sort of thing to happen it's not it's unclear and so I think the right thing to do if you aren't in a position to sort of do some careful statistics is just to say well I'm not sure it might just be one of the departures from the meter I don't want to make make too much of it and that's sort of what what Oldenberg did and then others you know felt freer and they just did whatever they wanted but Oldenberg was very a very sober worker so okay so what we're going to do now is first note that though that the look the localization of a word is partly phonologically determined in just by the nature of metrical composition and then we're going to compare the way that poets localize a particular word with the way that they localize all of the other words that have the same shape and when I say shape I mean phonological properties that matter for the meter so number of syllables syllable weight distribution and note that we also have to pay attention to the onset of the word and the rhyme or the end of the word just because of the way recelabification works across word boundaries and they so so if you look at this this word would be Lord but so there's that's an example of right word recelabification of the end and then here's an example of left word where my hip hip then being closed and counting us as a heavy syllable right so these are the things we have to pay attention to syllable count syllable weight template if you will of the word and then some things about its edges and so then here's the way that I'll be representing this way typically depends on what follows and so that's why words common words frequent words that belong to that shape class or purush to tell much praised paravati in the distance the tana y'all put imperative and so as you would probably have guessed where the poets like to put this sort of a shape is verse finally in a syllable verse where it gives a nice diet iambic rhythm purush to tell paravati and so forth and so what we'll do is we'll just we'll just you know look at all of them and then we will express the pattern the localization pattern of that class as a vector right and so this just means that there are six that they put it starting in the first position of an eighth syllable verse so the verse would start with purush to tell you see that you get a one in the first position right you see that five more times you get to sit zero time starting the second position one starting in the third zero time starting in the fourth and then starting in the fifth which is the latest you can put that in the verse purush to tell five six seven eight right that's where almost all of them they put almost all of them when they're composing an eighth syllable so okay so then in 11th syllable in this particular shape class we see that they localized most of them the beginning of the verse 17 of whatever that is 26 the post is their old position is starting in the fifth position is another spot they'll put them and then in 12th syllable looks like eight syllable they just put them line finally because the cadence there is likewise did in did so so okay so and then we just put the three vectors together into one long vector and so now we have captured sort of as a mathematical object or whatever so what you might think of is like the metrical fingerprint of this shape class so now we're in a position to compare individual items to the entire class and so that's what that's what we're going to do thank you so so and and I so I and I don't want to suggest that you know formerly people were doing bad work in this area and now finally I'm doing really good work in this area that's not true but but the you know that there are some some advantages that I think we can you know point point to and one is that we're just including a lot more information now we're not just looking at the cadence of the verse we're looking at the entire evidence and even though the earlier parts of the verse are not as strictly regulated they are regulated and so these are informative things that we're adding to the picture we're also taking account the relative frequency of a class in the three verse line types so for example the type that we just looked at the poets like to use that better in 8 and 12 syllable verse than they do in 11 syllable verse you can kind of come up with an expected frequency in each verse type just generated by the relative size of those three subcorpore and you see that they're either avoid they're avoiding them in 11 and or preferring them in 8 and 12 and so that gets sort of captured here as well and we can work with were shapes that just aren't fit for the cadence so they only are localized and in less regulated parts of the verse and if we do the math correctly then we're going to be treating the infrequently attested items exactly you know very exactly and we can even say things about things that are tested three or two or one one okay so I think I should well I think maybe okay so we could say something like well what's the probability that the Tatana expressed as a vector belongs to the class of you know light heavy light x items that are shaped the same way minus the Tatana and and so well we'll get a probability value and those will all be very small so we'll just take the log of them so that they're easier to work with and you don't have things like you know zero point zero zero zero one three four five or something like that and then yes and so obviously when you see something like negative 32.5 that doesn't mean anything until you put it relative to the other log probability values that you're coming up with and so just to give you a sense of this the the the log probability values for the individual forms of the rig data range from negative 918 that's at least probable to belong to its class to negative 30 which is you know most probable to belong to its class and for each class we can also come up with an average so the class the average for this class is negative 31.5 so very similar to the negative 32.5 that we have for the Tatana now classes depending on the shape and and other because of differences in shape and so forth the their averages are very different from each other so they range from negative 250 roughly to negative 30 and that's because what I refer to as with height and loose classes so the one that we just looked at is tight in the sense that there are only a few places where the poets can sort of fit them into the or do fit them into the verse and and something like CVC shaped mono syllables that's with a short vowel they put those in almost any position in the verse and so it's in these sort of loose classes that you really get to see the other things that are sort of determining word order and the rig Vita like like syntax so okay so all right so so that here I'm just reminding you of Isha and notice also Rossia is another like this where you would expect Rossia I would give over and so this will be our very brief case study so we'll just look at all of the optative forms of the first person optiforms it in and you can see they're just 15 of them and so 15 tokens and one two three four five six seven types so it's very this is a not not a well-attested class and so for each form we're going to make two comparisons we're going to compare the form with its shape mates and then we're going to compare the form with its putative shape mates that is to say or for example we're going to compare Asia with other things that are shaped like Asia is spelled and then we're going to compare Asia with everything that's shaped like Asia the form that we're considering reconstructing are restoring and then we're going to say well is it more probable that Asia belongs to its apparent class or is it more probable that Asia actually belongs to the class of things that are shaped Asia and so so here's so we I'm leaving Buxia aside for a second we we do find a kind of a distribution something it looks like it might be a distribution here so for Asia and Russia those two seem you know seem to have with a short eye that is to say something to reconstruct with the other ones you seem to have a long eye which is the classical Sanskrit form as well and the one that's transmitted in the text and it also seems to be the case that you get the sort of reconstructed form after a heavy syllable and after a light one that doesn't look crazy in a language that has a fair amount of morphophonology that promotes syllable weight alternation and is also in an environment like Iambic verse that promotes syllable weight alternation so and Buxia so but by this distribution we would expect you know Buxia with a short E because Bux is a heavy syllable but we find better evidence for Buxia with the form that we expect and so this is as far as this takes us it sort of suggests I would say the method suggests that we take the reconstruction of E with a short I as at least a variant pretty seriously and then now we just go back to doing what we always do which is internal reconstruction and external reconstruction and what I what I suppose here's a scenario that seems plausible to me I think we know what the etymology of the sequence is it's and so I guess that may have just that yeah with a short I may have been the relative the regular outcome of that that's also what Brogman thought but he didn't know about the two laryngeal so and and then you get Ea changed to Ea by by analogy and there are various ways to do this but if you wanted to do it as you know four-part analogy I guess you would say it's something like a top which is by the way this is a long-dowl always a top is Ea as E top is the X and X you would solve for Ea and you could also do it differently if you like and so that seems like a plausible source of the the younger form and then what we would have is another sort of a poet's choice situation where you have an older Ea that's still around that the poet's up to use especially after heavy syllables because they're composing in weight alternating meter and then you and then and then you also have the younger form that we know from classes as Ea so thank you for your attention that's that's it in short and I'd like just to remind you I am extremely interested if you can think of other places where we could sort of try this out and mainly we I think just need some sort of text which is in like the organization of which is fairly phonological obviously not completely phonological and where we need to figure things out about the phonology thank you


here on the 7th day of October Thursday. We're looking at the Dow having a little bit of a rally here. It's only a $470 and $34,887. We were down that much at some point yesterday. This is quite extraordinary, but the pattern that we've been looking for is the lower case H that goes to a lower case M, and we've got that H. It went exactly to the Chappan Wave inside track repellent zone about a week and a half ago, and then it pulled back from the $35,000 level, pulled back to $33,780, and now we've got another move up. And where are we? $34,930 to $34,970 are the two moving averages, the two trend lines we're looking at as resistance. Go above that. That'll be very impressive. In my show coming up, the Tiger State Editions Hour, I'll show the 120-minute chart, which has automated resistance levels coming up very soon. This is very important. A very good day so far. Of course, it's all news related, and the news turned out to be positive. Going into late yesterday, certainly overnight. And now we're looking at the S&P bumping into that trend line resistance Chappan Wave inside track. It's up 60 at $4423. It's up 1.38%. It does have 1.42%. If it can get by tomorrow's close, if it gets to the 44, 46, 44, 52 area, that's good. Let's go to the QQQ and the X100. It's retracing nicely to the upside. It's got a lot of catch up to do, and it is trading up 5.26 at $364. It really needs to get to the $368 level to say that it's got a further upside to go. Let's just watch and see what happens. IWM running very nicely. It's up 1.68% at $369.223. It was able to get to the $225.60 area. That's good. And let me just look at gold for a moment. It was pulling back when I last looked. Yeah, it's unchanged at $17.64. It's holding okay, but the dollar is actually moving quite nicely. It's holding its gains towards the upside. Crude oil is finally, finally, finally pulling back a little bit down 66 cents, and the TLT bonds. We'll talk about that when I get back a lower low. That means yields are going higher. Thousands of Chappan Tiger technicians are coming up. Check out the opening called 8 Years Better. That's pretty nice calls. Be back in a moment for my show. I hope you can be there. Otherwise, well, I'll see you back soon.


So this lecture is part of an online mathematics course on group theory, and we'll be covering the C-LOV theorems. So the first question is, how do you pronounce this name? He is a Norwegian mathematician, although it's often pronounced silo in English. I think Norwegians pronounce it something like C-LOV, and my apologies to any Norwegians if I haven't really got it right. So if we've got a group G, a finite group G, then any subgroup H has order dividing G, and we can ask the converse. So suppose a number D divides the order of G, we can ask, does G have a subgroup of order D? And most of the examples of groups we've seen do have subgroups of every order dividing D. However, this isn't true in general. Suppose we take G to be rotations of a tetrahedron, then G has order 12, but G has no subgroup of order 6. You can see this. For example, any group of order 6 has a normal subgroup of order 3. And when we look at the symmetries of a tetrahedron, you can look at all the subgroups of order 3, and you can see that none of them are normalized by some subgroup of order 2. So this just doesn't have subgroups of order 6, even though 6 divides 12. Well, there's one case, a very useful case in which you can show that G has subgroups of order. And this, so the Seelow theorems say the following. Suppose P to the n is the largest power of P dividing the order of the group. Then G has subgroups of order P to the n. Secondly, we can ask how many there are? Well, the number is one mod P and divides the order of G. And thirdly, they're all conjugate. In particular, they're all isomorphic. And fourthly, any subgroup of order power of P is in one of them. So these subgroups are called Seelow P subgroups. So this actually allows you to get hold of the subgroups of a group, because at least for order P to the n for n, the maximum order dividing G, there's a single conjugacy class of subgroups. So what we'll do is prove this theorem. Let's first show existence. So we're trying to show the existence of a subgroup of order P to the n, where P to the n is the largest power dividing the order of G. So first suppose G has a subgroup, let's say a proper subgroup of index, co-prime to P. So if this proper subgroup is H, then P to the n divides the order of H and the order of H is less than the order of G. So H has a subgroup of order P to the n, so G does. So here we're using induction on the order of G. We're assuming that every subgroup of order less than G has P Seelow subgroups. Well that was assuming that G has a subgroup of index co-prime to P. So suppose not. So suppose not. That means all proper subgroups have index divisible by P. And now what we do is we copy a bit of the argument that we used in proving Cauchy's theorem. We look at the order of G, and do you remember this is the order of the center of G plus sum over all conjugacy classes of size greater than one times the size of the conjugacy class. Now the size of the conjugacy class of size greater than one is the index of some proper subgroup of G such as the subgroup fixing an element of it. So all these are divisible by P, and this is divisible by P. So the center of G is divisible by P. So G has an element of order P in the center. So you notice this is almost exactly the same argument that we used to prove part of Cauchy's theorem. Anyway, now what we do is we look at G modulo, the cyclic group of order P where G to the P equals one, G is not equal to one, and G is in the center. So because G is in the center, this is a normal subgroup, so we can form this quotient here. So we now pick the seal of P subgroup S of G over P. So S is order P to the n minus one because the order of this group here is the order of G divided by P. And now look at, we've got a map from G to G over P. And this contains a group S, and we just look at the inverse image. Let's call it S prime. So the inverse image S prime of S has order P to the n, and it's a subgroup, so is seal of P subgroup of G. We've shown that every finite group has seal of P subgroups, which are subgroups of order of the maximal power of a prime dividing G. And so next we want to show that the number of seal of P subgroups is one mod P. Well, suppose S and T are seal of P subgroups with S not equal to T, then S cannot normalize T. Well, what does normalize mean? Well, we say that a subgroup A normalizes B if A, B, A to the minus one is in B for all A in A and B in B. In other words, the conjugate of B by any element of A is just B. So if A was the whole group, then the group normalizes B if and only if B is a normal subgroup. Well, why can't S normalize T? If S normalizes T, then ST is a subgroup because S1 T1 times S2 T2 is equal to S1 times S2 times S2 minus 1 T1 S2 times T2. So the product of two things in ST is an ST because this is in T. But this subgroup has order P to the N, so P to the K for K greater than N because it contains T which has order P to the N. And then it has an additional power of T because of S which is not possible as N is the maximum possible power of P dividing the order of the group. So if we've got two distinct set of subgroups, they cannot normalize each other. Now we look at the orbits of S on the set of P set of subgroups. So let's look at the orbits. There is one orbit with one element, which is just S itself, and all other orbits have P to the K elements for K greater than one. This is because the number of orbits is the order of S divided by the subgroup of S normalizing T, where T is some element in this orbit. And as we said, the subgroup normalizing T can't be the whole of S. So this subgroup here is strictly smaller than S, which means that this number here is divisible by P. So what we've seen is that the orbits of S on the other subgroups, there's one orbit of size one, all other orbits have size divisible by P. So the number of conjugates of S is divisible, sorry, is not divisible by P, is one mod P. Well, now suppose S and T are not conjugates. Well, the number of conjugates of S is one mod P by what is said above, but the number of conjugates of S is also not mod P, because since S is not conjugates of T, the number of conjugates of T on any conjugate of S under the whole group G is zero mod P, because these conjugates are not conjugates of T, so what we said earlier, the number of orbits of T on them is divisible by P. Well, this is a contradiction because the number of conjugates of S can't be one mod P and not mod P, so S must be conjugate to T. So there's only one conjugacy class of seal of P subgroups, so all seal of P subgroups are conjugate and the number is one mod P. So the fourth thing we want to prove, I guess that was the second and third thing sort of combined, is that any subgroup of order P to the A is contained in some P seal of subgroup. And the proof of this, what we do is we pick a maximal one, so if not true, pick a maximal subgroup, let's call it X of order P to the A. So we're picking a maximal subgroup whose order is a power of a prime but not contained in a P seal of subgroup and we're going to get a contradiction. So what we do is we notice the normalizer of X has index divisible by P, the normalizer of X is the things that normalize it, so this is the group of all elements G with GX G to the minus one equals X. So the number of conjugates of X is divisible by P. But by the argument above that we used for seal of P subgroups, the number of conjugates is one mod P, and this is a contradiction, the number of conjugates can't be divisible by P and also one mod P. So X must actually be contained in a P seal of subgroup. So we should give an example. We've said the seal of P subgroups are all conjugate and all isomorphic. That only applies to subgroups divisible by the maximal power of P dividing the order of the group. So suppose we take G to be the dihedral group D4, then G has subgroups isomorphic to Z over 4Z which could just be rotations for example. And it also has subgroups isomorphic to Z over 2Z times Z over 2Z. Because we can for example take the subgroup generated by flipping horizontally and vertically. So you see these two subgroups is both of order 2 squared, but they're not isomorphic, so they're not conjugate. So the part of seal of theorem about subgroups of order P to the N all being conjugate only works when N is the largest power of P to the N is the largest power of P dividing the order of the group. So in the next lecture we're going to use the seal of theorems to classify the groups of order 12.