I'm not going to mangle his surname, he's going to talk about challenges and increasing the network stack performance. Yes, thank you. This talk is also a little bit joined with Kristoff. When we reach around slides 28 or something, we might stop there and take it together with Kristoff's session. But we'll see how we are on time. How much time do we have? 25 minutes. Okay. I should have this timer here. Then let's get started. So today I'm going to talk about and make you understand the challenge of doing 100 gigabit and the insane time budget involved in that. And I also have some measurements and I also want to make you understand the crazy time budget. And so I'm going to give you something to relate this time budget to. I'm going to talk a bit about some recent changes which have been accepted. Where we do transmit bulking with X-MID more API and we do Qtisk DQ bulking. We only did the DQ part so far. Then we have some future work, which is like on the receive path. We have some work, we have more work on the Qtisk path and the memory management layer. I'm going to talk to that with Kristoff. It's going to give the next talk after lunch, I guess, about that. And we'll see if I'll do the information about the memory allocator limitations. If I will do that in the next slide.  Just to show that we can do something faster than the slope allocator. So 100 gigabit and Nix are actually starting to come. So it's challenging the network stack quite a lot. When we sort of increase the rate, the time between packets gets smaller. And we have to use less time in the network stack to process packets. If you take a full MTU size packet on ethernet, you can see on 10 gigabits, which we still have some room to work with. But when we go down to up to 100 gigabits, the time between packets arriving is only 123 nanoseconds. That does not a lot work. Inter-packet gap. The inter-packet gap. Yeah, it's actually also calculated into this because that's why I have the MTU size. So the inter-packet gap is 38 bytes. So I actually don't have access to 100 gigabit Nix. So I sort of did a problem solution. I'll just take my 10 gigabit Nix, which I've had a long time actually, and just send smaller packets. So the smallest packets I can send due to the inter-cap of the ethernet overhead is 80, 84 bytes. So that actually I can simulate something which is faster than 100 gigabit. So the inter-cap between the packets is 67 nanoseconds, and that actually translates into a 14.88 million packets per second. I'm going to use this number throughout the slides. I use that as a target because that's the wire speed at the smallest packet size and 10 gigabit. So CPU-wise it's only 201 CPU cycles at 3 gigahertz. That's not a lot. It's just kind of kind of a crazy task amount. But is it at all possible with the current hardware we have? That's also an interesting question. This is very important. I work in the industry and the time doesn't seem to be that important. Is that exciting? Yeah, thank you. So that is actually over the recent years. There are three network stack implementations which sort of bypass the kernel altogether. And there's one called Netmap, which FreeBSD has. And there's also some other solutions to DPDK from Intel, Package Shader. There's a lot of other stuff. The RDMA and Infiniband stuff which Christoph told me about, which has been there for some years actually. And they have shown that basically showed that in the kernel we are not using the hardware optimally because they can do quite significantly faster. So even though it's an artificial network benchmarking, they can show that they can forward 10 gigabit at wire speed at the smallest packet size on a single CPU. So that's worth noticing that they do it on a single CPU. That's sped across multiple CPUs. So the hardware should be fast enough. So the way I see it is that the kernel has been scaling out with the number of cores. And we have sort of missed out on adding latency and more code in the kernel.  And I think Christoph would agree with me that latency-sensitive workloads have already been affected by we adding more stuff in the kernel and the network stack. And then we say, no, it's fast enough because you just add more cores. But actually we've added more latency in the packet processing path. So my goal is to improve the efficiency per core. For example, today if I do a test, I only enable a single CPU. I do a CPU test. I do IP forwarding. I can only forward between 1 to 2 million packets per second. And the bypass alternative is like the YSB 14.8 million packets per second. And that's a really good compare because the network stack is obviously doing more than the sort of fake benchmark where they just forward packets blindly. But it does show that the performance gap is so big that we should do something about it.  So I've talked about these numbers before that we only have around 200 CPU cycles. So I want to give you some numbers so you can relate this to something. I've done a specific CPU testing. So on this specific CPU I measured that a single cache miss takes the 32 nanoseconds. So basically you can only take two cache misses and my entire budget is gone. So that's not good. Cache misses we don't want those. And what we have today is that we have the Linux SKB. We today are structured to our packets. It's four cache lines. And we insist on writing zeros to these cache lines. And we allocate the SKB. It's usually cache hot so it's not a full miss. So let's look at what the timescale or cost of the layer 2 and layer 3 cache misses are. So as I said, the SKB will usually be in cache. But what about the data? The smarts of you I have here is E5 something. Intel added us some hardware which can add the data part of the packet directly into layer 3 cache. It doesn't matter. It publicates it directly so that doesn't have to go through the memory. It happens transparently. I've had a bit trouble finding a measurement tool where I can actually see if it happens or not. But it's quite obviously when you get the cache misses that it's not happening. So I sort of went that way to see if it happens or not. So on this specific CPU I measured that the costs were around 4 nanoseconds and around 8 nanoseconds for layer 2 and layer 3, respectively. And that's the timescale we can work with. So what's the cost of the log and assembly instruction which we use for different synchronization purposes in the kernel? So I measured the cost to be around on this CPU around 8 nanoseconds for log operation. And I also did a test with a user. The optimal case where there's no contention on the log and just log on log on the CPU. And the measured cost is around 16 nanoseconds. I don't know if it all makes sense to have 8-point-something nanoseconds. What? Yeah, it's hard and cache. So this is the most optimal situation. There's no contention. So in my time budget it's still fairly costly if you'd have too many of those. Paul?  The other part is the same thing, different times, depending on how things are for the unit time. And so you'll, unless you, so you'll end up, you're maybe just on the right end of the box. And so what happens is it will take different paths with the hardware and get different lengths. Yeah. Yeah, so it can quickly cost more. So what do we actually want to talk to user space about this in the package, actually? So the overhead of a system call is quite large in this time scale. First, when I first measured it, it was around 75 nanoseconds. And then I figured out that the audit suce call stuff from AC Linux was quite costly. So I disabled that and I measured it down to 41 or 42 nanoseconds for a suce call. So it's a large chunk out of our budget. So what we can do about that is that we can sort of, I'm also size the suce call cost. And there's actually several existing suce calls which can do that. And you should notice the extra M here, the extra M here. So we can send several packets back and forth. I looked at these, actually to send M message and receive M messages. They're not as good as I expected. And we should actually improve the performance of those. Yeah, but it seems, my test at least, I didn't look at the code specifically. I looked, I felt a short time on it. And then I moved on to other stuff. But there's a lot of room for improvements in these interfaces. You also have the send file. Yeah, there's a lot of tricks you can do to get around this suce call overhead. So it's also interesting to look at some of the synchronization mechanisms we have inside the kernel. So we have, as before, we have the spin lock, the lock and unlock for the cost of 16 nanoseconds. Whereas the bottom half disable, it costs seven nanoseconds doing that. And for example, the IQ disable enable is there. It doesn't cost a lot. The sort of surprise to me came when I measured the IQ save and restore because it costs 14 nanoseconds. Which was a lot higher than I expected. And we use that quite frequently in the kernel. It's almost as costly as taking the lock, which was quite a surprise to me. And we have it in the slab allocator, right? Or I think I saw some patches where you tried to remove it. The slab allocator has none on it. No. Then it's the slab allocator, which has that in the hard part. So how did the other solutions manage to actually do this and use the hardware? So one of the general tools of the trade is batching as the main one. And we also have preallocation and prefetching. Obviously staying on the same CPU or numeral node. Generate and change in kernel entry. The what? It's not out of free. It hasn't been there for years. Yeah. So it says out of free. Yeah, out of free. Yeah, but doesn't it only work on InfiniBand? No. Okay. It was constructed by InfiniBand. And then the guys discovered that you can't use it. So they used the ID. Okay. Interesting. I should try to play a little bit with that. Yeah. Then there's, obviously, shrink the metadata and reduce the system calls. Yeah. But the batching is sort of a fundamental tool. So instead of doing the packet processing, we can do batching or bulking. But only where it obviously makes sense. And batching can be done at many different levels like just called batching and packet processing batching. So it's quite simple example. Like if you, the cost, the cost of blocking was 16 nanoseconds. So if you process a batch of 16 packets, when you take the log, the locking cost is sort of a motor size to one nanosecond. If you look at it that way. So in any questions about that, the previous document, I'm going to talk about some recent changes. So what we did recently was sort of unlock the transmit side of the drivers. So now with packet generator, the internal packet generator, we can actually generate the 14.8 million packets per second. This is the 10 gig wire speed I've been talking about. We're just spinning the same SKB. It's completely fake. We have no memory allocations. And what the primary trick here is actually we are bulking packets to the hardware. That's the trick. So the thing what is going on is that we are deferring the hardware tailponder right which notifies the hardware that packets are available. So what happens is that we are just filling up the hardware transmitter ring. And then when we have a certain amount of packets we call the flush operation to the hardware. And that makes us being able to hit wire speed with a single CPU with this fake workload where you're just spinning the same packet. I had a really hard time finding out that this was going on, that the tailponder right to the hardware was so expensive because it didn't show up in any performance profiling with the Perf tool. So the hardware didn't because I guess it was an external device being written to and it was non-casual memory and stuff like that. So the Perf couldn't help me there. So what happens is that the MMIO right will in Perf most of the time it was sort of jumping around to the cost. Most of the time we were playing the next lock operation. So it looked like the next lock operation in this case would usually be the cutest clock. So we'll pop up on top and say the cutest clock is very expensive. But in reality it was the hardware which was slowing me down. So they can tell us exactly how long all the hardware ops were taking and what lay across things like the NUMA interconnecting between nodes and so on and what. And so we could actually determine that the CPU music was actually coming from the hardware. And this was done as part of the more device bring up type process. So this sort of thing traditionally has been done at the hardware level and optimized at the hardware level before it's really got up to the point where we started looking at the network stack. Yeah. I would love to get my hands on some PCI analyzer. I don't have that. So I had to deduct it by other means. Yeah. I'll be very interesting to talk to you afterwards where I can sort of get my hands on such kind of hardware actually. I would love to play with it. No, no. No, no. But generally understanding what's available. So I could like, yeah, yeah, it would be really valuable for me to get some hardware. I can actually see this stuff. So what what we ended up with the we ended up with what we call the X mit more API. So the SKB has been extended with a single bit, which is a X mit more indicator. And what the stack does it uses this to sort of indicate a promise to the driver that another packet will be arriving immediately after this one. So sort of indication from the stack to the driver that we have more packets coming. So, so what the driver does is that it checks if the it's transmits ring queue is full. Then it would need to flush to the hardware or else it can simply add the packet to the hardware ring queue and defer the expensive indication to the hardware. That's that's the way we sort of have today and the cooperation between the stack and the driver. The driver also needs to implement this. So the hard part is we want to to do this use this API without adding adding latency. And so we should only do parking when it's really needed and based on some solid indication from from the network stack. So we should not speculate speculate that if we delay writing this tape pointer and sort of bet on packet schools maybe arrive shortly. It's and it's sort of hard to resist because benchmarking would look really good if you do some fake benchmarking. But we should really resist doing that. So what we do today is that we use SKB lists. So we changed the the XMIT layer of the stack to work with SKB lists. Actually David did that. So we simply use the SKB next pointer and and we can also use the the next pointer to see to use as a XMIT more indicator directly. So when once you build all the SKB lists what's going on is that the TXQ lock which is the hardware lock. There's no longer a per packet cost because we take the TXQ lock and and and we have the hard start XMIT which will send the entire SKB list while holding that lock. And yeah and that's that's that's how it's done today. Instead of sending individual packets down you can actually work with a whole whole SKB list. So this SKB list has to be constructed before reaching that point. So the stack already have packet aggregation facilities which is the generic receive of loading the generic segmentation of loading is sort of connected. And we have the TCP segmentation offering which is a hardware feature which comes in and they actually basically already have aggregated the packet for us. So by allowing to bulk these I'm not I'm not introducing any latency building these up introduces a little bit of latency actually. But at least I'm not introducing latency when allowing to bulk these. It's what? Is that hooked into the system? No it's not it's not hooked into the system yet. That's that's something I want. I want to work on so we can have that work all the way from from user space. It's it's it's on the drawing board but definitely there's a long way to go yet. But this is the only the transmit side where we're allowing we're allowing this we don't have it on the receipt. We have a little bit on the receipt of the generic receive of load where it can build up the list but not from user space. And then then we have the Q disk layer with this before transmitting packets out and we have we have to have it on for the hardware. When we have to push this packet cannot send any more packets we have to queue packets. So if we have a queue in the Q disk is a very solid opportunity for bulking. Really good indication from the stack. That is already delayed so it's easy to construct this list of skbs. It's one of those cases where we're actually reducing latency before because before reducing latency and increasing bandwidth. Because before we were doing a per packet cost for for the stuff in the Q disk. Now we are the cost is I'm also sized over several packets when we are taking them out. The Q thing Q disk logging is a bit nasty and has some extra locking costs are getting to that. And it also has other properties. So a bit about the Q disk path over it. It's sort of nasty because it has six assembler lock operations. And as we saw before it costs around eight nanoseconds. So we have 48 nanoseconds of pure locking overhead. And what I also did is then I measured what is the actual overhead. And it's between 58 to 68 nanoseconds. So we are using a very large percentage of the real cost of the overhead of using the Q disk layer on lock operations. So what we did was on the Q disk DQ side it's the stuff is now a multi-size over several packets. Because when we DQ we can pull out several packets and just make it to the hardware efficient manner. But that is only in case we have a Q. We have other situations one is the MTQ and we do something called direct XMIT. And it also has to take all these six lock operations. So in the future I'm going to work on fixing how we can reduce those lock operations. In the case of the MTQ where you can do a direct transmit. And the in-Q side of the story we still have a per packet locking cost. It might go away if we like we have all the way from user space we can send a bulk of packets and then we will in-Q the entire bulk. And that we will solve it that way by a multi-sizing the cost of the in-Q. So the Q disk most of you are this is all the detail slide but most of you are current developers so it shouldn't be a problem. But I sort of tried to sketch how the Q disk locking is working. This also happens for the direct transmit case so unfortunately. So the first lock would take the root lock and the in-Q and possibly we have to and be in-Q because we only allow a single CPU in-Q packets and multiple can in-Q packets. And we control that by the state running and other CPUs will simply exit if there's someone else running in state running and just in-Q their packets and go away. Once we have de-Q'd a packet or now several packets in this in the recent kernel we take the TXQ lock for the hardware and we transmit for the hardware. And afterwards when we come back we will take a lock again because we have to update this state running bit just to indicate if we are done or else we will also check if there has been in-Q'd some packets while we were in this section. We will check if there has been in-Q'd some packets and then either transmit them or if we are overquartered we schedule a soft I-Q. So the locking is a bit nasty also to fix them. So what we chose to do for de-Q'd is de-Q'd a packet. This is that we chose only to do it for the BQL drivers which is the byte-Q limit. The drivers was implemented that feature because the reason for doing that is we want to avoid overshooting the NIC capacity. Because once we overshoot and the NIC pushes back to us we have to re-Q packets. And the de-Q'd disk layer's re-Q facility is not that good because we can have a very complex Q-disk. And if we have a very complex Q-disk and we ask to re-Q it's difficult to ask the Q-disk to re-Q back in the right Q. So we will require some logic to handle that. The BQL facility shows that it's really good at limiting these number of re-Q's happening. So maybe we don't need to implement a better re-Q facility in the Q-disk. Because the BQL is very good at sort of limiting this. But I've done a lot of measurements and it shows that it's acceptable the head of line blocking occurring. So future work, what do we need to work on? As you've noted, it's part of the TX side I've been talking about. The receive side we also have some limitations. Like the user space stuff also has to be worked on as you pointed out. And we have the FIB route logup. It's very expensive with IP forwarding. Very recently some of Alex's patches went in. So Netnext which optimizes the FIB route logup quite a lot. And then on the receive side we also sort of limited by the memory allocators which we're going to talk about with Kristoff. So one of the future things is that I want a lockless Q-disk. And the motivation for doing a lockless Q-disk is that in the direct transmit case where the Q-disk is empty, we still take all these six locks which I think we should be able to do better. Then we also still have the in-Q cost which we could reduce. That is the case where the Q-disk length is something in the Q-disk. We could reduce that from 16 ns to 10 ns, not too huge an improvement but worth taking. So I actually did a lockless Q-disk implementation. So I did some measurements on that. And it's difficult to implement completely directly if I'm allowed to. There's a flag if the Q-disk can bypass in-Q. So it's difficult to implement quickly to avoid reordering of packets. But if I am not allowing the bypass stuff, I can save 48 ns on my implementation. So it seems like it's quite worthwhile doing. So the transmit side looks good and we also have to fix up the receive side. So the forward testing I was talking about before that's IP forwarding. I also did some bridge forwarding which also is around 2 million packets per second. So what we did some experiments tuning the receive side. So what I call here early drop is you drop the package right after receiving it. We only reach 6.5 million packets per second and that's on a single CPU. So we are kind of confused by that. If you just receive it and just drop it right away, why don't receive the full wire speed? So Alexi, the guy who also do the BPF stuff, worked a bit on optimizing the receive path and went up to 9.4 million packets per second by using the BuildSKP API and do some prefix tuning. But one thing is that the early drop stuff doesn't show the real memory, the interaction with the memory allocator. So how much time do I have left? What? 20 minutes. Okay. We could delay this stuff because this is mostly about the memory allocator for the next slot. So this is a little bit sort of important slide for how the network stack uses the allocators. So if I was just dropping the package early, I don't really see how the network stack uses the allocator because what's going on is that first we, on the receive side, we are sort of pulling the receive hardware ring queue and we will pick up too because we have a budget to 64 packets and when we are doing that, we have to allocate SKBs. So we basically allocate up to 64 SKBs. And then on the transmit side, we put the packets into our transmit ring and we don't free the SKB structure right away. There are some complications of doing that. And what will happen is that in number three, that once the transmit completion has happened, we were free up to a budget of 256 SKBs in one go. So that's how the network stack actually uses the allocator. And what I experienced is that the IP folding is sort of fitting, not the slow path but the slower path of the slot allocator. So what I did was I did some microbench parking of the KMM cache slot allocator. And so I was just, I would just in a loop, fast reuse the same element. So I'm really sure that I'm hitting the fast path, which is lockless. And the cost of doing that is 19 nanoseconds with a preemptive enabled kernel. Preempted enabled kernel. And so that looks okay for 19 nanoseconds. Then I tried to do a pattern of doing 256 allocs and 256 freeze. Based on this pattern I explained before. And the cost increased to 14 nanoseconds. Because I'm hitting a slower path, not the slower. That's the cost, what? Is that the cost per alloc in frame or the cost of all 256? No, that's the cost per alloc in frame. So I also tried to derive the memory manager cost by using packet gen. So I sort of hack implemented recycling inside the packet gen reader itself. But I'm touching all the usual data and area and zeroing and stuff like that. When I do that, because I'm just spinning the same packet, I'm not using our real device, which has this delayed free of the SKB. Because then it will free my data up and we have use after free and stuff like that. So I'm using what we call the dummy device, which what the dummy device does is to receive a packet of just free. So it's basically a free operation. So what I did was with no recycling and with recycling I measured how many packets per second and I just let this into how many nanoseconds it takes. And then I derived that the memory manager cost was 77 nanoseconds, which is slower than I expected. I'm quite sure I'm hitting the slop fast path. But what's going on here is the SKB data page. When I allocate that, it's costing more than the slop allocator. Is there a statistic subsystem? Yeah, I'm going to play with that. I haven't used that actually. So my measurements show that the cost is somewhere between 19 to 14 nanoseconds and the packet gen derived. But it's too large for my time budget. I need something faster. So either the memory management area needs improvement or I need something in terms of a faster memory pool. So I actually implemented a faster memory pool. So what I call a Q-mem pool because there was already something called mem pool in the corner. So it's basically a log free bulk and alloc free scheme backed by what I call the elf queue, which is array-based log free queue. And I implemented this Q-mem pool and I used it and plugged it into the SKB allocator in the network stack. In this geos second, save 12 nanoseconds on the fast path. If I drop packets in the raw table, when I do that I'm also making sure I'm hitting the slop allocator's fast path. When I do full IP forwarding, I'm saving 40 nanoseconds because forwarding hits a slower slop use case that we will be working on improving. So I did some microbench marking also against the slop allocator. So if we have the fast path use case where we just reuse the same element in the loop for the slop allocator, it costs around 18 nanoseconds. If I use it in soft IQ, which is possible because I'm restricting the use of the Q-mem pool to only run in soft IQ in NAPI mode, in the network drivers, I can do that. And my fast path is only 13 nanoseconds. If I need to use the Q-mem pool outside the soft IQ, I can do bottom half disabled and the cost. Then I'm up to a little bit more cost than the slop allocator. For the slower path, not the slow path, the slower path. When I'm using the 256 elements allocation before free, you can see the Q-mem pool even though I'm using the bottom half disabled here. It scales better up and it's around 24 nanoseconds compared to the 14 nanoseconds in the microbench mark. But the real use case for the network is that IPv4 actually showed me, gave me better numbers than the microbench mark. So what's the secret behind the Q-mem pool? Why can I beat the slop allocator? So it's primarily because I have bug support in my logless log-free queue. And I have something called a shared queue, which is a multi-producer, multi-consumer queue which supports when I'm bugging elements in and out with a single compare and exchange. And thus I'm authorizing the cost of the logged compare and exchange operation. And then per CPU I have a single producer, single consumer queue which doesn't require any atomic operations at all. And I actually speculate I could implement something faster with a simple per CPU stack actually. But with this I also exercised that part of my library. So what I implemented was what I call the ELF queue, the array-based log-free queue which is like the basic building block. And the killer feature is basically bugging with a single logged compare and exchange. And I support this multi-consumer, multi-producer stuff. Another feature of the queue is that it's array-based. So it's also cache-line efficient because it's just a queue for pointers basically. And on 64-bit I can have eight pointers per cache-line. So that's also some of the improvement comes from that. It also has the possibility of doing pipeline-optimized bug, DQ and InQ because when I'm copying out these elements instead of just having a simple loop where I'm just putting out elements, if I do loop unrolling I can measure that I'm using the pipeline efficiency of the CPU more efficiently. In practice I removed it from the op-film proposal because what I found out if I had a too advanced DQ, InQ pipeline stuff where I used it doesn't matter what I used, but in practice what happened was it would use more simple instructions to implement this loop. And because the code size were larger I would get more instruction cache misses so it actually slowed down my performance adding this smarter pipeline-optimized InQ and DQ so I actually removed it for code size purposes because it turned out that you don't want instruction cache misses more. So we could basically just view it as an array of pointers used as a queue but with bug-optimized lockless access. So the Q-Memple was sort of a practical implementation to find out if it's actually possible to do something faster than the K-Memperial-slop allocation which is quite optimized for quite a few years and sort of also to provoke the M-M people to do a generator code kind of stuff and say it's actually possible. Can we do something to make the K-Mem cache just as fast, integrate some of their ideas, perhaps extend stuff with bulking which the next talk will be about. Gustav will talk about that. Yeah, and we will talk about that in the next slide I think. I think this is the end of my talk. So we can postpone the memory manager discussions but any input on challenges I missed on the network area? The main thing is that you can address the performance issue. We have mostly the latency issue. So the problem is that the network operations for individual packets take too long and that's why we use, for example, the IB-Verbs implementation because we can map the device control structures into the user space and we basically flip the bits in the user space to trigger a device operation. So with this approach you may be able to saturate it to 40GB or 100GB link but you cannot really address the individual packet latency issues that we've seen. And I wish this would have been more integrated into the main stack. The IB subsystem right now is kind of a side guy in the kernel and I think one of the reasons was that the network guys felt that this wasn't kind of an off-road API or something like a TCP off-road API that I didn't want to support and there has been a history of antagonism. I wish there would be more integration there. And yeah, I was a bit surprised when I first read the slides and I didn't even mention the IB-Verbs API but please pay more attention to this. I didn't actually know it. I guess something had to be done. But I'm trying to reduce the latency of the stack. Yeah, I thought that. A lot of other people have been focusing a lot on just the throughput and just scaling on more CPUs which hurts the latency. Yeah, we also have the queuing problem. The queues are added everywhere and so the queue load, as mentioned before situations cause these high latencies. It's a significant problem. Okay, a bit of a different kind of question but seeing how you're working so hard to push the hardware and the CPU itself to its limit, have you been at times kind of closing your eyes and dreaming? What kind of instructions and other hardware facilities in the CPU would have greatly helped me here? What were you wishing for that actually the CPU had that doesn't have now to make this work significantly better? Do you have any ideas on it? Not really. I haven't thought about changing the hardware. Yeah, but I guess I should. So I was going to ask quickly, clearly your talks preliminary and you're doing a lot of work on Intel, have you looked at any other architectures, power or anything like that? No, unfortunately. I've primarily been testing on Intel hardware, that's true. I have been testing the lockless queue or the lock free queue on ARM processors and what did we also test on PowerPC and stuff because we had to test the correctness and we have to add memory barriers for different architectures. But it was only a correctness test for the lock free queue, not performance wise. So there are various approaches right now in the works on the hardware level to actually increase the speed. One in the ARM space is to map a register of the CPU directly to IO so you wouldn't have to go through any IO bus to reach the device. Intel has something called the Omnipath where actually you have an interconnect directly on the processor, it bypasses all the busses and goes directly out to IO and this will cause the ability of the processor to do much faster IO. On the other hand, the network stack won't be able to handle it, but now you have to use CPU instructions to do IO and we are utterly not prepared for this and I'm not sure how we actually could support this in the future. I was going to ask a related question. Intel as well have their DPDK stuff which is more towards the actual devices doing networking platforms rather than just an endpoint. It sort of seems like that in OpenVis which another, there's a whole bunch of these projects that somewhat intersect. How does your work interact with others in that space or is there better coordination happening behind scenes or something? Well, I actually haven't used DPDK myself but I have looked at it and some of the inspiration for improving the network stack does come from the number size I see DPDK can produce. I know other people in Red Hat is working on helping improving DPDK making it more easy packageable because now it's really difficult to just take and use. You would have to sit in the middle of the room. Just with the transmit more flag and stuff like that, if I remember correctly, that actually came from the block layer, the storage stack, the technique for actually being more efficient on your transmit. Have you looked any more at what the current multi-cube block layer code is actually doing from the storage side to enable high IOPS and whether any of that's actually relevant to the network stack? Well, I haven't actually looked at that and I guess I should look at what the other systems are doing. Yeah, because I mean basically that's the equivalent for the storage stack. Now we're dealing with flash devices and so on that are doing millions of IOPS. That's the infrastructure that is being used to scale both per CPU and across multiple CPUs for that sort of equivalent packet throughput that you're talking about here for the network stack. Yeah, but I haven't looked at it and I guess I should ask Jens about it. Yeah, definitely. Anyone else? Looks like we're done a little bit early. Thank you very much, Jesper. That's very interesting. I think you're crazy to try and get done that time.


Air purifiers and home sealing can reduce particulate matter levels in homes, even in highly polluted environments such as Kathmandu, Nepal. This article was authored by Leslie Edwards, Paul Wilkinson, Gemma Rutter, and others.


So now we're on, this is the mudstone bedding around us, this flat, nearly horizontal plane is the bedding, and if we look on the bedding right here, we'll see this thing that looks a little bit like a starfish, but it's not a starfish, it's what we call a Glendonite. And notice how it's a Glendonite and it has these little pointy ends, 1, 2, 3, 4, it can have numerous pointy bits. Each one of these is a crystal, so it's a crystal that in three dimensions would be sending up here, but it's been weathered off the top. Now this crystal was originally not Glendonite. Glendonite is a name for these things that comes from the Sidney Basin, a locality north of here, and what these things were originally is called Ikeite, which is a calcium carbonate water crystal. So these things only exist in fairly deep, very cold water. So these things established without any doubt that this water was very cold seawater. And what's interesting is these things get trapped in a mud, and then they get replaced by minerals that are stable, like magnesium and calcium carbonate, without water. So that these things are pseudomorphs, a pseudomorphic replacement of the crystals that were there.


Wherever you sit, stand, whatever you touch, there is memory and transaction happening all the time. It must be a conscious process. If it's a compulsive process, then you will pay the price for it. It is not that this is sin, that is sin. The question is, what is your priority in your life? Sadguru, how can a person stay committed to someone in a relationship? Is it natural to love someone and yet be sexually attracted to others? What should be the proper course of action that should follow or how does one handle this? See, there is a psychological integrity, there is emotional integrity, but there is a biological integrity also. Integrity does not mean morality. Integrity means you create a situation where it works best for you. So, when we say integrity, suppose I say there is a certain integrity to my body, this means it's strong and resistant to a whole lot of things, isn't it? It doesn't mean I'm morally stuck in something. So I'm talking about… I want you to understand the word integrity in that context. We're talking about integrity in terms of strength of this life. So if that is the thing, is it true? Do you remember how your great-great-great-great-great-grandmother ten generations ago looked like? Do you remember? No.  Yes or no? Body remembers, isn't it? Body remembers your forefathers a million years ago. Yes or no? So what you're calling as my body is a heap of memory, isn't it so? Hmm? Memory or no? See, now from Nepal you came to Shillong, you eat a lot of this Meghalaya food, your features won't change because your body remembers what is your genetics, no matter what.  Because there is evolutionary memory in this. Do what you want, it never gets confused, isn't it? You start thinking I'm a dog and start barking like one, still the body won't change. Yes or no? Mentally you can, but body has such a deep-rooted memory. So this entire body what you have is essentially a certain integrity of memory. If that loses that memory integrity then you will see it will become vulnerable to so many things. Now the nature of the body is such that anything that you touch with a certain level of involvement will naturally absorb that memory. Not mentally, physically it'll absorb that memory. I mean traditionally in this culture we call this runanubandha, you heard of such a word? Runanubandha. What this means is physical memory that you gather. Why people, you know, you will see this with people, let's say in their home, they will go and sit in one place. This will usually the older people you will see. They want to go and sit in the same place, they won't sit in another place. Have you noticed this?  Next time you chase him somewhere, he comes, he goes and sits in the same place because there is memory. Today there is forensic equipment where you're sitting here right now, you went away. After two, three, eight hours if they come, not with a dog, a dog can easily do it. But with forensic equipment we come here and just check this chair and we know it was you who were sitting here, not somebody else. So there is memory. Wherever you sit, stand, whatever you touch, there is memory and transaction happening all the time. Well, you come from Nepal, in India also it's very much there, in south is very strongly there, north maybe it's become weakened. We… people never give salt to another person. Do you know this, here also? If somebody gives you salt, you say, please keep it there. Because there are certain materials which transmit memory much better than others. Salt, sesame seeds, lemons, like this, if you give traditional people, they'll say, keep it there, I will take it. Because they don't want to develop runanabandha with you. Now in India, if you see older generation of people, if you try to shake their hands, they'll do like this because they don't want to get runanabandha with you. Because the idea is to keep the integrity of your body's memory in such a way that it doesn't become vulnerable to other things, that you become a very integrated life. If you want to nurture yourself to be a certain possibility, then you have to maintain the memory integrity. This is what runanabandha means, you keep your physical memory to the minimum. A sexual interaction is something where a huge amount of memory is taken from one to the other. So always, not in this society, everywhere else, forever people saw the advantage of keeping that memory to the minimum. If you make that memory very complex, you will see to be at ease will become very difficult after some time. There will be pleasure, but there will be no joy in your life. You can observe people, don't go by what I'm saying. You can observe people, they will have pleasure, they will giggle all the time, but you look at them, there is no joy in them, there's no ease. Because the ease will go away with excessive memory. This is not only with sexuality, there are many other things that you do like this. Right now you see this very much in the western societies. Wherever I go, especially in America, people will say, Sathguru, where is my hug? I say, it must be with you. Why is it with me? It is not like at a certain moment when you feel close to somebody, you hug them. It's like all the time you have to touch people. Because Sudeh's psychiatrists are analyzing these things and saying that is because they have not been sufficiently touched by their mothers and parents at an early age. When they grow up, they're desperately longing to touch somebody all the time. All these things have a serious impact on one's life. How much physical contact when the child is born, this will determine how much physical contact they will long for later on.  In many ways energy-wise, it's not a separate life. It's still attached to the mother's body. Actually, if by nature, if people go till then, they must be drinking the mother's milk and connected up to four-and-a-half years, that's how naturally it was. So the energy doesn't mature. At that time, more and more memory that you come get from the mother is better and better to strengthen this. But once the child begins to move out and becomes an individual, life is organizing itself. People come to me and say, Sathguru, can you bless my daughter? Can you bless my son? First thing I ask is how old? If they say fifteen, sixteen, eighteen, all right. If they're over twenty-one, I say no.  You may still think emotionally that's my child, but as far as life is concerned, it's become fully separate. Generally, the course of life is considered to be approximately eighty-four years, or one-thousand-and-eight cycles of the moon. The cycles of the moon and our body is very directly connected. Only because our mother's bodies were in sync with the cycles of the moon, we are born, otherwise we wouldn't be born, isn't it? Hello? So, a full life is considered one-thousand-and-eight cycles of the moon, which will approximately consider as eighty-three to eighty-four years. So if one crosses eighty-four, it's considered a full life. In this, the first quarter is the only time when it is connected to parentage. After that, the child must move. Because energy-wise, you cannot connect those two lives anymore. So that is one. The longing, if you have not created enough integrity within yourself, the longing for another body multiplies. Even though the hormonal phase may be higher between fifteen to twenty, the longing to bind and bond with somebody increases after twenty-one years of age, because unknowingly you have… you're like a satellite who fell off the main mother. You come off the motherboard. Now you want to attach to something unless you find some integrity. This is why between twelve and eighteen, one must do lot of sadhana to strengthen the body so that you don't desperately bind yourself to something or somebody. You must consciously, if you wish to take a partner, it must be a conscious process. If it's a compulsive process, then you will pay the price for it. This is not a question of morality. It is not that this is sin, that is sin. The question is, what is your priority in your life? If your priority is to make this life rise to a higher possibility, then you must be conscious. If you want to somehow live and go, it's okay. Thank you, Satgu. I'm sorry if I discouraged. You just answered my question, Satgu. Thank you so much for that.


his reading is actually quite short like I think it's probably 1500 words or something um Terry you're so funny okay so anyway um I think go forth and annotate uh and I will be looking into your annotations and I'm they may or may not make it into the actual keynote but I will let people while listening to my keynote know that this is a space where people have already been discussing these ideas and that will make it a little bit easier for me not to take more than 30 minutes for my keynote because right now my slides I have like 125 slides which you know I probably won't be able to go through but just also letting you know the article on third learning space so I didn't write an article on third learning spaces but there's a part in my article that you're that Nate just linked right now where I talk about first spaces and I've got an annotation there for you to refer to that so Michael is asking about the first step um to do to annotate well first you have to create a hypothesis account yeah and michael's michael's put a link in there to you and just to clarify uh my someone who was asking on the is there this article on the third learning space yeah so that's not an article I don't have an article on it it's just a line in this article and I haven't actually referenced the work on third so there's third places and third spaces and I they're not really interchangeable but I use them sort of interchangeably um uh homey baba talks about the third space as a cultural hybrid third space like a bridge between two cultures but I actually mean it's third place as in it's not the formal space of school and it's not the least formal space of home but it's somewhere in the middle like a cafe or like a social space or like the um in our institution the plaza between classrooms where you meet someone and you can have a chat and then move on that kind of space great yeah so for for anybody who hasn't been um participating before uh in annotation um you can follow the links that I've been putting in chat directly to the article and the article will appear with annotation already enabled um and you can start reading and looking at the annotations that are there if you want to start participating yourself you need to create your own hypothesis account which there's a little link up in the upper right hand corner to do and in fact I could share my screen mohawk that makes sense and actually go to the article myself sure unless you wanted to do that uh no you go ahead all right I'll check the chat there's a lot of chatter chattering going on yeah and I'll just say as a side note to folks who were um you know asking questions about blackboard integration and like how do I get the key and secret to install it in canvas and stuff reach out to hypothesis support um and my my colleague Michael is here uh in the chat and so they can connect with you we're not going to focus so much on technical details here in this session because we're really trying to kind of show off annotation but we also want to help you so um connect with support at hypothesis um and or ping michael in the chat and he can he can get you squared away let me go back to sharing here so um you should be seeing I got some toolbars in my way you should be seeing mohawk's article here literacies teachers need during COVID-19 and you'll notice that some of the words in it are highlighted in yellow right that means that someone has highlighted them and anchored an annotation to them if you click on one of those anchors you'll see that that little drawer on the right sort of slides open as we call it the sidebar and you can close or open that sidebar with that little chevron at the top there also it's a little bit hard to see this but you can also um adjust the width of the sidebar by dragging the chevron back and forth if you want to make it really wide or really narrow you can do that um and you'll see at the top I'm already signed in so I'm going to log out to show you but if you don't have an account yet you can use this sign up link at the top to quickly make yourself an account if you already have one and you're not logged in you can use this login link like I am to actually log in so now I'm logged back in again and so you can see that uh already 11 annotations have been made on top of the article here and so this is the time when we'll just kind of work together to read this text and start to either annotate by making highlights ourselves on particular words you see when I grab a word it calls up this little annotate highlight control and it's usually a bit closer to where you're you're yeah I know I'm gonna refresh I'm gonna refresh here because just to let just to let people know something that's a little bit tricky if you just highlight that's going to be private to you and nobody's going to see the highlights but if you write an annotation that's what's going to appear for other people and you can choose whether you want to make it private or public or you can send it to a private group if you're doing that for a class but right now we're doing it publicly yeah and I'm not I'm not sure why it's appearing so far away one of the things that's a little bit different about hypothesis than some of the other annotation tools is hypothesis comes to the content or where the content lives so like you'll see this is actually an article that ma published in alphanar media right I don't actually know that the outlet that well and so we haven't taken this article and put it somewhere else right what we've done is we brought hypothesis to the article and because it's a full blown web page and something sometimes there are tricky little things going on like we saw that advertisement pop up and so forth that can change the experience so when you're annotating out on the wild web the wild world wild web I like to call it here you might find some little idiosyncrasies like that and if you are having any troubles annotating or any technical help or assistance feel free to reach out in the chat there and my team of super helpful people including Michael and all will help you out I'm going to be quiet for a minute and let people read I'm a noisy typer with your powerful fingers yes it's because I learned to type on a typewriter when I was like nine or ten and with typewriters you have to like press really hard yeah me too and I also play piano a little bit so that's that's where my fingers that's a nice feeling yeah that's why I don't like those new keyboards that are all like there's no key movement in them and it's like tapping on a piece of glass or something yeah I think what they're probably going to do is that the keyboard will be flat but you'll have the tactile feeling of as if you've pressed something I think that's the kind of direction things are going right yeah seems like it Alice is asking how can I select a text to get started do you mean a certain amount of just highlight the text instead of getting a copy paste option where but you'll get an annotation but for some reason on nates it doesn't show it shows like very far down yeah see where that is yeah I don't understand why that is um so I try sharing mine and see what happens yeah maybe yours maybe yours will be better I'll just refresh try it one more time here okay nope mine's still far away okay let's try yours okay um and for people who are like installing this in canvas and so forth you can do that if you want um that's a couple of extra hurdles to get to just to annotate this article the easiest thing to do right now uh for uh Jay Easting who asked this um would be uh just click on one of the links in uh in the chat there to open the article and just start annotating here getting it in into canvas is uh maybe too big of a hurdle for the short time of this workshop so if people are seeing this so see as soon as I click here the annotate and highlight appear right immediately under the part that I'm highlighting so and then when you click on it it opens up on the right hand side yeah something must be weird about my computer today I don't know why mine was acting differently it does it to me sometimes too but usually on the phone Ryan is saying you had to highlight certain sentence several times just before it took that usually happens to me on the phone I don't know or maybe if you don't have the book maybe that's part of what the bookmark would does but I'm sure that Nate has the book yes PDFs can be annotated James so there there are two things that are interesting about annotating PDFs one of the most interesting things is even if you're if you download a PDF of an article because it's like behind the paywall or something and you annotated there's a way to to make it still sync with other offline annotations of that PDF once you connect and maybe Nate can share how that happens probably easier if you have the elements integration but it works even if you don't right yeah I mean the thing about PDFs it's um sort you can definitely annotate PDFs so web pages PDFs and there's also the possibility to annotate e-pubs in certain circumstances um PDFs are are annotatable in a couple of different ways so if you put them into uh some sort of web web delivery location like uh you know google drive or the file storage system in canvas or on your own website or something like that um and so that they open naturally in a browser then they become annotatable just like a web page would be and you can um either annotate them uh you know using like the chrome extension uh or through a special link like we're doing now with my house article um for the lms integration um yes there's a there's the ability to either annotate PDFs or web pages um by kind of adding them to file storage as you assign them in the lms as an annotatable reading so I'm sorry I said that Kathy Esumiller had um given some links it was actually Kate Mitchell Kathy was probably in another article of mine which was annotating sorry I don't know if Kate is with us today but I'm sorry Kathy but I thought that was here yeah and if anybody is having kind of questions or issues around technical support with their lms at their organization I apologize that we're not planning on addressing them here um but Michael is in the in the chat and he's um he's uh you know busy answering things away I also just pasted our support email in there and so um we'd love to help you figure that out we're just not going to do it so much in the context of this this workshop so Mo had asked uh if uh making a linked word or words into uh the anchor for an annotation is a good idea yeah Mo it could be a little tricky right then because when you go in to select the highlight it could also trigger the link that the highlight is on so um sometimes it's helpful if you want to annotate uh something that's linked is to highlight some words surrounding it too if possible um and that way it makes the annotation a little bit bigger than the link itself and gives you a place to kind of grab on to it maybe gave us a baby Yoda jiff oh he's so cute I admit that I didn't get on to the whole baby Yoda bandwagon but maybe I should I couldn't get into the whole Star Wars mind I tried I couldn't sorry we don't want to start a whole discussion about that today hey one roll has a pretty cool question here Michael that maybe you can address um wondering if they can change their name their username and hypothesis because they started out with a pseudonym and um and then wanted to change it later um and uh that's actually an interesting question just around um kind of account identities and so forth um because you know pseudonymic annotation can be a great way to have a kind of public participation without like giving away all the privacy at once um so pseudonyms can be kind of useful for sometimes for public annotation right but there are other times when you do actually want to annotate as yourself and and you know have your full human identity at play um so uh that's a it's something interesting to ponder now in the context of lms integration because it's a single sign on environment with the lms um the uh it sort of takes the uh the uncertainty of what your name is going to be a way right because the lms is providing it based on your your lms sign in so it's only in the wild that you really have that uh that flexibility to decide about what who you want to be and what you want to be and i'll i'll hope that um uh michael's already said that he answered you great yeah so debba brings up an interesting question about if everyone contributes a paper will have hundreds of annotations and how do you deal with that in classes you know that came up um in the in the earlier session today too and that led to a little bit of a conversation around the use of private groups um so first of all in the lms environment right uh every class and now in at least in cannabis every section can have its own and actually automatically has its own private group where everyone in that class or section is is annotating and so when that you put a document for instance a pdf or a web page in front of that group right and assign it as an annotation um exercise they're not going to see the annotations of um other private groups that have previously annotated that same document and in fact in the lms environment they won't even see public annotations on it either out in the wild it's a little bit more wild on the on the world wild web um where like on as we watch uh working here on name do you want to i'm gonna i'm just going to do something so i'm going to tell them what i'm doing right now sure so if you look now you'll see that it's like something like 11 annotations but you see this red button here there are 56 new annotations so we're 100 and something people wear the annotations and i forgot that i need to sort of refresh so now there are 48 it says oh that's 56 anyway yeah that gets that your question de macchina asked about um how quickly do the annotations update so they come pretty quickly but you may have to hit that button that uh just hit to to update your view and you can see there's already more new ones there since i've already got quite a few useful resources that i'm with um the first like for example someone mentioned um when i was talking about workload literacy someone mentioned assault noise work and assaults were from what i really respect and i i actually recorded a podcast about grading uh ungrading i guess with him and jesse stommel recently and i'm not sure if that podcast podcast is coming out uh before like you know i need to check with i need to check with the person whose name now escapes me completely who recorded the podcast with us but there's there's also a link to his book about labor-based grading in there and it's a very different very social justice focus approach that's asking that i think this is good looking at so it's interesting how you read something and you know someone really well but sometimes when you're writing a particular article you've forgotten that you know this person under what they did and imitation you can help a lot with that and i think this this really is a pretty good reflection of what it's like to annotate together in a class too right because you have different levels of experience with digital tools you have different levels of literacy on both digital and informational reading you have you know different perspectives which is one of the great things right is because those perspectives can start to come out and so the first time that everybody does this is always probably the most difficult and most painful but as you start to do it you become really adept at it i'm to the point now where i actually don't even like to read unless i can annotate at the same time so it's gotten that i can't really read offline i don't like to read on paper anymore one of the other things is that i think if students are reading across different articles i think it can help students who are struggling with avoiding plagiarism because if you're reading across articles and then you come back to write your own piece like a literature review it's it's it works like the way note cards used to work back in the day if anyone is as old as i am around that age where there was still no cards because you you can clearly see which part you're quoting from the article in which part is your own comment on it and so you're less likely to make that mistake of inadvertently reusing someone else's words without paraphrasing them i don't know if anyone used to have that problem but like before hypothesis i used to copy and paste texts to remember to use it and then it would be possible that you sometimes forget to paraphrase it unintentionally i always assume students don't intend to forget to do that yeah and i don't know if people remember way back at the beginning of the workshop when jeremy was talking about how annotation collaborative annotation can make reading visible active and social and i think ma just kind of hit on all of those in the sense that you know we can see now the traces of everybody's reading on top of this article so there's a visibility that's brought to it and there's a visibility in what they were thinking about right through the annotations themselves and the conversations that ensued from it and it kind of slows people down somebody was it somebody in this conversation earlier up earlier in the chat talked about intentional reading and i think there is there's a way in which annotation can slow things down and help people become more intentional about their reading and and actually maybe read less but read with a more active mode of thinking about how they can connect what they're reading to something else another idea another person linking out to another document i i'm sure some of these annotations must link to other things i haven't seen one fly by recently but it's it's obviously each annotation is like a little mini web page that has its own url and can contain links and videos and images and things like that as well actually ma can you show them how each annotation has its own url do you know how to find that i guess if i click on it i know it's that little um the little box with the arrow just below your cursor up nope down it down for the flag near the flag next to the flag yeah not the flag oh yeah that one yeah to share yeah so that your url and yeah sometimes i share these on twitter you know it says here like oh i really like this annotation i'm just getting impatient 25 minutes max so there's a lot of really great conversations going on in in both the q&a and the chat that i can't even keep up with they're coming so fast and furious so i know that michael's doing his best to answer things there and other team members um and i i hope that folks um see this as um first of all a way to get engaged with mahan or thinking and her keynote that's coming up um because there's a lot of really great uh sort of material that's come out of her process to working toward this keynote like she mentioned her twitter thread which we linked to in the chat before um this process that we're going through today all the thinking that she's done i know she's had a lot of conversations with with other people in various formats about it and it's really i would you say mahan that the experience of this keynote has been different for you than other other talks like that that you've given yeah definitely the the engaging with twitter before i've done before i don't think i've ever done a combination of engaging with twitter and hypothesis and and there's also a lot of back channeling going on in the large twitter vm with a lot of my friends um i think partly because of the social isolation again because i don't have that opportunity in my face-to-face environment as much as i used to like i can't just talk to people hey you know i'm thinking about doing this for my kid you know i don't have that anymore so i have to be more intentional about her and also just feeling like i can't be with people at the conference where the night before my keynote i could just you know let them know what i'm thinking about you know like usually that that's what would happen you arrive a day before the keynote you meet a few people or you're there for a few days of the conference before your keynote is happening um and i don't think keynotes should be this this isolated thing that's not part of anything else that's going on in the world and maybe also the thing that's really special about it is that everyone in the world is going through a similar thing there there are obviously other things on top of it um especially in the u.s but the the pandemic aspect and they need to go online and that suddenly those of us who have been doing online for a long time are sort of center stage in our institutions in ways that we've usually been marginalized before um but it's a very interesting space to be and i think that kind of makes me feel like more people would be interested in what's going on with what i have to say yeah and i can imagine a lot of the folks who would normally attend olc are in kind of in note in your shoes right where they're they are the kind of people who are engaged in this sudden move to remote delivery that everybody's been engaged in um and so as a community we've all been uh we've all been kind of riding that that wave hopefully surfing it so yeah uh refresh how many annotations are on there now there's at least 68 um 75 so you can see the things that confuses me is that i've got different um uh different sliders right there's a slider here for the annotations and there's slider here for the article yeah depending on which one you're looking at and obviously every time our official computing the beginning yeah and this is part of the situation right of uh because we bring the annotation to the website all the complexity of the website is there and then we throw the complexity of the annotation on top of that and sometimes it can seem a little confusing so you know uh working on a clean pdf you can even annotate a pdf that's just local on your own computer by just opening it in a browser let's say you have the chrome extension you can open a pdf um just locally on your own computer and annotate it there and it can be a much kind of quiet or less noisy experience than a web page that might have ads and all sorts of things going on in it um here's a really uh interesting point too about annotating pdfs because pdfs have unique fingerprints if i annotate a pdf on my local computer and maha annotates the same a different copy of but the same pdf on her computer in egypt our annotations will actually see each other and find each other and i'll be able to see hers in mind assuming they're public or we're in the same group because hypothesis ties them together behind the scenes knowing that they're part of the same document uh so it's a little bit of black magic it feels like but it's um it can be a really powerful uh a powerful experience and so that kind of gets to some of like mike duval was asking about browser compatibility you know we we have i'm sure michael could link you to um or if you didn't already he could link you to uh a statement about our browser compatibility uh sort of support at hypothesis we basically try to support all the modern browsers um somewhere more pesky than others the it is true that we only have a full-blown extension for chrome although there is a sort of almost finished firefox plugin as well but um that doesn't mean that one can't annotate in those other browsers it just means that we haven't enabled a you know native plugin for those browsers um there was another thing that i was thinking about as i was reading all of these annotations and thinking that this is like seven to something uh people is that if we were in an actual seminar stock costs there would be no way that every single person would get an opportunity to participate and there these are 78 annotations but there wouldn't be time for 78 people to speak for a minute or two and then for me to thoughtfully respond to them but what's going to happen now is that this is sort of a semi-synchronous space like right now we're all annotating at the same time but i can just keep looking at these for the next couple days and people can keep coming back and like i said in my key you know during my keynote if i can't get through everything because i think by the way you can just go and look at the hypothesis annotations later and continue to engage with these ideas right and that's something that when we when we valorize the face-to-face seminar classroom with its beautiful energy we forget that it's not a very equitable space and that it's a limiting space it's a limited limitation of time and space but online asynchronous work sort of solves that problem yeah a lot i think jeremy he might have even said this earlier that uh that people have described collaborative annotation as the closest thing you can get to having a small intimate face-to-face seminar when you don't have that ability like you have more people or you're at a distance or you're doing it at different times asynchronously yeah but when you say it's close to your you're making it say you're sort of saying it's close to but less than and i'm trying to say it's actually close to and more than i would agree with that as well um totally so i don't i didn't mean to i didn't mean to suggest that it was less of course there are different kinds of affordances you don't get the body language uh you know channel through the annotations as much unless you upload a video of yourself i guess but yeah there's so many there are so many rich possibilities in the kind of discussion and who who the discussion can involve uh in annotation that a lot of times you won't see you won't find in a face-to-face seminar where some people are just either not ready to or not allowed to speak yeah actually one of the things that i've been thinking about a lot recently is that if we go back face-to-face people will be wearing masks so we won't see their facial expressions well hear their tone of voice they can still use bodily gestures but from a distance but we won't be able to see their faces so it won't really be face-to-face like the face-to-face will be on zoom and the face-to-face like the in-person will not be face-to-face it'd be mask to mask right and i did teach a face-to-face student once and i know that you can sort of tell what a person is feeling from just looking into their eyes but i think that's a skill that we're all going to need to learn especially if you're meeting students you're knowing for the first time so you don't actually know what they look like without the mask um that will be an interesting situation but we're not used to it yeah i mean this is this is such a a unique time to be like we've the world has never had all the digital affordances and we still don't have them spread evenly across the world for sure but we've never had all the digital affordances and has a good reason to use them and it's it's i feel like it's stepped forward both in some really great ways but also some troubling ways and i think you get a you get a lot of that thinking in this article and i expect you will in your keynote as well there's still like hundreds and dozens of kind of minor technical questions too and i keep flipping back and forth from this the heady dialogue that we're having here to these can i ask a question can i ask a question sure that is a little strange about not strange i don't understand why she's preparing that saying can students know how many annotations or interactions are happening with their with their whole annotations and they can because when they when someone responds they'll get notified so i'm not sure why don't why don't you want them to know people interact and kathy i just uh i i guess i have the ability to allow you to talk with your voice if you want and so i did if you would like to respond with your voice you may not have been ready for that let's see it can you hear me now did i do okay okay so good thank you for coming to me so the reason i ask is uh think about like social media platforms uh instagram twitter and with the student somehow sometimes uh the status or the satisfaction that comes with seeing all the people that just like their tweet or something and then as opposed to tumble yeah whereas like in tumblr um there seems to be a lot you know the people seem to speak to the appreciation they have for not necessarily being able to tell how many people are interacting with with the blog or with the post um so just and i think what surfaced that was when you mentioned uh the valor of the face-to-face the fact that in a large class not everyone can contribute and i was just it in the face-to-face class it's so easy to see i i don't know i think it would be beneficial if you can't like look at whether or not 20 people have seen your tweet and ignored it or your well they can't i think they can't tell that they've read it yeah so they can't tell someone's read it but they can't know if someone replied to it and it requires a more substantial thing i don't think they can tell if someone's read it right exactly okay so they can't necessarily tell though like if um my friend kaya 65 people click on her annotation and open it up and expand it whereas no one expands mind okay that's that's my question all right thank you yeah yeah so just the responses which is okay because that's a substantial type of attention rather than the liking right and i don't think there was there was i just gave a heart to jim luke's annotation but i literally put in an annotation text and just drew a heart but there isn't the hearting type of thing that you have on twitter we can just like and people just like something they haven't really engaged with it so i think that's a good point that you're making there um yeah you don't want that difference between just someone's stuff gets attention on other one other people's doesn't yeah and i'll just i'll apologize in advance there are so there are so many different questions popping around now that we may not be able to systematically answer them all um really ranging all the way from um what kathy was talking about and in that conversation just there about you know what you can see quiet for a while so you can do that not necessarily i mean we like all the all the questions especially the bigger picture ones because that's why we invited you here uh maha even though you're obviously very proficient at the details as well um farhad did ask if uh when we were talking about the ability to annotate pdfs so yes if you emailed the same pdf file to everyone or distributed to them in some other way like maybe they got downloaded it from an lms or something like that um if they set up their their um computers and everything correctly with you know they'd have to use the chrome browser in this case and have the chrome extension installed and there's a little uh switch you have to flip so that it can annotate local files but if each student did that and we have help files about it but if each student did that uh they would be able uh and they were annotating either all publicly or all in the same private group they would be able to see each other's annotations on top of that document so it is possible to do do things that way farhan we have a lot of um great resources on our uh on our help website that um michael spent a lot of time stewarding uh so um definitely if you if you walk away from this experience with any remaining technical questions i invite you to visit our help section i'll put a link into it in that chat um and we also have a way there to like file a ticket and get a request and then uh and someone will we'll get back to you with an answer um debber brought up an interesting thing maha uh and we don't see it here on this page uh what is the difference between an annotation and a page note and i noticed that all the annotations so far on your article are our annotations or replies to annotations but there is the capability to add what are called page bless you all right uh to add page notes as well and page notes are annotations for the whole page so they're not anchored to any specific text on the page and if you scroll all the way to the top of the window is where you see the possibility of having um page notes there so you could add a page note just as ma is doing it right now as a great demonstration um and you'll notice also that a lot of people aren't out though she's adding a tag perfect yeah make sure people know you're being sarcastic right um dare you post it so now debber you can see that this this article has both that page note that isn't linked to any specific text and the annotations that are anchored in specific text and so people have been asking about like usage on a smartphone so it's a little tricky on a smartphone actually and part of it has to do with um they're just a screen real estate you have i bet ma you probably have some experience trying to annotate on your smartphone right yep i do it a lot doesn't always work yeah it works best on chrome again honestly i have to switch to chrome every time so it's possible but it's not it's not a flawless experience because we don't have a dedicated mobile app that's optimized for mobile use and just a screen real estate of having a text and the annotation interface together um is a pretty complex problem to solve so we haven't quite done that yet um we hope to but it is technically possible and we know that of course that would make a big difference because a lot of people are in environments where they need to read on their phones that might be the only way they could read a digital text and so we certainly understand the need um so i think for me and i don't know people know about this but there are lots of different ways of getting hypothesis to work one of them is with the chrome extension or the platform for whatever it's called one of them is to go to the hypothesis website and paste the link but the one that works that seems to work well on the phone but doesn't always work well in general is the via dot hypothesis dot hypothesis slash the url that's like the quickest way to like you're already on a url and you just add the via dot hypothesis that was at the beginning um so i'm gonna i'm gonna just can i demonstrate this name yeah give it a try this is a link that one of you uh posted for me so i'm just gonna go via dot hypothesis dot is and just this and as soon as i do that sure and just so you know if you don't if you didn't memorize that link that i'm gonna type in um if you just go to the hypothesis website there's a menu item paste the link at the very top and you can you can just paste a url or it will create it for you here let's go you can see there are no annotations on it right now but yeah another way which is i'm looking at something else is the bookmark where you just click here and that opens it and if there were annotations this h will have numbers on it so you see over here it's a 17 for some reason but maybe there's a 17 i don't know why it's a 17 it's an open path i think that batch may not update yeah there you go ahead nana i'm sure oh no i actually just caught about 159 uh annotations so far that includes replies hi john again i just lost the connection to this so john is actually um john works with us here at hypothesis um and he's uh he's asking a question back to you folks who've been asking about notifications in the in the lms so if anybody has any input for john um please give it back to him there uh thank you jennifer for coming and and anybody else too i realized that we're about five minutes away from the top of the hour um and it's been a really uh really great experience um i thank you all for staying so long um big thanks to all our our panelists um i know manica manica sorry i'm mispronouncing everybody's name today manica and cat are still here ben might still be here do yeah ben's still here i know uh ronica had to leave and obviously there are a couple of questions in the q and a that i think are probably have quick answers is there a table that compares hypothesis improves all and the question about can you tag someone responding i don't think you can tag someone but if you respond to the person they'll get a notification yeah i saw those and i was just um i'm not totally avoiding them but there's also there's dozens of questions that have been asked in chat that we haven't had a chance to address yet either um a couple of people did raise that question about perusal perusal is another annotation um tool that has an lms integration it has a couple of different things about it like you have to upload text to it or you can actually have your students buy textbooks through it that then become annotatable and it has some nice things about it um it's a completely different model uh than the way hypothesis works um uh and we uh have been talking about better ways to compare the two tools so we we don't um we don't have a really good summary usher yet of of that comparison um but we would try to try to have something for you i definitely recommend exploring both because they have just that the fundamental difference of whether you're bringing the annotation to the text or it lives or you're bringing the text to the annotation can make quite a big difference perusal also has some built-in sort of quote-unquote AI robotic grading mechanisms that some people like uh i'm not really a big fan of automatic grading mechanisms myself but if that appeals to you um perusal might be interesting yeah darlene has asked a really important question there around textbooks and textbook annotation and so um i will say uh darlene that that is like the a third chunk of a big area of content right that a lot of people do reading and the annotation would be really powerful in and so um right now if if the textbook exists as in pdf format and can be um accessed that way although it also could be a mighty big pdf and it might not work so great as a gigantic pdf for all those reasons um but that is one way that it could work if it uh some of the open textbooks work well because they actually you know appear to the world as websites um so if it's if it's an open textbook like that they can be annotated we are in the middle now of working with some of the big textbook platforms and providers to get hypothesis embedded into those environments so we we know that people want to use um hypothesis on textbooks and this is one of the big questions around stem use actually because a lot of the stem classes make heavy use of textbooks but we really need to have better ways to have hypothesis embedded in those native readers that where people experience the textbooks a lot like the vital sources and red shelves and macmillan's and piercings and center of the world so we're working on those um kind of partnerships and integrations um and so that is kind of the next area where we're expanding into but it's it's not it's not there yet yeah so here we are at 1 59 p.m pacific time uh what time is it for you maha it's 11 i think she's hungry yeah i tried to get her to eat something before she started but she's playing with her okay well maybe we should since it's 11 at night at chiro maybe we should let maha go to bed and and maybe get some food for herself and her daughter um it was so great to have you here what an honor thank you so much i really look forward to your keynote thank you thank you cat also you're still here good to see your smiling face um and everybody else who attended thank you maha did you want to say anything before before it ends just a very big thank you and i'm going to go back into these annotations hopefully i have a couple of days to sort of try to absorb them thank you all so much well again this was um probably the best hypothesis workshop i've ever attended so thank you all for making it be what it was i uh i learned a lot too and i'm overwhelmed we will um be capturing both video and text-based artifacts from this and distributing them out so uh y'all signed up with email addresses and as soon as we can we'll be emailing you with with outcomes and artifacts from this experience including cat sides thank you all i've been so inspired by all the panelists and all the people who've joined us today um you know it's just so nice to connect with everyone and thanks everyone for being here and monica jeremy anybody else want to say goodbye franny thank you everyone thanks everyone really great people people have turned off their cameras and are slipping into the evening i think great well i'm going to turn off the recording then and we're going to call this a wrap