So, hi everybody. My name is Falker Simonis. I'm working for a small company called SAP. We are doing virtual machines, Java virtual machines, a commercial one called SAP JVM. We are doing the PowerPCAX and S390 ports in the OpenJDK, which I'm leading. Since recently, we are also doing a binary distribution of the OpenJDK called submachine. You can grab some stickers after the talk if you want. Now, let's concentrate on class data sharing in the hotspot VM. So, the other speakers have laid a good foundation for this topic, and I will concentrate on how class data sharing is actually working in the hotspot VM, how the new application class data sharing, which will come with Java 10, will work and share some details about the implementation and the current restrictions. So, my slides, they are all on GitHub. So, if you know, sorry for that. Yeah, you can look at them later if you want. So, class data sharing is supposed to cache pre-processed class metadata on disk to improve startup performance and reduce memory footprint. We've heard that several times today. So, short history of class data sharing in the hotspot VM. It was introduced since quite some time in JDK 1.5 in 2004 already, but at that time it was quite restricted, so it only supported the client VM and serial GC, and it could only be used for caching system classes. It was done at that time. It was in the Sun JDK. It was done in the initial contribution for OpenJDK 6 and 7. Then sometime nothing happened on this topic since, but in JDK 9, there were several improvements. Support for the server VM was added for G1s, parallel and parallel old GCs. Also support for shared strings, so not only classes, but also strings could be shared. And with Java 9 support for application class data sharing was added, but in JDK 9, it's only in the commercial Oracle JDK as commercial feature. Commonly OpenJDK 10 with JAPS 310 application class data sharing has also been added to the OpenJDK. So this is actually a short overview of Andrew's talk just for people who didn't saw this. So we have on the right side a Java class, and for every Java class, the hotspot has to maintain a certain kind of meta data which is stored in the meta space. Previously, this was the permanent generation since Java 8 is in the meta space, and this is all C++ objects. So there's an instance class, and the instance class links back to the class object which represents the type of the full object. And then we have the class loader data which also links to the class loader in the Java heap, and we have a class loader data graph which links together all the class loader data, so actually all the class loader, so the VM has a chance to iterate of all the classes which it has loaded so far. We have the constant pool which was mentioned before, the constant pool cache which is used to speed up mainly the interpreter operation by caching a resolved object, the constant pool, and finally we have the method data, the constant method data which contains the bytecode and other things which aren't supposed to change, and finally then we have the end methods, so the compiled methods which reside in the code cache which is again a different area of the heap and which we won't touch in this talk. Okay, so how does CDS work? So before we can start to use class data sharing in the Hotspot VM, we first have to create the shared archive. So this is an offline step and this is also one of the problems of the class data sharing implementation in Hotspot we will see, I will talk about this at the end of my talk when it comes to the limitations. So by just calling Java minus X share done, there will happen a lot of stuff and I will briefly go through these steps. So first, the VM allocates space. You see the address, that's actually the address where the shared archive will be mapped into memory. So every instance of the Hotspot will map the shared archive at the same memory address. You can configure this with a command line option if you're not happy with this address but generally it's 32 gigabyte. It will be mapped into 32 gigabyte. Then it will load in the classes to share and you may be surprised but your JDK or JRE, if you download it it already contains a class list which is a plain text file with a list of classes. This is generated at build time. So when you build the JDK as it was shown in one of the previous talks there is a small Java application which tries to mimic other small Java application. It includes some of the containers, some of the util classes and so on and generates, and from this class this is generated which is part of the JDK and when you run share dump on your host this class will be used to preload all these classes and create this shared archive. Then a lot of stuff is done on the preloaded classes like class verification for example so this hasn't to be repeated later on when the shared archive is mapped into your space, into your Java process. You see the number of classes so there is about 1,200 classes in this class list file. Some unshareable information is removed for example if you remember the image I showed you before there are links to the class loader for example in the meta space and obviously these are different in every VM instance so this cannot be shared in the shared archive so such information is removed. Pointers which point from one class to another are relocated. Then as I told you with Java 9 we can also store some string objects and symbol tables in the shared archive as well these are also dumped into the shared archive and finally the links to the Java mirror class which is actually the pointer back into the Java heap is also removed because every time the shared archive will be mapped into a Java process space this link has to be updated because it's obviously different in every running instance and then finally the whole archive is dumped into a file and you see the archive contains read write and read only spaces because obviously some of the part is really constant but some part like for example the instance class has a pointer to the class object to the Java mirror and this has to be patched for every running instance so like every shared library also the libc library for example contains constant parts and read write parts which can be patched and this will bind these pages which are changed by the process to the Java process so they actually are not shared once more than one Java process is using this archive in the end you see the parts which are mapped into the heap so you see this is another address and this is why this only works with g1gc because g1gc has the possibility to map some parts into its memory region which contains the saved strings and in the end you see we generate this shared archive which again is stored by default in your JDK directory under lib server in the file class is jsr Java shared archive again you can change this name this location we will see that later on in the other examples and the size is about 18 megabytes of the shared archive so just a trivial hello world hello CDS sample Java program we just print out hello CDS and then we read in a read line so we can analyze the process once it's running so it doesn't stop now how do we use CDS we just put the x-xshare.colonon option into the command line and run our program and nothing special happens so how can we verify that now our classes really have been loaded from the shared archive well we can misuse the new logging API and instruct it to give us a log of the class loading and when we run it this way we get a line for every class we just loaded and every of these lines contains a reference to the location from where the class has been loaded and if this class has been loaded from the shared archive you see a source shared object file for example not all files have been loaded from there so for example the Java internal I don't know I've abbreviated it class pass file order was obviously not in the initial class list so it is not has not been dumped the shared archive file so this is loaded from the module from the base module right from the file system finally also my example application has been loaded from the file system because it was not obviously not in the shared archive in the class list we can just check how many classes have been loaded from the shared archive and if we grab for shared object files we see that 477 classes have been loaded from the shared archive and if we do the opposite so grab for all the classes which have not been loaded from the shared archive we see just about five so for toy application like Hello World which your estimation was right it's about 500 classes we see that most of them get loaded from the shared archive so that's fine so performance Christian told us that we shouldn't use time for performance measurement but this is not really a serious performance benchmark so I just used time to do some measurements so we see that when we run with shared archive to run we get about 9% our performance improvement for the whole application and if we measure the time until our application class gets loaded which is usually the last class for Hello World program we see that it's about 30% faster so I will show some numbers later for Tomcat but unfortunately I couldn't reproduce this 20% or 30% which Michael mentioned I'm not sure if I did something wrong but I think in my measurements it was not more than 10% to 13% improvement so let's see how much memory we can share again we run the Hello CDS program in background and then we use the PMAP tool so Christian introduced the PMAP tool I think it's really a great tool and probably the only tool is the real true information about memory usage because it's the right information from the kernel so it has a lot of options you could use even XX to get a lot of more information I just pull out some of the information which I think is interesting for us so if I run PMAP on our Hello CDS process you see the binary which is mapped into memory you see here the java heap so by default on my machine java heap has about 2GB but from these 2GB there is just about 129MB are really mapped so it's read write the other part of the heap it's reserved but not mapped so there's actually the difference between the RSS and the virtual space which can reduce people and then finally you see our shared class file which gets mapped to several locations so the first two lines with FF that's the part which contains the shared strings and which is mapped into the java heap by G1 you see and then again you see the different parts, the read write parts which can be potentially patched by the instance which loads the shared archive file and you see the constant file which is truly shared so this will always be shared between all the instances while the read write part maybe at some point in time privately mapped to your own instance of java and you will not have sharing so for example the byte codes that's a good example people think byte codes for example what do you do when you do debugging well you patch the byte codes so you cannot really put them into read only memory they are in the read write section and most of the time they are shared but when you start debugging your application and patch your byte code you will just make this page it's a copy on write you just make this page private for your process and only all the other process which don't touch this page will share the memory okay so this was a toy application just to show you how everything basically works so now I've tried to use CDS and the new app CDS which comes with java 10 with Tomcat and ngrinder it's some kind of application I really don't know what it really does it was just a big war file which I wanted to deploy in order to get a lot of classes loaded so Catalina options it's actually environment variable which influences the way how Tomcat starts and we can now launch Tomcat with class data sharing on it will use the shared archive which you have just created in the first step inside our JDK directory and we also use xlog to log all the loaded classes into a file just to get a baseline of how many classes have been loaded so when we run what count on that we see that Tomcat with this ngrinder application loads about 12,000 classes and when we now grab for the number of classes which are loaded from the shared archive we will see that it's just about 1,100 so it's if you remember the shared archive which we've created initially contained about 1,200 classes so most of them have been used by Tomcat but Tomcat again uses a lot more classes so how can we improve this situation well we have an online option called dump loaded class list when we run Tomcat with that the VM itself will print out a list of all the classes it has loaded and things it can share in a shared archive so let's just use that and dump that in a file and when we look at that file we see that again that's just about 3,000 classes from the 12,000 classes because CDS only allows by default sharing of system classes which get loaded by the boot class loader so obviously that's about 3,200 but all the other applications classes cannot be shared by the classic class data sharing so here enters the scene application CDS which promised to allow our class data sharing also for application classes and even for custom class loaders again we start Tomcat and add the use app CDS option so you can do this you can try this yourself with the early access OpenJDK 10 builds and when we look at the results that's actually pretty disappointing so we see that it's about 300 more classes can be shared now but that's actually not a lot so what's the problem here loaded class list still only dumps the classes loaded by the boot class loader and the platform class loader and the application class loader also known as system class loader so obviously Tomcat itself is a dynamic application and it loads not really many classes through the application class loader from the class class but it loads a lot of classes with own custom class loaders and unfortunately the dump loaded class list application doesn't handle these classes so what can we do here well let's first use the file that was generated with the 3500 classes and see how we can regenerate shared archive with these classes at least so we use the share dump option like at the first slide and we turn on application class data sharing and now we can tell them the location of our class list so this time we don't take the default class list which comes with your JDK but the one we have just created and then we can also tell the JDK the location where to write the created shared archive to so we don't want to store this archive into the JDK directory just to a second play and for some unknown reason this is a diagnostic BAM option after the first time I will file a bug to change this because I don't see why a shared class list file it's a normal product option and shared archive file is a diagnostic option which has to be unlocked first anyway that's how it's today and when you look at the created shared archive you see it's much bigger now and the shared archive which was created from the default class list was about 18 megabytes in size and this one is now nearly 50 megabytes in size it obviously contains more than 3000 classes so the first one only contained about 1200 and now we can use this shared archive again by turning class data sharing on using appcds the location of the shared archive so we don't want to use the default shared archive but the one you've just generated but actually appcds as I told you promised to support class data sharing of classes not only of the classes loaded by the application class loader but also for classes loaded by custom class loader so how can we reach this unfortunately we saw that the dump loaded class list doesn't help here so we have to sorry I'm just somehow my slides get out of order so how can we create this from by scratch a class list to enable application class data sharing also for custom class loader so when we look at this class list of classes and unfortunately there is no documentation for how we can use application class data sharing for custom classes we have to look in the hotspot code to the class list parser cpp file and there we see that this class list file cannot only contain class names but it can also contain another format like class names with an ID and with yes with with references to the super classes and the implemented interfaces and the source of these classes and when we look at the output of the class list load with the debug option we see that actually this output contains exactly this information so it not only tells us the loaded classes but also the class loader which loaded the classes and the source files where they were loaded from and from this lock output we can actually assemble easily class list file which contains the required information and I have brought a small tool called cl for cds class list for class data sharing which is on github you can easily google for it there is not many programs with this name but when we now we can now run Tomcat with the logging class load in debug mode and dump that to a file we can then just call the cl for cds tool to create a class list from the class loading log and then we can use that list and when we do that we see that now the archive is about 100 megabytes big and when we run Tomcat with this shared archive and count for the classes which are truly shared we see that it's about the whole number of classes and when you call for the shared classes we see that it's about 9000 now so that's better it's still not optimal but it's better and what's the problem well app cds still has some limitations so for example it doesn't support 1.java 1.5 classes and obviously this ngrinder application still contains some old classes also dynamically generated classes can be shared because they are generated dynamically at runtime different classes a class which is loaded several times by different class loaders can only be shared one time for one class loader that's another limitation and also classes which get modified by the class loader itself not by an instrumentation agent but there are class loaders for example netbeans that's why it's not easy to use up cds with netbeans which has class loaders which load a class and then do some bytecode changes to the class and then feeds it to the VM obviously that cannot be done with the cds implementation in hotspot the dumping of the shared alpha is an offline step which has no idea how the class loaders modify the classes another problem is that bytecode rewriting is switched off for the reasons I've told you before so interpreter speed might be slower so that's actually it here's another link to the slides and you have any questions please come to me I will be here all day thanks a lot


Hello and welcome to the session. In this session, we discussed the following question which says, find the area of a regular pentagon whose each side is 4 centimeters long and the radius of the inscribed circle is 5 centimeters. Before proceeding with the solution, let's recall the formula. For the area of a regular polygon of n sides, this is equal to n upon 2 into a into r square units. Where we have this n is the number of sides of the regular polygon. A is the length of each side of regular polygon. R is the radius of the inscribed circle. This is the key idea to be used for this question. Now let's move on to the solution. Now we are given a regular pentagon. So in a regular pentagon, we have n that is the number of sides is equal to 5. And we are given that each side is 4 centimeters long. So we say that we have a is equal to 4 centimeters. And also we have that the radius of the inscribed circle is 5 centimeters. So that means r is equal to 5 centimeters. Now we have the area of the regular pentagon is equal to n upon 2 into a into r square units. We put the value for n, a and r. So this is equal to 5 upon 2 into 4 into 5 centimeters square. Now 2, 2 times is 4. So this is equal to 5 into 2 into 5 centimeters square which is equal to 50 centimeters square. So the area of regular pentagon is 50 centimeters square whose side is of measure 4 centimeters. So final answer is 50 centimeters square. This completes the session. Hope you have understood the solution for this question.


What is happening guys? Staff Sergeant Massey here also known as R. Chisel on social media and welcome to another episode of US Army Basic Combat Training mini series, Your Future Starts Here. For today's episode we're going to be talking about the US Army uniforms. It doesn't matter if it's the OCP's, the Dress Blues, but for me what really matters the most is the beret. This is that moment where it completes your look as a soldier. But if you are a trainee halfway about to graduate, you can see right there the finish line is right there. Without further ado we're going to share you a few more stories as usual from trainees and the drill sergeant and what it means to them to wear the US Army uniform. So today we have them going through deferred issue basically where they're going to receive their ASUs and anything associated with that so berets, shirts, pants. Once they get their dress uniforms they start to see everything that they're receiving at that point and they're just looking forward more to graduation because that's the uniform that they're going to wear whenever they walk across that field. So they still have to learn how to shape and wear the beret. We actually take them up and give them a class. We watch them very carefully because they actually shave their berets and so if we just let them go loose they're going to put holes in them and cut the wrong parts and all that stuff. So we still shape their berets with them and we make sure that they're fitting properly and looking good. The best advice I could give for somebody that's looking to join the Army you're not going to know exactly what you're getting into but just go in with an open mind make sure that don't don't keep quitting on the back burner like that's an option because in your mind it shouldn't be an option. Whenever you join the Army you're going to be basically doing the best thing you could ever do to jumpstart your life. I mean whenever you're whenever you're looking at you know everyone that's your age right now and then you're going to see a noticeable difference once you graduate of everything that you've accomplished things that you thought you would never do you're putting yourself ahead of your own age group and I think that you know the country needs something like that right now because you know a lot of people don't understand what it means to serve. So today we got fitted for our skirts we got fitted for our shirts our berets our pants my pants were a little bit long they had to fix it we got size for right side shoes my 7.5 and we got our big coat and we got size for our graduation uniform. It made me feel very like to keep on going to not give up because you're one step ahead and it made me feel like you're this close to graduating so just keep up. I joined because my family right now they're struggling financially and I found out the Army could help me with my college especially since I wanted to do criminal justice. Drill sergeants my battle buddies my family a lot my family back home was seeing me writing letters telling me to keep on going and they're waiting for me back home because that will motivate me a lot. It feels amazing because I remember getting that letter from my mom and I just saw her handwriting it felt like she was writing she was speaking to me it feels great to receive letters every time. So my favorite MRE is this one right here it's the cheese tortellini and it comes with trail mix vanilla pudding crackers and peanut butter chocolate and the cheese tortellini which is not the good but I like the dessert.


Hi everyone I love all fishing methods which sometimes makes it difficult to decide which one to pick and the river season for fishing well for coarse fish opened on the 16th of June and I've been itching to get out but do I pick the bait lure or fly for most enjoyment I guess it sort of depends on what mood you're in ultimately I want to I want to know what type of fishing can I do that is going to provide a good challenge but also a result and up there the problem with vlogging is that it asks different questions that don't necessarily relates it to myself like how many big fish do you need to catch to make a fishing video I'm quite happy going out and blanking as you probably know but I don't think it makes very good viewing and yes it's an inconvenient distraction and as I only recently moved here I decided to spend the weekend sorting out and organizing my fishing kit as I didn't decide that I didn't end up doing it it took so blooming long but what I did make time to do was edit an old vlog from the Thames that was originally 20 minutes long and whittle it down to two and hopefully that shows some of the pros and cons of fly fishing and why you should get the lure rod out if you need more action I found I find that chub are considerably more shy and tucked away than the trout but maybe that's just a southern rivers thing where they get a bit more pressure anyway enough waffling make your own mind up thanks for watching and hopefully I'll post something of a recent a more recent fishing trip very soon cheers tightline I'm using the Maxcatch two-way I was going to fish a little black sedge little beauty got one on the daddy long legs not huge but it's just nice to see and come up and smash it affection in miniature little Jack I'm a tiny micro shed wicked scrap oh I've some of that


Hi everyone, pleasure to be here. Nice to see all of you and many familiar faces, some. Yeah, that's really good that all of you are here and my name is Alex and today we're going to talk about the importance of transactional metadata and the avenues it's opening for Apache Cassandra. So before we begin, I just wanted to mention that I have several copies of Database Internals book and I'm going to giving them out for the people who are asking good questions. So I have four copies on me and I have several more stashed, yeah. Mick is, yeah, they're gonna be handed out. Anyways, let's begin. Let me first give you an overview of the stock and we're going to start with and talk about the motivation, about motivation why we wanted to make the changes that we've made. Since the best way to plan the project is first to have clear requirements about what we want to achieve, let's set also clear requirements and because I'm presenting to a technical audience I'm quite certain many of you would be curious about the implementation details and in fact, many of you are probably here exactly because of that, okay. That was weird because, yeah. Lastly, since transactional metadata has already landed in Cassandra trunk we can also start talking about the avenues it has opened and the future steps and things that we have started working on. So let's first talk about how we came to the conclusion that we need to make the cluster metadata transactional. Around 2010, a new wave of NoSQL databases mostly inspired by Dynamo TV paper have appeared and all of them had several things in common. Having homogenous nodes running on the commodity hardware and implementing cluster metadata through Gossip was one of the features that they had in common in fact. With Gossip you can have huge clusters of nodes with spotty connectivity and nodes will be collaborating probabilistically propagating state throughout the cluster. In Cassandra Gossip was used for disseminating many things among which were schema and cluster metadata which consists of membership and ownership information. Membership information identifies the presence of the node in the cluster while ownership information identifies the target nodes for reads and writes. There is nothing wrong with disseminating cluster metadata through Gossip since that's what it's best for. The things start getting tricky when we expect Gossip listeners to have a coherent view of this metadata and act as if this was the case. So Gossip has become a subsystem that used to react to a membership and ownership changes announced by the nodes and it was creating among many other things something which was called token metadata objects. So anybody who worked with Cassandra for long enough probably knows about its existence. Token metadata is built from the information passed by the subscribers of the Gossip state changes and it is invalidated whenever the ring version changes which also means that it has to be rebuilt. Unfortunately, this was happening on the hot cot path which we ultimately would like to avoid with the transactional cluster metadata project. Now let's talk a little bit about the motivation. Most of the Cassandra clusters operated today contain only hundreds of nodes which is a relatively small number compared to the number of participants in something like a mesh network or some internet of things setups where it might be impossible to live without Gossip or spanning trees or something like that. We believe it may be more efficient to use simple broadcast as a first line of distribution and natural cluster communication as a backup to disseminate information that is currently being distributed by Gossip in older version which will make it simpler to reason about and lead to quicker time-bound convergence. Let's talk through the process of the node bootstrap. So when the node comes up, it contacts seed nodes which can give it the last snapshot of the membership and ownership information known to it but unfortunately we cannot know that there are, whether or not there are other nodes that are currently trying to make changes to the ownership of the range we're about to bootstrap. The most straightforward way to work around this problem is to introduce a delay so that other interested parties could make their intentions known. To reframe this a bit, shadow round and other delays you see mentioned on the slide here are necessary since we cannot really grab a lock on the cluster instantly or quickly ensure there is no other node that will attempt to modify overlapping ranges concurrently to us. Another source of complication is that the current implementation Gossip does not provide a way to ensure the strict ordering of these changes which may lead to the out of order propagation of the concurrent events. Say if two nodes are bootstrapping simultaneously, coordinator A may observe node one joining before node two joins and coordinator B may observe them joining in the opposite order. This also means that there can effectively be two disagreeing views of the ring. Of course in practice all of this is not super important and can be worked around since operators are usually careful and make sure to only permit safe operations in the cluster but unfortunately even if we're willing to ignore these things there are still things that we cannot easily solve with Gossip. I will give an example of a transient data loss issue later in my slides and besides because of the issues with automation software or human mistakes there is still some room for error in the current implementation which we cannot fully eliminate. So one of the things we've been discussing in the Cassandra community lately is a concept of token ownership. While an idea of hashering and a consistent hashing seems to be good on the paper there are still some things that aren't obvious at the first glance. First one is that it is using randomness in form of consistent hashing and load balancing for load balancing and distribution and this does not guarantee optimal load because unfortunately even though your hash is providing uniform well distribution your data is not uniformly distributed. There are partitions that outweigh the others in terms of size and load and there are two to the power of 64 tokens if you assume the murmur three partitioner and the space between these tokens is rather large which means that there are still many chances for hotspots. V nodes as you may know were an attempt to address the granularity issue and they did allow us to pick the multiple sources for streaming but this has not completely eliminated the problem with load balancing. The ultimate goal of load balancing is to have a way to gradually and dynamically relocate the data. The second problem is that token ownership makes it such that nodes position in the ring alongside with the replication factor dictates which ranges it is going to own because of the ring which means that you cannot pick a subset of nodes for a specific range and relocate the data to them in order for them to own this data without doing token arithmetic or relocate a range to a subset of nodes without having them owning the neighboring ranges as well. So I considered this a problem. It is also important for us to have a single point of reference. Any node turning the cluster should be able to quickly and easily find the latest membership of the ownership state and modify it without having to wait for other nodes to announce their potential actions. If you take a look at how for instance Paxos V2 is implemented in Cassandra its payloads contain a list of participants and because there is no other easy way to ensure that two nodes have exactly the same view of the ring and we should make similar information available for every operation even eventually consistent ones and we can make it very compact. A single number identified the ring epoch should be sufficient. Of course this applies not just to Paxos but eventually consistent operations just as well and coordinators and replicas should agree about the view of the ring and we should have mechanisms that allow lagging participants to quickly catch up to the latest epoch for the operation that is ongoing. And lastly we would like the ring consistency for the duration of every operation which means that a version of the ring available at the end of the read or write operation should be compatible with the responses that the coordinator has collected from the replicas. All of these things are required to maintain consistency levels honestly. We have for instance, if we advertise the replication factor of three we effectively are making two promises. By the end of the quorum write we will have at least two copies of data available and that the quorum read that follows the quorum write will observe the effects of this write. And let us see under which conditions this assumptions or promises may not be held. So I'm gonna walk you through the example of the transient loss of the write during the bootstrap. Imagine a cluster with only three nodes with a replication factor of three and nodes A, B and C present in the ring and we would like to bootstrap the fourth node which is X. And as you may know in Cassandra in order to bootstrap a node you have to add it to the cluster at first as a pending node, right? So it's going to be receiving the writes but will not serve any reads. So the reader still sees the node X as joining. Meanwhile, from the perspective of the writer X has fully joined the ring, A is already excluded from the write set and from its perspective the placements for this write are going to be B, C and X. And the result is the transient loss of the write during the bootstrap. So effectively the reader is going to collect the nodes A and B or may collect nodes A and B as its replica targets and since it's quorum it is sufficient for it to complete the operation and the writer may collect C and X as their write targets and since these quorums are non-overlapping we have a transient loss of the write. And one of the problems is that reads and writes are not monotonic with the ring evolvement and there is no order guarantees between these two and this may result into this problem that has existed in the system for a long while. Now, let's discuss other problems. So if you're in the community long enough you may have heard about Cassandra 1304 which was a bug that was resulting from the schema disagreement in two nodes corrupting the data during the read. So essentially a read repair was corrupting the data in the SS table. However, even if the results are not as drastic there are still problems that are difficult to resolve so there could be racing create table statements and like schema disagreements can be manifesting in many different ways. So let's now set up the requirements for what exactly we're trying to achieve. First, we would like to make sure that we remain honest about the consistency levels even during the range movements. This is a non-negotiable correctness requirement. We would also like to have safe concurrent bootstraps. Multiple nodes should be able to join and leave the cluster and you should have zero knowledge about Cassandra internals to make this happen. Next, we would like nodes to find agreement about the state of the cluster metadata quickly and easily ideally using natural internal traffic so there should be no additional traffic introduced to the cluster. Lastly, we would like the metadata changes to be completely transparent for the user. Clients connected to the cluster should be should see absolutely nothing while range movements are in progress not even a single unavailable or a timeout. That would be great. So let's talk a little bit about the implementation details. We have discussed two things already. So let's try to sort of incorporate them into the implementation. We will maintain compatibility with the tokens during the upgrade period and for as long as you continue using all node tool commands. So everything remains 100% backwards compatible. After migrating to the cluster metadata nodes however will own ranges and not tokens. Good news for Cassandra maintainers is that while doing that we have also made sure to remove the range wraparounds. So all ranges are now from minimum token to the maximum token. So there is technically no ring anymore. It's just like flat line from the minimum possible token to maximum possible token. For now we have decided to maintain full placement compatibility with the old token model but the plan is to allow dynamic relocation fully decoupling range ownership from token ownership. So these should be not interrelated in any way. This is technically possible even now it's just not exposed to the operators just yet. Right now token metadata changes are just re-computations triggered by the listeners of gossip state changes meaning that in 5.0 and previous versions instead of this we will now have explicit transformations that live in their own classes and explicitly describe what they're doing. So you will be able to inspect every change executed on the system in lots of detail. If we assume that cluster metadata is an atomic variable and say that we start with some empty state we can formulate the change events as pure functions that take cluster metadata from one state to the next one. No side effects in any one of them whatsoever. Results will be computed on all the nodes in the same way and there will be no way for two nodes to have divergent views of metadata if they have seen the same sequence of events which they will. This also means that every cluster metadata can be uniquely identified by the epoch. Epochs are monotonically incremented and comparable. So any two participants can quickly and easily understand who has a higher epoch and has to catch up. To preserve the history all of these transformations are going to be stored in the metadata change log. Long operations such as bootstrap, decommission, move, replace are now able to survive the node crashes and are now safely resumable. The metadata change log is also going to be strongly consistent. There is only one permitted order of the events and there can be no other history. The log is stored on a dynamic subset of nodes which are called a cluster metadata service or CMS for short. This service can be consulted at any point in time to find out the latest state and make the modification to this state. The participants in the cluster metadata service are managed by the cluster metadata service itself without any user intervention. Reacting to the changes in your cluster size and health of the metadata log owners. So you simply specify the replication factor for the cluster metadata service and the rest is done for you. Newly joined nodes only have to know a single node in the cluster in order to discover the latest state of the metadata. So there is no concept of CMS nodes or it's gonna be introduced to your scripts or setups. Since some operations have no visible trace in the four metadata such as bootstrap that was rejected because there was another bootstrap ongoing already so the ranges were locked or recreation of an existing table. The only events that actually make it to the log are ones that actually modify it. So your script which creates, I don't know, a table, if it exists will have no effects on the cluster metadata object because it's not going to grow it exponentially large if it's even running every single second and we only preserve the things that actually make relevant changes to the metadata. The rest are just transparent and do not bump the epoch. And finally metadata state is computed from the log as I have already mentioned. Starting from the empty metadata state we apply all the events in the order without any gaps and we this way obtain the latest version of the metadata state. Since we plan to allow rather many operations in for that can transform the metadata state in order of tens of thousands per day we decided to use data structures that have structural sharing in memory to reduce the memory footprint because if you have several copies of cluster metadata objects in memory we would like it to be as small as possible. And finally most of the time new cluster metadata changes will have no influence on the ongoing operations. Computation is going to be done completely off-thread and will be made visible only when it is complete. So when we manifest the epoch this is the first time the node is going to see it, experience it and there will be no additional computation required for it. All metadata elements including schema are now versioned. Changes to one key space or table do not bump the epoch for other tables and key spaces and placement information is also versioned. Epochs are changed only for the ranges that are actually affected by the operation. So if you have bootstrapped the node it doesn't mean that entire state has to be recomputed and the epoch for each and every range has to be bumped. So only the touched ranges are actually going to get their epochs bumped. Read and write requests now include relevant coordinator schema epoch and epoch when the range for which the current request is being performed was last modified. This means that nodes can quickly and efficiently check that their epochs match and the lagging node can catch up to the relevant epoch. And best of all, none of this requires communication with the cluster metadata service. Nodes can simply exchange the relevant metadata mutation among themselves without involving CMS nodes. So there will be no extra load on these nodes either. Let's talk about maintaining the consistency levels and how we have achieved whatever I have mentioned before with the loss of the transient writes. And I will simply walk you through the same things in more visual form. So let's talk through how the node X is going to bootstrapped with the transactional cluster metadata. We now again have the same setup. We have a three node cluster replication factor of three and the node X is being bootstrapped into this cluster. And here, the first and foremost step, we have something that we call prepare join. And during this, the ranges that are owned by the neighboring replicas are split and there are no other changes being made to the state of the cluster. So basically we introduce the new token where the X is going to reside, but apart from that, we do not make any changes to the ring. The next one, we add X to the write set. So this is very similar to the introduction of the pending nodes in the previous model. And yeah, we essentially say that, okay, now the write set is for the given ranges is expanded by addition of the node X. So if in the previous case, the range from zero to 100 had a write set of A and B, now it's also going to include X. For the next step, we had to introduce a concept of a progress barrier. It is serving as a way to ensure we maintain the promises about the consistency level as we discussed before. We can summarize the need for it in three conjectures. First one is the bootstrapping node has to receive every single mutation. So every single write that has been made. In other words, streaming must wait for the event that adds the target node as a write replica to be acknowledged by the majority of replicas owning the range before we can start streaming. Second one is the streaming should start only after a writing starts to the pending node. So before that, so streaming has to be blocked and it has to wait until it knows that the majority of nodes has experienced this epoch, manifested it. And the third one, all possible read and write majorities of replicas sets for both reads and writes, as I said, have to overlap their majorities, right? And this is done by essentially computing the majorities for each and every steps in a way that is done. We have a special simulator for that and you can check the Cassandra code for more details. Every subsequent event must wait for the previous event to be acknowledged by the majority of replicas owning the range before it can be submitted. So the progress barrier is essentially preventing the next operation to be executed before the previous one has been acknowledged and manifested by the majority of the existing replica set. The next step is something we called mid-join and in this step, we simply replace the node that is losing the range with the node that is gaining the range in the read replica set without any touches to the write replica set. And the last step, we manifest fully the X as a complete and rightful owner of the token. It has bootstrap four. And in this case, you see that we have moved from basically introducing the nodes through all the steps and now it serves as both read and write replica for the ranges it is going to own. So to summarize the bootstrap process, now we are bootstrapping using the ranges and node tokens. And in order to bootstrap the node first has to contact one of the ranges, one of the nodes in the cluster and this node is going to notify who is the CMS. The CMS is given the latest information about all the epochs. The node is catching up to the latest epoch, then it plans, splits the ranges, starts streaming and moves through all the necessary steps in order to get bootstrapped between every operation we have a progress barrier that prevents the progress until the majority of the nodes have experienced the epoch that precedes the next one. And finally, we unlock the ranges and the node is fully bootstrapped. Of course, you may be concerned about what is happening if you're losing the majority of the cluster metadata service nodes, it's not a problem. We have a series of scripts that allow you to quickly and easily recover from that. So essentially you can introduce new nodes by forcing a so to say two face commit over the existing nodes. We expect this to never be used by anybody. We wrote it in order to in a way placate everybody that don't worry if something really, really bad happens, you still have a way out, but we cannot really conceive a scenario where something like that can be necessary. And of course we've done a lot of testing in order to make sure that this feature is as rock solid as possible. As you may know, Cassandra has a state of the art simulator and we've done hundreds of hours of simulation on the Cassandra simulator using the fast testing tool that we wrote, which is called Harry. We also wrote special simulator for quorum intersections in order to analyze and sort of like conclusively prove that there can be no way for us to collect non-intersecting majorities for the steps that are being executed. So as long as we comply to the protocol that is given by the progress barrier, then can be no transient data loss. And we have created a new sets of tests which we call coordinator or replica tests. In this case, you are writing tests from the perspective of a single node. It is either a coordinator or a replica and the rest of the state of the cluster is completely simulated. So you can simulate arbitrarily large or small clusters by essentially talking and writing the behavior of a single node. So this is very useful for testing the scenarios of like this transient data loss and recovery from it. Yeah, so we have written a bunch of tests for DTCM as well and you can of course take a look at all of them in the Cassandra tree. And if you leave this presentation with just one word in your mind that is associated with the transactional cluster metadata, I would like you to keep the word elasticity in mind. And I think that Cassandra has to become more elastic, like more dynamic. And the things that transactional cluster metadata is making not only possible, but many of these things are actually becoming trivial are the following. I don't wanna read them from you. I'm personally most excited about the byte ordered partitioner and I have heard that for instance, Brandi Mir is also very excited, the person who presented right before me. Yeah, because it very nicely placed together with the tri-based MAM tables and SS tables. And it is going to serve as a better metadata for LWTs that are existing and for Accord. You will be able to improve your control planes and many of the operations that have been previously painful and had to be watched by a team of trained Cassandra professionals now hopefully can be executed completely painlessly. So in order to recap and sort of summarize everything that we talked about today, so we talked about the motivation, like why we wanted to introduce the concept of transactional cluster metadata and we discussed why we would like to stop using Gossip for this subset of things. Of course, we still are going to be using Gossip for things like load, failure detection and some other transient information about the nodes but it is not going to be used for cluster metadata anymore and cluster metadata is going to be transactional. We've discussed the requirements, what exactly we would like to achieve and we set the goal for ourselves before we set out with the project. You can also read the document of the CEP on the Cassandra Confluence or Apache Confluence and we presented it to the community and many folks have agreed that this is the way to go and we would like to pursue that. Then we talked about the implementation, some of the details of the previous implementation, some of the details of the new one. So basically moving from the token model to range model making the metadata changes strictly ordered and consistent and we've talked about the future and the avenues that transactional cluster metadata is opening for us and that's pretty much it. Thank you very much for coming to the presentation. I appreciate your attention and of course the books still await the persons who would like to ask a question. So go ahead and shoot. Yes, please. Yes, so in addition to the two simulations that I have mentioned, we have a third simulation that I failed to mention which is metadata change simulator. The metadata change simulator picks one of the possible operations given the setup of the cluster. So it first regenerates an arbitrarily large cluster and then just looks at the state of the cluster and thinks, okay, can I bootstrap the node in the cluster? Can I make one of the nodes leave, nodes leave? Can I make, like move one of the tokens in the ring and so forth? Then it makes a decision that, okay, I would like to perform this operation and it can also on the next step instead of continuing with the bootstrap, it can choose to, well, bootstrap one more node or have one node bootstrapping more node leaving or one node leaving and the other one moving the token. So we definitely have simulated the concurrent operations. This has been a huge area of our focus in the very beginning because we thought that if we cannot achieve that, then what are we doing at all? So we definitely did a lot of that and yeah. Yeah, so I will repeat this one because I think I haven't repeated the previous one. How do we expect this to affect the cold restarts? So I'm assuming that cold restarts is like rolling bounces whenever you need to restart every single node in the cluster, is this? So once again, would you like to bring up a whole new data center with hundreds of nodes and none of them have been a member of the cluster before that? Is this what you would like to achieve ultimately? Okay, so in this case, we have two scenarios. One scenario is first you bring up some of the nodes that are going to be the cluster metadata service. You can bring up one node that is going to be CMS. That's the quickest and simplest scenario and easiest to achieve. I would personally probably do that. Then you just start all the other nodes and they start registering and registering is as easy as simply submitting a single LWT or PAXOS operation. It is very quick, it's currently two round trips because we are using PAXOS V2 and I expect maybe there will be some contention but even for a thousand node cluster, like we actually simulated way more contended scenarios with PAXOS V2. We have not simulated anything like that specifically with the TCM in this case but I don't expect any problems to be happening and in the scenario when you don't have any data in the cluster instead of using the multi-step operations that I have presented previously, you can save something that you can use something that we call unsafe join which is a single operation which means that everybody just joins in one step. So the register, they join and after that you have a thousand node cluster and I expect it to be.  You will not even notice anything on top of that so that's my take on that. Anybody else? Josh, sorry? I think I'm about ready. Okay, so that's the difference between the concept of a pending range which means that you have to write to the majority of the previously existing nodes plus one. We have changed it and we essentially expand the replica set by one and the majority is gonna be number of nodes divided by two plus one which means that for three nodes and one bootstrapping we have four nodes so the majority is of course three nodes but it can be three existing nodes. So you're not good with your old quorum. You have expanded your quorum and you will have to write to all of them effectively. Yeah, in this case, you have absolutely no problem. As you expanded the quorum, you will have to write to three nodes instead of just two. You can quickly revert the operation meaning that you can sort of like abort the bootstrap and it's a very easy and quick operation. But yeah, so that's the answer. Yes, please, in the white t-shirt. Okay, the question is what is the upgrade path from the existing gossip clusters to TCM clusters? The process is as follows. You bring up all the nodes with the version that is aware about the existence of transactional cluster metadata and you start the upgrade process by running a simple node tool command. You don't remember, it's initial ICMS or something like that. There is a documentation on the Cassandra website, of course, for that or like entry even and this command does the following. It ensures that every single participant in the cluster has the same gossip view of the ring meaning that they have converged in terms of gossip because otherwise what state can we base our transactional cluster metadata upon? So that's prerequisite. If you have some nodes that have corrupted view of the gossip, you can actually ignore them. You can manually inspect, like in fact, during the upgrade we print the diff for you and we say that okay, here is the disagreeing nodes and here is where they disagree and you can still say that you know what, I think that these nodes are wrong. I don't know why and I don't have time to figure it out. So let's just say that this gossip state that main one sort of looks fine for me. So let's go ahead with this one and after that we are using a two-phase commit which requires all the participants to respond during the first phase and say yes, I promise you that I will upgrade to a single instance of transactional cluster metadata because if you accidentally launch two upgrades then well, we would like to prevent that and we would like nobody to think about like how to avoid something like that. So we have to consult not the majority of nodes but actually every single participants. This is the point of the phase commit and after they all of them have promised they you go through the second step and you essentially instantiate the first instance of transactional cluster metadata and after that everything go on as previously just not with gossip but with TCM. There is a single limitation that during the upgrade process you cannot like the metadata is essentially locked. So you cannot bootstrap or decommission the nodes but the word on the street is that like if you are upgrading, you better not have also like bootstraps while doing that because there can be several things in flight that yeah, it's just many people don't do that. So yeah, that's the only limitation. There is a way to downgrade as well. So you essentially say, okay, convert the transactional cluster metadata back to the gossip state. The gossip state is propagated and you downgrade but I don't really anticipate anybody needing this barring the issues with something else in Cassandra. Yes, sorry. I'm really sorry. I understand what you're saying. I just don't hear the question. Could you phrase it in a form that could be like what are you asking? Like are you trying to achieve something or you think that the way we've implemented it has some, yes, thank you. Okay, now I see. Of course, the transactional cluster metadata can be used in order to dynamically move individual partitions between nodes. This is absolutely true. The only reason we have not done that is for everybody in the community to be able to keep using their current tooling Cassandra tooling because I'm assuming that you have actually like concept of tokens in all of your scripts, right? There is absolutely nothing that is preventing you from saying that certain ranges are owned by like a single node or even a single token is owned by a subset of nodes. You can even say that the replica set for a given, I don't know, token is not three but five or seven, like anything goes really. So from the perspective of transactional cluster metadata, everything is just like a range and a set of nodes that owns the range. So this is already implemented. The reason we have not exposed it is that we wanted everybody to have an opportunity to transition first to the, well the system where this is, like everything is transactional and after that we will start introducing new tooling that will allow dynamic repartitioning, different ways of load balance, reintroduce the binary ordered partitioner which would mean also that you will be able to essentially have shards and Cassandra and so forth. So this is in the books, everything for that is essentially implemented as a prerequisite but like the features itself are still to be done but this is planned.


Thanks everybody for coming. Our next speaker is Tiffany Lee giving a talk on facial recognition, DNA and biometric privacy. Stop Facebook from buying your brain. Alright, hi everyone. I'm Tiffany Lee and I'm here to talk about biometric privacy in all different forms. Let's see if this works. There I am. So I'm a tech attorney and I'm a researcher at Yale Law School. I focus on privacy, AI and tech platforms and I've written and spoken about topics ranging from Facebook and Cambridge Analytica to if there's a legal right to AI and AI and this microphone is on and off but here we are and a lot of other topics like that. But today I'm here to talk about one specific topic that's really close to my heart which is biometric privacy. And also the main topic in the title of this talk. Can Facebook buy your brain?  Even if you think you know someone who has a completely unused brain that's fresh off the market for them, not happening. And not happening anytime soon. But I use this example when I talk about biometric privacy because it brings up a lot of the really very dystopian notions that could potentially happen some day. Things like whether or not a company can own your brain waves, a company can read your thoughts.  But that scary dystopian vibe is also apparent here today in current biometric technologies and that's something I think we should keep in mind that the world is already a not future. So here is what I'm going to talk about today. First going over biometrics generally, what they are, what the law and landscape is for it. Then a few special cases that I think are especially important. So facial recognition, DNA privacy and then data related to brains. And I think these cases are important because they're very unique I would say and even unique for biometric privacy. And then finally I'm going to go over a few things that we can potentially do to prevent the most horrifying of the sci-fi scenarios from happening. So what are biometrics? Biometrics are the physiological and behavioral characteristics of individuals. This could include fingerprints, voice, face, retina and iris patterns, hand geometry, gait and so on. What we often say with biometrics, um, so biometrics often are described as data that is derived from the body. What this means is that biometric privacy then is privacy for all data derived from the body. What makes biometrics special and why is that biometric privacy is a special category of privacy with unique and important risks. Again, biometric data is data derived from the body. And that's the main problem. You can't delete your body. You can't change or modify a lot of the data that comes from your body. And this is why we often say biometrics are user names and not passwords. Biometrics are distinctly identifiable to a certain individual. And the risks from biometric data are unique and potentially worse than other forms of data and data privacy. The good thing is there are some biometric privacy laws. So I'm a lawyer, but I don't think laws are the only solution. And as we can see, just for biometrics, they're really not complete. In the United States, a few laws exist in some states covering biometric privacy. Texas, Washington, Illinois in particular, have biometric privacy laws. A few other states are considering new laws. But again, this is only a small subset of all the 50 states. More states in that look at biometrics as part of what's called PII, or personally identifiable information. PII is a really specific category that's important in privacy law. Because many laws only refer to PII. So often you only have privacy rights, the data that is identifiable to you. Some laws include biometric privacy or biometric data as part of PII, and those protect some biometric privacy rights in that way. In the EU, the GDPR, which is the general data protection regulation, protects biometric data as a special category of protectable data. So again, there are some laws that exist for biometric data broadly speaking. Additionally, there are a few laws that exist specifically for facial recognition, for DNA data, and even for brain-related data. For that category, the most laws that we see right now in the books refer to brain scan images, which are something that have shown up in some jurisprudence revolving criminal justice. So the first special category of biometric data is facial recognition. And this photo is actually taken from an advertisement for an app that uses facial recognition for dogs. So can we protect the privacy of your dog for facial recognition? Is a really important question that I will not answer today. Privacy for people I think is more important. Facial recognition tech, for those of you who don't know, is tech that identifies or verifies a person based on an image or a video. This is distinctly different from facial detection technology. Facial detection technology merely recognizes if a face is present. And I mentioned this because I feel like many times when someone talks about facial recognition technology or positive applications of facial recognition tech, what they're really looking for is facial detection technology. Facial detection technology is much less privacy invasive and can be used in many of the same contexts. For example, you can consider an application that opens a grocery store entrance door if a person is walking past. For this sort of program, you don't need facial recognition. You don't need to know which customer is walking past your door. All you need to know is that a human being is walking. So facial detection could be used instead of facial recognition. This kind of tech solution is something I talk about a little bit later. But it's one of the big things I think we can do to minimize the privacy risk for biometric data. So facial recognition is used in both corporate government surveillance worldwide. And it's really becoming more widespread by the minute. A few examples that you've probably used or have heard of, face based logins on your phone or your computer, photo apps that manipulate your photo to look older or younger. And also law enforcement uses. A lot of law enforcement around the world relies on facial recognition to identify people they believe to be suspects and so on. So some of more of the risks of facial recognition data. First of all, facial recognition is widespread. And that in itself is a risk. The fact that facial recognition is everywhere right now and people aren't aware of that is a huge risk. If you're not aware of what's happening, you can't be aware of how much, how many rights you have and you have no way to know when you can actually advocate for your rights. The Georgetown center on privacy and technology has studied that one in two American adults are already enrolled in a law enforcement facial recognition network. This means that everyone in this room, probably about I would say a third to a half of us just broadly speaking are probably already in a law enforcement database. This is also DEF CON so probably more of us are in the facial recognition database already. And this data is pulled from many different accounts. From driver's license images, from CCTV surveillance and more. It's really hard to know where the data is coming from and for a private individual, it's almost impossible to have recourse against the government for these uses. The second risk for facial recognition is that facial recognition tech right now is often very flawed and can be biased. Joy Bulamwini, a researcher at MIT Media Lab and founder of the Algorithmic Justice League conducted a study of facial recognition tech from some of the largest companies and she found that error rates for darker skin women range from 21 to 35%. While the same error rates for light skin males were below 1%. So why does this matter? If you think about the fact that facial recognition is used by so many law enforcement agencies around the world, you can definitely see the risk for criminal justice. It's more likely that a law enforcement process using facial recognition could generate false positives for people who are darker skinned or who are women. And this creates obviously an unfair discriminatory impact on these people. And finally, facial recognition can be used to harm human rights. This is often the case, especially with countries that don't have democratic rights or don't have open rule of law. One example that you've already seen happening in the world all of today is when governments use facial recognition to scan photos of protests to identify political dissidents and then take action against those dissidents. So facial recognition tech already has a lot of risks and we're not adequately protecting against it. The next category of special biometric privacy is DNA data. So DNA is essentially the code to your body. And I found what I think is the creepiest pictorial representation of DNA. But I think it really tells you what DNA is, right? It essentially carries the genetic information to your body and is unique to each individual. So one of the biggest cases that happens recently is about the Golden State Killer. So many of you have probably heard of this case. This was early last year when investigators identified a decades old cold case serial killer based on DNA sampling. They had a very old DNA sample of the suspect that they then ran against a publicly available DNA database. They found a match with a distant relative. And just based on that distant relative's DNA match, they were able to then eventually identify the killer. This seems probably like a positive outcome. I think a lot of people who hear about this case think it's great. That it's wonderful to have DNA out there because we can identify more serial killers, more terrible people and so on. But I think many of us in this audience already know that that's a huge risk to everybody involved. That if tech like this can be used for good, it can also be used for bad. The Golden State Killer case brings up one really important risk with DNA privacy that isn't the case with other types of privacy. Your DNA data can be used against you even if you don't have data anywhere. The data of say your distant relative can be used to identify you. So in this case the government isn't the real threat, the companies aren't the real threat, but Uncle Bob at Thanksgiving dinner might be the real threat, especially with all the 23 and me ads out there. So DNA data I think is a special case for biometric privacy that we should be thinking of more clearly. Aside from the privacy risks, we also have some interesting discussions of who can own DNA. So this is a picture of Henrietta Lax and she's probably the most famous test case for medical genetic information and how that information can be owned and then used by other people. Henrietta Lax was an African American woman whose cancer cells are the source of the HeLa cell line, the first of what they call immortalized human cell line. She went in for testing for medical reasons, her cells were collected and cultured without her consent or without her knowledge. And eventually this led to many scientific breakthroughs. Again, it seems like a positive outcome, right? Like the Golden State killer case, there was a positive outcome involved. The problem is that there was no compensation for Henrietta and her family, no consent from her for how her cells could be used and no rights against the companies or the medical information agencies using her data. So while it's amazing that there is a positive outcome, there's still a question of who has the rights to that data. And broadly speaking, who has rights generally to DNA data? The good thing is there are some laws right now. DNA privacy laws include the 21st Century Cares Act, HIPAA, which is a general privacy act broadly covering health privacy, and also GINA, which is a genetic non-discrimination act. Finally, the last category is BCIs and Neurotech, which are really the crux of what this talk came about from. The inspiration for this talk was when I was thinking about what companies like Facebook are doing with Neurotech. So in 2017, Regina Dugan spoke at Facebook's big tech showcase, and she asked this question, what if you could type directly from your brain? What if you could type directly from your brain? Just imagine sending Facebook messages just by thinking them. What this entails would be that Facebook would have to be able to access your brain data somehow, would you be able to ping the interface with your brain and Facebook has access to whatever sort of pings that occurs then. This involves what's often called a brain-computer interface, which is a computer-based system that acquires brain signals, analyzes them and translates them into commands that relate to an output device to carry out a desired action. That's a lot of words. It's really just brains talking to computers. Easy. I think you can already assume a lot of the problems here, but I think that we have to take a step back before we talk about the privacy issues and realize that a lot of this tech gets a little over-hyped. So I don't know if Neuralace is the new blockchain in terms of hype, but there is already a lot of it out there. Recently Elon Musk said that his new company, Neuralink, is going to meld humans with AI, which is kind of a ridiculous prospect. It could, let's say, might happen in the far future. It's not happening in 2019. It's not happening in the next 10 years. Those are not the issues that we're discussing. The issues we're discussing are more closer to 2019. Here's a not-so-extreme hypothetical. If BCIs become common and we use them for communications or for work and so on, it's likely that companies will own and operate the brain communication interface platforms. At that point we have platform regulation issues. We have privacy issues. We have communications issues. We have questions about who can access this data, who stores the data, how secure is the data. If there are communications being performed across this platform, then we have similar questions as we have with current communication platforms. For example, we can see that Facebook and Twitter moderate content. Will future platform providers be able to moderate the communications across a BCI-informed platform? These are questions that no one has the answer to. In addition to finally the questions of Simply Again, who owns the data? Can someone own the data emitted by your brain waves in a BCI-informed platform? Those were a lot of questions and here is the solution part of the talk. What can we do? I think this boils down to a few different categories. First I'll discuss legal and regulatory solutions, next some tech solutions, and then a little bit else about what we can possibly do to stop a lot of the worst of the privacy violations. In terms of legal solutions, I think there are a few things that are maybe not easy to do, but things that we can consider legally. So first obviously pass new laws or make better laws for facial recognition, for DNA data, biometric data, BCIs and so on. We consider updating the GDPR guidance in the EU to include more of these categories and in the US we might consider more regulations or even just one federal privacy regulation that includes biometric data. Also this gets a little in the weeds, but I think it might be interesting to think about new classes of legal privacy harms not directly related to the rights of the data subject. This could cover for example when Uncle Bob uploads his data to the 23andMe database, right now you have no rights to that, but there could be laws that give you some protection. And finally if we get to the point where there are platforms for BCI communication, we need some sort of standards for how those platforms can be moderated and how those platforms can be operated. There are also of course tech solutions and I separate this broadly between data level solutions and system level solutions. That's an incredibly broad and oversimplified way to divide this, but this is how I think about it. On the data level we have to protect the data through security and privacy protections. This can include better security and privacy tech including encryption, differential privacy on device analysis and so on and so forth. These are interesting technologies that make sure we can secure the confidentiality and the integrity and the access of data, but they're not enough. You can have the most secure system, the most privacy protective system, but it can still be a terrible system. You can still have terrible tech that creates real world impact. So generally we have to figure out how to design better systems and how to deploy better tech that keeps in mind issues of criminal justice, social justice, bias, fairness and so on. The problem though of course is first, re-identification is always possible. No system is truly secure. Laws are always behind technology. So tech can be legal, have privacy and security protections and still violate core human rights. So what do we do then? And I think the solution is ethics. And it sounds a little cheesy, but one thing that we should all think about whether you're designing the system, implementing, critiquing it or whatever, is think about the ethical issues at play. So think about the ethical issues before you design your new technology. Think about the ethical issues before you purchase a new app. Think about the ethical issues before you talk about this tech. As a society we have to figure out what we believe to be privacy for the future. We have to figure out if we believe that DNA data is intrinsically linked to the body or if brain communication interfaces should be using communication or someone should own that communication. Laws won't solve these problems. Tech won't solve the problems. We have to figure this out as a society. So here's what you can do. Design better systems, deploy better tech. It sounds super easy but it's very very hard. Do your best to use your technical expertise and many of you in the audience have a wealth of expertise here. Use your expertise to design the better systems and critique flawed systems when you see them.  Be a conscious consumer and an informed voter. Support companies that protect privacy. Vote for policies and policy makers that care about tech and good tech regulation. And finally, you can advocate for change no matter who you are or what position you're in. This can be as small as talking to the people around you about and informing them about their rights and it can lead to supporting large groups, advocacy groups or even speaking your mind out in speeches and letters and articles or even running for office. You can always be an advocate. In conclusion, here are my steps for how to stop Facebook from buying your brain. Understand the current landscape of technology and laws. Update the current laws and create better laws. Design and deploy better tech. Remember ethics and always consider ethics first. Be a conscious consumer and an informed voter and advocate for change. If there's one take away from the talk that I would like you to have, it's that yes, the biometric tech isn't proving and biometric privacy is at a greater and greater risk every day, but there's still time. There's still time to stop the worst of the privacy violations from happening and you can be a part of that change.  So you can change the future and that decision starts today. Thank you. Alright, thank you Tiffany. We have time for questions. Please come up front and line up by me. Hi. So what are your thoughts? Obviously like these are excellent steps, but what are your thoughts on the fact that a lot of big data companies like Facebook don't make the decision to track these things and like store the data public until long, long after it's happened. So there are a few issues of that. You asked about issues of Facebook and other companies not making their data or the decisions to store data public. And there are actually some legal potential there that you could solve things. For example, people have said that companies can't really tell you how they store data or how they use data because they're afraid of liability. We could create legal safe harbors that would allow them to have more transparency reporting to tell us what they're doing with the data and how they access data. But a lot of this I think is on the companies themselves. Most companies at the end of the day are motivated by profit. And if we as consumers don't really seem to care about privacy, they're not going to care about it either. So we either have to have consumer pushback or regulators enforcing something to change. Other questions come on up. Thanks for your talk. Really enjoyable. You mentioned a little bit about efforts or ideas around regulating data that doesn't necessarily belong to one person but is shared. That comes up in a lot of conversations I have with peers who are interested in this stuff of like you and I are having a conversation right now which one of us owns it is the Unity pairing. Can you give a little context into like what influences the popular thinking around this and what progress has been made? Sure. So the question is about data that's not owned by you and about some of the privacy implications there. So one issue I bring this up and I don't talk about it too much in the top because this can get very technical. But there are a lot of privacy laws out there and most of them specifically just give you rights as a data subject. There are very few laws that talk about what happens to people when data that's collected about them but not from them impacts them. So some people are doing work on what they call for example inferences in algorithmic decision making and how those inferences can impact someone regardless of how the data was collected. I mean that's really important. I also think some other categories we should be thinking about in terms of data include for example data that's not collected on you or even about you but that's related to you. For example if there is a company that collects all the data of different DEF CON participants you're not included in the data set maybe but whatever predictions are made based on that data set will likely impact you because you are part of the larger group. You have no rights to that. The law doesn't talk about that. These are the sort of edge cases I think we need to think about in the future. Hi. Thanks for the talk. When you were going over your questions for how to prevent a really terrible future with BCI I kept thinking all of those questions are applicable today to virtual reality and will be soon applicable to the rumored platform shift to augmented reality. Have you thought any about how those same questions should be applied to virtual reality? Definitely. So virtual reality VR and AR tech right now it's similar to a lot of this tech in that it's still pretty new and it might be overhyped in some cases but I think it's definitely coming and I actually have a paper in progress right now specifically about VR and AR and that being a sort of new type of tech platform. So right now we regulate ISPs like Cloud Flare or like Facebook as a sort of platform for communication or for the transfer of data. And in the future if we have VR and AR tech becoming more common place that raises a whole class of new issues similar to how BCIs raise new issues. For example in the really crazy future scenario in which we all live in the matrix what happens if Facebook owns the matrix right? If Facebook literally owns reality then we have a lot more problems to deal with but that's a small snippet of some of the issues of BCIs. Hi so to your earlier point about like the privacy subject laws to what extent do you think it's possible practical or reasonable to say like have Uncle Bob require my consent to leak his DNA or have some kind of like recourse or restrictions for me in those types of scenarios. That's a good question. I think you honestly have very little ability to control the data that you specifically don't contribute to a data set. So you have no rights to stop Uncle Bob from doing anything. That's why I think the law should look at those other classes. They should figure out some sort of rights for for example for the algorithm decisions that are made based on that data. We have to think of the harms that happen after the data collection because we don't know what the points of data collection might be. So I know that like all about 30 thousand or whatever of us are here at this conference. You know we really we care about this stuff. But how do we get the normies to care? How do we get the people that like look at me weird because I have a Mohawk out in the casino. So it'd be like oh hey yeah I want to I want to protect my privacy because I feel like let's say all about 30 thousand of us had Facebook accounts and we all deleted them. I'd be like a drop in the bucket.  So how do we get them to care and see that like oh yeah the government's spying me on me is probably not a good idea. Yeah so how do we get other people outside of Dev Con to care about privacy? That's a really big question. I think what's interesting is that a lot of research has shown that people do care about privacy. Kids care about privacy. Adults care about privacy. People from all different spectra care about privacy. They don't know what's going on though I think is part of the issue. So when something like the Facebook or in Cambridge Analytica scandal happens people suddenly realize that their rights are being violated. Or when there's another big hack like the Capital One scandal recently people realize that their data is you know potentially out there. So I think that a lot of this has to deal with informing the public. So education programs, journalists presenting good tech reporting can help a lot. As individuals I think what we can really do most is just talking to people. So I know that not your entire circle is made of Dev Con people right? Like you know people who maybe aren't as plugged in as you are. And explaining some of these concepts or just bringing some of this to people around you is often really helpful. All right that's all the time we've got for questions. One more big hand of applause for our speaker.