texts for software-enabled Flash, a newly announced technology from Kyokusha. Today I will be covering what software-enabled Flash is, why we created it, the concept and technologies it contains, and I will also show some demonstrations of the software-enabled Flash technology running on our prototype FPGA. I will finish by adding a little section on coding examples and then directing you to where you can obtain more information. So what is software-enabled Flash? Well, software-enabled Flash is a media-based host-managed hardware approach. We'll talk more about this in a few slides. We're trying to redefine the way the host interacts with Flash to allow applications to maximize performance and provide functionality that would be difficult if not impossible to achieve with existing interfaces. Giving the host control over media enables the host to define the behavior of the media and the performance characteristics for Flash. This can enable new storage functionality that maximizes performance and efficiencies or stated simply the ability to extract the maximum value from your Flash resources. Coupled with this media-based hardware approach is the software-enabling API. This API is designed to make managing Flash media easy while exposing all of the capabilities of Flash. So software-enabled Flash technology isn't just software, but it is software-enabled. The highlights of this API are, first, it is an open-source Flash native API. Next, it's designed to make it easy to create storage solutions with powerful capabilities. It abstracts low-level details that are vendor and Flash generation specific so that your code can take advantage of the latest Flash technologies without modification. Making an open project, any Flash vendor can build software-enabled Flash devices that are optimized for their own media. And finally, even though just the API and documentation are published today, Keelkshow will be releasing an open-source software development kit with reference source code, utilities, and examples. So before we go into more details, I'd just like to reinforce the message that software-enabled Flash was created to enable the development of custom storage systems that provide innovative solutions to storage problems. Software-enabled Flash is free from legacy disk interfaces and paradigms. It offers total control over the Flash media and can extract the full performance and efficiencies of Flash by providing solutions that are Flash native. Software-enabled Flash is extremely easy to adapt to new storage applications, to foster innovation in storage, and also to enable rapid deployment of new storage solutions. And I'll show some examples of this too. So in summary, Software-enabled Flash is shedding the legacies of hard disks, allowing applications to have much more control, to have predictable performance within the Flash device, and enabling flexibility and scalability for software developers. As previously introduced, Software-enabled Flash consists of hardware and software components working together. Over the past several years, I've had the fantastic opportunity to meet with the storage developers of most of the world's hyperscalers. Taken with the input of other engineers at Kyokusha, it has allowed us to distill a list of basic requirements and features for hyperscale customers. I should mention that although the hyperscalers face similar problems, their individual priorities and approaches vary significantly. On the requirements side of the slide, Flash abstraction is not just about code reuse. There can be major economic and performance advantages to transitioning quickly to newer generations of Flash at scale. Scheduling is an item that's going to be covered in depth later. Access to all the Flash media. In hyperscale environments, they refer to RAID within a device as a RAID tax. Their systems are designed with redundancy at the system level. And so they are trying to access all the capacity that's native to the device since they're going to be ensuring resiliency at higher levels in their environment. Also, for these hyperscale environments, CPU offload is important because they view the CPU as a sellable resource, particularly in hosting environments. And finally, device DRAM configurations need to be flexible. Device DRAM has to handle worst-case scenarios, but often that leads to stranded resources in their environment. On the functionality side, data placement is important to try and minimize right amplification for Flash media. Isolation is both for security and noisy neighbor problems. Latency control is a topic that we're going to cover in depth later in the presentation too. But for management, well, that's tied back to the flexible DRAM configurations, and we'll show more around that later. And finally, adaptability. A hyperscale environment is a very dynamic environment and changes quickly behind the scenes. The preferences for standard configurations that can be provisioned and deployed in real time to meet changing workloads. In response to those requirements and required features, the software-enabled Flash API covers these major areas and a few more that aren't listed. Group broadly, the areas of the API are concerned with the hardware structure, buffer management, programming of the media, and error management. Not listed separately, but touching many of these areas is latency control. In order to maximize the performance of Flash storage, it is necessary to be aware of the geometry of the Flash resources. The software-enabled Flash API allows storage applications to query this geometry in a standardized manner. Some of the characteristics of those are listed here. The API also allows control over how many Flash blocks may be open simultaneously and can control the management of the associated write buffers. When discussing the programming of Flash, it's important to note that the optimal programming algorithms vary from vendor to vendor and often between generations of Flash from single vendor. The software-enabled Flash handles all the details of programming and lets the Flash vendor optimize for their own particular media. The API was created with consideration for other Flash vendors, as well as all the foreseeable needs of Kyokushia's own future Flash generations. Finally, with respect to error management, software-enabled Flash allows vendors to optimize error reduction techniques for their specific Flash media characteristics. And it also controls the time budget for error recovery attempts. This is a high-level block diagram of our software-enabled Flash controller. It is one possible design and other vendors are free to implement different architectures as long as they comply at the API level. For example, although we currently use the toggle interface at the bottom of this block diagram to connect to our Flash chips, other vendors may implement on-fee interconnects or a different style of interconnect. What that our design uses standardized interfaces wherever possible, including the PCIe interface at the top of the block diagram. The controller itself is focused on management of the Flash media, programming, lifetime management, and defect management. The controller has advanced scheduling capabilities on a per-dye basis and hardware acceleration for garbage collection operations if needed, as well as ware-leveling tasks and other administrative tasks. Another important call out here is the use of on-device DRAM is optional in the software-enabled Flash technology. Host memory resources may be used instead. So as we just mentioned, the use of DRAM on our software-enabled Flash devices is optional. Why is that important? Well, the answer lies in the fact that hyperscale customers often require thousands of simultaneously open Flash blocks on each Flash device. The actual numbers vary from customer to customer, but we've heard requirements between 4,000 to up to 16,000 open blocks on each individual device. Since each open block requires a write buffer, this can create a demand for a lot of memory within the device, potentially tens of gigabytes of DRAM dedicated to use as a write buffer. It's often the case that the actual number of open Flash blocks can vary over time as a function of the system load. So sizing the device DRAM configuration for a worst-case scenario creates what we call stranded DRAM resources within the device. They must be present to handle peak load, but in normal operation, that's DRAM that is essentially idled and not used for other purposes. Software-enabled Flash supports device DRAM configurations. It also supports host-only DRAM configurations and, most interestingly, hyper-configurations that allow the DRAM to be sized within the device for normal usage and for the device to drop on host DRAM resources during periods exceeding the limits of the drive's internal resources. We should note that in systems that use host DRAM resources, there are system requirements to protect against data loss in the event of unexpected power loss. Many hyperscale environments already have this in place with non-volatile memory resources, system-level mirroring, and system-level erasure coding. So in summary, we support both DRAM within the device, DRAM only in the host, or hybrid solutions that contain DRAM both in the device and in the host. There are benefits to each approach. With the unified write buffers that can draw upon host-side resources, we can dynamically adapt to changing requirements and workloads, and we can optimize DRAM at the system level without stranding any DRAM resources. The conventional architectures are more appropriate for systems that cannot meet the resiliency requirements at the system level and rely upon the drive for data loss protection in the event of sudden power loss. Now I'd like to introduce the software components of software-enabled flash. Keopsia has created and will soon release a software development kit. This will provide open-source reference block drivers, open-source reference flash translation layers, and open-source device management utilities. Bundled with the software development kit will be an API library, once again in open-source, and also will have a device driver too. This block diagram shows how the pieces of offer development kit interface with each other. Two notes on the software layering. First, as you can see, it's possible for a user application that is software-enabled flash native to interact directly with the device, bypassing file systems and traditional flash translation layers and block drivers. We have a couple of proof-of-concept applications running today on our own FPGA prototypes. These include a software-enabled flash engine for FIO, as well as a version of ROXDB and a version of Firecracker that are software-enabled flash native. Although these applications are currently just proof-of-concepts, in the future we plan to include open-source native applications as part of the SDK. The second note is that the block-labeled SDS stack is a software-defined storage stack. Hyperscalers are already running their own software-defined storage stacks in their environments. These software-defined storage stacks can be modified to interface with the software-enabled flash library and are not dependent on the use of the SDK. In such cases, the SDK serves as the detailed example of the best practices for using software-enabled flash. The APN library is once again open-source, runs on Linux, it has both C and C++ bindings, it supports use in kernel space as well as user space, and provides easy access to all the features of software-enabled flash. And now for a system-level view of one possible deployment of software-enabled flash. Note that the items outlined with the red dashed lines are items that would likely be customized for different environments. Let's start at the top and work down. So at the very top of this diagram, you see unmodified applications running within a virtualized guest. Those applications typically use the POSIX API to talk to a file system, and the file systems use block device IO to talk to devices. In this particular case, since these are virtualized guest environments, they're actually talking to a hypervisor within the kernel. In this case, we're calling out that we have built a software-enabled flash QEMU block device driver. This interfaces with our reference flash translation layer and the software-enabled flash library to bring block-level features and functionality to the virtualized guests. The software-enabled flash reference FTL communicates to the software-enabled flash library using the software-enabled flash API. The library, in turn, uses IOU ring to talk to the software-enabled flash driver in kernel space. This allows us to bypass system call overhead for greater performance. And the final thing to note is that on the far left side of the diagram, outlined in red, there are soft native applications, software-enabled flash native applications that can interface directly with our library, utilizing the IOU ring mechanism to allow applications to interact with software-enabled flash devices without system call overhead for high performance. Finally, the software-enabled flash QEMU block driver, as I mentioned, it's part of this SDK, is useful because it allows unmodified applications to take advantage of many of the features of software-enabled flash, such as isolation, latency control, and die-time allocation, even without modification. So I encourage people to look into the possible uses of the software-enabled flash QEMU block device driver. All right, at this point, we're going to introduce some concepts and features of the software-enabled flash technology that are going to be necessary to understand later examples. This diagram is a representation of a possible software-enabled flash device. At the top of the diagram, we have the actual control logic. And then this particular device has 32 die that are arranged on eight channels with four banks per channel. The first concept that I would like to introduce is that of a virtual device. A virtual device is a set of one or more flash die that provides hardware isolation. So when you define a virtual device, you get to define how many channels the device is spread across, as well as how many banks.   The next concept is that of a quality of service domain or a QoS domain. This is a mechanism that allows us to impose capacity quotas and scheduling policy as well as to provide software-based isolation. Note that unlike the virtual devices, you can have QoS domains that share a single virtual device. So in this example, QoS domain 0 and QoS domain 2 are sharing virtual device 0. Likewise, QoS domain 4 and QoS domain 5 are also sharing a virtual device. The final concept is that of a placement ID. And this is a mechanism that we'll talk more about later that allows us to group data at the super block level within a QoS domain. This slide describes how the concepts introduced on the previous slide provide control over data isolation and data placement. Super blocks within a virtual device start out in a free pool. As QoS domains allocate storage, a block is drawn from the free pool and assigned to a QoS domain. The device can choose any free block from the pool, so contract blockware and block health and assign the optimal block to maximize device endurance. Super blocks are never shared between domains. There is no mixing of data at the block level. Once a super block is released, it's returned to the free pool. So over the lifetime of the device, ownership of a super block may change several times if there are multiple QoS domains within a virtual device. So the virtual device provides physical isolation at the dye level and can be used to shape both the capacity and IO throughput for an application. Within the virtual device, blocks are combined to form super blocks that are striped across each dye in the virtual device. This controls the amount of parallelism and the amount of potential throughput for the virtual device. To summarize data isolation, the two main mechanisms are listed here with their benefits and restrictions. So we have dye level isolation in the form of virtual devices and block level isolation in the form of QoS domains. Dye level isolation is the most effective isolation. It provides full hardware isolation between applications. However, typical devices only have a small number of flash dye per device. This can be in the range of 16 to at the high end, low hundreds. So there's a small number of potential virtual devices, and it's limited by the actual number of physical dye on the device. However, block level isolation using QoS domains is far more scalable. It can scale into the thousands of tenants. And while the isolation is not total hardware isolation, we can guarantee predictable performance and extend lifetime and keep data separated at the block level between different applications for either security or performance reasons. Closely related to data isolation is data placement. Here we're introducing the concept of a nameless write mechanism to control data placement. Why is a new write mechanism desirable? Why did we do this? Well, although the system benefits can be realized with control over data placement, if physical addressing is allowed for writes, the host becomes responsible for device wear. Flash memory is a consumable. It wears out with use. And so poor choices for physical data placement can wear out flash devices quickly. So even though the host would like to be able to control data placement to minimize write application and group data from applications or tenants that have potentially similar lifespans, the host really doesn't want to be responsible for the lifetime management of the media itself. How can the host have control over data placement without this responsibility of ensuring the device helps? Well, the answer is nameless write. When a new superblock is required for a placement ID or if a new superblock is manually allocated, the device chooses the optimal superblock to use. This is the framework for nameless write mechanism. Now let's show how the actual nameless write works. Nameless writes allow the device to choose where to physically write the data, but allows the host to bound the possible choices for where the device will put it. As mentioned earlier, QoS domains are mapped to device nodes in the system. And so the nameless write operations must supply a QoS domain handle as well as either a placement ID for auto allocation or a superblock flash address returned by a previous manual superblock allocate command. The QoS domain maps to a virtual device, which in turn specifies which die can be used for the write. So by virtue of the QoS domain, you've specified a subset of die where this write can be performed. The placement ID or flash address specifies the superblock, which is owned by the QoS domain, that should be used for the write. If a placement ID is specified, a nameless write command can span superblocks, and additional superblocks will be allocated to the domain as needed. In manual allocation mode, nameless writes cannot span superblocks. As illustrated, the device is free to write the data to any empty space within the bounds specified by the host. And when the write is complete, the actual physical address is returned. This enables direct physical read path with no address with translation required. Direct physical read optimizes performance and minimizes latency. The nameless write operation automatically handles all media defects so that you always are given the actual physical address at which the data resides. Similar to the nameless write operation, the operable flash has a nameless copy operation that can be used to move data within the device without host processing.  The nameless copy function takes as input a source superblock, a destination superblock, and copy instructions. These copy instructions are powerful primitives, supporting valid data bitmaps, lists of logical block addresses, or even filters to select copy data based on logical block addresses. A nameless copy command can operate on entire superblocks with a single command. This animation illustrates the difference in impact for the host to implement a garbage collect using standard read and write commands versus the nameless copy. As you can see, for the manual copy operation, the host had to issue 20 commands, 16 of which were reads, and fours were right to harvest the data for garbage collection. The nameless copy operation issued a single command to perform the exact same work.  So on the left-hand side, manual copy is being used for garbage collection. On the right-hand side, nameless copy is being used. The two workloads are identical. The blue rising graph are the actual garbage collect operations. Over the course of this workload, there's a dramatic difference in the impact to the system. As you can see, from the center graph, CPU utilization was much lower for nameless copy. Why was it lower? Well, to perform the exact same workload, nameless copy only needed 24 commands to do garbage collection, versus the manual copy needed over 800,000 commands. Likewise, nameless copy only transferred 120 kilobytes of data. These were the copy instructions, versus over three gigabytes of data that flowed through the host for a manual garbage collect operation. Pretty dramatic results. Another key feature of SAPRAEVAN enabled flash is the advanced queuing and scheduling features. We'll spend a bit of time here with several slides. Scheduling and queuing controls how die-time is spent. This means how much time is spent for read operations, versus write operations, versus copy operations, as well as the prioritization of multiple workloads. Consider a multi-tenant environment. There may be business reasons to enforce fairness, or give certain tenants priority. And these business needs may change over time. These tenants can share a device, and weighted fair queuing can support the performance goals. The host is allowed to prioritize and manage die-time through the software-enabled flash API, and the device enforces scheduling policy. This is the basic architecture of our software-enabled flash scheduler. First, I'm going to go down the feature list. Each virtual device has eight FIFO command queues. These are eight input queues to the scheduler. And since a virtual device can be as small as a single die, it can get down to a one-to-one ratio of a scheduler, eight-way scheduler per die. The device scheduler handles all suspend-resume operations for programming and race commands, giving read operations priority. The host can specify which of these eight queues, input queues, are used for each type of flash access command. So for example, you could configure all read operations to be submitted to queue zero, all write-off or program operations going to queue one. The assignment of flash operations to scheduler input queues is done on a per QoS domain basis. So one QoS domain can be submitting its reads to queue zero. A second QoS domain could be submitting its reads to queue two. Anything's possible there. Every queue can specify die time waiting for both read operations, program operations, and a race commands. It's a credit-based system that allows us to allocate die time based on very complex and flexible needs. And finally, the host can override the defaults for both queue assignment and operation wait for each individual command. Now let's talk a little bit about the functionality. We mentioned that each one of the queues can have separate waits for each type of operation, an erase, a program, or a read. If you set all the waits to zero, this works as an eight-level priority scheduler with input Q zero being the highest priority and input Q seven being the lowest priority. When all the waits are the same, it works as a round robin scheduler. And when all the waits are unique, it works as a die time weighted fair queuing scheduler. And now another demo of weighted fair queuing running on our FPGA prototype. So some things to note. We started off with QoS Domain 1 in red, having a wait of 198 and QoS Domain 2 in blue at a wait of 202. We did that so the lines wouldn't overlap each other. We're now going to alter the wait for read operations, giving a much lower wait for QoS Domain 2. This graph is a graph of read response latency. And so as we lowered the wait, the latency was reduced in real time. Now we've gone ahead and shifted the wait over to favor QoS Domain 1. And you can see the latency response curves for the two QoS domains have now inverted. QoS Domain 1 has priority over QoS Domain 2. I promised we'd talk about latency control and it's a really complex issue. These two graphs have the same average latency, but the top graph has a better minimum and a much worse maximum latency. So which is the better response curve? Well, the answer is it's a trick question. It really depends on the application. Some applications prefer consistent latency and others need the best possible performance and can tolerate long tail latencies. With this in mind and with the theme of host control, software enabled flash allows control over the latency response curve. We demonstrated a bit of that in the last slide. We do this using many features that we've already introduced. So we can isolate workloads to make sure that they do not interfere or impact the latency of other workloads through the concepts of virtual devices or QoS Domain scheduling. We can map commands to different die queues per QoS Domain. Once again, prioritize and shake latency response. We can wait the commands per die queue and per QoS Domain to prioritize the different commands. For each individual command, we can provide an override. So in the previous demonstration, that's what we were doing. We were altering, in real time, the weights assigned to read operations by using this override mechanism. The system also provides automatic program and erase, suspend and resume functionality. Software enabled flash also gives host control for garbage collection with the offloaded nameless copy accelerator. You can prioritize and assign die times nameless copy operations separate from user read and user program operations. And you can adjust the weighting to account for right amplification to give consistent latency response curves. We also provide host control over where leveling and patrol operations. Once again, to allow the best possible response curve to the needs of the individual applications. This is the final demonstration of our FPGA prototype. This is just a single still frame taken from a live demo that we gave a while back. In this demonstration, we define three separate virtual devices and they were running three different workloads with three different storage protocols. In this example, we were running the ZNS protocol, a standard block protocol and a hyperscale FTL protocol in the different virtual devices. The graph is a heat map of die activity. The blue bars represent read operations and the red bars represent write operations on a per die basis. None of the workloads are impacting each other's and the same device is supporting the three different protocols simultaneously. In this first domain, which is circled, the workload was a mix of reads and writes, a fairly balanced workload. The second domain was doing a redominated workload and the third domain was doing a write dominated workload. We really don't think that running different storage protocols simultaneously on a single device is a common use case, but the value is that a single device can be deployed at scale and then provision and configured in real time to match dynamic needs. As new storage protocols evolve, software enabled flash can adapt quickly. And now we're going to have some actual examples that are taken from the software developer kit. SCF CLI is used to configure a software enabled flash device. It is a command line tool, it's open source and included in the SDK and any of its functionality can be incorporated into an application if needed. There's extensive built-in help and this is the top level of the health output. In addition to allowing configuration of the device, it also supports all of the API primitives. You can read and write data with this command line tool if you want, but probably the neatest feature of this is the built-in Python shell for interactive programming of the device. So in the target section of the help, you'll see that the different functions supported by this CLI tool are grouped by functionality. So we have functions that deal with virtual devices that could be creating a virtual device, listing virtual devices, deleting virtual devices or changing virtual devices. We have functions that apply to super blocks, the shell, which is the interactive Python shell, unit level controls, the block layer controls, root pointers, many things that we aren't going to be able to cover in this introductory webinar. But getting back to that Python shell, it's extremely useful tool for examining the device for diagnostic purposes while you're debugging your software development. You can even supply Python scripts to the SEFCLI program and have it execute those scripts for you. This is really helpful, once again, for common debugging tasks. We use this all the time in-house. So now that I've introduced the SEFCLI program, let's go over an example of its use. Note that in all of the coming examples, there are many possible settings that aren't illustrated in the pursuit of clarity and simplicity. When you don't supply an argument or possible setting, there are default values specified for each parameter. So let's go over how you would create a virtual device with the SEFCLI tool. As you can see, you can specify the starting channel and the number of channels. This defines the channel level parallelism of the virtual device. Likewise, the next arguments start bank and number of banks determine the number of die per channel that you've specified that will be assigned. The combination of channels, number of channels and number of banks defines the total number of die in the virtual device, which in turn defines the maximum parallelism of the device and the size of a superblock for the device. When we create this virtual device, we assign it a virtual device ID in this example. It's being set to virtual ID zero. And we can also set a parameter to limit the number of QoS domains that can be created within this virtual device. In this example, we've set that to one. Once you've created a virtual device, you can also use the SEFCLI command to list the virtual devices and that's illustrated at the bottom of the slide example. Next, we're gonna create a quality of service domain or QoS domain. With a QoS domain, although it's not shown here, you can specify the Q assignments for each of the flash operations. So here we have told SEFCLI, we want to create a QoS domain. We are creating this domain minus S zero is specifying which software enabled flash device we wanna create this domain on. We have to say which virtual device on that software enabled flash device, we wanna create this QoS domain in. So we're going to create this QoS domain in virtual device zero, the one we just created in the previous example. We're gonna give this QoS domain an ID of two. We're going to specify the capacity of this QoS domain. This is always expressed in terms of atomic data units or ADUs, and we're gonna specify the size of the atomic data unit. In this case, we are specifying a 4K atomic data unit size. The last two parameters are fairly interesting parameters, number of root pointers. The number of root pointers is a argument that allows you to specify how much metadata, how many metadata root pointers you want to store within this device. So if you were implementing a key value store with software enabled flash, you may have a set of key value mappings that you need to store or persist. Or if you were implementing a standard SSD with software enabled flash technology, you would have a lookup table for the LBAs. That metadata you would want to be able to store and retrieve. The problem is, is where do you store your metadata? Well, our answer is you store the metadata within the QoS domain itself because QoS domain can be running potentially a different protocol and have a different type of metadata, unique metadata in fact, and that poses the problem. If you store the metadata within the device itself, how can you retrieve it? Well, you need an anchor point and that's what a root pointer is. It is a piece of metadata that we guarantee that you can retrieve on system initialization that can point to all of your other metadata that you need to initialize your storage application. And then the number of placement IDs, as mentioned earlier in the Nameless Write discussion, one form of Nameless Write is to specify that this data needs to be grouped with other data and all data with the same placement ID will be put in a set of super blocks that are actually allocated for that placement ID and we will not mix data with different placement IDs within the same super block. In this case, in this QoS domain, we're specifying that we can have four different placement IDs active within this QoS domain simultaneously. And so that is what it takes to define a QoS domain and once you've defined the QoS domain, you can list out all the QoS domains that are present on the device with the list QoS command illustrated below. This next example is a Nameless Write and this is the minimal program necessary to form a write of data into a previously existing QoS domain. So if you would use the SEFCLI command to define a virtual device and then later to define a QoS domain within that virtual device, you could write this program, simple C program. The first call is to get handled on a CEP unit. So this says I would like to interact with this particular software enabled flash device. They are enumerated by the system automatically. And of course, you would need appropriate permissions to be able to open the device. But once you've opened the device, you get a handle to it. Once you have a handle to the device, you can then call SEF open QoS domain. You supply the handle to the device and the ID of the QoS domain you would like to open. You can also optionally supply a notification function to get asynchronous notifications for that QoS domain as well as a context, which is a piece of data that we will preserve through the operations that we don't use internally at all. It's user context, very helpful for maintaining asynchronous state. Finally, key was the encryption key for the QoS domain. And the last argument is a address of a QoS handle that will be returned by the open command. So first you get a handle to the software enabled flash unit, next you open a QoS domain and get a handle to the QoS domain. And with those two pieces of information, you can issue as many writes as you would like. And so here is a call to SEF write without physical address one. It takes the QoS handle that you just opened. In this case, we're doing auto allocate. When you're doing auto allocate mode, you need to supply a placement ID of what placement ID you wanna group this data with. You can supply a user address, which is a piece of metadata that will be recorded along with the user data. You specify the number of atomic data units you're trying to write, an IO vector mapping out the write buffers for the write. In this case, the one is a count of how many entries are in the IO vector. And we will return the permanent address. Once again, you've set the bounds for where we can perform the write. The device actually chooses the actual physical address and returns that to you in the permanent address field. And then we also return the distance to the end of the super block in case you're doing manual allocation mode. It's helpful to know when you're approaching the end of a super block and need to consider allocating another super block. And the final parameter to name was write is an override structure, which allows you to override either Q placement or Q waiting. I mentioned when you opened a QoS domain that you could supply a notification function to handle asynchronous events. There are several asynchronous events that a software-enabled flash device can generate. Once again, in this example, we're showing that you open a QoS domain and you supply a handle to the asynchronous event notification function. And then beneath that, we have the actual asynchronous notification function. This is just a small stub of a very large switch statement showing some of the different types of asynchronous events. There is an address update event, which is used when data is moved. Data can be moved by the device under a number of circumstances, but one of the most common would be if in fact there was a right error when performing the right to the device, we could have decided, we could have told you that we intend to place the data at one location and end up that we have to write it to an alternate location due to a media defect. And the next example there is a block state change notification. So you can tell when a block has been filled and closed. There are also some capacity-related events. When we defined the QoS domain, we gave that domain a capacity quota. It is possible to do thin provisioning and in those cases you can hit events for running out of space on the device if you've overcommitted data and need to take action. This next example is an example taken from our reference flash translation layer and it shows how we update the lookup tables, the translation between a logical block address and a physical flash address for a block mode implementation or traditional SSD implementation or block mode driver. And the important thing to note here is that you see once again a context which is useful for managing multiple independent flash translation layers. This tells you which flash translation layer you're operating on, the user logical block address, the actual physical flash address. And what you can see in this example is that we are using atomic exchange operations and we are updating in this example, both the valid bitmap showing that this is valid data within the lookup table and then we are performing the atomic exchange on the actual lookup table itself so that we're entirely lockless upon our table insertion. The next example is an address update. This could happen once again, if in fact we had to move data internally as a result of some internal event in the device. Lots of comments here, but the key takeaways are that the address table update is also lockless. We supply the expected flash address as well as the new flash address. And this allows us to handle race conditions between data overwrites and data movement without doing any locks whatsoever in the FTL. So once again, the reference FTL is a very high quality, high performing FTL. This example is an example of a direct access read. Once again, the housekeeping of opening up a unit, opening up a QoS domain, and then reading the physical address by specifying the QoS handle and the flash address you wanna read. And I think it should be fairly obvious. Once you've opened up the QoS domain, you can issue as many read and write commands as you want, you only have to open it once. But those operations are listed for completeness here. And there's also a little bit of a note here that the error recovery mode for a QoS domain is set at the time of creation of the QoS domain. And this determines if the software able flash unit will perform automatic error recovery on the QoS domain. And so you can actually set a read deadline that allows us to limit recovery attempts. And this is very useful in environments where you have mirroring at the system level. It's oftentimes quicker to retrieve data from a mirror upon a read error than it is to actually do the heroic recovery attempts on read-read trial. I'm sorry to jump in here, Rory, but we do just have a little bit of a break. But we do just have four minutes left in the presentation. So yeah, feel free to continue on here, thanks. Thanks, I knew it was gonna be close. So this is actually the last of the API commands. And this is just an illustration that all of the previous command examples I showed were synchronous command examples. They all have asynchronous counterparts that take IO control blocks and operate asynchronously. So let's get on to the summary and how to learn more. So hope you've seen that software-enabled flash technologies is a pretty interesting new approach to handling flash media, giving control to the host. So it is purpose-built hardware. It is an open-source flash native API. It's leveraging the industry standards wherever possible. And it can be used as a building block for multiple drive protocols. So we've actually demonstrated traditional block SSDs built on top of software-enabled flash technology, ZNS drives built on top of it, custom hyperscaler FTLs implemented on top of it. And as well, we've gone flash native with several different applications. And we're trying to combine full host control with ease of use, it's a very delicate balance to strike. So for more information, the first thing would be to direct you to the microsite. That's www.software-enabled-flash.com. There you can find the white paper, as well as pointers to the actual repository for the header files and API documentation that have already been published. And with that, I'm done with the presentation. Thank you. Awesome. Thank you so much, Rory, for joining us. And thank you to all of our attendees. Hope you enjoyed the presentation and we look forward to having you join us next time. Bye, everyone.


Good morning and welcome to the sixth meeting in 2016 of the SPPA committee. Can I remind everyone to switch mobile phones and other devices to silent? The first item on our agenda this morning is cross-party groups and we will be taking evidence from Gordon MacDonald MSP on the proposed CPG on independent convenience stores and then from Jackson Cannell MSP on the proposed CPG on building bridges with Israel. I would like to welcome Mr MacDonald to the committee this morning and I would like to invite you to make an opening statement about the proposed CPG. Thank you, convener, and thanks for the opportunity to attend the committee and highlight the need for a cross-party group on independent convenience stores. In session 4, there were 85 cross-party groups registered and yet despite that large number, there was no CPG focusing on retailing and, in particular, the independent convenience store sector, who are the lifeblood of many small rural communities and urban neighbourhood shopping areas. The Scottish local shop report published late 2015 was the first-ever report into the importance of convenience stores to Scotland and was produced by the Scottish Grocer's Federation. The report found that the majority of the five and a half thousand convenience stores operating in Scotland are run by small business owners and that the sector provides over 44,000 jobs with their value to the economy in terms of gross added value at over half a billion pounds per annum. Scotland has more convenience stores per head than any other part of the UK. Yet, at present, there is no cross-party group that can provide a forum for retailers and those in the supply chain to interact with MSPs, raising awareness of the opportunities and the challenges facing the sector. The importance of this sector was reflected by the number of MSPs who asked to join the group and the number of retailers who attended the inaugural meeting. In total, over 50 people representing the retail trade gave up their time from running their businesses to highlight the importance of establishing this forum. It was agreed that, if recognition was granted to this group, subgroups would be established on the economy, retail crime and community support, providing background papers in order to inform MSPs and form the basis for discussion at subsequent CPG meetings. In session 4, we had sectorial CPGs covering aviation, construction, oil and gas and Scotch whisky. Retailing is the biggest private sector employer in Scotland, providing 13% of all private sector jobs. Given the number of jobs that the independent convenience store sector provides and its importance to many communities across Scotland, I ask that you approve the creation of this new cross-party group. Thank you very much, Mr MacDonald. Do any of the committee members have any questions for the member? Mr Scott? Mr MacDonald, thank you. Do you think that there might be any overlaps with any other groups that are already established or not? I am not aware of any cross-party group who's sole focus is on the retailing sector. There is a cross-party group on towns and town centres, but if you examine the session 4 registration document for that cross-party group, you will find no mention of retailing within its purpose. The list of members that this group includes only one retailer. There are more local authorities that are members of that group than actual retailers. I notice that Professor Leigh Sparks is the chairman of the town's group as well, but anyway, that's terrific. That's fine. Thank you very much. Any further questions from the members? I have it on record that I am a member of this cross-party group, so I am just really declaring that interest. Thank you, Mr Johnson. Mr MacDonald, we will be considering this at item 2 on the agenda, and you will be informed of our decision as quickly as possible. Thank you for your time this morning. Good morning, and again thank you very much for coming to the committee this morning and I invite you to make some opening remarks. Thank you very much. Convener, the Jewish community settled in Scotland at the beginning of the 20th century. Many of them have subsequently come here as refugees fleeing from persecution, unlike all others then and before and since. They should be made to feel welcome and safe in Scotland without fear or favour. That community in the post-war years amounted to 48,000 people. It now amounts just to 6,000 souls across Scotland, predominantly in the west of Scotland, but with groupings in Edinburgh and other parts of Scotland too. They are a fiercely loyal community. It is estimated that over 90 per cent of the Jewish community in Scotland will have visited Israel, and the purpose of this cross-party group is to build bridges with Israel. What it is not, is not to be a chair-leader for the state of Israel. There are very strong and passionate issues that are raised and which have concerns across a much wider range of political considerations. There are other cross-party groups that represent areas in that region that also have a challenging political background to them. However, it is designed to do two specific things. One is to build bridges in the cultural, economic field in particular. Culturally, the Jewish community helped to establish the Edinburgh Festival here in Scotland. Economically, many of the pharmaceutical products that we use in this country come from Israel. Secondly, it is designed to tackle antisemitism here in Scotland. The Jewish community in particular has felt claustrophobic in many respects, sometimes under pressure, very often through a feeling that, because their community is concentrated in quite limited areas, the broader discussion about antisemitism and the broader discussion about Israel does not take place. The purpose of the cross-party group is to represent both those bridges that can be built but also to tackle antisemitism here in Scotland. Just to understand the purpose, it seems from what you are saying and what is written in the paperwork that the purpose is threefold. One is to build bridges culturally, academically and so on, as CPGs might do with many other countries. The second is to present an alternative viewpoint. I think that that is the way that it is phrased in the paperwork. An alternative viewpoint is that singular one, which is to the four at present. The third is to further the goal of countering antisemitism. I am just wondering whether you felt that there was a reason why those three distinct purposes should not be reflected in the title rather than only one of them. No, I think that the intention underpinning the group is specifically to build bridges with Israel across all of those fronts. Building bridges in terms of understanding as well as in terms of the specific areas that I have identified of education, culture and business, but building understanding as well. It is building understanding with the state of Israel in the broadest sense and building bridges between the community here in Scotland, the Jewish community here in Scotland and with Israel as well. I am just wondering what you are saying about the balance between those different aspects. Obviously, presenting an alternative viewpoint to the one that is to the four at present, as phrased in the paperwork, is something that would perhaps attract people who support that other viewpoint, whereas countering antisemitism, I would hope, is something that all of us, whatever our views on the issues of the actions of the state of Israel, would want to support. Therefore, I would be hoping that that aspect of the CPG's work is explicitly open to all, regardless of viewpoints on issues regarding Israel. This group is not at any way intended to try and create any division here in Parliament between members. It has a broad representation from a number of different political parties. If you look at the groups that have chosen to support it, it is not seeking to focus on the international situation exclusively. Many of the people who are seeking to represent us come from completely different perspectives. One of the things that we have talked about in the preliminary meeting that we had is the potential for this group at some stage to hold a joint meeting with the cross-party group on Palestine. It would probably be beyond the Scottish Parliament and two cross-party groups to resolve an age-old historical problem. However, in terms of the spirit of the title of this cross-party group, anything that the cross-party group could do in due course. I think that those things would have to be sensitively and carefully handled and with an understanding of what might be achieved from them. However, if it built bridges in that sense as well, I think that that is something that the group would welcome to do. There is no suggestion, of course, that if the broader agenda of the group, in the same sense that I have a great deal of support for the state of Palestine and I am not worried about the cross-party group on Palestine, it is not to suggest that any political grouping or individual who felt that this was not a group with the broader agenda that they would wish to support, that in some sense gives any suggestion that they are endorsing anti-Semitic behaviour, that the whole of the Parliament I understand and believe is united against that. That is very helpful. In that regard, do you have any indication of the topics that you may have under discussion in your upcoming meetings? We are very keen, because of the cultural aspect of the group, to seek to have some reflection of that in each of the committee meetings that take place. Culture in the broadest sense that can be music, drama, creative arts, but to try and introduce the cultural aspect into the Parliament. Yes, there are a number of areas that we want to discuss. We want to look at how, with the development of groups who are part of the centre for Israel relations, who are a group of people who are not Jewish themselves and who are seeking to try and ensure that the promotion of Israel takes place across the whole of Scotland, how that can be better facilitated. We would hope to hear from individuals in the business community, the cultural community and the educational community who would be able to help, identify and take forward the links that we are seeking to promote. We have certainly had a lot of interest from all those areas, from people who would like to participate in that work. Thank you so much. I wonder if Mr Carlaw could give us the reasoning for a group particularly on Israel and something that is not covered by other CPGs? Have you your content that that is required on that basis? There are a number of groups that have been formed in the Parliament representing, I imagine, similar objectives. I know that there is one seeking to re-establish itself in Russia, there has been one in China, there is one in Taiwan, there is one in Palestine, there is one in Germany, there is one in Poland. There have been developments, and many of those are underpinned in the first instance by a relatively strong immigrant community who can trace their roots back to the country that they wish to seek Scotland to develop links with. I think that the greater the promotion of those, the better. I have spoken with Visit Scotland and we hope to discuss with Visit Scotland as we have with some of the commercial airlines the opportunities for the development of tourism with Israel. There are many people who make that journey, but I think that there is a tremendous opportunity for not tourism to Israel but tourism from Israel into Scotland. In the same way that the promotion of individual countries can be progressed with the establishment of cross-party groups, I believe that that is also true of Israel. Are there any further comments or questions from members? Thank you very much, Mr Carlaw. We will consider this under the next item on our agenda and inform you as soon as possible of the decision of the committee. We now move to agenda item 2 for the committee to consider whether to accord recognition of the CPG and I will firstly discuss the independent convenience stores. Are there any comments from committee members? I think that it seems like a very reasonable thing to do. Mr McDonald has given it a great deal of thought and I would wish him every success. Thank you. I think that I would like to concur with those comments because I can ask that the committee is content to approve that CPG. Thank you very much. Next, we move to building bridges with Israel. Are there any comments from the committee on this area? I think that it is perhaps just worth reflecting on the fact that Jackson Carlaw has clearly considered some of the potential controversies that could arise from this group and so long as it is clear that the ethos that he is suggesting is the way things work and that those who take a more critical stance in relation to the actions of the state of Israel or issues, for example, around boycott, divestment and sanctions policies would not in any way be excluded from participating in its work on countering anti-semitism and that it is not seen as a conflict between those two objectives. I do not think that I have any objection. I think that Mr Carlaw has given us a flavour of what he is trying to achieve and it might be a difficult balance to achieve, depending on what the groups decide to do when they attempt to build bridges, but at the same time, I think that the idea and the concept works well and would enhance tourism and culture and all of that. It does have a real opportunity to make a contribution. I agree with what Mr Harvey and Mr Stewart have said. We are content to approve the cross-party group on building bridges with Israel. We will now move into private sessions to invite members of the press and people from the gallery to leave.


All right. Hello everyone. Welcome. Thank you for being here for the Tech Focus Caring for 3D Printed Art Symposium. I'm Sarah Satron. I'm the Education Manager for the Foundation for Advancement and Conservation. And we're really happy to have you here. We have a great international audience and an international group of speakers, which I'm so happy to welcome today. And so without further ado, I'm going to turn it over to the organizers who I just would like to give a huge thank you to. They have done a huge amount of work to put this program together. They are Alex Nichols, Martina Hyde Vogel, and Emily Hamilton, and Alex and Martina will be moderating the symposium. So I'll turn it over to them now. Thank you, Sarah. And hi everybody. So hello and welcome to the fourth instance of the conference series Tech Focus on the topic of caring for 3D printed art. My name is Martina Hyde Vogel. I'm Alex Nichols. And together we will accompany you through the next days as we as a community will embark on putting our heads together to discuss, consider, rethink, and possibly establish the care these artworks necessitate. Three conference instances have happened in the past on the topics of video, film and slide, and the latest one on software based art. As you can tell, these had a clear focus on media artworks and their technology. The conference name itself is a play on tech archeology, a groundbreaking conference that address the preservation of media artworks in a collaborative manner between curators, conservators, technical experts and artists. Tech archeology was held at the San Francisco Museum of Modern Art in January of 2000. In the same way, Tech Focus too is designed to bring together various stakeholders from across disciplines. And it is the artworks that require this approach and their care pushes us to manifest these ways of working in our institutions. And it fills me with enormous joy that in our conference today this collaborative way of working also extends into several disciplines within conservation. In the audience today we have objects conservators, media conservators, paper conservators, textile conservators and conservation scientists. We also have artists, educators, technical experts, collection managers, scientists, art producers and the list goes on. It was a joy going over the attendees list and seeing how rich our audience was today and seeing how these artworks of 3D printing make us all come together. Thank you all for coming. This conference has been in the making for over two years. It was originally to be hosted at the San Francisco Museum of Modern Art, hence the not to Tech archeology, but then the COVID pandemic forced us to reschedule it and move it into the digital realm altogether. While this saddens us tremendously and prevents us from having coffees together during our breaks. It allows us to draw an international audience of over 130 people from four continents. This conference was organized by Emily Hamilton, Alex Nichols and myself with incredible support from Sarah satron and our colleagues from the tech focus team, as well as numerous other people with whom we've been having conversations are wonderful and incredibly knowledgeable speakers, printers, artists, colleagues, and friends. So thank you specifically for the many hours we've spent together in conversation, leading us through dark valleys and new beginnings are most heartfelt things go out to all of you, you know who you are. And I also want to take a moment to specifically thank both my co organizers, without whom this program would not be standing as confidently as it does. Emily and Alex. Thank you both for the many thoughtful conversations we've had leading up to this day, full of giggling and concern, humor and care tears and laughter. It is always about people. And Alex, I want to thank you specifically for being just plain awesome on this journey, on which you stepped in halfway through this conference planning space, when Emily had to step out. And you stepped onto this moving train that one could see from a mile away was going to be a bit of a bumpy ride. Alex, thank you for writing it out with me, and for the many beautiful and touching moments we've shared this conference would not have happened without the generous support from the National Endowment for the Arts and FAC, the foundation for advancement in conservation. I want to give special thanks to Sarah Satan, Satan, and Eric Prusho from FAC for their support and applying for these grants. We want to want to thank James Hughes and Feliz Graciela Robles from New York from NYU, who have helped us to send out the 3d printing sample case, as well as as of moment for their initial support, and they're offered to host this conference on their premises. Thank you so much, Martina. So just to give you a really quick structure of our three day conference, it will be as follows. Day one will focus on the history and technology of three people printing as a medium. So during this time, we will also hold our school of seeing series of talks. We will discuss the methods and materials used in manufacturing 3D printing. Those who registered prior to September 1st were sent a 3D print sample kit created from various materials and with different printing methods. And these samples will be referenced during the school of seeing presentations. Day two will focus on 3D printed artworks and their lives once they enter the museum. We'll hear from a variety of museum professionals, artists, and stakeholders about their perspectives on collecting and acquiring 3D printed artworks. We'll continue this theme on day three with case studies and with talks focusing on the condition issues with 3D printed artworks. We'll also discuss some of the ethics and the approach to caring for 3D artworks or 3D printing and culminate the whole conference in a panel discussion led by Jill Sterrett. For those of you who stepped in just a little bit late here, we have some quick housekeeping rules that we'd just like to remind you of. And we're on Zoom webinars, so the format might be a little bit different than other Zoom platform to use before. For questions to the speakers, please use the Q&A box down to the bottom of your screen. But you can also use the chat box to add comments to the general audience. Thank you so much to everyone who's already introduced each other. I always think it's really nice whatever you get a good discussion in the chat box really forms a sense of community. We will also be recording all of these sessions. So in case you want to revisit a talk at a later time, you'll be able to access them using the same portal that you use to join the conference. We also have a few references which can be find on the FAIC platform. So we have a collaborative notes document, a references document, a key for the 3D printed sample kit, and STL files for the sample kit itself. And I think Martina might be putting a link to the collaborative notes document in the chat. But on the same portal that you use to enter here, you can see the handout section. So you can find direct links to all of these right here as well. We have also created a mirror board for this conference as well. And mirror is a free platform, so it does not require login at all. And with this, we've created some activities for everyone just to help connect people during the conference.  We should just do a little demo right away. So I'll stop sharing my screen and I'll just show you what is here. So this is what the mirror board looks like. And on the lower right hand corner, you can see a little zoom in area as well. And you can move around. And so we'd like you to start out. It's always fun to just see where everyone is coming from. So to do this, I'm going to hover over to one of these little red dots in the corner. And I'm personally coming from London. So I'm just going to copy it. And I'll see how good my geography skills are. And place it right there. I just think it's a really great way to show everyone where you're coming from. So so we can see what we're doing. But in addition to that, we also have, whoops, I'm just using your mouse, you can scroll over. If there are any questions that you have, you can add them using either sticky. So on the left hand side, there's a sticky note option. Choose your color. Place it down. And just put it in a note. You can also use the text box as well. So if you want to say add something here, say hi, I would like to say something cool. This is very small. So I'm just going to increase the font size like that. And then you can also go down and just for fun, we can just draw some pictures. So for this one. This is a very poorly made cat for Halloween. That's always fun. Great. So let's get back to business. So once again, here's the mirror board link and it's also the chat as well. But without any further ado, let's all get started.