All right, then. I think we're just about ready to start as I see most of people who are interested have already filed in So thanks everyone for coming. It's nice to see so many people here It that's interested in this particular topic and yes to make sure everyone Wants to be are actually here since there's actually a minor misprint in the pamphlet if you're interested in Grease pencil in VR. This is actually the wrong session if you want to learn more about that I would suggest that you head down to the theater instead And without further ado Let's start get started. I Should warn you though this talk actually it seems like some people actually misread So let's wait a few minutes to let them file out and there. That's it going again I like would like to warn all of you though This talk is going to be a little bit of a technical or rather a Talk about free technical topics as such the supply of need to renders is actually going to be running as little bit sparse But I hope to keep you all entertained and Hope all of you learn at least something from this. Oh, right But first some kind of introductions is probably in order My name is Gustav Valde-Marsson a software engineer at arm where I typically help out with various bits and bobs in the Mali graphics driver stack and especially the parts that are related to hardware accelerated ray tracing and To top it all off. I'm a so-called industrial PhD student at Lund University in Sweden Where I also work with various ray tracing related topics some of which will actually make a small appearance later on in this talk All that said I am obliged to say that I do not represent arm during this event Nor is any of the content here sponsored by them as such any views that I express here in the presentation are strictly my own So let's get into the meat of things then shall we as I hope you can imagine The topic I will discuss is about managing custom data inside of GLTF files and using blender as our interface for doing this As such I can imagine that the audience is split something like this on one side we have people who know and work with GLTF in one way or another and on the other we probably have blender developers or plug-in writers and Lastly we have some Group of people that probably have some kind of data that isn't natively inside of blender or in GLTF But somehow want to make them work together And this could be really any kind of data. It could be some tree data Some funky textures or data used to a completely elsewhere in some kind of database And my goal here today is of course not to make any of you an expert on any of these topics But what I want to do is slowly pull all of these groups together just a little bit So there's a bit of overlap such that anyone who wants to do something similar to what I have done just a little bit easier and Of course, I hope to do that with a few compelling examples With that we have a few different areas to cover as such I will naturally start by talking a little bit about GLTF itself What it is what kind of data it actually stores and so forth Second, I'll briefly go over how blender works with GLTF files and talk a little bit about the add-on that drives that process As well as a discussing at least one of a tool that you may come across when working with these kind of files Such as when you're trying to pull GLTF data into your own rendering frameworks After that we get into the custom things themselves Here, I will present the extension mechanism in GLTF that makes all of this possible And after that I will go into detail how you would construct a simple plug-in That simply performs a bit of watermarking on each image that we are exporting After that, I'll also show some of the other functions or hooks as they are called that we can modify inside of this export process I'll then move on to my primary use of these facilities Namely generating a thing that is known as a micro map during the GLTF export process And finally, I'll be wrapping things up by saying a few words about the importer side of GLTF as well So where should we start? Well for completeness sake, we obviously should start with GLTF itself In short, it is a royalty-free format for transmitting and Loading 3d scenes and models while keeping the data itself about these models as small as possible Both while it is in memory and while it is stored on disk Additionally and most interesting to us. It is very extensible Which is what makes everything else I will be talking about here possible That said GLTF is not a universal 3d modeling format The base primitives it supports are distinctly geared towards real-time rendering use cases In fact, many of the constants and enumerations in the format map directly to equivalent API values in OpenGL All to make it as seamless as possible to load all the data in a file and send it directly to the GPU for rendering But of course the format is not limited to that as an example the material model it uses is pretty general and With the aforementioned extension system. It is well extendable I should mention though that I will not really go into the detail of how to offer GLTF scenes But I will instead focus on some of the more technical aspects that we need to know about to write plugins like this That said, Cheuviens du Roor, and I hope I'm not butchering your name too much Is what one of the primary offer of the GLTF importer Exporter add-on that I've literally built everything on top of will actually have a session right after this in the classroom venue That will deal with this particular Aspect in much greater detail so starting with the GLTF files themselves essentially The there are essentially three versions of GLTF which in blender are exposed as GLTF embedded separable or binary and the first two of these are essentially two versions of a JSON file and Only really differs in how some of the data are packed inside of that file For the embedded format all data even images are packed inside the same file and Binary data is just encoded using base 64 for all strings and then it's just as JSON string inside the file and this is all to make it easy to transport and share the file while also being able to change it slightly afterwards Change some of the other aspects is more easily But of course if you want to actually change the binary data You basically need a separate tool to pull it back out again to re-offer it For the other one all the data or the binary data is instead packed into separate files That are stored next to the primary GLTF file And this is mostly to make sure that it's easier to re-offer Some of these files basically the textures if you want to change them after the fact and The last format is an entirely binary format and thus is also the most compact version of this and That is a format that I like to think of as a binary equivalent of JSON But of course there are more new and nuances to it than that, but it's nothing that I have time to get into in this talk So let us take a look inside of the GLTF file then shall we? In brief the entire file structure is actually really well encapsulated inside this single Relationship diagram which lists essentially all types of objects that you can find in the so-called GLTF world And while it looks a little bit complicated We can actually start to pull this apart to identify some key components that we may care about in some contexts For instance this top part of the file basically define the scene graphs inside file and that is plural Basically, we have a single file that can contain or a single thing that can contain multiple scene graphs and On top of that it also actually contains the whole bone hierarchy for skinned animations but that is another side note that we won't be getting into and Just to show you roughly how this is structured This is a scene that just contains a single scene graph and you can see in the topmost Node basically you have a list of indices which represent the scene and in this trivial example The scene that it represents is just a graph with a single root node with two child nodes with separate transforms associated to them one with a rotation and one with a translation and Then of course It wouldn't be a proper scene graph if we didn't have measures associated with them and those can then be stored inside of each node separately and This use of indices that you can see here is a reoccurring theme in GLTF To keep everything as compact as possible Objects are practically always stored in a linear array Which are then crossed reference with indices when we need one of the other objects and Moving on this next part defines the shape the base shading model that is used by objects in the scene by default a Physically based rendering model that is called the metallic roughness model is used which essentially defines up to free Textures that we may care about a color a metallicness factor and a roughness factor Again, these are more details that I won't really be going into And lastly we have the so-called data storage model or data acquisition model And as you can imagine from this talk, this is something that we will actually have to care about a little bit Since it's quite important for us when we want to store custom data inside of these files so most of one Most of the work we want to do in this talk really just boils down to this We want to take some kind of data from some other source that we already have somewhere and pack this into something that GLTF understands and Since it is a JSON file, we could really pack anything into it But beyond the JSON primitives themselves there are these are their basic building blocks to packing other kinds of data inside this files and We can basically break down into a few different levels at the lowest level We find the buffer which is really just an unstructured source of bytes with a length associated to it and Just above that we have the buffer view which is really just a bit of metadata on how to index into a buffer using a offset an optional stride and length and Above that we find the accessors textures and images which describes Exactly what kind of data we want to access and how it should be accessed through for instance a buffer view So what becomes of all of this when it actually reaches blender them? well this is where the GLTF importer exporter add-on comes in and The way that it struck is structured is really straightforward Each object that exists in GLTF has an exact corresponding class as so to say that with it and That and any entity that was previously cross-referenced with indices that I mentioned before gets resolved through a proper object references object reference and all that to make it easier to handle this kind of data So thanks to she again and the other contributors of the GLTF add-on we can actually ignore many of these low-level details as That is automatically handled for us Even the low-level buffer handling and image objects gets automatically converted to classes that are called binary data and image data That automatically handles the conversion of well a byte array in Python to Well classes that we can read later on But I would be getting into that in much more detail a little bit later So Why if you are actually working with the format though keeping the GLTF specification and the object reference sheet That I have actually linked down here is actually really helpful because there's a lot of objects And you might care about some sub parts of them. So using them is an absolute must So in summary these classes will act as data containers if you need the data or a Rapper for reading and writing the various GLTF files However, if you are on the receiving end of a GLTF file, you may actually have to care about these details If you are exporting files from blender using GLTF and then using for instance the tiny GLTF C++ library to load the file in that case all you get are direct struct based a direct struct based representation of the GLTF file and It is entirely up to you to actually read back the data that you actually want Including handling this index resolution that I mentioned earlier Which as in in this truncated snippet that I wanted to show requires quite a bit of code in my case This takes roughly another hundred lines to correctly read back a single array of values which could be anything from float values to vectors and While it is bare bones tiny GLTF is the best tool that I've found for this application all you want to do is really read back GLTF files and If we obviously compare this to the facility lease we have Available in blender through the GLTF add-on everything is just much nicer in comparison To write a buffer and at the same time create a buffer view we simply construct a binary data class and If we want to read data back, that is just stored as a unstructured Byteray or as a binary data actually then we simply call the decode accessor function But the pet so depending on our context life may or may not be so easy for us and The final thing that I want to mention about GLTF is of course the extension mechanism As I mentioned earlier the GLTF file is essentially just a JSON file as such We technically can add whatever data we want to it Obviously the specification requires some stuff But it also gives us an official way for adding new attributes to it in a structured fashion This is done by adding new key object pairs under the special extension object in the file The key here in this case is the vendor prefix Basically the letters key khr for Kronos in this case followed by some kind of descriptive title Which in this case is light punctual which refers to allowing storage of various point-like sources inside of the file For this talk though Everything that everything that we will be working with is obviously going to be custom in some sense as such Wherever this prefix come into play, we will just be using the prefix none So it's up to you to choose whatever prefix is applicable for you and These extensions then essentially comes in two different flavors and they are appropriately named required and optional and as the name suggests a Compliant GLTF parser slash renderer should abort the process if an extension isn't supported For instance, if measures are compressed with an unsupported compression algorithm, they naturally cannot be handled in a Satisfiable way Optional ones though could be things like meta data containers or optional rendering features For instance, if the client does not support it, it should still be able to use the rest of the data Although renderings may be slightly different different from what the author intended and Then these extensions that are listed at the top of the JSON file like can see in this example here Then depending on the actual extension itself More extension keys like these can be found inside of the file attached to the other kinds of objects for instance seen For instance point lights are typically attached to a scene graph in some way as such It makes sense to actually attach these extension objects to the node objects in the file So that we can actually manipulate point lights as if they were actually nodes in the scene And of course some of these extensions are useful to are sorry useful to many people out there So okay, they are occasionally standardized as in the case of this punctual light extension And a collection of these can be found this link that I've included in the bottom However creating official standardized extension in this way is well beyond the scope of this talk But for anyone interested This link provides any details that you want in this case And finally we get to the plugins themselves and a quick disclaimer before we go on I would not like to claim that I'm a very good Plug-in. I would claim that I'm not a very good blender plug-in writer In fact, I'm pretty sure that there are much better ones in this very room But with that I would like to present a first example of some of these gltf tools that we can use a Simple plug-in that basically watermarks all textures while we're exporting them from blender And this is of course Admittedly a bit of a facetious use case, but it is a good example to see all of these things in action So with all that let us go over how to create one of these exporters exporter plugins in detail So let us start with having a clear goal in mind we want to know We are we already know what we want the water market to do. We have a nice little visual cue up here in the corner And we want this to happen to all images as we are exporting them But obviously we want to be able to control this plug-in and process in some kind of fashion To that end. We obviously want some kind of panel Add some kind of panel to gltf exporter that looks something like this Basically, we want the ability to enable or disable this plug-in so we can stop watermarking and Want the ability to select what image we should use as a watermark But let us start from the top first We need a special variable called BL info that blender uses to pick up various meta data about the plug-in such as the name Description current version of it as well as which Verse which blender versions it supports and so on and this is of course not unique to our type of plug-in But probably applies to all blender plugins and after a bit of digging I even found a specification for this So you can see what options are available for this Dictionary and what things you shouldn't be adding to it And next we get into the so-called property class that actually contain these control options In short, we want some kind of Boolean option to properly enable disable the option the plug-in itself and a string to use as an identifier for the work watermarking image and This property class is then used to help us create the actual buttons that change these options Which what is what we'll look into next and to do that? We actually need another class for the so-called look and feel of these buttons or controls This one uses a slew of variables and methods to define how to create the buttons how to set the properties and Even how they should be created and look And of particular interest to us here is the so-called BL parent ID variable Here this is set to a special key which basically signals that we are a child of the main gltf exporter panel Obviously, we could set these buttons to be anywhere in the blender UI by controlling that But this is a kind of logical place for it since we probably want to control our gltf options along with well the other options for it Then to actually create the buttons itself. There's a few things that we need to do and I only want to highlight a few things Basically, we get these draw methods that basically draw our panel itself and for that We get a layout where we actually add the various things that we want to create and to do that We have to call this prop method and give it the property class that we had Created previously and then select which key or option. We actually want to change and here is one caveat I really want to highlight It is strongly recommended to actually make sure that these two classes are uniquely named as such you can otherwise you can get enough Well Otherwise you can get into a weird situation where only the last plug-in that you loaded is actually going to show up in your export panel and Then finally to actually get them into blender in some reasonable fashion We basically need to register them and these are done with using these collection of methods and Of course if we'll register the classes We probably also want to unregister them if we should disable the add-on itself And that is done with the corresponding unregister methods One for the panel and one for the properties and then we finally get into the class that finds the gltf plug-in itself In your script you should define a class with the exact name gltf to Export user extension and then add methods to these that corresponds to the Methods that we want to customize or the data that we want to customize So in our case we want to customize all images that we export Something that is done using the appropriately named gather image hook Where we simply receive the gltf image and all other Blender data that's equivalent to this exported image and For completeness the actual function that does the watermarking process looks something like this Basically, we first convert the encoded image because that is just unstructured binary blob at this moment and in this case I do that using Well a minor utility function just to get back a NumPy array Representation of the image and then I basically applied the watermarking by just slapping it on top of that array and Finally to convert all the images back to binary. We use these special classes that I mentioned before Although we have to be a little bit careful depending on what the user wants we can actually get Different things inside of the image class Basically, if we decide to have an embedded gltf file the so-called URI Attribute in the gltf image will not be populated as such we should probably replace the what they call the buffer view of the image rather than the uri and that is simply done by testing that particular attribute and Setting the appropriate class in this case and the main reason we want to do this is that It's basically that We want to respect the user's choice if they have selected an embedded format We probably should respect that and make sure that everything still is that after we have run our plug-in We could override it, but of course a good plug-in should respect the user's choice And lastly, I want to mention a few words about the special Extension class that I didn't actually get to use in this watermarking example This is a special class that is used to signal that we are adding extension data to a file in some way Such as the exporter can pick this up and add it to the list of extensions in the file and As I showed before we can place this extension class pretty much anywhere such as in the node hierarchy for this very small example And it can essentially contain anything that can become valid JSON data And this includes any of the other GLTF classes such as the binary data class that I mentioned before and Now is probably a good time to add a little reminder for completeness sake I typically install add-ons by dropping the file that is represented by it into the add-on directory of Blender But then I completely forget to actually enable it and sit around scratching my head wondering where all my stuff has gone So this is easily fixed by opening the edit menu going to the preference dialogue and Opening the add-on section then I recommend enabling all sections and searching for GLTF And then simply click enable to make the add-on usable And I'm sure there are ways of doing this automatically From the command line, which I typically prefer to work on but this is a good way and a good reminder of how to do these Kind of things and if you have done all of this correctly You should have populated this extra section inside a GLTF export panel with all of your options Moving on to the actual hook functions then As you probably know Blender has loads of primitives and GLTF supports many of these and these are exposed in a long List of methods that you can see here All of which you can actually override but in short each of these map to one of the objects that you can find in the GLTF world So beyond the image hook that we looked at for the watermarking plugin We will only really talk about a few more methods or out of all of these Namely the mesh and the primitive hook and while this is pretty Restrictive I think that just the mesh hook alone is a good example Such as it's pretty easy to infer what else could be done in any of the other hooks In general, I try to think of each of these methods something like this It is a method that receives both the blender version of the object and the GLTF version And then it's up to us to actually customize and look at these things in order to decide what we want to do with them So in the case of a mesh we get the blender and the GLTF version of the same mesh But we also get objects that might be related to it in some way for instance We get for meshes. We also get the blender vertex groups the modifiers and active materials And finally all methods also gets this last object you can see here, which are export Export settings, which is controlled by the GLTF add-on itself, which are things like the Export directory where we are exporting things and other things like that And this then finally moves us on to my actual use case for GLTF As I mentioned, I use GLTF as my primary 3d modeling format for my research renderer As such, I often investigate various new techniques or types of data However, creating new formats for loading and manipulating this data is really annoying As such, I eventually realize that the GLTF extension mechanism could actually be used to simplify this matter quite significantly As such, allow me to present one of the things that I've been working on during the last year A relatively new type of rendering primitive that exists in Vulkan and DirectX That is known as a Micromap And basically this is the kind of thing that we can visualize something like here to the right And in this case, this is a texture file that is taken from our classical sponsor scene And what we want to do is be able to generate whatever this thing is here on the right And if you were to actually apply these to our traditional sponsor scene We can get an effect that looks something like this inside our ray tracing pipeline And this is entirely without using any kind of alpha mapping The Micromaps are effectively something that makes our geometry much more granular And it does that almost for free So without digging into too many of the details, and I mean this is a Vulkan extension after all There are loads of details to go around But we can still pretty easily describe it as follows Take any triangle in your scene Split this triangle along the midpoint edge of each Split each edge of the triangle along their midpoint Thus forming four new virtual sub triangles And then repeat this process up to n times Which becomes our so-called subdivision level Next, we can associate a value with each of these sub triangles And then we can use a special space filling curve to actually linearize each of these values Giving us a single array of values Which is our Micromap In short, you can then view this Micromap as a very specialized per triangle texture And what I want to do then is generate some kind of realistic data for each of these triangles Now in Vulkan currently, there are currently only one official type of Micromap available Namely the so-called opacity Micromap That is a map that stores opacity values for each of the sub triangles And only very specialized ones at that Basically, we have zero for a fully transparent triangle One for a fully opaque triangle And two different values of unknown We don't know the opacity values And want to have an escape hatch for faster processing in corner cases Naturally, this is a very specialized type of opacity And in fact, it is targeted at improving the performance of hardware-accelerated ray tracing applications By reducing the number of times you actually have to call the any-hit shader But that is a discussion for a completely different time And I want to stress this here While it looks like we are storing an array of values The kicker here really is that we only need to store two bits for each value In fact, there is another mode to these Micromaps Where we only need to store a single bit per value Hence, these Micromaps can be really compact Especially when you compare these to typical alpha maps And for completeness' sake, at rendering time We then have a very efficient algorithm for actually going from a triangle's barycentric coordinates To a Micromap index And consequently, the Micromap value that we stored in that specific virtual sub triangle So, to get back to my case then What I want to do is create a simple plugin That can generate these kind of Micromaps And embed them as part of the GLTF file Such that I can manipulate this data in some fashion And test out this new extension in various ways But, of course, I also want these to be realistic in some sense That is, we want our exporter to find any existing alpha texture Associated with the triangle And then generate a representative opacity Micromap for it So, in short, you can summarize this with this pseudocode For each triangle, add Micromaps to it If it has an alpha texture associated with it And yes, so we're clear exactly what we will get out in the end We want to be able to generate GLTF files That contains extension objects that looks something like this That is, the primitive, that is the list of triangles in the scene Now have a list of Micromaps also associated with it With any other meta data that might be interesting in this case Such as the subdivision level that I mentioned earlier So, on the face of it, it looks like we should be able to just add customization methods For primitives, and we should really just be off the races Unfortunately for us, or me at least in this case GLTF structures primitive something like this Basically, the meshes themselves contains the actual material And the primitive doesn't really have access to it So we actually need to move ourselves up a little bit in the hierarchy And customize the mesh exporter method instead And another good reason for moving up to the mesh level Is that we can much more easily attach custom primitive Custom control attributes to each mesh inside of Blender That actually allows us to control this Micromap generation process On a per mesh basis And this is something I do by just adding custom attributes Inside a mesh object down here And in this case I just name them according to what the attributes are Such as the subdivision level that you can see here Thus, to adapt our little loop to the GLTF world We want to do something roughly as follows Each time we export a mesh We want to check if the triangle contains alpha textures And if so, extract the GLTF triangles Basically the indices, the UVs, materials and textures And then use these to map the triangles to the alpha textures And generate Micromaps for that mapping And then finally store these as extensions inside the GLTF file And this is pretty easy on the face of it And all we really need to do is actually replace this loop With the gaver mesh hook, essentially And then we're really off to the races There are a couple of caveats here That makes this a little bit more challenging in practice For instance, each time we actually subdivide these triangles We basically quadruple the number of sub-triangles That is, it scales by a power of four And that can easily be very computationally dependent However, each of these sub-triangles are actually computationally independent So we can actually use any kind of multiprocessing In order to accelerate this process to generate them And of course, in Python, we can use the multiprocessing module In order to handle this for us And of course, if it's computationally heavy We probably also want to use profiling To help us find some problematic snippets And of course, in the essence of time Multiprocessing itself is a very complicated topic With numerous corner cases that I don't have time to go into But just for completeness' sake I want to include some general advice If you end up in a similar situation like this Remember that local variables and functions Are typically not available to you Avoid nested parallelism Try to flatten any nested structure that you have And provide those as arguments instead to your What do you call them? Parallelization functions For instance, for my micro-maps I typically want to parallelize over all primitives And all sub-triangles Which are basically give these methods as indices And remember that all data that are sent to these functions Are essentially copied As such, this could actually be really expensive Who are sending away large textures to each of these functions And finally, set appropriate chunk sizes The Python default of 1 is often inadequate So try to divide your tasks equally Between all of your cores And profile your applications I cannot stress this enough Python has loads of tools to make this easier So try to use them when possible And, including And to further help things The GLTF add-on actually has two more special methods That can help us with this in some regard These are simply called, as their names suggest Before and after the export process themselves As such, there are excellent places to start Or stop the profiling Or to actually print out the profiling results But moving back to the micro-maps themselves Obviously, I didn't learn everything about GLTF And micro-maps at once So I actually ended up with more than one format To store each of them in But as different extension objects So I basically ended up with two different use cases For really small micro-maps that I used for testing purposes It was much more easily mapped By using just a string It was much more easy to just store them as a string And then you could just change some of the few bits inside of it In order to change the objects that were testing However, larger micro-maps That actually represent some kind of texture Are probably not going to be modified in that kind of way So for them, we might actually use The existing GLTF buffer system instead To keep it much more neatly contained So in the previous example that I had up there We basically stored all of our micro-maps As strings inside a JSON object Which, of course, is terribly inefficient Effectively, we're using eight times as much storage As is necessary Instead, it's much better to use The existing GLTF buffer system for this use case Which we can very simply do By just converting the micro-maps to a byte array And then adding them to a binary class Like I do in this case And obviously, this talk only really concerns Opacity micro-maps However, NVIDIA have defined At least one other type of micro-map That is known as a displacement micro-map Which is already available in DirectX and Vulkan As a provisional extension On top of that, it also seems that they're looking They're working with more types of micro-maps As they have defined GLTF extensions For micro-maps with much more general attributes For each of these sub-triables This, though, is something that I have Not really explored in detail just yet But, of course, it would make sense To modify my plugin in some way Such that we can actually generate something Of the same format Additionally, displacement micro-maps themselves Might be something to keep an eye on in the future As this thing could be something That might fit neatly into the sculpting baking process But this is really just me speculating I don't really know enough about Either the extension or the baking process itself To have a valid opinion about this And, lastly, I want to talk briefly about importers Most of this talk has obviously focused on exporters Which makes sense, as that is what I've primarily been working with But it is also the most common use of GLTF Typically, we have a detailed model With loads of data In Blender or some other framework That we are effectively filtering as we are exporting it There are, of course, exceptions And the most compelling of these use cases Are probably compression Say you have developed a new compression algorithm That you might want to use the importer As an effective way of testing The decoding process of this mesh And, really, the structure of these add-ons Are very similar to the exporter With many of the same hooks Available for it Before many of the same hooks Available for it And the main thing that can differ Is that we now sometimes have Both a before and after hook Since we can't really get The Blender representation of a GLTF mesh Before we have decoded it And, obviously, I'm not skilled enough Really to develop something like this For a talk like this So we'll settle with showing a much simpler importer And this simple importer Basically looks at a GLTF file And looks if we have used any of these Micromap features that I showed you before And if they are present Then we basically add those custom attributes That I showed you before Basically these ones And setting these are really easy We basically just look inside our We basically look inside of a GLTF mesh For the presence of the extension key That I mentioned And if it's there We just set these on the Blender mesh attributes And they will be populated As I showed you here And just to quickly summarize everything then In this talk I've talked about GLTF What it is How it's structured And what kind of data it can store The GLTF add-on That Blender uses to import And export GLTF files And plugins to this add-on That allows us to customize This import and export process Along with at least two examples Such as a water marker And a Micromap importer and exporter plugin And finally I provided a small Collection of tips and tricks For how we would handle Things like multi-processing Or profiling inside of Blender And with that I think it's time to open up for questions I guess you could use it for rasterizing There's not really anything that stops that As it's effectively a texture of some kind But currently It's really only used for the Vulcan ray tracing hardware Accelerated ray tracing pipeline As far as I know It's not added to the rasterization pipeline I don't think there's As strictly any benefits to it But there could be I haven't really investigated this In detail It could be For instance in my case This GLTF extension Are optional So if you try to actually load This in a different Basically if you try to load this In my framework You get the Micromaps themselves Because obviously I support the framework But they are optional So if you load it somewhere else You basically get the base Thing without Micromaps That's a very good question Essentially On GitHub There is a Similar to many other Cronus things I should probably mention I'm not actually affiliated With Cronus in any way So this is just me Looking at what is accessible On GitHub Basically the GitHub repository Contains a list of all Currently official extensions But of course you can also check All forked repositories On GitHub And many forked ones Are the ones that contains these Extensions that are currently in works Such as the NVIDIA ones That I showed here I haven't encountered any limitations Yet But I haven't really worked with All that many just yet One thing that I did encounter Is that the hooks Can actually change definition Depending on which version of blender You are actually using Hopefully that is something that Would be rectified If the GLTF add-on Is more tightly integrated into blender But it might be a good thing To actually have a layer between Your GLTF GLTF extension plugin The last thing that I showed on the example And have that one basically Check which blender version you are using And changing which method that you actually Apply for that attribute I don't actually One more question down there It's absolutely fine Do you have your answer And be sure to check out Julian's talk Just after this Any other questions? If not Then call the session


A fiery horse with a speed of light, a cloud of dust, and a hearty high old silver, the Lone Ranger. In the early days of the western United States, the boys and girls who grew up in the new territory heard many stories about the masked rider of the planes. Courageous, daring, and resourceful, he championed the weak against the strong and brought law and order to a lawless frontier. It was he more than any other man who made possible the winning of the west. Return with us now to those thrilling days of yesteryear. From out of the past come the thunning hoofbeats of the great horse Silver, the Lone Ranger rides again. The Lone Ranger and his faithful Indian companion, Tonto, sat their mounts beside the trail to Laramie. Their gaze was fixed upon the cloud of dust that marked a stagecoach approaching in the distance and that should be it, Tonto. They should have left last Thursday to a two-day trip by stage from Yellow Ghost to Warshow. Warshow, they'd catch this Laramie stage, another two-day trip. This is Monday. If something happened, it'll lay them there in it. Not right. Hello. Carefully as it goes by, Mrs. Clements is an elderly gray head woman. You've seen the marshal. Me, watch him. I won't be able to see as well as you. I want to stay behind this rock. I don't want the driver to think he's in for a hold of. Be careful. Here they come. Watch closely now. Them on stage. You sure? Uh-huh. You see Laman. Good. What do now? The stage will be in Laramie in 30 minutes. It won't be long after that before the law goes into action. We're riding to Blackie's place where we can keep an eye on things and see that they go as they should. This is one strata that Blackie won't escape the law. Come on!  Come on! Half an hour later, in Laramie. Oh! Oh! Oh! No. You ain't.  I reckon I must be more than 20 years younger than you, and yet I'm only... I can't think of rest until I see the man who killed John Hanged. Oh, sure. Sure. I'll save him. Maybe we'd better be getting out. There.  I'll help you down. Thank you. There you are, ma'am. Oh, then. That must be the sheriff over there. Here, this way, ma'am. Sheriff! Howdy. Howdy, ma'am. I see you're wearing the United States Marshals badge, Mr. Your sheriff, Otis. Yes, ma'am. This is Mrs. Clements. A pleasure, ma'am. How do you do? I'm Marshal Cleary from Yellow Gulch. Yeah? Well, well. I've heard of you. What brings you to Laramie? Sheriff, my husband... Pardon, ma'am, but maybe it'd be better if I was to explain. Sheriff, Mrs. Clements is from Yellow Gulch, just like me. Last Wednesday, her husband... Well, her husband was killed. And I never heard of a more cowardly killing, Sheriff. The polecat never gave Mr. Clements a chance. All we got, Sheriff, is a description and a tip from a masked man that the fellow we want lives in a Laramie. So that's what brought you here, a masked man. Some crook gave you the tip, huh? I suppose he's an outlaw. Don't imagine he'd be a masked if he weren't. But anyhow, it's the only lead we got. Tell me what the killer looks like, and I'll tell you if he's around. And if he is, we'll get the polecat, whether he's a killer or not. Right. Describe the fellow, Mrs. Clements. Well, I'd say he was just under six feet, but he was stuck here. Just a second, ma'am. Billy. Huh? Come here. Well, me, sir? Uh-huh, you. Westman, wasn't you? Gee, I was just standing there. I wasn't doing nothing. Nothing but spreading your ears to the wind to pick up everything that was said.  They get larynxed. Now that moves. I'm going. Better keep going, Billy, till you get home. And don't stop to spread anything you heard. That's a boy for you. All is underfoot. Nice looking, lads. Sure, Billy's all right. Feeling kind of sorry for the young one, with no maw or paw, and only Blackie to look after him. Not that he don't think the sun just about rises and sets on Blackie. Well, this ain't 10 in the business. We'd like to get it over with just as soon as possible. Sure. Ma'am, I want to hear the rest of what you've got to say. But I reckon this ain't the place for it. A little too public. Your office? It's just across the street and down the block. My arm, ma'am. This way, we'll have this settled and then jiffy. To a boy of Billy's age, the little he had overheard of the conversation between Sheriff Otis and Marshal Cleary was exciting news. He wrote as fast as he could toward his home, a small cabin about five miles south of Laramie. When he arrived there... Oh, Comanche!  Blackie! Gosh, you know what I heard just from Tom just now? Do you? I bet you'd never guess. Hold on, hold on there, young fella. But, but you know... Hold on. Whatever it is, you can tell it later. I've been waiting for you. Any notion why? Huh? It'd been nothing you've been wanting kind of special lately. That I've been. Oh, gee, Blackie! You don't mean to... Sure do, come on. Let you and me see if we can find something waiting for you out in bed, okay? Oh, golly! It is Blackie! You bought him! It's the house I seen at Mr. Whipple! Oh, boy, he's swell! Like him? Oh, gee, he's the swellest house I've ever seen. Did you really buy him for me, Blackie? Did you hide it? Uh-huh. He's yours for keeps. Just look at him! Hear that? He knows me already. Takes to you, don't he? Oh, Blackie, I, I... Oh, I don't know what to say. You've been so swell to me and... Me? Sharks kid, forget it. I ain't hardly done nothing. Least ways, nothing to all I'd like to. Was this what you went to your claim for, Blackie? Was it to buy the house for Mr. Whipple? Claim? Oh, yeah, yeah, that's what I had in mind.  Won't you someday, Blackie? We'll talk it over some other time. It didn't you start to tell me about something happening in town? Oh, gosh, yes. I was near forgetting. Blackie, the sheriff's gonna jail a killer. Yeah? Come on, we better get back inside and start rustling up some grub, huh? Uh-huh. Now, what's this about the sheriff? Well, he was talking to a U.S. Marshal and some old lady. I was right there and heard him. Well, that sounds interesting. U.S. Marshal, huh? Sure, from a yellow goat. From yellow? Kid, did you say yellow goat? Well, sure, Blackie, what... Who was that there was after? You hear him say, did you? What's the matter, Blackie? Quick, did you hear him say or didn't you? Well, no, they didn't mention no name. What else did they say? What was the after this, fellow bird? You hear that? Gee, Blackie, you're hurting my arm. All right. Sorry, kid. You said I didn't notice that a hole in you. And go on, tell me about it. I'm kind of interested. Well, the Marshal was telling the sheriff that back in yellow goats, there was this old lady's husband and some skunk held him up.  Well, you ain't listening, Blackie. I thought you said you was interested. Here, then, I just thought of something I meant to do and didn't. But... It's got to be tender to you. You go ahead and fix the grub. I'll be back later. But, Blackie, wait! Blackie!   Gee, what got into him? When Blackie left his young friend, his destination was a remote clearing in a wood some distance from the cabin. There, leaving his mount saddle, he took refuge in a crudely built lean-to. All day, he remained in hiding. Then, just as dusk was beginning to fall... A horse.  It's me, Blackie. It's Billy! Billy! Oh! What made you look for me here? Blackie, it was you I heard the Marshal talking about. Did they come to the cabin? Yeah. Gosh, kid, I don't know what to say. Let's not talk about it, Blackie. You've got to get going. They're on your trail. They are. I, I watched them. The Marshal can read sign real good. They'll be coming along most any time. You better leave. Yeah. That's so a rick and a bitter. Oh, look, kid. I guess I've smashed them, Zena. There wasn't no claim. Nope, there wasn't. It was all lies. You wasn't buying me things with gold, you'd payin'. It was with, with cash you'd stole. Oh, why'd you do it, Blackie? Why'd you have to? Everything was so swell. I was so happy and, and, and now I... Billy, just listen for a second. Then I'll clear out. Kid, first of all, I want you to know I don't mean to whitewash myself. I'm pretty bad. I always have been. I was a crook when I first found you. I always thought you was the swellest fellow there ever was.  What you see young when I, I kind of felt toward you like, well, I reckon like a man that feel toward his own son. No matter how bad he was, he wouldn't want his boy to know it. Nor he wouldn't want to give up his boy neither. Well, that's the way I've been. I guess I ought to be shot for it. Won't you go, Blackie? Please, I don't want him to catch you. Sure, I'll go. But before I go to you, you think maybe you could say you ain't mad at me? Well, I can't blame you, none only. Oh, gee, Blackie, I couldn't be mad at you in a million years. Oh, kid. But you gotta go. You just gotta. Here, boy. Blackie, you take my horse. He's the one you bought me in. He'll give you a better chance. I don't know. I'll go on, please. Well. And, and Blackie, I'm, I'm awful cold. If you just let me wear your hat and your coat. Cold? Why didn't you think it was? I'm, I'm shivering. Why, here, kid, take him. I just didn't think it was that cold. Thanks. Well, Billy, I reckon this is goodbye, all right? Goodbye, Blackie. And, kid. Yeah? You'll be hearing a lot of things about me. Bad things, most of the truth. But if they ever try to tell you I didn't mean to be square by you. Well, they'll be lying and don't you let them convince you different.  Marshall Cleary, sheriff orders and the deputy approach the woods. Suddenly. There he is. That's him. That's the totem hat he almost wears. And that's his horse. Trying to get away. Shoot him down. I got him. I knocked him from the saddle. Keep your guns handy in case he's tricking us. Get up, get up there.  Get up, boy.  Get up there. Get up there. We got your Blackie. Don't try nothing. Pull up.  Sheriff, it ain't him. We shot the wrong fella. The wrong fella. It's Billy. And he's hurt pretty bad. The curtain falls on the first act of our Lone Ranger story. Before the next exciting scenes, please permit us to pause for just a few moments. How to continue our story. When the three lawmen discovered it was Billy they had shot, the deputy mounted his horse, and the sheriff and the Marshall carefully lifted the boy into his arms. The deputy was ordered to get him to a doctor as quickly as possible. When he had disappeared in the direction of town. Come on, Silver! Hear that? I thought I heard something. Hurry, old boy, hurry! Hold it. Whoa, whoa, whoa, whoa, whoa, whoa. Who is it? Can you make him out, Marshall? There's more than one. Wait till they get clear of the trees. Well, I'll see. Can you get one fellow masked? Yeah, but that's Blackie with them. Right there! Here's your prisoner, Marshall. Pull up there. Please, get your hands up, stranger. You and the engine both. No, you don't, Sheriff. We caught your man after you've gotten away from him. Take him. But you've got nothing untoward on me and you know it. And you don't hold us without a fight. Look here, mister. No masked fellow can tell me what it is. Just a second, Sheriff. Stranger. Well? It was a masked man that gave me the tip to come here. I was just wondering if maybe you warned him. It doesn't matter whether I am or not. Turning Blackie over to you, that's all that concerns you. Well, well, Blackie, talking won't get you free. The Marshal's told me all about you. You'll hang sky high. It had to happen some day. Eight dollars. Ah, I ought to know anybody low enough to let a kid like Billy get shot on the count of him. What's it about Billy? Hey, no. Please, answer me. They said Billy had been shot. If you scunked out that kid out... Well, I did. It couldn't be helped. Why are you... Listen to reason, man. How is we to know it was him in your coat and hat and on that horse? The horse? My coat and hat? Why, he must have been scheming it right then. He was.  But that kid seemed to think for some fool reason he was all right. Least wise, he aimed for you to have a chance. He was willing to give his life to see that you did. Sheriff, how bad's Billy hurt? That's pretty bad. But he's on his way to the doctor now. Then take me to him. Let me see him. Sheriff, I'll do anything you ask. But just let me find out how the kid's gonna do. I don't know. I'm sorry. It's out of the question. You're going back to Yellow Gulch with me. But don't go to Marshal. No use hogging. We catch a stage going through the night. Now get moving. But I... On your way. You ride between us. Strangely, we're thanking you for your help. Forget it. Get up there. Get up there. Get up there. Where we going? To Laramie. From what the sheriff said, Billy's in danger of his life. I want to learn if he live. Get him up, scoundrel! Come on, silver! Come on! In Laramie, kindly old Dr. Ewing had taken young Billy into his own home and worked upon the youngster with all his skill. Billy's wounds were not serious. And yet when the sheriff called the next day to ask how the boy was doing... Hello there, Billy. I've got someone to see you. Howdy, young'un. Hello, Sheriff. Do you like talking for just a short while, son? Oh, I don't know. Here now, young'un. Where's all your pep? Golly, I never looked for the time when you wouldn't be full of spunk and ran to go. Sheriff. Huh? Blackie. What's gonna happen to Blackie? Why, uh, he... Billy, you shouldn't exert yourself. But I gotta know, Sheriff. What have they done with him? Why ain't he been allowed to come and see me? Ah, Billy. He's on his way to Yellow Ghosts with a monster. He wanted to see you the worst way, but it just couldn't be. They're gonna hang him. Oh, why don't you... They are gonna. They are. I know they are, but... But I tell you... Oh, they can't do that to Blackie. They can. The sheriff, you gotta keep... Why, son? Really, the sheriff's your friend. If he could do anything to help you, we would. But don't carry on so. Try to sleep, won't you, son? Uh, I can't sleep, Doc. I just can't. Don't you realize that if you don't, you won't get well? I don't wanna get well. I don't care what happens to me. Blackie, they're gonna hang him. I have to go out, Billy. But I won't be gone for long. And I want you to promise me something before I go. Promise you something? Whether you think you can sleep or not, really try to. It's the only thing that's going to help you. If you won't try it for yourself, then try it for my sake. What do you expect folks will think of me? If I lose a patient, as husky as you are. I'll try, Doc. There, that's better. Come on, Sheriff. Remember now, that's a promise. You see... It's the dog gone to sleep. He's not badly wounded, Sheriff. He is healthier than even you are. But he won't rest. He won't stop worrying about Blackie. And medicine can't save a life when the patient refuses to live. Twelve hours later, on the stage bound for Washoe, Mrs. Clemens, Marshall Cleary and Blackie were startled by the sound of firing. Yes, man! Pull up! Do you hear me? Pull up at once! Those are the men who helped us catch Blackie here. Hello there! What is it, stranger? The life of the boy was wounded. It depends on Blackie returning to Laramie. There's no time to waste. You've got to turn back at once. Turn back? Are you local? I said there's no time to waste. Then you go to places. I won't argue. One move and I shoot. Blackie, climb out here. You bet I will, mister. Get up behind me. You'll be tailed for this, stranger. You'll get yours, and that's a promise. Marshall, I don't want to oppose the law. But in a case like this, I have no choice. Ready, Blackie? Ready. Come on, come on, Stone! Come on! Blast him! Speak it and you do what I say! I think you hear him, Marshall! The Marshall's bullets had taken effect. Both the masked man and Blackie had been hit. And once beyond range, the lone ranger and tato reigned in their mouths. How, how bad you hurt? Don't mind me, Kimus, I'll be... Mine's a flesh wound. Cut me in the shoulder. Have a look at Blackie. Hey, I'm scared I'm doomed for, mister. Oh, and you've got heat that wounds. Then we can't go on. We will. I'll make it somehow. I've got to. Before I cash in my chips, I can do one decent thing to make up for all the bad I've done. That's to give Billy reason for living. It's a hard trip to Laramie. But just the same if it was twice as hard and twice as far I'd still last it out. Well, it's not talk. Let's travel. All right. Kill him! Get him, Stone! Get him! The next morning, Dr. Ewing hurried toward the front door of his home. Yes? Blackie! How was the boy? I'm afraid of you who come here. Doctor, look at Blackie. He's almost finished. Yet he came back here because he thought Billy needed him. Don't delay us because of my mask. Blackie! Good heavens, man. You rode here with a wound like that. Don't mind me, Doc. How's the kid? But you're in bad shape. Doc, don't waste time. I can't hold out much longer. So, miracle, you've lasted as long as you have. Blackie, the boy is delirious. I haven't been able to get him to sleep. But he certainly won't do him any good to see you like this. Blackie, you just leave that to me. Let's go in there, huh? Blackie, I don't know what to say. A minute ago, I hated you. But now... Well, come on. You too, stranger. Of course. Stay here, Tyler. The Marshal and the Sheriff come. Keep them out. Ah, how do I do it? Jeff, I feel good. Let me talk to him alone. Oh, Blackie. Just stay here. Yes. Come on, kiss me, Blackie. Kid. I can't hang you, Blackie. It's me, Kid. Oh, Blackie. It's Blackie. I don't care. Can't you hear me, Kid? I don't know what you did. It's me, Blackie. Come on, Blackie. I'll come back to you. Blackie. Shucks. You think there was anything in the world that could keep me from you when you was alien? Did you really think I wouldn't come back to you? Is it really you, Blackie? Is it? Sure is. But what they said you was going to hang. They said it... Someone said that about me. Well, of all the fool things. My kid. They won't hang me. I'll give you my word. Cross my heart. I've just been telling you things it ain't so. Honest, Blackie? Honest? Honest. And I'll tell you something else. Yeah? I've changed, young fella. I'm all through being a crook like I was. From now on, for as long as I live. Gee. Gee, Blackie. That's swell. I've been awful tired, but thinking of you and wondering what was going to happen. Ain't no need to wonder now. That's all over with. Now what you're going to do is get some rest. So you and me can be together again. Rest, kid? Sure, Blackie. Sure. You've done it. He's sleeping. It's the crisis. He'll live, Blackie. Don't. Don't wake him up. Stranger. Yes? Don't let him know too soon, will you? No, Blackie. And after I'm gone. If you could kind of look after him a bit. There's nobody here. He can turn to him. And you've been right kind. You have my word. I will. Thank you. Yes, Blackie? It's no use, Stranger. He can't answer you. He'll never answer anyone again. Sheriff. Your prisoner's inside. This blasted engine wouldn't let us by. By it's under this time, Blackie, won't get away. Not if I have to handcuff him. You're right. Where is he? You're right, Sheriff. He won't get away from you. But I'll tell you something else. Blackie will never hang. You have just heard. There's a copyrighted feature of the Lone Ranger Incorporated.


It's one click on who's the March 22nd 2022 so you must be watching Science at Soast, live streaming from beautiful downtown Honolulu. I'm your host Pete McGinnis-Mark and every week we have young scientists from the School of Ocean Earth Science and Technology telling us about some of the exciting research which they get to do here at University of Hawaii at Manawa and I'm really excited today because we have University of Hawaii going into space. We've got Joseph Ben-Goshon who is a staff member at the Hawaii Space Flight Lab, which is part of Soast as well as part of the College of Engineering. So, Joseph, welcome. It's great to have you on board and really excited to hear a little bit about what you have to tell us today as well as a bit of your background. So welcome. You are part of the Hawaii Space Flight Lab. So maybe we could start off just by telling us what is Space Flight Lab. Thanks for having me, Pete. Glad to be here. So the Hawaii Space Flight Lab was developed and established in 2007. As you mentioned, it's part of Soast and also a collaboration with the College of Engineering and is embedded within the Hawaii Institute of Geophysics and Planetology. And I graduated in 2017 with my Bachelor's of Mechanical Engineering and since then I've been working with the lab here as first a mechanical engineer and then now I've been doing project management, project engineer related stuff. So the lab, our main goal is to develop aerospace workforce development and satellite development here in Hawaii. And we'll talk about that a little bit later. And we really want to just kind of promote the collaboration of science and engineering between the different disciplines that we have here at UH as well as kind of build an aerospace workforce and industry here in Hawaii. So this is really STEM education on steroids, right?  We've got both engineers as well as earth scientists involved in this program. And it's really a well kept secret at the university that we're actually part of the space program. Now, I know you only joined Space Flight Lab back in 2017, but I think there was a little bit of history before that, which we can see in the first slide if we could go to the first slide. Now, here you are. I presume we're going to be talking a little bit more about what you're working on here in some of the related discussions. So I got the slide sequence wrong. If we could go to the next slide, slide number two, that would, there we go. This will give us a little bit more of the background. So Yusuf, as I said, you weren't with Space Flight Lab back in 2015, but do you know anything about what we're seeing here in the slides? Yeah, so that was the first satellite that was developed here at the lab with some of my co-workers who are still working here. And on the left, you can see Hyakuset. That's our 50 kilogram size satellite. It's a small satellite, which just means relative to larger ones. It's a specific size category. And they developed that as a precursor to one of our missions we're working on right now. And it was launched on that rocket you see on the right as part of the ORS-4 from PMRF on Kauai. And unfortunately, the rocket had a failure, so our satellite did not make it into orbit. But that was the first kind of real large project that HSFL worked on for the Air Force. Right. So I got lots of questions related to that slide, for example. Where you say it weighed 55 kilograms or I guess about 100 pounds in weight, what kind of size is that?  It's about the size of like a desk chair, maybe. And the black rectangles up on the sides. It looks hexagonal shaped, the satellite itself. What are the components we're seeing here? Those are the solar cells. So each rectangular shape, that's a solar panel, which just consists of those smaller pieces, which are solar cells. That's how it gets the energy from the Sun while in orbit and powers the satellite by charging the batteries. And unfortunately, it caused the rocket didn't work. But where was the spacecraft going to be launched? I was going to be launched to the low Earth orbit. And it had a hyperspectral thermal imager on top of that. So it was going to be taking images of the Earth in thermal imaging. And low Earth orbit would be a few hundred kilometers. So the altitude, the space station, something like that? Yeah. The space station is in low Earth orbit at about 400 kilometers. It can go a little lower. It can go higher. It really depends on the terminology you want to use. But yeah, in the hundreds, hundreds of kilometers. Hundreds of kilometers to 300 miles up. So that would have been really impressive. And it was launched from PMOF, which is the Pacific Missile Range Facility on West Kauai. I was actually there for the launch. It was really impressive what we could see. But obviously, it didn't work too well when it got to altitude sort of thing. Well, let's go back to the first slide. And that features you working on a different type of satellite, right? This looks smaller. And you're wearing almost a bunny suit. Explain to the viewers what it is you're seeing here. Yeah. So that's a specific size, standard size of small satellite called a CubeSat. And a CubeSat standard size is 10 centimeters on a side cube. And what we're looking at there is a 3U CubeSat. So it's three of those put on top of each other as the total volume. And that right there is about the size of a loaf of bread, I'd say. And that's neutron one. So that's our first satellite that successfully made it to orbit. Hawaii's first satellite as well in general. And that's currently still in orbit. So it's just over a year now since we launched. It's nearing the end of its life, which is about just over a year, just under a year and six months. So it's going to be coming back down and burning up in the atmosphere in the next couple months. But it's still up there and still beeping at us and letting us know it's still alive. Terrific. And I think we'll actually see later on in the show an image of neutron one being deployed in space. But this is quite an unusual capability that the university has. Let's take a look at some of the facilities which have been developed and I'm understanding much of this is student built or student run. So in this slide, for example, what are we seeing? So these are called ground stations. And those are used to communicate with the satellite. So we have to use radios to communicate up in space, right? Similar to like our GPS on our phones, those actually bounce up to the satellites in geostationary orbit usually, which is much higher up. And so these we have a few different ground stations that we help support and one as a university in general. So the first one there on the left is Honolulu Community College. That's usually, I think, mostly used by NOAA. The second one is that well being the the National Oceanographic and Apospheric Administration. Yes, yes. So that second one is our S band and UHF. That's what we're currently using to communicate with Neutron One. And that's also an amateur ham radio receiver. It has a few different bands that it uses that so the amateurs can communicate with it as well. And they have all over the mainland as well as here in Hawaii. And then on the right, we have the one here at UH Manoa, which we currently have UHF operational as part of another ground station network. Now, seeing that we've got these ground stations at several different places around Hawaii, is this a UH system wide endeavor? Or is it just that so? How big, how many people are involved or how many students at least? It's a lot of different groups. It's not just us. We maintain that one at Kauai for them for the most part. But there's other groups involved along with us. There's dozens of students every year that work on different projects of ours, including the ground stations, managing. We have students undergrad and graduate students that maintain contact with Neutron One, download the files and manage the contacts that we have and we've been getting. That's impressive. And do they get degrees or college credit for this kind of work? It's a mix of college credit through certain classes and experience real life experience is kind of the main. There's also cab stock projects that can be involved with those. I'm sure it must be great background training for getting good jobs when you graduate, sort of saying, because if you're working with the space program, that's high tech. That's an exciting career to think about. Yeah, that worked for me. Exactly. And the facilities aren't just scattered around the community colleges. I think that the next slide, slide four, we'll see some of the equipment which you guys have now. Where is this? It looks specific to the Jet Propulsion Lab, but I don't think it is. That's not as similar. But yeah, we have a clean room facility, which is what we're looking at there. That's in the basement of post where I'm at right now. And what we're looking at is the attitude, determination, and control system test bed. So that Hemholz cage, you can see those big circles, those create a magnetic field. And we use that to counteract the magnetic field from the earth and then put a simulated magnetic field of what the satellite would see while on orbit and what the sensors would read. So we do that along with that motion system. We use that to track the satellite as it's rotating on an air bearing there, which is kind of a, it's a friction, mostly frictionless bearing. So it uses air and you have like a hemisphere or a puck that sits on the air, on a cushion of air to give it a very low friction coefficient. And the motion tracking system just tracks that to see where we think it's more pointing. And the sun simulator, that's a fancy, you know, just a powerful light that we shine on it to simulate the sun that shines on the satellite while on orbit. And then the GPS simulator also puts in that GPS coordinates of where it would be. So the satellite essentially thinks with all its sensors that it's in orbit wherever we tell it it is. And then we use that to test our algorithms and systems and functionality of all the components. Okay, I got a whole string of questions related to that. But then let's start off. What do you mean by a clean room? My bedroom is pretty messy. So it's not one of those. But what is a clean room? That's a good question. So clean room is a standard. There's a bunch of different standards that you can use ours is a class 10,000, which I believe means that's 10,000 particulates per million or something like that, where it's a level of cleanliness. And it's the amount it includes how much particulates are in there. It also includes how often the air is exchanged. So it's really to avoid any dust and particulates. And the reason we wear that bunny suit is to avoid any of our, you know, skin cells or hair or anything getting in and around all the electronics. Okay. And how big is that room? You've got, you know, just the test instrument with the blank circles looks pretty big. It's at least one and a half times the size of a person. That's a fairly large room. That's, I don't know, the exact square footage. But you only see about maybe a third of the room in that picture. And this is on the Minoa campus, which is a remarkable facility to have. Now, all of the various attributes of that system, I think you're trying to say this is how you can put a satellite and test it in that environment before you actually put it into orbit. So for example, I think you were saying the air bed simulates the lack of gravity. A lack of friction. The lack of friction. So how would gravity actually influence the way the satellites would work? That's a bunch of different things. If you could figure out how to get rid of gravity for testing, we'd love to know. No one's really figured out how to counteract that yet. But we can do certain things. Since there isn't much gravity out in orbit, things only rotate, right? So they aren't being pulled down, you know, satellite while orbiting is just falling. So because it's in a free fall, it doesn't experience gravity, you know, as we do here on Earth. But in space, obviously, there's no atmosphere. So you're going to have vacuum conditions and probably the temperature changes. And I think your next slide, number five, shows that you've got other really impressive pieces of equipment there. And I'm looking at the silver cylinder there. What is that? Yeah, so that's the second half of that room, the clean room there in the basement. So yeah, like we're the other picture was showing that it was the frictionless environment. And that's really to get rid of the atmosphere to simulate that. This there simulates the it's a thermal vacuum chamber. So it simulates the vacuum as well as we can heat it up and we can cool it down to what the satellite would experience in orbit. It's usually down to like negative 20 Celsius, you know, a little lower, or it can go up to about 70 Celsius, depending on the orbit, of course, it can go much higher and much lower, especially if it goes out into the solar system and not just around the earth. Right.  It's relatively close. You know, there's specific terminology for that. It gets down to about negative 10 to the negative six to 10 to the negative eight tour, if you're familiar with that measurement for vacuum. And it's good enough for us for our testing. Depends what exactly the satellite is trying to do. If it's a spacecraft that's going out into the solar system, maybe you want to go lower and get an even higher vacuum, which can be achieved. But for us for low earth orbit, since there is a tiny, tiny bit of air molecules still, it is definitely good enough for us. Wow. And how long do you have to leave one of your satellites in there to gain enough information about whether it will survive in space? We try to put it through kind of what we call day in the life. So we leave it in for several days. And of course, takes us longer to warm it up and heat it up, warm it up and cool it down, depending on the size of the spacecraft. And that chamber is fairly large. It was meant for Hyakosat. So these small cube sats we're using are a little bit smaller and they can heat up and cool down pretty quickly compared to the size of that chamber. So we actually have a smaller chamber as well that we can use for smaller components. But that one is what we use for, since it is in the clean room there. How many universities have this kind of equipment? This looks, as I said before, almost like the Jet Propulsion Lab, right? Absolutely. I'm not sure. I wouldn't be able to say how many, but I'd say more don't have than do. Okay. Wow. Impressive. And I think the next slide will show the other part of the puzzle. Of course, you've got to launch the satellite and it's kind of a rough ride. So this blue monstrosity, what's the story with this? Yeah, so all the other components that were in the clean room that we talked about, that's more to kind of simulate the environment in space while it's in orbit. So this is a vibration and shock table. This is to have it test the experience on its way to orbit on the rocket. So you can have very different vibration profiles depending on the rocket launch that you're using. And who's launching it, whether it's SpaceX or United Launch Alliance or any of the others. And it totally depends on what that rocket does and what the satellite experiences. But this can do a wide range of different vibration. We mount the satellite to that table that you can see there and then it just shakes it at different frequencies to simulate what it might experience and make sure nothing breaks. So you can actually do various vibrations at simulated different parts of the launch sequence. So you don't just put it on the table and shake it up. It will go through a simulated launch profile with first stage separation and second stage or whatever. Yeah, you can totally do that depending on the requirements for that specific spacecraft. We have what we call random vibration that it kind of goes through a bunch of frequencies and does different accelerations. You can do sine waves, which kind of sweeps through a bunch of different levels. And there's also shock, which is a little higher, higher Gs. Unbelievably impressive. And there wasn't a scale or person in that image, but presumably that's what you use to test high accuracy or people used. So it's half a meter across or something like that in size. Yeah, I believe that one is about, I want to say two. So I can almost, it's like four foot, let's say, give you the exact numbers. And okay. Now, how long does this whole process take?  Are we thinking about weeks or decades here? How long tell a viewer how long it takes to build a satellite like this? Yeah, so it's been getting shorter and shorter relatively to what it's been in the past, especially with this small set, kind of like, quote unquote, revolution, where things have been getting smaller and faster. You know, we just had James Webb get launched and that was, you know, a 10, 10 year endeavor just for the build and 30 years in the making. Pretty intense and incredible and expensive. $11 billion as well. So it's a little different. We're on a totally different scale.  We also take, you know, anywhere from maybe a couple years for a small CubeSat, maybe even less, maybe a year for a 1U CubeSat, and then up to several years, depending on the size. I'm not sure exactly on the HiakaSat, but I'd say maybe around, you know, get like maybe five years, give or take a year. And that must be really great for a student because she or he can actually conceive of the satellite and then build it and have it launched during their career, right? So this is a relatively short time scale. Yeah, and we have different programs right now where we have vertically integrated projects here at, as part of the College of Engineering. And one of our professors that works with us, Miguel Nunesh, he has students working throughout, you know, anywhere from, you know, freshman or sophomore to working on a satellite. And they're going to be looking to launch their satellite in the coming year. And they'll have started the design, gone through the entire process, build and launch, all while still doing their undergrad. Perfect. And speaking of launch, the next slide, you worked on Neutron 1. I have to mention you didn't fly on the rocket, but it was sent up on some astronaut carrying mission. We're seeing the space station on the left. This is just over 18 months ago. So this is Neutron 1, the spacecraft which we saw you building earlier in the show. Yes, that is Neutron 1. And you can see the space station there on the left. And that's Neutron 1 being deployed from the space station. And that picture there on the right was taken by the astronaut, Kathleen Rubins.  And it's still working ever since November of 2020, huh? Yep, still working. So I went up on an ISS resupply mission. Astronauts took it out from the pod, put it into that deployer there you can see, and then launched it out with that arm. Okay, well, we're running a bit short of time, but I've got to ask, what's next? And I think the final slide will show us a little bit about what is next after launch pad literally, right? Yep, so that's our current six U-Cubes sat. So that's two of those three U's kind of put together. And it's the size of maybe a shoe box would be the closest. And that has a hyperspectral thermal imager, hence the name Hytide. That's our current mission. It's NASA funded. And our principal investigator is the director of HIGP here, Rob Wright. And we're working on that right now. We're actually in process of integrating it in the clean room downstairs as we speak. Fantastic. Now, you mentioned it was a hyperspectral thermal imager. What does that mean? So we would have to get into the science, which is not my specialty. What it really means is that it's seen infrared light, and it's using that to look at the earth, and it can measure crops, crop health, it can measure what different gases we see, to check on the volcanoes or wildfires, anything like that really. So lots of science projects as well as an engineering endeavor.  So do you talk to geologists or ecologists about how they would be using the measurements? Yeah, absolutely. I think that's one of the kind of benefits that we have here at the university. Since we have all these wonderful and amazing scientists at our fingertips, and we kind of talk to them, ask them what do they want in a science mission. And then we as engineers kind of decide what's the best way to do that and put it together so we can actually do something useful. Fantastic. And is this going to be your career? You may not stay at UH the entire career, but are you keen to stay in the space program? Yeah, I believe that's kind of feel like it's my calling. And I'm originally grew up on Maui, so I'm looking to kind of stay in the islands and work with the university to continue our efforts with our workforce development and our kind of bringing our space industry here so we can have more STEM careers and opportunities. That's terrific. And I would imagine helping more junior students learn some of the engineering skills that you have would be really important as well. Absolutely. So definitely useful. And we have dozens of students every year that work with us on different projects, internships, and kind of getting real-world experience. Well, you said if I'm afraid with one at a time, but given that you're launching another spacecraft in a few months, I might invite you back again sometime in the future. But let me just remind the audience you have been watching Science at Soast. I'm your host Pete McGinnis-Mark, and our guest today has been Yusuf Ben-Goshan, who is a staff scientist at the Hawaii Space Flight Lab within Soast and the College of Engineering. So that's all for today. Thank you for watching. And please join us again next week where we'll have another exciting student project to talk to about. All right. Goodbye for now. Thank you so much for watching Think Tech Hawaii. If you like what we do, please like us and click the subscribe button on YouTube and the follow button on Vimeo. You can also follow us on Facebook, Instagram, Twitter, and LinkedIn, and donate to us at thinktechhawaii.com. Mahalo.


Big money Monday welcome back 14,700 dollars in one week selling file expense. No surely not Yeah, I said about 26 appointments ended up running about 20 Sold 12 apps. You know that big money Monday. We talked about that one sale. There was five hundred and five dollars per month over six grand of total AP from one sale That was a part of that week, but about 12 sales total had an incredible week. The difference was between $1,000 a week and a $15,000 a week I said almost 30 appointments bigger numbers Result in bigger commissions. Believe it or not when you think bigger when you do more when you see more people You end up making more money. I know it's rocket science. It's amazing. It's crazy. Try it


Please association Coaches participants, cheerleaders, officials, and spectators are expected to assist in the promotion and achievement of sportsmanship When keeping this contest in a high level affair on this competition, all athletes are therefore not against, positive, not negative, or disrespecting Good luck to both teams Before we get started here, we will have the National Academy


Before we cover recurrent neural nets, which we will do in full detail, I need to do an important and slightly lengthy digression on embeddings. Because many of the recurrent neural network models and basically all of the natural language processing models use some sort of embeddings. Ways of mapping words or other objects to vectors, such that similar words are close. Now there are lots of things, objects, things we might want to map to a vector so we can take it as an input to a standard neural net or to a recurrent neural net. The idea of embedding is that objects that are similar should be close in the vector space. They should have a small distance between them for some notion of similar. That will allow things to generalize. Now, lots of things can be embedded. You might think that images don't need to be embedded because images are already a vector, but that's wrong. Two images may be quite different in pixel space, their original input to a neural net. But if you take the output of their penultimate layer, the second to last layer, the last set of activations of the neurons before it goes to the softmax, then what you could find is that the embedding, the outputs of the activations of the neural net are, in fact, more similar for two images that are both elephants or both plants, than if you were to look just at the raw pixel space. So often it's useful to embed even images that are already vectors. But certainly if you have words or sentences or documents, sets of words, really everybody now embeds those discrete objects like words into vector spaces, such that two words that are similar, we'll define this more precisely later, should be close in the vector space. You can also embed products, iPhones, androids, people. Any discrete object can be embedded in a vector space, such that objects that are similar are close in vector space, and this provides a really good way to do inputs to a neural net, as opposed to actually trying to capture the object itself. It also means you can often take objects you've never seen before, find their embedding, and use those. We'll see examples of that with words. Cool. So this week we're going to focus on the simplest of word embeddings, word-de-vec, or it's very similar to another method is called glove, which map each word, typically we'll use say a million different words. Does English have a million words? It does actually, if you include numbers and names and places and all these things, map each word to a vector, a 300-dimensional vector and embedding, such that words that tend to show up in the same contexts, which are words that we'll define as being distributionally similar, will be close to each other in the embedding space. Now you might ask, is there enough room to take a million words and put them into 300 dimensions? And I'm saying, yeah, 300 dimensions is really big. How big is the 300-dimensional vector? Well, if you put ones and minus ones in them, or ones and zeros, you could represent two to the 300th possible combinations, two to the 300th way bigger than a million. So there's plenty of room to put all the words you could ever see inside a 300-dimensional space. We will cover a bunch of different, more sophisticated word embeddings next week, but I want now to lead up to word-de-vec, which is what we'll use in the exercises in this week's notebook. Now the key concept behind embeddings is distributional similarity. And the idea is that words that occur in similar contexts are similar. So if you look at the sentence, he ate the sandwich and he ate the shridlu. I'm assuming you haven't seen a shridlu recently. What do we think a shridlu is? Is it a person? Is it a book? Is it a computer? It's probably a food, right? Something you can eat. How did you know that? Because of the context. Similarly, if I had shridla ate the sandwich, what is shridla? Probably an animate being. Maybe a person, close to he.  But it's certainly something that looks like that. Without understanding the word ate or sandwich, we can just take statistics on what context does shridlu show up after? After ate the, it's something that's edible. It will look similar to other things like meatloaf or noodles or dosas that show up after ate the. So note that words which are distributionally similar tend to be similar, not just in terms of part of speech like being a noun, but also similar in terms of being something that can be eaten or something that can eat. And we will use this over and over this week and next week. Now back in the battle days before deep learning, people treated each word as a unique symbol. And if one were to do that for a neural net, a word like hotel is going to be a million dimensional vector. I didn't show a million dimensions. Which is it the word A? Is it the word apple? Is it the word aardvark? And so forth. And eventually it's the word hotel and it's not the other 999,999 words. It's one, one and a bunch of zeros. Similarly another word like motel is a different one and an equally large number of zeros. And if I ask how similar are hotel and motel, all pairs of words in one hot encoding are equally similar or dissimilar. But that means there's no generalization. It means there's no way to easily learn from them. So we want first of all something that's lower dimensional, not a million dimensional but say 300 dimensional or next week we'll do things that are 768 dimensional but low dimensional compared to the words. And we want things such that we can actually compute similarity so we can learn that if we learn to recognize hotel that motel would be similar if it is. Now the classic method for doing vector embeddings back from my childhood is latent semantic analysis, LSA. And this takes each word and looks at the documents it shows up in. So we take a whole bunch of documents. They could be web pages or text messages. And we're going to take a big matrix of word by document. Every word and for every document is it in it or not. We will run singular value decomposition on this or take this document matrix times this transpose and run PCA if you're more familiar with PCA. And we will then get a reduced dimension principle component or singular value decomposition that maps each of my million words to say a 300 dimensional vector, k dimensional vector such that words that tend to show up in the same documents will be close in that embedding. This will make all related words close. Words like doctor, hospital, nurse, cancer will tend to show up in the same documents. They will then be related. They will then show up together and have similar embeddings. So let's look at that. Here's a trivial LSA example for documents. First document is I ate ham and cheese. Second document you ate cheese and crackers and so forth. You see a matrix here. Each row is a word for every single word in my vocabulary. Each column is a document for every document in my collection. My corpus use the fancy piece. You can see the word eight shows up in documents number one and two but not in documents three and four. If I now take this matrix and do singular value decomposition what I will find is that the first principle component will tend to load that eight and cheese will be close to each other. They'll have similar reduced dimensions and hospital will be close to doctor and nurse in other words operation that will show up and we'll see that related words have similar embeddings. Now most people don't use LSA anymore. More modern is to use something like word to beck or glove which instead of using a word in a document takes the context of a word as being the words right before it the left or right after it to the right and the early versions used say five words before the document the token and five words after modern ones use hundreds of words on both sides large contexts but that's next week. So now we're going to do is the same idea as LSA. We will take a word by context matrix which I'll show you on the next slide run singular value decomposition and we will then map each word to a low dimensional say 300 dimensional embedding such that similar words are close with LSA it was related words doctor nurse hospital cancer with word to beck it will be similar words doctor nurse so what do I mean by similar if you look at I8 ham and you ate cheese ham and cheese are distributionally similar they show up in the same immediate context I and you are very distributionally similar they show up in the same sorts of context before the same sorts of words and what is Eigen words or if you think of the way I'm doing it formally here or word to beck which we'll cover in detail next week do more generally it takes every word like eight and says what word did it show up which word showed up before it I showed up before it once you showed up before it twice what word showed up after it cheese showed up once after it ham showed up after it once these other words didn't it takes each of these words and each word is now a big vector of the context it's been seen in and now if we do a singular value decomposition then words like cheese and ham that look similar in this big space will be embedded to similar vectors we can then take all million words in our English vocabulary take billions of words of English text Wikipedia, the web if you're Google and now we can just take for every word what's all the tokens it sounds scenes around all the context it's in run a big SPD which could be run very quickly and we can then get a vector that represents every word if we then take any small set of words and you'll do this in the notebook in just a second and take say here's a set of words that I picked and you can take these words and plot them taking their first two principal components any sort of plot that makes close words close remember each of these words like river is a single word remember each of these words like river is a 300 dimensional vector but what do you find you'll find that words like boat and car and truck are close in this 300 dimensional space words like dog and cat which you can't see underneath the truck there are close in this space home and house are not too far away nouns are close things like listen and carry and eat verbs are in a different space because verbs are distributionally different from nouns and words like