Fy oeddwch. Felly bynnwch y wpersiau tynnu yma yn ymryd yn ymrydd yma yn y vlog yn y ffordd yn y ffordd. Mae'r ystraddiol pan gallwch ar y cyfnod yw 10 min, a'r rhywbeth yn ymryd yn ymryd blaen, ac yn ymryd sy'n 100 ydi. So, ynghylch yn y fhrifwyr, y panfodol y ti'n niech ei wneud yn ymryd. Ond erna ychwaneg y gallwn i fod wedi tro, a'i ddatblygu bod rhywbeth pwysig iawn. Felly, ydych chi'n gweithio'r ystafell, rwy'n gweithio, a dyna'r amser amser, rwy'n gweithio'n gweithio. Rydych chi'n gweithio. Chris, dyna'r momentau chi. Mae'r gweithio'r awddiant. Rydych chi'n gweithio'r gweithio, rwy'n gweithio'r gweithio. Rwy'n gweithio'r ystafell, CEO, y policymaker, y pwylltau'r ysgol yw'r blwch. Wel, ychydig yn ychydig ar gyfer y gweithio'r gweithio ym wrthidau? I asko, well, what keeps you up at night? Well, we don't know what teams are really doing, setting and changing policy is slow, hard to communicate, and people just go off and do their own thing. They think they know better, and often they do, but then I'm left playing catch-up with the risk that they've signed me up to. Second person, product manager, the whipcracker, well, what's important to you? So, managing risk, most of the opportunity risk, the fear of missing out, getting features out the door and avoiding getting bogged down with bureaucracy that seems designed, seemingly, to slow me down. Next person, dressed in overalls, they could be in a trendy part of town, they could be the CTO. Before I ask, cleaner, they say, ah, okay. Well, how did you get in my imagination? I'll come back to you. My attention to the last person, hoodie, headphones around the neck, ah, my stereotypical developer, yes, I know you well. Well, what's important to you? Well, writing consistent quality code, avoiding technical debt, and the rest of my team being able to cohesively work as one. So, we use lintas, test coverage tools, the usual to help with that. Great, I say, I write code too, let's be friends, and I hand them a printed QR code, my public GPG key, so they know that they can trust me. Back to the cleaner. How do you get told what to do and when it changes? Well, something gets stuck to a notice board, or so like last week we were told, all the meeting room whiteboards were needed to be wiped down every night. It's then up to us to then keep everything in sequence. Like when we haven't updated the meeting room, ah, the, the, the, the, I haven't updated that the meeting room on the third floor was being used as a dedicated wall room, and we wiped all their boards down. So I looked to the dev, does this sound familiar? I nod. Well, turns out we're not all special snowflakes. So, if any of this sounds familiar, and you can at least relate maybe one of my imaginary friends, then maybe I might have something resembling answers for you. So, what if I said that you could update policy easily, even releasing several version updates, not just in a single day, or all in a single day, seamlessly communicated without derailing anyone? Visibility on compliance, perhaps using tools that you might already be using. And multiple concurrent versions of the policy are all supported. Okay, I'm Chris Nessot-Smith, so I'm an instructor for Learn KX, and also for control plane, a consultant to various bits of UK government and a tinkerer of bits of open source. I've spent a fair chunk of my professional career now working in Gov and large organisations where problems like these arrive. This is a lightning speech, so we probably won't have any time for questions, but I'm pretty obvious I'll be the only one with pink jeans, and you can find me afterwards. So, by show of hands, who's with my CIO and has set, written, or applied any sort of policy before? Like anything, like a coding standard, or anything for your team, or anything like that? Cool. Okay, next round. Who sorts exemption or consciously bent, broken, circumvented, ignored, bypassed, whatever a policy with at least some good intentions? Cool, thank you. You fell for it. So, thanks to the organisers, we've got all of your names and employers' details down, so put your phones down, the stakes just got raised. So, policy usually comes in one of two forms. So, security, like data at rest being encrypted, for example, or consistency, such as code style. Both generally are intended, at least in principle, to mitigate a risk of some sort. However, with the best of intentions, these are often emotionally led, rather than grounded in proportionate controls, which often becomes the open door to case-by-case exemptions being required when you come up against a situation that you weren't expecting.  But the devil's in the detail. Throwing some curly braces or yamel at something doesn't inherently fix things, especially if it leads your engineers who are all hopeful plenty smart people are found finding inventive, should we say, ways around the computer, says no response that they've got. Sure, you might say that you provide some warnings on less important issues or new emerging policy, but that's only useful if anyone's actually seeing them. Okay, so yes, as you know, all presentations this year are contractually required to reference a lock 4J, even when it's entirely out of context. So these are my slides, get over with it. So in just a few short months, I'll be able to remove them and just broadly point to a list of CVEs in order to command your behaviour through fear. So I've just covered a lot of ground and hopefully sounded at least vaguely convincing, and it's not just this fictional utopia painted in PowerPoint. And I know you really all came here wanting to see a million words on a slide and not just the odd emoji or two. So I'm going to be talking about two things to prove this is not just one tech or one tool. I've picked Terraform and Kubernetes, but I could have probably picked anything, really.  But again, these could be any or some or even all probably. So Chekhov's going to be doing my Terraform and Coverno's going to be doing my Kubernetes. I've created an example GitHub organisation here. I'm not expecting you to read or groch the code on screen. It's just to prove that it's a real thing. And the link will be at the end as well. So the policy is stored here in this repo of the org. So here's where my policy starts at v100. I've got policy that requires a department label on all resources, so long as it's set, doesn't matter what it is. I've written some tests for this, so passing test cases become great examples of what good and bad looks like. I've pushed a tag in Git. We've added some release notes. Obviously, they've signed it. And version two looks similar. Only now that field has to be from a predetermined list. Tests, release notes, tags, signed, et cetera. 210 is where we correct a spelling mistake in that list of departments. And 211, we've added a new department to the list. It's a few more repos in that organisation. There's app one and infra one. So these both depend on version one of the policy and not compliant with version two. But how might I know that? Well, I can use renovate, in this case, to automatically make some pull requests. Could use dependable or something else, if you like. So with new versions of the policy, so I can update my dependency just for my pull request. And I get feedback about when I'm not compliant. And I can also see the pull request over the organisation as a whole, so I can measure the compliance of my policy. A couple more repos, so app two and infra two. Well, these depend on version two of the policy. However, we could merge the open pull request all the way up to 211. And lastly, app three and infra three, well, these are all dependent on 211, and they get a gold star from the CIO. I've written some bash, sorry. So now, from my laptop or in CI, I can evaluate my code against the right version of the policy that I've declared in my resources. And the last piece of the puzzle then is then managing the life cycle of the policies and allowing multiple versions of the policy to be accepted and evaluated within a single runtime. I've cheated a bit here. So, Cube gives you admission controllers. I haven't found a sensible way to do the same evaluation in Azure, GCP or AWS. All of their policy is doing its own weird stuff, and you can have to do it for real. So, the way the policy is designed and distributed lends itself well to coexist with previous and future versions of itself within a Kubernetes cluster. So, I've got a few more repos, so cluster one. Well, this describes a cluster that accepts all the versions we've described so far. Likewise, cluster two. Well, this only accepts 2.00 and greater. And to demo this, I've just used kind in order to deploy the apps and prove what does deploy and what doesn't. And then we have it. A full org, all done, all compliant policy, all versions, CIO, all aware of what's going on. So, this is great. When some new privacy regulation comes out or you acquire more data, the risks and the appetite will stand still for no one. And now, neither does your policy. But the most important thing that I want you to remember from our time together and please do feel free to humor me and say it out loud with me, that purposeless policy is potentially practically pointless policy, which I've been practising saying far too many times. I've been Chris Nesbit-Smith. Thanks so much for your time. You're now free to leave. I will delete the photos, maybe. If you're guilt-admissions earlier, like, subscribe, whatever the kids do on LinkedIn, GitHub. There'll be little or no content. Sometimes I'm awful at self-promotion. And talks.cns.me. The point contains this and other talks and they're all open source. And cns.me just links to my LinkedIn. Thank you very much.


Hello everyone, my name is Ellen Koike and welcome to this presentation about ImageSignal Processor, ISP, Drivers and how to merge one upstream. A little bit about me, so I work with Collabora since 2016 and which we are an open source consultancy company and I'm mostly working on the kernel mainly in the media subsystem. I also have experience in other subsystems such as any VME, also device mapping and here and there in general but mostly inside the media subsystem. Right now I am the maintainer of the RK ISP driver that we're going to talk a little bit during this presentation and I'm also the maintainer of the virtual media controller driver. I am really proud to say that I was an outreach intern in 2015 where I started the VINC project there, Lohan Pansharra was my mentor and who proposed this project and right now I am the co-coordinator of the Linux kernel project in Outreach together with Vaishali who is the current coordinator so if you have any questions regarding this this program or if you want to become a mentor so feel free to reach out. The main goal of this presentation is to give you an overview of the camera ISP memory pipeline then we are going to give another overview about the media framework, some design choices when implementing our driver, also some lessons learned when I was upstreaming the RK ISP driver and I also want to talk about this specific user space tool called Liby Camera. So I hope that this presentation is going to be useful for those who are not entirely familiar with the Vita from Linux community but want to get started and upstream their own ISP driver and the subsystem. Okay so let's start with the camera ISP memory pipeline. On our phone usually we have the camera and this camera is composed by a sensor. This sensor is composed by smaller light sensors that's represented by this gray grid on this image which comes from Wikipedia. These gray sensors they are sensible to light and on top of it there's a filter a color filter that only allows a specific color to pass through which means that the set event is basically a color sensor and the readings of all those color sensors is sent to your computer to your SOC processed in some way and the image is made available to your application. So what is an ISP? An ISP stands for image signal processor and the common use case is that the ISP receives the readings of all those small color sensors and transforms it in an image that is usable for user space. It can also perform several other image transformations such as format conversion, the wiring for instance, so the format with all those small color readings is that we call a buyer format and transforming it in actual pixels is the process that we call the wiring. We can also convert to RGB, YUVE and several others. ISP can also perform crop, resize, white balance, compose, image stabilization that we're gonna talk a little bit about it, effects, filters, flip, rotate, several others. So those are processing that's performed by the hardware and it helps of loading your application so that you don't need to perform software. An ISP can also generate some statistics such as histogram, error contrast and several others that they are used by application to implement algorithms that can read those statistics and reparameterize the ISP to improve the image on the fly. So some examples of those algorithms are histograms, histograms equalization, what we call 3A for autofocus, auto exposure and auto white balance. When an ISP is not, so an ISP is not a codec, ISPs work with raw and uncompressed image while codecs that we can divide into encoders, they transform raw image to a compressed image format such as A264, JPEG or VP9 and there are several others and decoders do the opposite, they transform compressed image into raw image, back to raw image. Now let's talk a little bit about different kinds of ISPs, so inline versus offline. Offline ISPs can be divided in two phases, one is retrieved information from the sensor and place it somewhere in memory, the other one is getting this memory that's usually in a buyer format and do some kind of processing and place it back in memory. Usually those two phases are implemented into separated drivers and communication between those two drivers is coordinated by user space. An example is the IPU-3, the Intel IPU-3 that is divided in IPU-3 CIO-2, the camera interface drivers that gets the image from the sensor and place it in memory and the other one is the IPU-3 image units driver that process this image and send it to user space. An inline ISP is when the data reaches memory only in the end, so the sensor is directly connected to the ISP without touching the memory, then ISP does some kind of processing and then it reaches memory for user space. One example is the RK ISP-1 driver, this is not entirely true because this driver can do this hardware can do both actually, but in the driver only inline is implemented, so we can have hybrid devices that can get the image directly from the sensor or from the memory and it can behave as inline or perform the second phase of an offline ISP. One example is the MediaTek 8183-PES-1 driver which is not yet upstream but it is published in the mailing list, so the image can come from those two paths, so one is represented by the DMA input there is the memory and the other one is a connection directly from the sensor, so sensor camera interface. This data goes to the Byring processing and then it reaches memory for user space. Mipidify, let's take a look at this, so Mipidify is a very common bus used in the market for camera and displays, it is specified by the MIPI Alliance and it is a physical layer with high data rate that can process 4K image, so ultra-high definition image, ultra-high resolution with a really good frame rate. In this image there we have Raspberry Pi that is connected to a sensor through this flat cable bus, so this bus is the Mipidify. This bus can have up to four data lanes and it also has an I2C bus for configuration, so in this image on top we have the camera connected with the host and we have several data lanes, some clocks and then the I2C bus. On top of this bus we can have up to two protocols, so one is the DSI2 for display serial interface when the computer wants to output image to somewhere, usually a display, and the other one is the CSI2, the camera interface to capture image from somewhere to the computer, usually from a camera sensor. And this Mipidify CSI2 is a really frequent term in the ISP land, so I mentioned this because you're gonna see these acronyms in some other diagrams during this presentation. Now I would like to talk about the RK ISP1 driver in specific because I have most experience with this and I will also use this driver as an example during this presentation. So the RK ISP1 is the driver of the ISP block that's present in the Rockchip RK3399 SOCs. These SOCs can be found in several devices such as the Scarlett Chromebooks which is the tablet in the image, Rock Pieboards or Pinebook Pro laptops. The driver was originally written by Rockchip, it was merged in kernel 5.6 under drivers staging with more than 9,000 lines of code. So this is the hardware architecture of the ISP, the Rockchip. The image can come from those two buses, one is the Mipidify that we mentioned and the other one is Parallel Bus, but the parallel is not implemented in the driver right now. Then image goes to the ISP which performs some kind of processing, then it goes to some image enhancement blocks and the image can go to these one of those two paths or both of them at the same time. Those two paths can perform cropping, resizing, do some RGB flip rotation also and then it reaches the memory. The main difference between those two paths is the use case. When you have your phone and you want to take a picture, you can see there's a preview and this preview is really fast. If you rotate it can rotate quickly and it is display ready. You can see that there's almost no delay, it's live preview so it's really fast and if you noticed you can see that the resolution that it shows is usually not the full resolution that your camera supports. So when you take the picture it will take a little bit more time and the final image is a really higher resolution. So basically this preview would come in this case from the self path picture and the picture itself would come from the main picture path. So the main picture path supports a higher resolution, it doesn't need to generate RGB display ready images, doesn't need to flip rotate while the self path needs to be fast for display and doesn't need to be a higher resolution. So one is fast, another one is lower. The ISP comprises with image signal processing, many image enhancement blocks, crop, resizer, RGB display ready image, image rotation and we have those two paths. So the self path for preview and the main path for picture as we mentioned of course those are use cases, user space is free to use whatever it wants. Now let's take a look at the kernel side, the media framework. In the Linux kernel in the media framework there is this concept that we call topology. So user space can query a node inside the VFS let's say slash dev slash media some number x to retrieve how the inner blocks of the hardware are interconnected and the order of the image processing. For instance in this image user space can know how those blocks and how they are connected it can know that we have a sensor that is directly connected to the ISP that is connected to some DMA engine where the image reaches memory. In topology we have two types of nodes so what we call sub devices in green that represents the inner parts of the hardware and user space can perform some configurations there and the video devices in yellow that represents the DMA engine where user space can perform some configurations but it can queue and the queue buffers contain the image or metadata to and from the hardware. In this example it is from the hardware because we are retrieving an image from the sensor but we are going to see some other examples that the user space needs to inject an image into the driver or some kind of metadata. Those blocks are connected by what we call links and links always connect pads so those zeros and ones in the image are what we call pads. Some interesting thing to note is that usually the sensor block is a driver that is separated from the rest this is because we can reuse the same sensor with different kind of hardware so in my development setup I have the same sensor that can be used with the Raspberry Pi that we saw in the previous image but I use with the RockPi board so the sensor is the same but the ISP driver is different. This is the topology of the IPU3 CIO2 driver the camera interface that performs the first phase of the offline ISP. We can see here four blocks of the IPU3-CSI2 sub-device it means that we can we have four buses and we can connect we can retrieve image from sensor for four sensors at the same time in this case we only have a single sensor so the IMX355 that is that is connected to the first block and the other ones are not connected and the yellow blocks are where the user space is going to interact so in this case slash dev slash video zero to retrieve images from the kernel. The camera interface will place the image in memory in some format that is very specific to this Intel driver so it is up to user space to get this image and feed to this other driver so the IPU image unit that performs the second phase of the offline ISP. Then user space is going to retrieve the image from the previous driver and feed it to this first node there the IPU3-IMGU0 input to inject the image inside the buffer inside this driver then it will perform some kind of processing and it will make the process the results available through one of those two paths so we have the output there and the viewfinder. If I recall correct those are the equivalent of the main path and the self path so one is meant for the picture itself with higher resolution and the other one is for preview to be fast. This driver can also generate some statistics through the node down there so 3a stat and user space can read those statistics and put some parameters on that node called parameters so in this case we have two instance of the ISP it means that it can process two images at the same time. Here is the topology of the RK-ISP1 driver which is in line in a specific this is the topology of the Scarlett Chromebook which is a tablet and we have a back camera and a front camera. Those camera sensors are represented by the top blocks starting with OV and just one of them can be connected to the ISP at a time that's why you see one line is dashed. ISP performs some kind of processing and it outputs the image through one of those two paths so main path and self path as we mentioned before and this driver can also do something similar to the Intel driver generating statistics and receiving parameters from user space. Now I want to talk about some driver configuration architecture that you should think a little bit about how you design your driver in a specific case auto versus manual configuration propagation. In the auto configuration scheme user space will do all the configurations and operations on the DMA engine node so in the yellow blocks there. When user space want to set a resolution it will set the resolution through the node slash dash slash video zero and it's up to the driver to propagate this configuration on all the blocks of the topology including the sensor that can be a separate driver. So as you can see the other blocks doesn't have any nodes exposed on the VFS. In the other hand in the manual configuration propagation scheme user space is the one responsible for configuring all the pads through all the the pipeline or the image pipeline. So for instance if user space wants to configure a resolution it needs to configure a resolution that the sensor up there needs to generate on its pad zero. Then it needs to configure the same resolution to the ISP block in the pad zero to inform the ISP which resolution should expect from the sensor. Then user space needs to configure the resolution that the ISP needs to generate and it needs to match the resolution that the resizer expects and needs to configure the resolution that the resizer should generate so can could be bigger or smaller to zoom or shrink the image and finally setting the resolution that is expected inside the memory buffer for the final image. As you can see this increases along the complexity for user space since it needs to perform all those configurations and if formats don't match when you try to start the stream it will fail. On the other hand it provides you a finer grain configuration in the inner blocks of the hardware and I'm going to show you an example when we need this but also the more blocks that we have exposed to user space more complex it becomes since we have some we have more points that we need to perform the configuration. Also manual configuration is extendable since if we come back to the previous slide we cannot mix also and manual configuration so if you choose one scheme you need to stick with it we cannot have a manual configuration on the sub-device on the sensor block there and auto configuration on the ISP doesn't make much sense so if we want to add more blocks in the future and you chose auto configuration then you realize that this new block could benefit from the manual configuration then you won't be able to change it it's it's or maybe you could add some parameters when loading the driver but then it won't be compatible with other applications from user space anymore. Why our KSP is manual? To answer this I want to talk a little bit about crop so how do you crop the image you just select a sub rectangle in the main image of the sub image that you are interested in the media the media subsystem allows you to expose an API that allows user space to select a sub rectangle on all the pads actually but we are going to see that we could allow user space to select this sub rectangle under the node the final node there slash dev slash video zero but we're going to see that we have some problems because this driver allows cropping the image from the sensor so before doing the processing in the ISP and it also allows cropping the image just before the resizer so it can shrink or zoom a specific image and exposing this crop only once in the final video node would be confusing for user space since it wouldn't know which crop being the driver is using so instead of exposing the crop only on the final node and allow auto configuration we expose we expose the API of selecting a sub rectangle on the topology in specific pads to make it clear to user space where this cropping is taking place so in this example we allow user space to select the sub rectangle on pad zero of the ISP so it can select which part of the image from the sensor it should work with and also in pad zero from the resizer so you can select which parts should zoom or shrink now about the image stabilizer how it works so usually we have the main image and the image stabilizer just selects a sub rectangle in the main image the idea is that only the outer rectangle would shake and not the inner one so if you shake your phone the image shouldn't appear that shakes that much just to relate a bit when you open your phone your camera and you select the video mode you can notice that the angle of the image shrinks a bit it seems that it zoomed a little bit it means that it's just showing you the sub rectangle is not showing the outer part so it can have some space to work with that means we need to allow user space to select another sub rectangle and we expose this one in the pad two of the ISP so we have three points to select sub rectangle then exposing this just in a single point and allowing auto configuration doesn't make much sense that's why in this case we have manual configuration propagation files to the system I want to talk a little bit about some design choices these ones the original topology of the driver when I started working with it we had this block exposure so rock chip sy me pd phi and we removed it and I'm going to explain you why so this block represents the me pd phi bus right in the manual configuration propagation as we saw if we have more sub devices more complex for user space so we need to rethink a little bit more about which blocks we want to expose and the five block there doesn't expose any image configuration this just represents a connection point ideally in the topology the use for information is just the image processing steps and also the same processing steps could be used in different buses in this case the rksp one it supports the parallel bus it's not recommended yet but could support and the me pd phi si2 so the idea is that that one doesn't provide much information so if we come back to the original topology when if we want to add support for the parallel bus we would need to update topology dynamically depending which bus you are using replacing that block from with the parallel block which would be confusing for user space that needs to perform all the manual configuration or we would need to expose another block to represent the parallel bus and another more block more complex for user space that's why we decided to remove it some lessons learned if you have so I see this and some other drivers that people post upstream the code from the bus is usually integrated with the code of the isp so if you can separate those in two different drivers one inside the media subsystem and the other one to the five abstraction layer subsystem under drivers slash five then with these you will have a more generic topology for any bus it is less complex for user space and the isp driver itself becomes much cleaner since you separate those two drivers and the five driver can be used for other protocols as well for the dsi for instance if you can have both protocols under the same line so some more lessons learned not only in the perspective in the technical perspective but I also want to talk a little bit about the community perspective the video for Linux community is very open to accept drivers in staging with the condition that you work on it to move it out as soon as possible you also need to have the tail to the list with requirements what is missing from moving it from the the from staging the advantage is that it make make available to other people to use and also improves your workflow it's much easier to get contributions from others if it is already upstream somewhere otherwise people will need to send you patches directly to you and you will need to integrate on our patch set every time and reposted every time so it's much easier if it's already upstream also it makes easier for people to test to send you bug reports and it also decreases the maintenance cost since you don't need to keep rebasing all the time and you can work on it step by step with also the the collaboration from from the community who can follow all the all the progressions so just still about the staging I really recommend you to do that with the condition that you work to remove it as soon as possible so some more lessons learned don't be afraid to reorganize the code if you are going to maintain the driver and make sure that you feel comfortable with it it has your your way right so don't be afraid to change the files namings the code or the rewrite functions also I would recommend splitting the code between different files pair implementation nodes or pair per block in the topology some cases this is not entirely possible because we reuse a lot of code from one block to another but at least separate the code between the video nodes and the sub device nodes since those are different they have different hooks inside the media framework and it makes much easier to reveal them if they're separate of course those are all tips take it with a grain of salt check if those applies through your case I would also separate the codes that configures the hardware from the codes that implements the video for Linux API mostly because when I am reviewing other people's code I don't really know the hardware so I can I wouldn't know if the register that you are writing to is the correct one so if they're well separated doesn't need to be in different files it could be just in different sections inside your your a single file I can focus on the video for Linux implementation for review also I recommend removing all the code that you are not using using or that you can't test for example the rk isp one driver also supports the rk 3288 soc but I wasn't using it I wasn't testing it so just remove it but keep the code in a way that is extendable it's easy to add to put it to put the support back also the driver defy part it had support for all the mipi device csi ports and it also had support for the dsi port and I was not using this one and most of the hardware that I saw most of the boards were also using just a single port so just remove everything the code was already huge so the idea is to simplify the code but keep extendable we also had lots of macros in the headers a lot of headers small headers that was not that easy to navigate some several macros that were not being used so just remove everything make it smaller and easier to reveal now let's talk about this specific project the libcamer one from user space as we can see not all features are auto discoverable from user space an example is the rk isp one driver that we can select the sub rectangle for cropping or sub rectangle for the image stabilizer and in the media api the api there's no way for user space to know that this sub rectangle represents a cropping or the image stabilizer which means that the driver in user space needs to know a little bit about the driver in the kernel also the metadata data buffer structures that we have in the statistics and also in parameters are usually in some kind of format that is very specific to the to the specific driver this means that it requires user space to have specific implementations for specific drivers for so this means that we would have a specific application for a specific hardware and the problem is that code is not very reusable it is very hard to test since you need a specific implementation you cannot get a generic one a generic application and usually those applications are proprietary that's where libcamer comes in so libcamer is an open source camera stack for many platforms with a core user space library it has user space drivers that has the knowledge of those specific drivers on the kernel and it also has a ways for image processing algorithms that knows how to deal with the parameter and statistics and what is nice is that libcamer allows you to plug your own image processing algorithms some as a plugin so you can plug your proprietary algorithm and it separates really nicely the open source part from the proprietary part we can think libcamer as the equivalent of the mesa project for the graphics world but for the camera world here is the architecture of libcamer that you can see under the libcamer docs i'm not going to pass through all those blocks i just want to mention some specific parts so down there we can have we we can see the mc and video front end support block mc for media controller so this is the part responsible for talking with the kernel we can have buffer allocators and on top we have the camera device with a driver specific code so we have the pipeline handlers that is responsible for configuring the whole topology and it knows which subwork tango mean cropping and which ones mean image stabilizer we can also have image processing algorithms that can that allows plugins and can be proprietary so a tip that i can give you is if you are working upstreaming an isp i recommend you to add push or update support for your hardware under the libcamer project it makes easier to test because sometimes configuring the whole topology is very painful you get more users more developers involved since libcamer is a very recent and very active project people are involved in both sides so in libcamer and also in the kernel community so you're probably going to receive more feedback from for your driver reveals guidelines the design design guidelines to improve the quality of your driver and you also contribute with this awesome project i foresee libcamer being used everywhere in the future so not only in our desktops but on our phones chromebooks and several devices and that was it that i wanted to present to you thank you very much for watching and feel free to reach out if you have any questions any comments my email address is ellen.quake.com thank you very much


Discovery, Houston with you through Hawaii and we got America shuttle fleet is playing a vital role in preparing people and machinery for a more permanent presence in space. Space station freedom. On-orbit testing of new hardware like this restorable solar array has helped engineers design lightweight compact structures for the station. Astronauts also gain knowledge in handling large objects and connecting space station like components. Even devices for transporting crew members along freedoms 350 foot truss have been tested. On several occasions the shuttle's payload bay has been converted to accept space lab similar in design to the scientific laboratories plan for station freedom. Here life science experiments focus on the body's adaptation to weightlessness. Without gravity, bones and muscles deteriorate rapidly. Astronauts conduct research to find out why and test possible counter majors to prepare for extended stays aboard freedom. Microgravity research gives America a competitive edge in today's high-tech marketplace providing valuable new developments in such fields as materials, electronics and biosciences. For example, protein crystals can be grown larger and more uniform in space. Researchers want to unlock the molecular structure of proteins, large molecules that perform any key functions in our bodies so they can design drugs targeting specific diseases and viruses. Shuttles spend only a few weeks in Earth orbit while space station is designed for a long-term stay. Its state-of-the-art laboratory will house more than a hundred scientific instruments, including those to develop better alloys, more efficient semiconductors and solar energy systems. This research will revolutionize our way of life in the same way the Apollo era gave us technology for computers, laser surgery, long-distance calling and even ATM machines. Already prototype components are being built and tested. Engineers are also fine-tuning power systems and simulating the station's demands on electricity. 18 shuttle missions will carry prefabricated sections of the station into orbit using robotic arms to assist astronauts during assembly. Once completed, a single habitation and three laboratory modules will be clustered together, shared by the United States and its international partners Canada, Japan and 13 European countries. Freedom's four-person crews are scheduled to spend three to six months at a time living, working and observing from its unique vantage point. The station will regularly be visited by shuttle, ferrying food, supplies and new scientific experiments from Earth. One thing is certain. The key to this country's space future is tied to the ability to live and work there on a permanent basis. Station Freedom, a unique outpost built on the strength of America's space program. The half a million Americans suffer sudden cardiac arrest. The heart begins to beat erratically, losing its ability to pump blood. Often the only hope for survival is a defibrillator, sending electrical shocks to the heart that can restore normal rhythms. In the past, those fortunate enough to live through an episode like this faced more bad news. A 50-50 chance of it happening again within two years. Today, these odds can be reduced to about 2% thanks to this automatic implantable cardioverter defibrillator, or AICD. Incorporating a variety of NASA developed technologies, the device consists of a pulse generator in a series of wire leads all implanted in the body. The generator about the size of a deck of cards is inserted under the skin in the abdomen. Leads connected to the generator are sewn on to or placed inside the heart. Once implanted, the AICD continually monitors cardiac activity, delivering corrective electrical shocks any time erratic rhythms occur. According to Dr. Rick Veltry, Chief of Cardiology at Sinai Hospital in Baltimore, more than half of the 250 patients he has treated with the device experience a shock within the first two years. If we equate that first shock as being potentially a life-saving intervention, then obviously 60% of those patients would not have lived to tell about it. Michael Letterer is one of these patients. He runs his own heating and air conditioning company. Mike describes the shock his implanted device delivers as akin to a kick in the chest, not pleasant, but something you recover from quickly. His job requires spending time at different construction sites. His active lifestyle, both in and out of the workplace, made the AICD a logical choice. Physically, I'm doing the same thing I always did. I've been thrown off my lawnmower and I've played golf and volleyball and I've been knocked down and what have you, it's no effect. I'm glad I got it. The AICD is manufactured by Cardiac Pacemakers Incorporated, St. Paul, Minnesota. Microelectronic, microcomputer, battery and telemetry technologies originally developed for the space program are all incorporated. Dr. Morton Mauer, a vice president at CPI, is co-inventor of the device. Our work with the device actually was in the same time period as the putting a man on the moon and bringing him back safely within 10 years and it was the push that this gave to science and engineering, which finally benefited the device. There are currently over 25,000 people like Don Gallagher who have received AICDs and a new lease on life. I can't imagine the circumstances where a person wouldn't want to live and certainly it's a device that gives you the opportunity to continue living.  So it definitely saved my life and many occasions. In 1990, Shuttle Discovery launched NASA's Hubble Space Telescope, the most powerful observatory ever built. Able to image detail up to 15 billion miles away, Hubble has provided astronomers with a clearer window to our universe. Soon after Hubble was deployed, scientists discovered an aberration in the 94-inch primary mirror. Even so, the huge space-borne telescope has been returning a steady stream of new information, particularly with the aid of computer enhancement. Similar processing has been used on data from other spacecraft, like Voyager, which traveled past the planets. Enhancement now brings out more detail for Hubble scientists. This is the telescope's original image of a galaxy 52 million light-years from Earth as it is cleaned up to its final version. Here, light pulses in a straight line between Earth and a distant quasar. Years ago, Albert Einstein suggested that if a uniform object crossed the path, light would bend into a ring. But if the object were not uniform, the ring would break up into four patches. Known as the Einstein Cross, Hubble's image reconfirmed this part of Einstein's theory of relativity. Hubble also showed scientists a celestial X, marking the exact spot of a black hole with the possible mass of one million of our suns. Astronauts are already training to fix the Hubble mirror problem during a shuttle service call scheduled for late 1993. By substituting a special component in the telescope, the aberration will be repaired. Called co-star, this corrective optics package contains a series of mirrors that work like contact lenses to restore Hubble close to its full imaging potential. In a clean room at NASA's Goddard Space Flight Center, technicians also practice removing the spacecraft star tracker, which helps point the telescope. In the meantime, Hubble continues to send back fine detail of the planets, faraway galaxies, black holes, and distant stars. Rewriting the textbooks on how we view our universe. Crawl from an airport runway, achieve speeds over 17,000 miles per hour, and fly directly into low-Earth orbit. Called the National Aerospace Plane, or X-30, this sleek research craft will lead to a new generation of vehicles providing affordable, flexible access to space in the 21st century. To help ensure that this high-tech concept becomes a reality, a group of about 50 engineering students, faculty members, and technicians at Mississippi State University set out to build a one-third scale mock-up of the Space Flight. They were selected for the job in a competition open to all 350 engineering schools around the country. The goal was to create something that would help communicate what the X-30 is all about. A model folks could see and touch at air shows and museums around the country. It had to be rugged enough to withstand the rigors of both the road and mother nature. This wasn't a problem for the project team. They built it just like an actual flying prototype. Professor of Aerospace Engineering, Masood Reshrahani. It is quite realistic. We did not make any shortcuts or we did not take into consideration that this thing is not gonna fly so we don't have to make this part strong or that part strong. We did not think that way at all. Building it like a real plane gave Aerospace Engineering students who worked on the mock-up an opportunity to apply what they'd learned in four years of classwork and get a better idea of what to expect once they graduated. Senior Latunya Pat. You can sit in class and listen to lectures, I mean lecture after lecture after lecture, but you can't really get a good feeling for what's going on until you've actually had hands on experience. One of the keys to designing and building the model in just one semester was the use of a robotic router to cut molds for key components, an approach that spurred real interest in the aerospace industry. Professor Kenneth Hall. We actually have people waiting in line now that they see our construction techniques to apply these techniques to their aircraft prototype development. At a ceremony in the school hangar with some 500 supporters on hand, the 50 foot 5,000 pound mock-up was presented to the public for the first time. Congratulations accepted. It was time to ready the model for the road. Its wings were stowed and landing gear removed. Once loaded on to a large flatbed truck, the two-and-a-half ton mock-up was bound for its first air show, beginning a journey that will help usher in a new era of space travel.


But the governor of Beyoncé, Stade de Yer-Diri, has been declared as the people's democratic party's flag bearer for the upcoming governorship election, shattered for November 11. The announcement was made by Governor Adi Mola Adiliki of Oshun State, Husev, as the chief returning officer for the party's governorship primaries. Adiliki announced that Diri emerged victorious with 305 votes out of 315 delegates who were present at the Gabriel Okara Cultural Centre in Yenegwar. Adiliki announced that Diri was elected as the governor of the country under state capital. By the power vested as the chairman, chief returning officer, Senegal, Diri, the election. Today we have heard one of the best primaries in the history of our party in Beyoncé State. It is therefore with this utmost joy and a very grateful heart to all of us in the people's democratic party, the leaders, the followers, and all who accept your nomination. Candidates, big party, big party, to contest the November 11 gubernatorial elections in Beyoncé State. I have not heard from anybody who wanted to contest with them and was deprived of it. The man has done well and all of us feel he should go back without any antagonism, without any disturbance. So what is going on? See what the achievements are making in the state? They are marvelous achievements, silent achievements. It is achieving all the local commentaries of the state. Dori Diri is somebody who has come to be fired by Assad.  Hello, hope you enjoyed the news. Please do subscribe to our YouTube channel and don't forget to hit the notification button so you get notified about fresh news updates.


So this story happens approximately in 1735. France have laid siege on Parma in Spain and things have become so bad that the army cannot fight their way out of it and so the Bishop of Parma is sent to see the commander of the French armies, the Duke of Vendor, to plead the case for Parma.  He's violent, deported, insolent, basically he's repulsive and when the Bishop of Parma comes he decides he's going to be at his worst.  The Bishop is so offended that he leaves immediately and he refuses to go back. The problem however is that the siege is still on. Somebody still needs to go and talk to the French army, somebody still needs to be the diplomat but after this incident no one is prepared to go. Enter Albarone. Albarone is a con man who somehow managed to fake both a title and a position for himself at court. Nobody knows how but the most important thing about him is that he's the guy who will do stuff that nobody else is prepared to do, all the dodgy stuff and so in the end when no one else can be found to go and see Vendor, Albarone is sent on that mission.   Long story short, after the Oscars saying Vendor and Albarone had ended up becoming the best of friends, some say they were even lovers but who actually knows about that right? What we do know however is that Albarone from having started life as a mere con man and having to invent his own title and trying to slide his way into every little position in court ended up becoming one of the most powerful men in the Royal Court of Spain. The longer I live, the stranger the stories become.


This is Beyond the Lines on Think Tech Hawaii. I'm Rusty Komori. We broadcast live every Monday at 10 a.m. from the Pioneer Plaza in downtown Honolulu. My guest today is Toby Tamayi. He is the president of AT Marketing and he is recognized as one of the top marketing professionals in Hawaii and often referred to as the digital Superman. Today we are going beyond marketing. Toby, great having you here today. Great being here today. Yes. You know, I had a feeling that you were going to dress up in a suit and I needed to kind of keep up with you. Thank you. You look very good my friend. You look very good. This is the first time I'm wearing a suit on my show. You should wear suits more often. Yes. Suits always make a man like that. So you always look good. I see all these networking events and you always look good. So I have to keep up with you. Thank you. Thank you. I'm glad to wear a suit. Thank you so much. I want to ask you, Toby, about your background. Can you share with me some of your history? Absolutely. So I was actually born in California, San Diego. Oh, wow. Missed my chargers, but it is what it is. I lived there for a couple months and then actually moved over to IA. Went to IA. Intermediate as well as Alpha Scott Elementary. Then most of my childhood in IA, after my sophomore year of high school, I actually moved to Kona and graduated from Kona Wainah. Spent two years there. Very interesting neighbor island experience compared to actually being in IA where I was right across the street from Colorado Ridge Center. But anyway, that was how I spent my childhood here in Hawaii. Did you have a far drive in Kona? Yeah, actually. So when I was going to school, they didn't have the second high school in Kona, so I actually had to travel 30 miles to go to Kona Wainah. 30 miles? No, bus. Wow. Well, so you appreciate having cars now? Yes. Well, I appreciate living in town now and just moving around my biky and Uber. It's much more easier than going 30 miles on a bus. Awesome. Now, you went to University of Hawaii. That's correct. And what did you study and why did you study what you did? So, one of the things that I did is I majored in management. I've always kind of wanted to start my own company and I've always been in marketing, events, promotions. You know, even when I was in the dorms, I was throwing some of the biggest parties there. And I'm still doing those kind of things today and I had a big focus on Japan. So a lot of my electives are always built toward Japanese business, Japanese speaking and writing. And that's kind of where a lot of my focus is today for my company. Great. And I want to know, Toby, what boards have you served on or you currently serve on? Currently, I sit on the board of directors for the University of Hawaii Shilor Alumni Association. I've been on previous boards in the past, including the Chamber of Commerce of Hawaii. That was a very intimidating board for me. But you know, I learned a lot and met a lot of great people. We're always looking for ways of diversifying my skills in terms of different boards and helping them do things that are really beyond marketing that they really don't know about. Yeah. And you run some of the best networking events in all of Hawaii and I love attending your networking events. Thank you for coming. Thank you for inviting. But how often do you tend to do these networking events and why do you do it? So one of the things about Hawaii that we're very unique is that we just have a certain population that allows us to really be able to have a big network, yet we're not really lost in the shuffle like being in Tokyo or LA and San Francisco. And so a lot of times I advise professionals take advantage of living here in Honolulu and going to different events. I probably go to between two to four a week. Sometimes it's two to four at night, depends what's going on. But you know, we're always trying to build our presence in the community, our presence in terms of who we're connected with out there. And it's very also, it's very good to just be out there and support different organizations that are really trying to get people to get involved. And it's always over good food and drink as well, so it's very nice. Before I see you, you always look so happy and you have an awesome job. Thank you. What is a typical day like for you? So one of the great things about what I do for your AT marketing and I have a great staff behind myself and we're really built on relationships that we can carry over by actually working at home. And so a lot of the things that we do is wake up in the morning, I'll check emails and I'll do those kind of things. I'll go to certain meetings during the day and at night usually there's some type of event that I have to attend.  I've got to make sure that I'm there and make sure I tell my people about what's going on with those restaurants. But a typical day is about food, it's about meetings, it's about emails, it's about networking. I do a lot of business in Asia as well, so a lot of the big bulk of my afternoon is spent communicating with some of my constituents in Tokyo and Hong Kong. Great. Now you're the president of AT marketing. That's correct. And how and when did AT marketing begin? So this is now my 18 year of business. I've been lucky enough to develop different programs in terms of marketing to both the residents and the visitors here. But of course, nine, ten years ago I really got involved with social media and that's became a big part of our marketing programs. So about 18 years, it's always diversifying. We're always trying to find out what's going on. We're always trying to utilize what we have and what we know to help people today but also figure out how we can help them tomorrow with different programs. Spam Jam is such a huge popular event in Waikiki and you're the organizer of that event and you've done such a great job with that. Can you tell me more about Spam Jam? Yeah, the Waikiki Spam Jam is now the largest one night food festival in Hawaii attracting about 35,000 people. We have 20 restaurants, three stages, it's a really great environment for both locals and visitors. In fact, I run the Spam Jam Facebook page and about every other three days I'll get another person saying, hey, you know what? I am making my flight to see the go to the Spam Jam in 2019. And that's exciting. People are actually loving this event so much that they'll actually come down to Hawaii and plan their vacation around. It's really just a festival about fun. One of the things that we're trying to develop is AT Marketing. It's developed a lot of these food festivals here in Hawaii that are not necessarily a paid festival. It's an open festival that really has more fun about food. I think the food here and the culture is amazing here in Hawaii, probably one of the best places in the world. And so I think we need to explore and use events more to really get more people involved in seeing really amazing restaurants and dining we have here in Hawaii. Spam Jam is just part of that. Yeah, and when I attend Spam Jam, and I absolutely love that event, it's very important for the businesses and for tourists and locals. Now, what are you hearing about why they love it so much? Well, it's a corny event. And I tell you, one of the funny things is we actually have a Wikipedia page for it. Really? Yes, it's that popular. Wow. And I just think that people are always looking for those unique experiences when they're traveling abroad. If I go to Tokyo or if I go to LA, I'm always looking for events that weekend. In fact, when I go to Tokyo next month, later on in a couple weeks, every weekend I have some type of bone dance that I want to attend or some type of festival that I'm interested in going to. So I think people are really planning their experiences around these kind of events. And as Hawaii produces more of them, we're going to be seeing much more happy visitors. Yeah, and it's very unique for sure. And it's a great draw.  And you've organized, you're one of the main organizers for rice fest and noodle fest. That's correct. Can you tell me more about that? So I've been helping out rice fest for a couple years. It's currently at the Howard Hughes Corporation. It's a celebration of rice. Why not here in Hawaii? We have a fantastic lineup of food and vendors. And then recently in March, we started a new one called the Noodle Festival. And that was actually very successful at 15,000 on our first event. That was also with the Howard Hughes Corporation. And so our job right now is to really partner up with restaurants, partner up with different venue spaces, and create amazing free festivals that the public can go to, families can go to. Hey, we're pet friendly events with the amount of people that are growing in the Kakaoka or Alamoana area. We're giving them a walkable event that's going down to them. That's really exciting about what we're trying to do at the Rice and Noodle Fest. And we're looking for more festivals to start developing in 2019 and 2020. Great. Your company offers such a wide range of marketing. Why are you so good at public relations? Public relations. Again, Hawaii, we have a certain type of space that we're in. And we're PR. We're very lucky that there's just so much media that are out there. And so relationships is all about. I've been doing this for 18 years. I've built relationships with some of the people out there, including, I think, Justin Cruz here one day. So we're having a lot of great people that we're connecting with all the time. We're also always looking for brand new things. We're looking for new stories and new angles and trying to be part of that. And I think that's what makes us so strong as a publicist, is that, hey, Toby, something new. Toby has this. And this is kind of things that we're doing to help us grow our, I guess, our network with the media. And so what really makes us good is really relationship. It just takes time to build relationships here in Hawaii. But when you build them, you can be a fantastic success. You've definitely built tons and tons of great relationships with a wide range of people. And your company develops marketing programs, obviously in English, but also in Japanese, Korean, and Chinese. That's correct. Can you share more about that? So one of the interesting things that we've been doing over the past five years is developing more programs into the digital marketing for tourists. Right now, we're getting about 2.5 million tourists from Asia, which is a big amount. And they mostly come here to Waikiki. So I'll just say, right now, we develop actually Facebook in English, Japanese, Korean, and Chinese languages. Twitter in Japanese and Korean. Instagram in Japanese, Korean. I have blogs in Japanese, blogs in Korean. And then I also run the micro website Weibo for China. And so we hand these packages over mostly to hotels and shopping centers. And we're really developing a comprehensive program to be able to communicate with tourists before they come here and after. One of the things that we're also trying to do is we see a lot of differences. Today is about beyond marketing. Beyond marketing yesterday was print, online, radio, and TV. And today, we have such a huge market of digital that we have to really grasp. But people don't understand that that digital marketing is different in every single country. Japan is very similar to us. Korea is China. China is not even close to us. Oh, wow. But people would be surprised that Hong Kong is the number one user of Facebook in the world. Really? Yes. Why is Hong Kong the number one? I think they just don't want to use Weibo, like Chinese and WeChat. I think they just want to separate from China as much as possible. So they're not using actually their WeChat and Weibo there, and they're using Facebook. So it's Taiwan. Taiwan is huge Facebook penetration. And so understanding these kind of things, and we don't expect everyone to understand that. And that's why maybe you should look for an expert like us that can really help you and guide you in terms of what is available for you, what kind of costs are available for you, and how does it maintain all that kind of stuff. We're doing a lot of translations now in four languages. And so a lot of the companies here are exciting for us. But it's also going to be a challenge for a lot of the smaller businesses here that are not able to do all these kind of things. And how do they market themselves to the new Asian visitors. And that's going to be a real challenge for a lot of people here in Waikiki and the Netherlands. You're definitely in touch with the current vibe that's happening in the United States and in Asia for sure. And I mean, you're doing your part to really keep Hawaii up to date in what everywhere else is doing. Thank you. You also do ticketed event development. Can you talk to me more about that? So one of the things that we're trying to do as AT marketing, or meaning personally as myself, is to assist events to become more relevant to today's market. I think the days of us having these sit-down 10-seat dinners and a gala and a performance is dying. And a lot of the non-profits and a lot of event planners know it. So what I've done is I've actually developed new events and new concepts and changed the concept of the events to be more relevant towards, I guess, our millennials and Gen X ticket buyers, as well as people that can possibly sponsor. And so you have to understand now what they're looking for. And what they're looking for now is they're looking for interaction. They're looking for an experience. Experience is not a sit-down dinner of four courses. They're looking for 12 restaurants. They're looking for wine. They're looking for socket. They're looking for some type of ambiance that creates an experience for them. So nowadays, what we're trying to do with ticketed events and we're talking gala, fundraisers, silent auctions, anything with those kind of premise, you need to really figure out your event today. And you need to figure out how you're going to survive the next five to 10 years. And you're not going to survive the same way you're doing it today. And if you don't change it, and it takes a couple of years to change it, you're going to be stuck with an event that's not going to be working. Now there's a problem now that we see here is that a lot of the people are not like us. They want to keep that same system of the 10-seat tables. And that's what they love to, that's what they're used to. And that's fine. So our job now is to find that balance and to find how we can keep both parties happy, how we can keep their traditional ticket buyers and sponsors alive, but also build their portfolios. And a lot of events nowadays, you have to start thinking about that. And how to even reach this market, right? It's not gala or email. It's a totally different way to reach this millennial Gen X market. So that's one of the big projects that we're working on right now is to really look into certain non-profits events, jump on their boards, or just jump on their committees, give them some insight about how to improve their restaurants, how to improve their drinks, how to improve their experience, how to improve their sign auction, how to improve their marketing, and how to improve, basically, going beyond marketing. No, that's great insight. And Morimoto's had their grand opening in Waikiki that I attended. And Joy of Saki is such a great event, too. It's such a popular event. Toby, we're going to take a quick break. And when we come back, I want to talk to you about why you are the digital superman. All right, let's do it. You are watching Beyond the Lines with my guest, Toby Tamaye. We will be back in one minute. When I was growing up, I was among the one in six American kids who struggled with hunger. But with the power of breakfast, the kids in your neighborhood can think big and be more.  One in three teens to smoke will lose years of these moments. It's your life. Don't miss a thing. Hello, everyone. I'm DeSoto Brown, the co-host of Human Humane Architecture, which is seen on Think Tech, Hawaii every other Tuesday at 4 PM. And with the show's host, Martin Desbang, we discuss architecture here in the Hawaiian Islands and how it not only affects the way we live, but other aspects of our life, not only here in Hawaii, but internationally as well. So join us for Human Humane Architecture every other Tuesday at 4 PM on Think Tech, Hawaii. Welcome back to Beyond the Lines on Think Tech, Hawaii. In case you are just tuning in, my guest today is Toby Tamaye. He is one of the top marketing professionals in Hawaii and often referred to as the digital superman. Toby, you are the digital superman. Shhh. And you're an expert in so many things digital. Can you tell me about that? So one of the things that my company, Team Marketing, is really trying to strive for is becoming the digital experts. And when we're talking digital, we're talking so many different landscapes of digital, from social media to websites to online advertising, influencer marketing of social media, bloggers, news sites. I mean, it goes on and on, advertising on AdWords, advertising on Google, making sure your listing is correct. Just keeps going on and on and on. And we're just talking, again, one language, English. Let's separate this and go into multiple language. And we've got another way that we have to really figure out how to communicate digitally. So our job, just keep learning. We know that this is the way that the market is moving, or a lot of the shift to mobile as well. And so as we talk, as the viewers are watching this today, there's such a big shift in the landscape in terms of digital. And even for people like us, it's still hard to maintain and keep up. There's the virtual reality part of where marketing is headed to. Can you talk about that virtual reality? Sure. So VR is a big project for ours. I'm actually traveling. I'll be in Hong Kong from July 15 to 17, visiting a VR studio there called Shadow Factory. It's actually owned by a local boy from the Big Island. He started the company about two years ago. Now he has 65 employees. It's a very large production now of developing VR and augmented reality. They actually have relationships with Facebook as well. So they're developing some programs at Facebook. Our job right now is to learn what they're doing in Hong Kong, learn what they're doing in their satellite cities, such as LA and Tokyo, as well as Taipei. Get that information, and bring it here to Hawaii, and show them, OK, this is what other people are doing. This is how they're interacting with the market. This is the products that they're selling. This is what they're doing for trade shows. This is what they're doing for events. This is what they're doing on YouTube. How do you get that information out there? And how do you build it? That is what we're trying to learn for now. VR is really what I think is the next step of marketing. First, you had social media that came in a little while ago. I think now VR and augmented reality and mixed reality will be the next medium that people have to learn from marketing. So you touched on augmented and mixed reality. What is that? So augmented reality, for people to understand Pokemon Go, it's basically just using a real world look and putting different type of filters in it. For example, what I think is going to happen one day is that you're going to be on Amazon one day. And Amazon will say, take a 360 pound of your room. Then you do it. And then you'll see, pick which lamp you want. And you can put the lamp in there inside an augmented reality. You can see exactly what it will look like you put on. You can put in beds. You can put in TVs. You can put in pianos. I mean, they're going to really look at this whole system as a way of you developing your lifestyle through what you see, but placing things in there that are not supposed to be there. That's exciting. Yes. And that's going to happen. That's happening already. It's happening now. It's happening now around the world. Hawaii is always a little bit late, but we're going to get there. Are you going to bring it to Hawaii? We're going to bring some good stuff here. We actually brought some programs with DFS and Michael Kors last winter. We had a gaming system where they actually started to catch Michael Kors bags. It was through VR. And it was for the winter at the DFS Waikiki. And so it was a really interaction for the guests. And what did they see? They see the bag. They catch it down. Oh, there's the Michael Kors bag around the ground. Wow. Ready to buy. That's amazing. So it's really about interaction and creating more engagement with people today. That's great. Yes. Now, regarding social media, a few weeks ago I had Emma Woe as a guest on my show. And I refer to her as the queen of social media. I could see that, yes. And I know that you are the king of social media. Can you tell me about why you're the king of social media management and advertising? Well, thank you for that, first of all. I give honest feedback. So you know what I do? I've worked hard. I've built a huge following on my Instagram, my LinkedIn, my Yelp, my Facebook. Huge, huge following. Thousands and thousands of people. And so what that has allowed me to is to really express myself in terms of promoting events, promoting restaurants, promoting people, promoting the show. Make sure you guys watch my show today. Really getting that together in terms of who I am, what my company does, and what I want to deliver through these mediums. I'm really excited about what social media is. I really advise, when I talk to a lot of the classes at UH and KCC, I always tell them, you need to build a social media network. How is that going to make me successful? It's going to make your living more successful than you are today. So I advise people right now to really look at their network and how they build. And if you get built, maybe the king of social media where I can promote stuff and bring people any time I need to, which is what I can do right now. With a point of a button? Totally. That's really exciting things to really know that's happening to this world. Do you feel that social media is just going to get bigger and bigger? Absolutely. We're just we're in the early stages right now. We haven't seen a lot of entrance come in. Snapchat was probably the last one that came in. But it's really Facebook and Instagram here in Hawaii, as well as around the world. So it's just going to get stronger. There's going to be more interaction when you're going to see what Facebook will augment reality, as well as what you're going to see how Instagram is delivering stories to people and how we're using them as actually influencers to help promote our brands. I want to ask you about that. What insights can you share about Instagram social media influencers? So right now, locally, we have about 250 people on our database that deal with food health. I'm sorry, food or fashion. And so we'll invite them to parties. There's one tonight. There's another one tomorrow.  And so what we're expecting them to do is post one on their Instagram and maybe an Instagram stories. What I'm trying to do right now is I'm actually trying to build. And if you guys don't know what Instagram posts and Instagram stories is, you should really know the difference because it'll help you out. We're really trying to build Instagram stories. So we're trying to build influencers. Recently, last week, we had six influencers from Japan. They all had 165,000 or more followers. And so I sent them to a restaurant and just asked for some Instagram stories for trade. And so a lot of people right now are really looking at how do we get more influencers into the market from Japan, from the US, even here locally. Maui Big Island, that's where the control is of messaging now. But how do you know them? How do you meet them? How do you grow them? That's the challenge that we're having today. And you yourself is a major influencer. I do have some draw, yes. So how did you build your network? So one of the things that I do right now is, let's say I met you seven years ago, I would ask for your business card. Today, when I meet you, I ask for your Instagram handle. Really? Yes. Why is it like that now? I'll be connected to you. A business card is basically I put it on my desk, and that's it. And I probably won't see it again. But I'll see you in my Instagram feed. I'll learn who you are. I'll learn. And it's not like you're telling me, Toby, look what I'm eating. Toby, look what I'm doing. You're not telling me any of that. I'm actually going there and looking at it. And so much people come to me and say, you know what? I went to this restaurant because of you. I went to this place because of you. It feels so great to know that people are listening and being engaged and showing enthusiasm to do things that we're showcasing all together on social media. You have a great connection with the Japanese people.  I love Japan, yes. What do you know about the Japanese people that others don't? Well, so I'll be flying to Japan next week. OK, cool. I'll be there. That doesn't surprise me. I'll be there for two weeks in July, which is so hot. Don't go in July. But I go there every July. And so I travel there a lot. I know the food. I know the culture. I study music. I study news. Right now, unfortunately, I'm recording the World Cup against Japan and Belgium. Awesome. That's how much I love to really be connected to the culture. So people have to understand, if you really want to understand the culture, really be connected. You've got to do homework. And once again, beyond marketing, you've got to go beyond what you know, what you want to do, and keep learning. I wish I could learn Korea. I wish I could learn Taiwan. I wish I could learn Hong Kong. I mean, there's so many other cultures I want to develop. But of course, definitely Japan is my expertise. Flights to Japan are very cheap now. So why is that? Crazy cheap. Last year, two air carriers came in, Air Asia and Scoot. They're actually flying in through Osaka. And because of that, their low-cost airlines, they actually were able to drop the rates dramatically to the Osaka region. Unfortunately, it's not been happening within the Rita and the Tokyo market. However, starting next May, Air Nippon Airlines is supposed to be flying three daily flights, these large air buses that are double-deckers. And that's going to create more seats, and you might see better pricing coming starting next summer. So it's a very exciting time to go to Japan. Right now, Osaka will be priced very low for a while. And I see Tokyo being priced very low starting next summer. Great opportunity for the people of Hawaii. And the food is much cheaper. You'll be surprised. Hawaii is just a very expensive place. Sorry. And delicious. The food's great in Japan. I love even the food at the airports in Japan. Oh my gosh, yes. Yeah, in fact, one of the famous restaurants I just called Tsurutontou, and they're in the Haneda Airport. And they just actually opened up at the Ro Hawaiian Center a couple of weeks ago. So Hawaii is getting a lot of great restaurants. I mean, we're so lucky to live here, especially if you love Japanese food. And if you love Instagram, put it up and show everybody what's going on at these restaurants. Help them out. Really helps people out when you promote them on their Instagram.  What inspires and motivates you every day? What motivates me? I definitely got some motivation reading your book. Thank you so much. Beyond the Lines really gives me information and insight. I refer back to that book quite often. Thank you. Looking back at different things, you need to work on yourself all the time, and you need to really understand what you're doing. And for me, your type of book that self-help is very helpful. One of the things that I'm really saying right now is for people that want to be successful, go out and just learn how to be successful. Google will help you do that. You're actually self-taught. I'm self-taught. Everything I've done, I've learned myself. I'm actually now going to Google and learning how do you do mixed reality in an event studio? I got to keep learning. Thank goodness for Google. Thank goodness for Google. Yes. Now, you shared some great insights on where marketing is headed and just going beyond the conventional marketing. Absolutely. If people or businesses want to reach you, how can they reach you? Right now, the best way to reach me is to just connect with me on Instagram. I know it's kind of tough for a lot of people. You can also connect to me on Facebook at toby.tomai. But of course, Instagram and Twitter is probably my major ways of really getting connected with people these days. We really want to make sure that people understand that with social media, it's about connection. It's about delivering information. It's about supporting businesses. It's really about beyond marketing nowadays. Awesome. Toby, thank you for being here today. It was great having you share all these insights into marketing and why you're the expert and why you are the digital. You're too kind. You're too kind. You're too kind. And you're the king of social media. You're too kind. You're too kind. Thank you, Toby. Thank you, Rusty. Thank you for watching Beyond the Lines on Think Tech Hawaii. I'm Rusty Komori reminding you to outdo what you have done and to find your greatness and help others find theirs. Aloha.


Is the world we currently experience all there is? Or is our universe being computed by beings in another sphere altogether? More importantly, would knowing this make our lives any less meaningful? Would we try to break out of the matrix? Regardless of the deeper nature of our existence, our experience now remains the most real we know. We continue in pain and pleasure, misery and joy. Our memories neither decrease nor increase in poignancy. Our power to help or hinder is unchanged. Our relationships matter more than the technical details of the implementation.  Sailing over a cardboard seat, but it wouldn't be. Yes, it's only a canvas sky, muslin tree, body, monkey tongue.


Okay, pop quiz. Aside from being incredibly dark, what do all of these diamond grids have in common? The answer in the back of the book is that each one is alone in the universe. And it's because the Hard Cell 3 grid, the matrix, is fragile and the diamond is gathered around one root and if another one comes in and they interact that's bad. So I did some work on it this week. Take a look at how that went. The reason is I'm not sure of the grid is still running on the UPS. It's still shooting. It's still on battery. Fans are out. So these are going to start heating up. Why does it matter that we have multiple cells on it? Because the reminder cells are the master plan. Even if we start out with only one diamond grid in the universe, the whole point of this process is we end up with two. So we've got to have some way that when we've got multiple ones that they can avoid crashing into each other, avoid interfering. So that's what I was trying to work on with the border foam. I went through several iterations of it. We saw three of them there. The one that actually got killed by the power outage, the UPS ran out of juice and we grid died. That one actually hadn't messed up but these things are messing up all over the place because it's hard making a cell membrane. A cell wall is hard. It's got a bunch of different often conflicting categories of properties that we want it to have and I came across this. This is a 3D electron microscope or something. I don't even know how they do this of an actual cell membrane. It's super tiny.  This raspberry-strawberry thing in the middle is the clathrin cage. That's a bunch of molecules that are put together just for the purpose of transporting something else through the membrane. It's unbelievably complicated because it's got to do on the one hand it has to resist outside stuff but on the other hand it has to be open to allow influences to come in selectively. And in the case of the T2 tile, the whole thing has to move as well so we can do it. I started working on it and still got a few bugs in the system. This is the title card for this update. Cell walls are hard to be soft. It takes a lot of work. These guys crashing into each other. This is in the simulator but it's a lot like what we just saw in the opening video as well.  I'm having fun. But it is progress. All right so the artificial life development challenge. I had been expressing it in terms of days. It was 180 days when it started but I've now re-expressed it in terms of T Tuesday updates like this one. And when you work out the dates, this is T minus 10. Here's the entire chart going from today down to January 3rd, January 2nd, when we're going to see the code running which creates copies and then those copies using the code running will create copies and we'll have grandchildren of the ancestor. So what we were seeing today, well what we will see, we'll see some stuff about the loops, actually the foam stuff, the cell wall stuff, that was borders. That was actually supposed to come in August but I switched to it a little bit early because the loop stuff was driving me crazy although I did go back to it. So T minus 10, it'll be over before we know it. Here, where we are for today, these were our tasks, code segments, the loops, the chain segments that could act as instructions. They can act as instructions but we haven't actually built any instructions yet, that's still to come. And then the loop spikes to actually be able to make one of these chains that forms a loop, that was the subject that took a ton of time on. The SFI review, oh I made some progress but I still didn't ship it. That is now priority number one going forward.  Depending on how you define fun, I guess I did okay. So to remind us the whole thing about loops, that was supposed to be on it here. So what is it? The idea is a loop laid out in space so that we can do loopy things with it. How are we going to do it? We're going to use it by having special atoms that are going to represent the segments, the links of the loop that are going to be laid out, not actually a circle, just any kind of closed path in the pockets of the HC3 matrix. And why are we doing this? Well, many purposes but the main one I like is to represent a loop in programming. In programming when we learn how to do programming it's always you go step by step and sequential programming is really valuable, really powerful because each step takes place at the same place. We get to use the same hardware over and over. Now I'm doing this, now I'm doing this and if we can actually complete a loop and have something that comes out of it then we can multiply our gains by just running the loop over and over again and stacking up the results. How are we going to do it in the T2tile project? We're going to have a single seed, a seed loop is what I'm calling it at the moment, that's going to pop out and make the minimal loop, which in our case is two atoms that are both upstream and downstream of each other. So they're up in the down from this guy points that way, the up and down from the other one point that way. So it's a little two atom loop and one of those atoms is going to be this special thing saying please grow the loop and that's what I was working on. The reason it's a challenge is because you know we have this big event window 41 sites that we can change everything all in one time and that's great but when we're doing it on the HC3 and we're saying neighboring pockets, one segment of the loop per pocket within the HC3 matrix. You can't do it all at once, you can't reach it all at once, you have to do multiple steps at it and it got very complicated. Let's look at a quick demo in the simulator. Okay so here we are, let me see I'm going to do a couple of things. I'm turning on the event tracking which is particularly useful and I'll make a, all right there is a seed small diamond so it takes a little while to actually grow this debugging setup because you know step one we have to let the HC3 grid expand and get out there and do stuff so that's going to go and here we've got the seed is still there it's basically waiting on a timer counting events until it gets to 25 or something like that and it's going to pop out all right so now it popped out a seed loop which is just wandering around and the seed loop is very silly it doesn't actually respect the pocket structure because I didn't code it too but once it finds enough empty space which should be quite quick no that's not it there you go all right bam so when it happened it was all really quick so this is the advantage of having the event tracing running is that we can now go back in time and watch what happened all right so now so there it is so the first thing that happens is whereas all right there we go so the seed loop finally finds a fertile spot and it pops out and it makes a new grow in you I hate naming classes like new or modern like that because of course it goes wrong almost immediately but I did it because this is like my fourth attempt just in since the last t Tuesday update to come at this in various different ways so this is the new grow approach and over here we've got one of the demos this is actually the same class that what we were doing with the worms that were zooming all around in the other demos um and but this one is special because it's going to try to build a box and let's see what happens find all right the demo moves around because that's what demos do okay so now the new grow is going to get an event and it pops out a new corner so what it's going to do it's going to go from here over to here down to here and then come back to here and try to hook up that entire chain and it might not to you know something could be there could be things clogging it up but in this case it actually worked because we've already seen it so eventually the the demo diffuses around some more the new grow diffuses around some more all within their pocket but eventually the nc where is it it's going to have to get an event uh see now the other thing we have we have the the hc3 grid is is still growing out or all around us oh this is getting boring all right there we go back it up and so there's so the corner gets an event and it pops out a side and that now gives us a path from the grow to the corner to the side and back to the uh demo where we can grow the loop we can make it we go from just two elements of the loop to one two three four we can add two and and that's in fact what happens so when the ns when the new side gets an event up there it was it did it um it oh oh no no uh actually i got it wrong the new side had actually done its work already and we were waiting for the corner to notice that the side had done its work and when it did the the corner got the event you know we can tell because we've got this this green uh event window around the whole thing it uh relabels everybody so now this one is actually the original demo that we had and we've got one more temporary things moving up and down and as we let it go forward now uh actually it will just jump ahead to the future all right uh so now in fact the uh those are updating around and around in a circle in a little four loop yay uh now there's a bunch of reasons why this is actually not going to work perfectly a lot of robustness that isn't there uh go through it more in the live stream if anybody cares so that's the progress um in other news um i have picked a first book to publish i'm feeling pretty good about it i'll talk about it a little bit when we're a little bit further down the road but but here's the teaser i didn't write it let's see how that goes also uh in the process of all of this code development lately you've turned up some some issues some little bugs in column five alana has now fixed them so uh we're going to push out a minor release you know 5.0.1 or whatever the next one is i don't care about the exact numbers so that'll be coming up soon and and that's it so the next update will be on august 2nd uh uh the goals are number one get the chain loops to grow beyond four links right now it starts out as two grows to four and then can't go any further we want to go many further we want to build the cell wall to do do better than having the uh ships crash in the middle of the night i'd like to refresh the obs workflow in particular because it's still taking me too much time to get this stuff set up and i would like to be able to do more video clips and get in and out of them stuff so it's just you know i've learned a little bit about obs at this point i need to do some work to smooth everything out and and indeed to have some fun so i hope to see you next time