This is a really quick overview of what are the current solutions-based applications we have. We have mentioned before that Cell Broadband Engine has got its applications in digital video surveillance. It's got its application in aerospace and defense sector, financial services and banking applications, also seismic computing, high-performance computing. Let's see what I'm missing. Electronic design and automation, medical imaging, and digital media. Digital media and digital content creation. So these are some of the examples of what IBM already has demonstrated workloads and what are some of them that these are in progress. Each of these areas are being worked on by a different team in IBM. So there's this first one application which means basically it stands for FAH. It's got some applications in life sciences and biomedical. And they have already demonstrated some results based on the Sony PlayStation 3. So imagine just having like a $600 or a $500 to $600 appliance with you and being able to do something really useful beyond just gaming. You might have heard about this very famous deal that we have made with the one of the US's leading defense organization, LANEL, Los Alamos National Laboratory. Basically there's two major defense labs in US. One is the Los Alamos National Laboratory and the other one is the Livermore. So we have signed a deal with Los Alamos and the final goal by 2009 or 2010 is to produce one petaflop of performance. So 1.6 petaflop peak double precision performance, floating point performance. So it's basically a hybrid environment where cell broadband engine is used as the diagram shows can be used as a Linux cluster with 8000 plus blades. So that will be 8000 times 16 SPEs, same in power. And the AMD Octaron system is used as the master cluster. So the reason we have to club up with AMD is because the main bottleneck on the cell broadband engine which we cannot avoid is basically the power PC, right? It's not really powerful, it's just a puppet master which basically just runs the initiates the application and tries to offload work. It just pulls the right threads. So AMD Octaron processor was proven to be much more better and for this particular application cell broadband engine is used more like an accelerator. Also there's finite element solvers and computational fluid dynamics. There's some workloads already implemented in the area of fluid dynamics on cell. This is in process, we don't have the, I don't think the FM is in available in market today but this is something that we are working on. There is some smaller workloads, not a full end-to-end solution proven yet. But we have for large objects, as it says, we have demonstrated performance of 52 gigaflops using 8 SPUs. And there is a website at the bottom. We are working with a company called Digital Medics in Germany. So this website will give you more details. DVS, this is another major play with a few billion dollar revenue potential for IBM. So basically now in the airports or in major stores, how are the security cameras there? There may be like 10, 12 or maybe 5, 6 only security cameras in different areas and all the data that's captured from this camera is stored in these many disks. And by the time you look at the disk, you know, you're revined and find out, then there's a person sitting manually in the office reading each of these disks and trying to see if there was any activity. So naturally the productivity of the human eye decreases after watching so many tapes back to back. So with cell, what we're proposing is all these different things that can be done. Object detection, 2D object tracking, classification, multi-scale tracking, multi-camera handoff, face cataloging, metadata representation. Basically how the cell broadband engine will be basically used for this thing, the video compression and analysis phase. So with cell broadband engine there can be maybe like 50 or 100 cameras in different, or maybe more than that actually. I don't know the current numbers yet. But they'll be there all in the airport. And real time they'll be reading in this video input from the camera, compressing it, analyzing it, and you know sending in, via the surveillance middleware and via this external terminal it will send an alert. So in other words, within a minute you will be knowing that okay, there was some suspicious activity and it will raise an alarm.  But instead with the cell broadband engine, with the cell processors and this current model with the right middleware and the end user equipment, we can actually do real-time alerts and do a lot more than what's being currently happening in the industry. You can do things like, you know, show all the cars that drove north on 10th Avenue to if you want to go ahead and, you know, to back trace the activity, you can see that beyond a certain line, there's a trip line, beyond a certain line in this parking lot, show me all the cars with this particular, with the license plate from Texas or something like that. So a lot of neat things can be done. There's a website for IBM. This is based on the IBM Smart Surveillance System, which was basically a people vision project from IBM Research Labs. Very well recognized. There's recently a really nice news article also about this. It's getting worldwide recognition. So there's a lot of, there's detailed descriptions available on the IBM Research website. I'm thinking it's not internal, it is an external website. If you need to know more, please do let us know. There's more details about what the Marvel system is. It's called Marvel, the Smart Surveillance Engine. So this is basically modeling of data and producing 3D, high-resolution 3D graphics images, reading the data. And now the most outputs that we have is all two-dimensional. The whole world is progressing towards 3D outputs, right? 3D graphical display, high-dimension data. So this is another, I Plus is another solution area that we're currently working on, where you can observe this input, model it, and basically reproduce it on the computer in a three-dimensional manner. And again, there's some more numbers about if you are from a banking sector, or if you're working on applications in the banking sector. Again, we've already produced workloads in the Monte Carlo simulation to the team in Germany that did a really good work on this. So we have some benchmark results available on that algorithm. Again, the way the cell processor deals with random number generators, random number generators are a big part when it comes to any play in financial sector. Again, cell processor is very good for that. And here there are some results. Over here, over eight cores versus 16 cores, and the peak gigaflow performance for FSS-based solutions. This is the company we have already talked about this. RapidMind joined hands with RealTime Technologies. RTT is also a Europe-based company to produce something called DeltaGen. RTT has got something called DeltaGen, which basically does complex 3D visualization for cars, really upscale cars. I don't know if they're doing something else for shoes and things, but I think mostly it's for cars, automotive, airspace, and consumer goods sector. So yes, consumer goods also. And they did demonstrate a very really nice application for the SIGGRAPH conference last year in 2006. Accelerator, this is just explaining the accelerator hierarchy for DCC digital content creation. Basically, this is just demonstrating that how cell broadband engine, when used as an accelerator, there will be obviously an encoding device, a video capture device, and how getting that data from the device, moving the computational part of it over to the cell broadband engine, can be just excellent results when it comes to 3D visualization. It's basically just the device that's doing the data capture of it. And then over here, I wish we had an image of a real blade center. I think we did have in one of the day one presentations that Duke had gone over. But this is exactly what you would buy from IBM. So you would buy this, and then seven of these blades go into two-use slots. Right now it's a two-use slot. Next year we're going to make it a one-use slot. So in one blade center chassis, it's roughly the size. It's very portable basically. It's just this high, and you can have seven blade centers plugged modular-wise. So the chassis itself has got the IO connectors, network connectors, everything else. And there's a management module that you can open up, and you can very neatly configure, find out how many blades are there, what are the resources available, how do you configure the network, give them all different network IPs. But this one thing has got two cell processors and 16 SPUs, and it just looks just this thin enclosure, which has this cell processor. Again, with more details on how you do the rendering of the data from here, move it over here for the data management part, storing and doing something else, which does not involve a really high-level computation. When it comes to the computation, move it over to the cell blades to do the analysis and compression. Basically, I think this device, this thing, it looks like a system IRP or some kind of that kind of server or an HP server or something. So it will basically read all this input data, put it in frames, and this is basically the device which will be sending the cell broadband engine all the frames, frame by frame to process. Again, more details on remote visualization. And this is basically some terrain rendering engine. I think we have a demo. I will show that demo in order to explain this further. Basically, there is, because with this high satellite imagery, recently there's been a lot of progress in satellite imagery, right? So you have all these satellite images of terrain. So what the cell broadband engine can do is capture the data, and then on the blade, it will basically do the server-side rendering. And to produce the output, and basically to produce it over this monitor, it would do, it employs some technique called ray casting. So it will try to determine the depth, combine it with the colors that get reflected from the surfaces to produce the final image. And this is just showing the advent of processing, how we have been going on a steep, on a high slope and just making significant advances when it comes to information processing. Started with the mainframe batch, right? Batch processing was the coolest thing back in the 1970s. And then moved over to mainframes and then a mini-computer, multi-tasking. And then the Linux introduced the concept of multi-tasking. And then the whole world, with the 2000s and the dot combos, everybody moved over to the internet. And now the revolutionary concept of cell broadband engine, or so we say. Okay, so developing an ecosystem around the cell. Yeah, our mission is to develop programs, be it programmability tools, be it a scheduler, be it a debugger, be it an IDE development kit, you know, anything that can help the programming community and make the programming a few steps easier. And also fill up the solutions tag. If there is an application and say imaging, right? There are a few sort of core algorithms that are needed in the imaging area.  So in every solution area, we want to complete the solutions tag. So by that we were trying to build the cell broadband engine ecosystem. The tools must address the creation of new code. Any tools that we need, they have to address the porting of existing code, portability tools, demos, proof of concept of proof of technology. That's another thing that we do for different workloads. In other words, if I want to see if Black-Scholes algorithms are... Let's pick up an Euler integration formula, for example. Is it a good fit for cell or not? So in order to determine that, we do something called a proof of concept. We put it over to cell and then we try to develop the... Get the performance numbers and benchmarks. And it becomes a benchmark when we're actually comparing it apples to apples with another architecture, which is equally compliant. Either an equivalent G5 or a dual-core processor or something like that. So we do a lot of POCs and proof of technologies. So what is the strategy? Provide development training. We did our part, step one, and leverage the academic interest. If there is any interest from the academic community, produce demos and papers, that is the key to all technologies, have as many papers as possible, right? Verify the performance potential of cell BEA. So in other words, if there is an application which I think is a good fit, right, verify that yes it is and then see if we can get the results out. Produce libraries, example code. Make sure that everybody gets a chance to play with the development kit. And then target ISVs in markets. The more assets we have, the better presence we have. It's very important for any technology to be able to survive to have ISV presence, academic interest, and open source assets in the community. So we do have lots of university programs that we do. Some high-tech universities that we work with. So high-tech programs and medium-touch programs and low-touch. For medium-touch, we have something called, well, actually let me go one by one. For the high-tech universities, we have one focal point that works with every university. We give out loaner systems to the universities. Again, this is all, we're not saying that we do that with every university. This depends upon what kind of collaborative research we can do with the university. We definitely give remote access. In other words, we have some QS20 blades available in what is called a virtual loaner program. So if you have an application that you have developed and you want to really run it on the real hardware, if you write it, if you send us an email, we will try to get you an account with something called Partner World and then get you set up with remote access so you can directly log into the cell blade and develop and benchmark your application. We have something called SIR grants and faculty awards. And of course we provide training worldwide and Duke is the lead for worldwide training. It does an excellent job. Already has done connected workshops in numerous places in China. Shanghai, Beijing, Taiwan, Singapore, Malaysia, Brazil, all over Europe. So anytime you have any training suggestions or any additional venues, please do let Duke know. Media Touch universities, we provide VLP support, virtual loaner program where students can remote log into the hardware. We have forums where anyone can post a question and there are developers actively monitoring the forum and they'll let them know. The forum is a really healthy community. You can see all kinds of really nice technical questions and discussions going on on any subject, be it A&E, be it financial sector, be it anything. And lots of really low-level technical questions too. I'm trying to do this DMA operation, it's not working. You can ask your DMA risk question if what we provided it was not satisfactory enough. Any question that you have that you're not satisfied with and your bulb goes on and you're like, oh, how does this work? Just go ahead and post it in the forum and somebody will respond to you. If they don't, please do write to us and let us know. There's a Lotus approach also where we provide online education. Rukh has set up this online education system which is called IBM Education Assistant. So there is a website link. I think it was there in the initial programming slides. You have it on your CD. And also you can purchase equipment. So if you have something and if you're trying to build a cluster, if you're trying to do something for university and you want to purchase a blade, please do let us know. So sample of events we did. They recently did the one-day workshop at Georgia Tech in Columbia University, a two-day workshop at Minnesota. Again, worldwide workshops everywhere. We have this university challenge which is getting a lot of recognition we already have in one month over 30,000 hits. Third-party development tools. We are working with a number of third parties. So in order in our continuous effort to make self-programming easier, we are working with a number of tools, ISVs, to provide programmability tools. So if you don't want to do the DMA, don't want to worry about all the stuff splitting and everything. We are working with better programming models, vendors that are providing better interfaces to make it easier for you. Provide more abstraction levels. So you don't have to worry about so many low-lying details. So we are working during the course of this year and next year we will have a lot more tools that we can talk of and showcase around this technology. Some of these are in the pipeline. The last libraries are MassLibrary, FFTs, FFTW. We already have support for open MPI, open MPI. Several multi-core frameworks available. Mercury Computer Systems has produced quite a few use programmability tools, something called TATL, something also called multi-core framework, which basically abstracts the responsibility from your hands to do any kinds of memory, data streaming or data partitioning or DMA operations. So in other words, you don't have to worry about synchronization and memory management. Continuously working on debuggers and performance analysis tools, we also have play in embedded systems. So in other words, we are working with our vendor, Mentor Graphics for developing real-time OS around cell. So in other words, we are targeting from the smallest level that starts with embedded systems and therefore we need some real-time OS support all the way to the most high-performing clusters to hybrid programming environments. This is a good example where cell SDK is a starting point. Then we have something called the Block Management Library. This is developed by one of our colleagues at IBM that works with me and Duke. He has produced this library. We are releasing it into open source. Basically, again, another abstraction layer, it takes away the responsibility from your hand to split the data buffers up and do memory management like DMAs at all. So you can use it. It's a very user-friendly, high-level API that you can use. We are currently in the phases of getting open source licenses and releasing it in the community, licensing and everything. So if you are interested in BML, do let us know. It's just a high-level API that will abstract the need from you to do the DMAs yourself and synchronization and everything. With the Pipe Library, BML is a two-sided data transfer. Pipe is a one-sided data transfer. Both of these came as requirements from the A&D Aerospace and Defense sector. We're working with Raytheon and they needed something that was not available. So this proved very useful for their requirements. That's also another framework which abstracts responsibility from your hands to do synchronization. Again, there's documents and documentation available about all of these in the SDK. And if you don't find any documentation tool, let us know. Another thing called ALF, it has got a similar use as BML, just a more low-level. Again, Mercury tools are available today and RapidMind are complete abstraction. So here, by going on this, this is like a line. It starts from the low-level tools, low-level, high-level, high-level, and this is complete abstraction. Today in RapidMind, what they provide, the environment, you don't even need to know how to vectorize. You don't need to know how to split the task of nothing. You just do your scalar program, take your scalar program, convert the data types, build it with your favorite compiler, and that's it. Okay, IDL cell applications. So right now, this is the best, I guess we have. We don't have a list of projects right now, but we will certainly follow up with you. So target areas, obviously, digital and media, image processing, media processing, any applications, algorithms that fall under these categories are an extremely good fit. And if you see, DSP is one of them over here. All right, and also graphics, floating-point intensive applications of pattern matching. This year, this is not our highest priority, but still it is a good, it is an area where we already identified over 4x of performance benefit. So we will cover programming tips and techniques really quick, some porting methodologies, and then move over to SPU overlays. Another effective technique, because the main challenge, a big question that is there in everybody's minds is that I will definitely have more than 256 kilobytes of code and data. What do I do? So Duke will cover that. There's also a very good paper. Dan Brokenshire is another senior technical staff member in IBM. He has written a paper in DeveloperWorks, 25 techniques to exploit the cell broadband engine. It's a very neat article. So if you can, you can go to DeveloperWorks and search for papers, articles, proof points that we've already developed. You'll find a lot of neat material. There's no dearth of material on anything in IBM. So we've covered the strong points. Obviously, everybody by now, everybody knows that, yes, it's 8 plus 1 is the magic number. And actually it's 8 plus 2. These all, these SIMD engines are decoupled. That's another key thing. There is a very simple architecture, no complex hardware-based features available. Decoupled SIMD engines. Obviously, the PPU is the two-way multi-threaded. The chip area is very small. And that makes the power consumption also, brings the power consumption also very small. Not the chip area, but the simplifications that we have done in the hardware. That is, by separating the PPU and the SPUs and by the memory architecture, all those things add up to, for the less power consumption. Because in a way, with the advent of technology, the limitation has not been CPU power anymore. You can go on building deeper and deeper and deeper pipelines. The main roadblock now is memory latency, several thousands of cycles in most cases. So that's our key challenge and that's what we're trying to address with this technology. As my performance goes up, my power needs to be low. So 128 registers, local store, the DMA bandwidth is 25.6 gigabytes per second, 16-way SIMD fashion, fully coherent buses and memory, peak bandwidth for EIB is 204.8 gigabytes per second. Programming tips. So we now know that, okay, the level of programming is not the most high level, right? It's not Java. So there is certain intelligence and awareness of hardware that's needed, not hardware as such, but the hardware features that's needed as such. Again, as Duke mentioned, this is a new technology. We are evolving right now. We're working on making more and more tools that address the area of programmability available commonly to every developer. So dual issue rate, we have dual issue pipelines. I have to overlap DMA with computation. Design for limited local store, the code has to be compact. Loop unrolling, we have to keep in mind that you cannot keep on just unrolling the loops, right? Because it increases the code size. So it's a delicate balance between how many times you want to unroll the loop versus what is the benefit, performance benefit that you want to get out of it. Shuffle byte instruction is a very key feature available. Avoid scalar code because, again, when you're using scalar code, what are you using? What is the disadvantage? We're not using all the 128 bits, right? We're not using the full cache lines. So avoid scalar code. And choose the right SIMD strategy, load and store only by quadward. Instructions scheduling, different instructions take different, have different lengths and stalls. For example, single precision have zero stall, and the latency is six cycles. Integer multiplies is seven cycles. Double precision is seven plus six, so 13 cycle latency right now. We're trying to change that. And there is pipe zero, which is the even pipeline, and the pipe one is the odd pipeline. All the memory-related instructions are there in the pipe one. So in the SPE, there's a high penalty for branch misses, right? 18 cycles. You can use branch prediction. There's software-assisted branch prediction that's available. If it's successful, it will completely avoid the branch penalty altogether. So use software-assisted branch prediction, and I think we have an example coming up very soon here. So if you have an Eiffel's loop, normally in today, this is how you would do the Eiffel's loop, right? If A greater than B, C plus equals one, or C plus equals B, is C equal to A plus B. Instead, what we're asking you to do, and this is definitely a branch. If you don't do any kind of, you know, branch hints, the branch will happen, and it has its own penalty. So we want to save that altogether, right? So what do we do? So we do, first of all, we do a compare greater than A comma B. It will find out if, you know, if A is greater or B is greater. The mask, the result is stored in the bit form in select. We do SPE underscore add C comma one, SPE underscore add A comma B. So in other words, we are in the scalar code, or in a normal code, you only do one of the two computations. You either do C plus equals one, or you do C plus is equal to A plus B. In this case, we're saying do both the computations, right? And leave it on runtime on the select mask. This avoids the branch. And C equals SPU underscore self. So depending upon this mask, the select, it selects if it should take the AB value and store it in C, or if it takes the C1 value and store it in C. Now, one key thing to remember over here is that sometimes it's not as simple as C plus equals one, right? There's a whole bunch of code in the if loop versus the else loop. That's again up to the programmer's decision that, you know, which one, is it safe? Is it worth it to do both the computations? Does it take more cycles to do both the computations versus does it take more cycles to basically just let the branch happen? Unroll the loop. Again, remember the code size. Every time you unroll the loop, you are saving cycles, but you are also increasing the code size. So do always measure the code size.  SPU unroll the loops to reduce dependencies. And there are some, the transform light workload is already demonstrated. There is a paper out on developer works again, which basically demonstrates different workloads and how they were done. If there was a matrix multiplication workload and which showed like the 10x or 12x benefit, how did they get the benefit? They go over the strategies they used. There's papers, there's a paper on FFT, both from Mercury and from three developers. We showed the pictures also from IBM, which demonstrate, okay, if I did produce the results, how did I do my optimizations? Like the approach and all. There is different papers available on different suggested workloads. Again, all this material is already out there. And this is actually some data using the XLC compiler, some demonstrated results for the transform light workload. Again, this is an example of workload, which is hardly this small. The code is like hardly what, 12 lines, no, not 12 lines, maybe 20, 25, 30, less than 50 lines of code. So it's a small workload, but it's a really crucial workload in a lot of areas. But by demonstrating this workload and putting huge amounts of data on the workload, you can pick up a sizable program and demonstrate that, yes, by optimizing and doing techniques like this, you're seeing a resultant performance. Function inlining is another thing. If there is a function that you can inline, we highly recommend you to inline it. It eliminates the two branches associated with function call linkage. The first one is branch and set link, to store the address where it's going off from and then branch and direct for the function call return. But then over-aggressive use of inlining can result in code that reduces the locals that will consume a lot of local store space. So do things, techniques like software pipelining, this is just a general concept, not very, not very particular to self-programming. A lot of times we try to enforce pipelining where we are trying to do two things in one loop. The way you will write the code is basically just like double buffering. So while you're doing this part, you're also doing a second part of the computation at the same time. Avoid integer multiplies. FPU only has 16-bit integer multiply support. So if you wanted to do a 32-bit multiply, you will need five instructions, right? Like three adds and three multiplies and two adds, something like that. So avoid 32-bit multiplies. That's another thing if you can. Keep the array element size to a power of two. Cast. Okay, so basically in order to keep it to a 16-bit multiply, try to do operands, cast it to unsigned short before you do the multiplying. And because FPU only supports the quad-word loads and stores, and it always does the loads and stores on 16-byte boundaries, so try to avoid scalar code. The FPU code should always be vectors, vectors and vectors. Only if it's absolutely required, if it's a counter for loop or something, is when you should use scalar variables. There's instructions available to switch from scalar to vector and back to pick up, to store locations in particular areas on the vector or to extract values out of a vector. Choose a SIMD strategy appropriate for your algorithm, evaluate an array of structure organization. Basically, if you want to store all x values in one vector, or you want to store x, y, z, w values. So it's your trade-off. It's your choice as to what is appropriate. It's actually not your choice. It's the algorithm's choice as to what is a better strategy. So to make all these design decisions early on and spend a lot of time making these decisions, it will really yield a good result. And this is another example. Basically, it's just taking an example of what kind of results you would expect if you did a vector across computation versus a parallel array versus parallel array for one subdivision point at a time for four independent triangles versus four subdivision services, vertices at a time for a single triangle. So it's just another demonstrated workload. There's more documentation also available for this on the web. And again, there's different strategies for partitioning and work allocation. There is something called, I think we have the next slide that covers the reporting techniques where your programming model can vary. It can be, actually, do quality programming models, the PPE centric versus SPE centric, the master and slave kind of configuration, the fork and joint kind of computation, the FFT largely use fork and joint, right, because it's numerous small, small, small computations and you try to break it up into the simplest phase and at the end you combine the results from all these computations into one result. Offload as much work onto the SPE as possible. Accommodate potential data type differences. Find your data and see if you really need 64, you know, doubles or floats versus, you know, 32-bit versus 16-bit integer multiplies. Exploit the SIMD. Write your code so that there is no false dependencies. Let the compilers do the auto-symdization. If there is false dependencies in your code, the compiler will... compiler's hands are tied. Write assembly. If you're an assembly person, write assembly. Programmer-specified user generics. There are features available. The branch hints, the function inlining, loop and rolling. There's techniques available. So it's quite interesting to deploy all these techniques and see the difference it can make in your code performance. Use SPE lets for quick prototyping. So, for example, you just want to know, okay, I just want to see if SPE can do this computation fast enough for me. I have very high expectations, right? You don't want to deal with compiling your program on the PPU, send the data all to the PPU, down to the SPU.  And then see if... Or maybe you just want to split it up the whole data already into the SPUs and see if it's fast enough. The vectorizing part of it, right? You can do something like SPU let, which is a standalone program, right? You just... The data is already there in the SPU main program. You don't have to do the sending and everything. So that will give you a quick overview. And the optimizing part, enough. Then, if it's good enough, then you can add the DMA part and everything else. There's software cache handling applications. Basically, the software managed cache. The source is available in this directory. Documentation is available in this directory. And then there's examples also. So we're giving you everything. Documentation, the source code, everything. And test programs that demonstrate the effectiveness of it. So several levels of parallelization available on the cell processor. Now, this is a presentation that deals with a step-to-step approach to porting an application. Some real-world examples of what problem we had in hand and how did we actually tackle it. So... SIMD processing. So several levels of parallelization available. One is SIMD processing. So we have the ability to vectorize our code. Dual issue is there. So in other words, we can make sure that we're not only... We're executing only one instruction at a time. We're executing more than two instructions at a time, basically. Multi-threading support available. Multiple execution units with heterogeneous architecture. Right? And then we have shared-memory multi-processing. And then distributed-memory multi-processing. This is the cluster kind of configuration where you can couple a lot of... Not couple, but you can use cell broadband engines in a cluster kind of configuration. The EIB gives the support. The high bandwidth plus enables us to do that. So basic steps for parallelizing my program. First of all, what do we need to do when we have an application in hand? We have to try to find out if there is their concurrency. Right? If there is no concurrency, then basically it's really hard to parallelize if the whole program is serial. There's serial flow of data. So the key points are what is the data flow? How is the data coming in? And how is the data getting out? What are the dependencies? We have to find our dependency if there's any concurrency of data. In other words, if the same loop if there is one section of data that's repeatedly operating on a big bunch of data several millions of times. So we have to... If there is still some room for concurrency but the application is written in a very serial manner to structure the application so that concurrency is either increased or you have means of exploiting the concurrency. To infer performance, challenge data dependencies, try to reduce the dependencies, overhead in synchronizing, try to reduce the synchronization that you really have to do. The less synchronization you do, the more time you save. So in other words if everything is fast, the memory transfer is fast you're doing double buffering, you're probably using DMA less, you're vectorizing the code but then in order to synchronize one task with the other task and the other recipes, you're doing a lot of back and forth DMA messages mailbox messages and synchronization variables it will negate the effect that you're getting out of everything else or neutralize it. So watch out for that is the message. And when you partitioning work make sure that you do at least some kind of initial decision about load balancing. And there's obviously if you're working migrating from another platform, there's differences in bandwidth of buses topology, so observe all those differences and plan accordingly. So locating the concurrency, the first step, analyze the program and its algorithms and data structures what is the effect of overhead of synchronizing, how much synchronization do you really need? Is there any way you can override these dependencies? Identify the program hotspots. There is a very cool, I don't know have you ever used Gprof Ganuprof? So it's a very neat profiling tool. Once you run that profiling tool, it gives you an output it will give you a total number of loops that are there total number of time taken on all the functions it produces a flat file with all these different different parameters you look at the parameters you're like, oh yeah so this is the code that is being entered maybe, you know, 100 times or 1000 times. So this is the part that I really need to optimize, the rest of it I don't need to care. This is the high traffic part. So tools like that are available to help figure out what are the hotspots in the scalar code. So do all processor elements update shared data? Is there any shared data? You know, the shared memory usage in that data? Does one task need another task data? Are there any race conditions where the data might be read before the other processor wrote it back? Is there too much synchronization overhead? Is there the interactions among tasks, synchronous or asynchronous? So and the processing advantages are they worth the data communication cost? So we observe all these things to locate the concurrency and then structure the application to exploit the concurrency there are several considerations, right? Task or service level parallelism, data partitioning, task grouping, how can you group the task together? Divide and conquer, right? Divide it into several tasks and see if you can and assuming that there is less dependencies between these tasks, it can be a very effective result. So it could be a master slave, it could be a fork and join. So considerations for distributing application workload and data, processing load distribution the program structure data flow, data access patterns, observe all these things. And I'm sure there's lots of open source tools available to help achieve these results so that we don't have to look at the code, hundreds and hundreds of lines of code. It's impossible to look at 10,000 or a few million lines of code and say, okay, where are my hotspots, right? It's very hard to do, it's unrealistic. So there are tools available that will do it for you. There's again different application partitioning can be basically PPE centric where the main application resides in the PPE and it will either stage it like in a pipeline like you do this part and you do that part and you do this part. Versus parallel stages, like pipeline is one below the other. So it splits it has one task and it spawns of three threads that are working at the same time. Versus parallel stages where there's dependencies between the first stage for the second stage. First we create thread finishes and then the next SP thread can be started off. Steps of parallelizing a program first understand the program, choose the programming tools and technology, high-level parallelization strategy, low-level parallelization strategy and then design the data structure, iterate and refine, fine tool. So we have this video, right? Basically this is about surveillance, surveillance of video. So we have a video, right? And we are monitoring this. There's techniques called segmentation and background subtraction by which you can detect activity. So in other words, the video looks like this. You try to build this background over a period of time so that you can find the differences. How do you monitor activity? How do you monitor change in the activity? The only way we can do that is to find all the differences between the foreground and the background. Right? So there is this background frame that gets built over a period of time. And via segmentation what you can do is detect the foreground objects and differentiate them with the background image. So the end result would be something like this. There was probably something over here. I don't know why she's cut off like this. Now the complexity, right? There's several complexity with this. In general, in real world there's all the security cameras on top of all these buildings, right? There's moving rainbows at sharp edges. It's called the NTSA effect or something. Then you have to adjust for weather changes. There might be wind. There might be very low visibility fog and rain. So the camera has to adjust for all this. And the data analysis has to, not the camera, but the data analysis has to adjust for all this. In other words, even if there is rain there has to be some mechanism by which you can still analyze the data. Handling swaying of cameras if the camera just moves off. How to make sure that the software is capable of detecting that and reanalyzing the images. And doing things like auto gain control. I don't know what auto gain control is. And then auto white balance. So basically in this BGS application which is BGS stands for background subtraction, you can build the background model over time. And then in image processing or in general in imaging there's nothing called an empty frame, right? There's no scene is empty. It's always a mixture of some kind of pixels. Weed grayscale or 8-bit color or grayscale. So tracking the illumination changes and adjusting to it accordingly. There was all these stages in a full fleshed application. So this is via input image and you could do smoothing to it. You clean it up do motion energy process it for shadow and highlights texture. Do that find out the texture differences pixel differences and again all these processing that you could do quiescent area addition, slow update blending. I don't know all of this but some parts of it yes I have worked on stationary healing and then we get our background. So there's all these stages that go into and even when the almost final stage comes out you still have to do the statistical thresholding, contour smoothing to smoothen the edges and then component pruning finally come out with a mask. There's some algorithms and there's detailed information available if you are interested in this particular workload we can provide you some documentation. So now our phase was to do that video analysis compression and analysis part right. So if we have to do the image preprocessing part and the salience detection part and the mask generation and then at the same time do model maintenance and then that gives us the final objects. Now we have at that time there were many so our design choices right. We have this full application with a few hundred thousands of lines of code. Many functions involved right and then functions tend to access multiple frames. Every single function needs some access to some kind of frames at different times and then some frames contain color images. It becomes much more complicated because it's not just dealing with 0 to 55 pixels anymore but we're talking about 8 bit color maybe more 16 bit and then the original code was written in C++ and then processing one frame touches 4 to 6 megabytes of data just one frame. So we had a few three alternatives right. Complete the BGS the background subtraction in a single SP for one video stream. So take one stream and then finish it in one SP or we can organize it into groups with one group per SP or we could do individual BGS function partitioned across multiple SPs. So what did we finally do? These are the alternatives. So take one video stream and give it to one SP and this case we needed code overlay because we're trying to forgive the entire processing responsibility to one SP at a time. This is when we used code overlays which Duke will cover in the next session and it made some difference and so the other option was to make it as a pipeline this didn't work very well because load balancing comes in. Some phases might take longer than the other phase. Third one we could add data partitioning across SPEs. In this particular application it was not so easy because there were lots of dependencies, there were lots of functions and all these functions were accessing all these frames at different times. So this we could not do. So we considered all these design considerations and went with the first alternative. So this is what we finally did. Data partitioning was not feasible for all functions and all images stored accesses from main memory. Code partitioning criteria code size and the compute time are the two key criterion to find out the code partitioning strategy. So we figured out there is four compute intensive pieces plus the coordinator. So each module ranges from 180 kilobytes to 240 kilobytes with code and the temporary buffers that we had used. And this was the final application, the way the process that we came out with. First of all we allocate the data block in system memory, right? Obviously we have to bring it up into the RAM and then pass the address of the data block to the SP function. The pre-process stage for the third stage the SP initiates the DMA to transfer data to local storage. The fourth one SP initiates the DMA to start transfer results back with MFC get input and then the SP notifies pp of the job completion. So this was the rough structure. This is one critical application that IBM has a big plane with some surveillance based airports and stores and everything. Again another example is image compositing. Base image, we have the base image B, this is overlay image A and we want to come out with this resultant image. So these are the, for all the images there is an RGB and alpha value. So this was the formula that was supposed to be applied to all these pixels, all the RGB and A values, the alpha values. So this was to be done for these several hundreds of thousands of pixels and so this is how we did the data flow for it. In other words read A and B, right? So read the image A and B like if you go back to this image right, we have the base image B and then the overlay image, right? So read both the images and try to do the formula but do it such that you can overlap different stages and form like a pipeline. It's just a basic pipeline it's just that we're trying to enforce the pipelining concept into our code right here. So while we do A equals A by B you write it to A, right? And then you write the A image back to main memory. At the same time we try to do the next two blocks from the image A and B and then while you're doing the right you're trying to do the compute which is on the next two blocks you're doing A equals A by B and at the same time you're trying to do another DMA operation initiation, right? So this is how we bring out some good performance out of some applications and these are the performance results for 120 images, A over B 120 images per second for four SPEs and there's some more numbers for more images also. So multi-beferring help, now remember you do have VMware but I strongly strongly recommend that if you have an X86 PC right try to partition it, create a Linux person, it's high time switch to Linux so create a Linux partition there's several partitioning tools available, create a Linux partition and install SDK 2.0 in a native environment. VMware is good, it gets you feet wet, it gets you up to speed real quick but when you want to do real applications, real workloads it's slow you will realize that you can't even open up a browser and check your email because it takes forever to come up, VMware takes all your memory, so really if you want to just get down and you're developing a real workload please do it natively ok, too much overhead and creating threads, so the lesser threads you create, the better ok, that's it


Hey guys, how you doing? This is rich here. We have a rich TV live and another member brought me this pic Rdgl is pretty wild because I did my analysis on it yesterday and told our community that it was gonna explode and It did it went from two cents to 24 cents over the last just What? couple weeks So and today specifically this big green candle here this big massive green buying candle. Yeah, that's today up a hundred and thirty six percent Congratulations to all of our members on Rdgl and let's talk about why okay? So right here vivos ink applies for breakthrough device design designation for radio gel trademark Vivos ink Rdgl on the OTC QB now. This is OTC QB stock which for a penny stock I like this was only a two cents stock announced at 20 cents It's an OTC QB issuer, which means it's fully reporting fully reporting issuer as opposed to a non-reporting issuer This is a much higher tier company vivos ink announced It has completed its submission to the FDA the food and drug administration seeking breakthrough device designation for radio gel big news breaking news and the reason why I believe this penny stock needs to be on everyone's Radar despite the fact it's already exploded. I think it could go higher This is a voluntary program for certain medical devices that provide for more effective treatment of diagnosis of life-threatening or irreversibly debilitating diseases or conditions It is intended to help patients have more timely access to these medical devices by expediting their development assessment and review While preserving the statutory standards for pre-market approval or de novo marketing authorization The breakthrough designation process takes priority over other FDA meetings So we elected to submit this application prior to our plan Our FDA pre submission meeting We intend to schedule the pre submission meeting in the near future with no impact to our overall IDE submission effort It gives us time to gather more data to share with FDA Dr. Mike Corenco stated in our opinion radio gel trademark meets the criteria for breakthrough device designation But we cannot be presumptive about the FDA determination if granted Designation would give radio gel trademark priority review opening up enhanced FDA Communication channels and potentially saving months in the IDE Approval process now. Let's just look at their share structure because that's vital You know that I like tight float stocks. I do not like deluded pigs Look at the market cap only a 20 million market cap a baby market cap 143 million shares in the float This is relatively tight for a penny stock QB company. I think the upside is tremendous we told our entire community at two cents and We started to really talk about it at eight cents and now here it is today Our members getting in early and today it explodes up a hundred and thirty five percent Congratulations to members and winners on vivos Inc. Rdgl a screamer I think we could see this at a dollar in twenty twenty one I'll come back in twenty twenty one to celebrate If you're not winning you're not watching we bring in the winners and we bring them to you first Rdgl a screamer penny stock up a hundred and thirty five percent Get your popcorn ready. I'm out. Peace Another big winner is true


All right. Well, hey guys. Welcome to the testing and automation dev room today. I'm going to be talking about Writing good tests writing go tests and writing good go tests probably in that order I tried to my best to cater this talk to a wide range of experience with go So if I have experts listening out there, please just be patient with some of the introductory material and I promise we'll work our way up to some of the more complex cases and And to any novices if you're new to go don't get don't get discouraged by any unfamiliar go Concepts or packages much of what I'm going to cover is like directly from the go doc So if you want to reference those at the end to revisit anything you're welcome to do so Hopefully regardless of your experience level you're all visual learners I'm going to have a lot of code on the screen and kind of be like tutorial based For each concept so hopefully by the time we're done here You guys will walk out with a better understanding of best practices in general testing and in goaling testing My name is Nikki Atea and I'm a work-from-home dog mom who hacks on distributed systems and workflows I'm based in Southern, California And you could find me at my handle listed on any social media You could just look for the baby Yoda avatar sipping a soup and that's me I Why am I talking about go tests? I have a long history with testing and go So I at the power and performance team at Apple I maintained automation frameworks for iOS and watch OS performance testing and we used Ruby and Python for that and currently I'm a software engineer at Sun Sue Which is an open source and open core monitoring tool kind of like Nagios on steroids if you're familiar with either of those And for about the last three years. I've been contributing to the sense you go Project which judging by the name is a rewrite and go Sonsu Inc the company I work for Has supported all of my efforts to come here from halfway across the world So a huge thank you to them and their support of open source So if you're curious about what we do at Sonsu at all you could check out our github right there All right, so let's start by answering the first question You might have on your mind why go Believe it or not There's a lot more to go lang than the cute little gopher mascot such as this Belgium styled gopher by Ashley McNamara For starters, it's an open source language, which is pretty important designed by Google. You could find it at github.com slash going So it's a comparatively young language especially compared to the ones that more of the popular ones we use today And it's been almost eight years since its initial release to the public so as a result the original offer authors were able to address more Current and relevant problems in the landscape Like at the time that they were designing it So these include runtime efficiency high performance networking and multi-processing So as such many consider it like the hot new language and distributed programming Including those big companies like uber twitch Netflix, which all require like high concurrent performance. So It's a good language Now that we have a little go history. Let's talk about the design of the language For the most part it's considered Statically and structurally typed it's type safe requiring a strict type for each object in field memory safe Allocating memory for each object in field. So in these instances the compiled binary is a lot faster at runtime Which of course is a performance win But the caveat to this characteristic is that go also supports some instances of dynamic typing So if you're used to like Ruby and doing really dirty like Things in Ruby go is capable of doing that But it's just more considered on the statically typed language side So using techniques such as reflection you can treat go objects generically at a little bit of a performance trade-off. So Generic functions if you decide to do that They encourage code reuse and can compile the program faster But might be a little slower at runtime because of the additional dynamic processing it's doing So for the sake of testing that will focus more on the statically typed cases because they're very straightforward It is a compiled language which with a very large standard library It produces tools such as go build go run and go test Therefore when you download the language itself, it's quite large. I think comparatively the other languages as well The most recent version of go is a hundred sixteen megabytes on Mac OS and Linux systems But due to the enriched Standard library that they have a lot of the functions you're going to need on a day-to-day basis such as IO calls OS exec encoding networking syncing and of course our testing operations Are all contained in these built-in packages. So it limits the need for external dependencies So Speaking of containment It's self-contained. So you say go build and it's good to go So when you run go build it produces a static binary Against a target operating system and architecture. So the resulting binary includes any external dependencies you have with your dependency management system and It'll run on systems that don't have the language installed. So it makes distribution of go projects super super easy and Finally, it's concurrent and asynchronous with go routines programs can handle tasks simultaneously And with channels and weight groups programs can execute tasks asynchronously So both of these properties improve performance and responsiveness for your program Regardless of the pros and cons of these characteristics as a whole they all have a different impact or implication on how you would write tests So tests like I mentioned in Structurally type languages are a lot more straightforward because your input and output types are already defined and You would probably have a compile time error if that was incorrect Therefore testing fixtures are a really common practice in go They can be read in from a JSON file or compiled in your go file directly But essentially they'll provide a quick reliable and easy way to invoke test artifacts in your tests The standard library makes writing tests really really convenient The testing package is built in and it's really huge. It's got Functions to make assertions and they can be made without any external dependencies It also has packages for mocks and things such as like a test HTTP server Which we'll get into a little later for more of them more involved integration tests Self containment how this relates to testing is it helps keep tests organized Unit and integration tests are written as go files and they live directly in your code These tests of course are not built and packaged and shipped when you run go build But you can easily run the test in a CI pipeline and the end test However capture more of a larger scope, but go build kind of empowers you to do that a little easier in a CI pipeline So you could go build and then run and end tests on that resulting binary The concurrent and asynchronous features of a go program are some of the biggest challenges to testing and go Because they're more prone to race conditions and blocking calls Therefore running these kinds of tests with a race detector and a timeout is typically encouraged So we'll get into some examples about how to deal with concurrent complexities and tests But for now, I hope I've convinced you the pros Outweigh the cons when it comes to go design and tests So whether you came in new to the language or not, I hope that background was useful Because it's gonna help us understand at greater depths You know the background of the language and help us write better tests So regardless of the language we choose even if you're not choosing go and you're just trying to get a good seat for the next session We still need to be able to recognize what actually makes a test good and what techniques we can use to write them simply and concisely So what are some properties of well written tests? Similar to a scientific experiment a good test will only test one thing at a time so control variables are designed to isolate the effects of a single independent variable on that it has on that outcome and Just as there are control variables and like a scientific experiment like this a test should also contain controls for each iteration So for each thing you're stressing You should have some constants around it Even in larger scope tests such as end-to-end testing you want to focus on Testing just the happy path or just a networking failure for example Otherwise you could end up with multiple points of failure, which makes it really difficult to isolate which component caused what? While we want to make sure we're only testing one single thing at once We also want to make sure we're testing many things together and that's not a contradiction. I promise software components can have many like entangled interdependencies So for this reason we should exercise as many permutations as possible in our tests So each permutation of code like in this image is going to be unique in context and order And although the permutations permutation itself is unique. It can still yield the same outcome as other tests so for example like having the same number of blues and reds in each column and Just because we have a redundant outcome doesn't necessarily mean that the test is bad And we're being redundant because we're actually Creating multiple success and failure scenarios Triggered by all of these different variables working together. So The variety of the code paths that get touched by these different combinations are what makes a test really really good Are you guys familiar with Marie Kondo in Europe some of us, okay So we want to be able to test a single thing at once and additionally multiple things together But how are we supposed to do that especially in systems that grow in complexity over time and often like per commit? Simply put we can Marie Kondo our code. She's a Netflix documentary person and she helps people like organize their garage and stuff It doesn't mean we want to erase the code that like doesn't spark joy like we still need our code But for example if a project Resembles a messy garage that hasn't been cleaned out in years It's going to be really difficult to verify the contents and usability of anything that's in there So on the other hand if we Marie Kondo to garage will probably have Transparent containers. They're clearly labeled organized by season or activity And they're easily accessible for when we need it. So the same goes for code in that metaphor Organization and compartmentalization will help simplify our code and therefore our tests So our unit tests are smaller the testing smaller things By dividing complex tasks and functions into smaller and more manageable pieces We can encourage some of that same code reusability and isolate those specific problem points So obviously those smaller code blocks are far easier to unit test Then large like run-on integrated code blocks that just start getting really like intertangled Another component to writing good tests is writing failing tests This tweet says you from another talk. I'm not sure where Says you should never trust the test. You haven't seen fail I'm not sure that Joe or Gwen here were the first ones to say that I don't know if they're gonna be the last ones to say that because I'm saying that now But we shouldn't just write tests to pass we should be making incorrect assumptions on our code So we can actually observe that failure So personally my favorite way of doing this in practice is with bug fixes So if I've narrowed down kind of to where I think the problem is I'll immediately write a test for it Without even attempting to like fix the bug Observe that it fails ultimately reproducing the bug that's reported And then I can Basically confirm that the code was behaving improperly before and properly once the fix is shipped So your code reviewers will thank you when your test explains exactly what's going on So that last slide is really just a funner and like longer way to say test-driven development Some might argue that it's just a vicious cycle of endless handoffs or maybe riddled with like developer bias But in many cases At least in unit case unit tests it will help you achieve really good code and really good coverage So in practice writing a test prior to fixing the bug or implementing a feature sounds great, especially for those smaller scope tests But in reality for real-world applications and larger scope tests They're not as easily testable through this method. So don't beat yourself up over it I'd recommend using a test-driven development when possible, but focusing more on shipping organize and compartmentalized code Which in the end makes it more testable So that's kind of my overview on writing good tests and best practices I write and speak a lot about testing in general So if you want more of that be sure to check out the resources at the end, but for now Let's shift into go tests specifically Before we start writing go tests, we'll kind of discuss how we would run a go test I mentioned earlier that going produces tools such as go build go run and go test and you guessed it go test is meant to run our tests So it includes other features such as benchmarking and parallelization Which we won't get into in this talk, but they're great resources Into gathering and reporting some more context about your code So to test you simply run go test from the directory where your tests are stored or you can utilize some other attributes of the tool So here's a couple use cases for how one might use the tool You can run all the tests in the current working directory You can run a single test against a fully qualified package name, which you have like the github path there You can tag go files with a simple comment at the beginning of the file So if you want to just run integration tests, you could tag all of your integration files with that and that's just going to run all of those I Think I skipped over dash v is just going to verbose the print any output any additional output From your go testing tool You can enable race detectors with dash race and timeouts for tests that might depend on blocking calls So if you're calling something with dash race, you're probably going to want to time out as well You can also get a snapshot of code coverage with dash cover I know code coverage can be a pretty controversial topic While I believe we should all be shooting for a hundred percent code cub at least on unit tests it's really not feasible in some instances and Writing high-quality code in meaningful tests is a lot more important than just checking that box and saying like okay a hundred percent code Cove we're good And lastly some editors such as VS code, which is what I use can even highlight the covered and uncovered lines of code if you run the tool through the editor So it's kind of really helpful to identify if your test is actually hitting those lines Some basic syntax and project details that you'll need to know about writing go test before we get started First is the file naming convention All shipable go code should be in a file ending in go just like that Well all non-chipped test code should have the postfix Underscore test go so it's still a go file. It just has that test When writing fixtures for your go-structs that are going to be using your tests I would actually recommend to organize those fixtures in the go file not the test file In the same package where that struct was defined because this is going to help limit circular dependencies that could arise When importing those fixtures in tests across packages Secondly the test naming convention a go test is written as a regular go function like this With the caveat that it always follows the same function signature So it's prefixed with the word test with capital T and it accepts a single testing dot t parameter Which is like the core testing package in go The first letter it should follow just like regular camel case. That's kind of goes Styling preference so the function name will identify that test routine like that test xxx and It should be capitalized So with all of the prerequisites we discussed in mind let's move on to writing tests and go We'll start out with very basic tests and work our way up to more complex use cases with like race conditions and all of that So paint patients as a virtue for the sake of Consistency I'm going to try and use the same example throughout so this Gophers getting pretty lit on beer beer seems to be pretty popular here in Belgium and I love beer So we'll go with that Our business model for our program can be an online beer store and subscription service For some context I created a few structs To represent each object in our business model So a cart is a shopping cart that contains a list of cases a case is a pack of beer that Can be a specific amount such as a six pack or a 30 rack and then a beer represents Details about the specific beer such as its brand its name its size and fluid ounces So on and so forth. So I've also written some preliminary fixture functions I've only pictured one here. There's some other ones in the code, but they're a little Intuitive once you see them in the tests And this is just going to allow us to rapidly initialize this struct So I don't need to take up six or seven lines of code creating this every time I want to create one of these objects Lastly the function we're going to test first is really simple It's called add case and it simply appends a single case of beer onto the list of cases in the shopping cart So our first test is pretty simple. We initialize a new empty cart again That function wasn't pictured, but it just initializes a cart object Before we do anything else. We're going to make sure the length is equal to zero with if statement and Call t dot fatal in the event that it fails So it'll print expected empty cart if for whatever reason there was beer populated in our cart to begin with. What's up? Yes typically yes only used for the test but if you if you want to use that fixture in Other packages like when you're say I'm testing in another package and I want to use that fixture I can't export that in the test in the test code itself It'll only look for the code in the go test or the go file. I'm sorry. Does that make sense? Okay, so custom error We'll create fixtures here So we can add those fixtures into our add case function The bat blue light is my favorite beer. It's a Canadian Pilsner Typically found in 12 ounce cans. So we'll fill that up in the case and If it works properly our test pass the second assertion as well because we're making that if statement on the length of the cart So as I just demonstrated in that last slide using the go testing package alone We can explicitly mark tests as past or failed depending on the if conditions This makes code more human readable But it can kind of create all of these like dangling conditionals that make it a little bit more tiresome And we're going to be writing a lot of tests today. So I want to introduce you to testifies assert and require packages Testify is a lightweight external dependency that acts as a testing toolkit around the go standard library testing package So it wraps the testing object T That's your top-level parameter in all of your tests and it extends common assertions such as equal not equal Error no error nil not nil and many other conditions Every function takes that object T as the first argument and as a result it can write those same errors out to the go tool the Assert and require functions also return booleans indicating if the assertion was successful or not So if you want to build off of each assertions and only make Assertions based on like what you just previously asserted you could do that. We're not going to do an example of that here We're just going to ignore and swallow the returned booleans, but for future reference if you have special code paths I would recommend that So the same test we just did it can be written in a few less lines of code simply by invoking testifies assert package So rather than making that if statement we can just assert equal the length of the cart and have This empty or expected empty cart message. So assert can take Three to four parameters the first being T the second parameter being the expect expected value the third parameter being the Actual value and then the fourth parameter being an optional like custom error message So that's what it'll print to the go test tool in the event of an error The require package is almost identical to the Assertion or the assert package the only difference being that if you call require dot Equal and that fails. It's similar to running T dot fail now which will short circuit the test in the event that it fails and it won't continue on So I'd recommend you kind of use your best discretion when deciding between an assert condition verse require condition But for any like non catastrophic failure scenarios It's probably easier to use assert so any failed test cases in some algorithm or some mathematical equation that you're computing Will still be surfaced despite any failures prior So we talked about test-driven development a little earlier now that we've gotten our feet a little bit wet with some go examples We're going to see it in practice So let's say I want to write a function called subtotal that calculates the subtotal of the contents in my shopping cart Instead of jumping directly to the dot go file. I'm going to start with the test dot go file writing a case that I know is going to fail Writing it before my code is going to encourage me to adhere to the original Expectations I have for the function and probably the original function signature as well so Test subtotal similar to the add case test I can fix your beers and cases and add them to my shopping cart Google tells me a duvel triple hop is a very popular Belgian beer and although it's quite expensive for a pack of four I'll be worldly and try it out and throw it in our cart so We also might be playing some drinking games later. We need a 30 rack of LeBat blue For 2499 so I'm expecting the subtotal function here Which we haven't yet written to accept zero parameters, so empty parentheses and Return a single value, which is a float If this function works as I would expect it to the calculated value should be 3998 but let's round up to 40 for the sake of this test so we can first assert the failure and then fix it later on So our test assumed the function signature was shown here. So this is how it should be defined So it should plug and play perfectly We're going to do some really intense math here to calculate the subtotal We'll iterate through each case in the cart adding the price to the subtotal and then return that value So it fails with an expected Value of 40 and passes with the corrected value of 3998 So while TDD kind of gets us to decent coverage in the small helper function I'd like to believe we're a little bit more thorough than that and we want to test more than one instance of the calculation so Although it may have passed like what happens if our cart is empty what happens if there's a negative value for some reason more than two cases duplicate cases in our cart and Then lastly, how can we capture all of these edge cases while simultaneously optimizing our code? reusability So for this purpose a common practice for unit tests and go are called test tables so you can create a test table which is basically a slice array of Input and output variables Defined directly within the test. So the struct doesn't have to exist anywhere else. It's just right here We can take the name of our test It'll accept a cart as an input value and the output value for the test case is going to be the subtotal Which is that float? So using normal go syntax and logic we can actually just iterate through all of the test cases here and T.run is going to Run each function as a subset or subtest of T in a separate go routine, but that go routine will block until each test is finished Doing this allows us to reuse this line of code But if you have longer tests if it's an integration test, it's going to be nice that you have Your test tables here because you're only going to have to define your testable stuff once So instead of it writing an entirely new test we can validate all of these multiple Edge cases as subtests so empty values multiple values negative values. They all abide by the same function signature, and we're good to go I'm not sure under our business model, but circumstances would create a negative value or like a free beer But in this case We're at least like aware of what we need to add and we probably just need like some additional validation in Either our fixture or in the struct itself So while we aim to compartment compartmentalize our code we can do the same for our tests So as a code reviewer, it's a lot more clear to identify like what your tests are actually trying to accomplish When they're broken out like this So the more complex the setup and verification the larger trade-off you get with some of this code reusability HTTP test is another standard library package which provides utilities for HTTP testing The new server function starts and returns a new server the caller of course being responsible for stopping and cleaning up that server It gets created with a unique URL So it's important that the component that you're testing has a way to pass that variable in so we can make our test against the test server Rather than a real server The function also accepts a HTTP handler that gets invoked each time the URL is called so we can Basically mock out the server and have it return different status codes So for example, let's think about how our business model might interact with like an external or arbitrary HTTP API So say we have a third-party service to process payments We can simply just do an HTTP post such as this to send the payload there It could include other things such as like the total amount That's due any credit card information if it's secure And whatever else the payments are we needs And this function as is is actually really difficult to test because pay.me is hard-coded directly Within the function itself, so we have no control over what the server is going to do So we wouldn't want to do this and Then it would just leave us really prone to like intermittent service errors and we wouldn't be able to actually like change the outcomes of the service so Passing the server address as an additional parameter is much more suitable for this test So the payment server here, it could still be a constant, but we're just going to pass it at the highest scope in the function So this will allow us when we're testing a process payment to pass the URL of the test server up here And we can control the custom handlers that are going to respond So as such we can simulate things such as auth errors server errors and of course like a successful payment processing So if we have more code down here that like is dependent on the response of HTTP post we can exercise those code paths as well The handler func type that you see up here is defined in the HTTP package And it basically is an adapter that's going to allow us to use ordinary functions that we write as HTTP handlers So if it abides by the appropriate signature here with a response writer and an HTTP request We'll be able to pass it in as the handler The signature it looks maybe a little weird if you've never seen this before But it's the same signature that if I was writing my own a go server I would run the function serve HTTP and it just mimics that as a real server So in each test case we'll write a custom function here and these two different handlers One for exercising a 200 status code So we'll do that with the right header function and we'll pass the status Okay, and then W dot right. Let's us pass a response body Through the test server and then here for our internal service error We'll just write that header and it won't have a response body Kind of similar to how a server might act in the wild And each subtest we'll start by initializing a new HTTP test server right here This handler funk is from that test case. So those little custom handlers we built Immediately we want to make sure that we defer T s dot close which means at the end of all of our tests or at the end of each iteration It's going to close the test server So it's ready to create a new one So for both of the test case that we defined in the last slide the expected errors And the expected body are asserted as well So it should match up But it actually doesn't and it's kind of a caveat to how the HTTP post method works Does anyone know why our test might fail? So HTTP post actually doesn't return If if it's like a 500 response code it still returns a nil error because it was actually successful in processing the post command so What we actually need is Some error checking or some checking around the response status code So if anything greater than a 400 we want our actual function to return that error So we can have this custom payment server error and give us what the status code is so The previous test would fail under I'm sorry it would pass under these conditions now and remember never trust the test You haven't seen fail So that caveat kind of reminds me of another gotcha that I've come across in the go world and it has to do with contexts Specifically how it relates to text testing So the context package in go is responsible for carrying like deadlines and cancellation signals across API boundaries and processes It's extremely powerful, but equally as important to understand So you want to initialize a parent context and you can call one of these two methods to do that They actually both initialize a non-nil empty context the main difference being Context dot background is typically done in the main function initialization and tests whereas to do kind of acts more as like a to-do like comment with the Idea that you will eventually like replace it in the future So maybe at the time that you're writing the function you don't have a context That's actually being passed down through a higher scoped function So you'll put a to-do there or your unsure of which context is appropriate to use But the intention is that we're going to replace that So it doesn't really make sense to use it to do in tests because we have no intention on replacing it And the test is already at the highest scope so If you ever notice inconsistent uses of to-do and background in tests that you're reviewing in the future Save someone from getting nerds sniped and direct them to background While they're programmatically the same if you use them correctly You can actually use like static analysis tools and it'll help validate that you're using and passing around context through your components Correctly, so it'll help surface problems early on So now that we know what context to use and when we can start creating functions which require them and test them out So context should always be the first parameter in a go function So in this start subscription timer, it's the first and only parameter Here I've implemented a subscription timer, which basically will listen for a context cancellation signal if it's done will just return from this function if not on every Interval of this ticker it will send a cart Along the message channel So you see it doesn't return here after so each each time that timer or that ticker expires It'll keep sending those messages on the channel So as a beer subscription service will have multiple orders to fulfill concurrently So we'll likely have to call start subscription timer through a go routine for each active subscription When the order is ready to be processed We just send it on that message Chan and we'll have another component of our code actually responsible for receiving those messages So how do we test channels? To test this function will create two unique carts there at the top For the sake of this test it makes sense to have a short interval duration such as one second or even like a higher granularity Like millisecond duration But that's how we set it through the subscription itself The timer is designed to run continuously and it doesn't return So like I said, we need to run this as a go routine in the test if we don't it's just gonna sit there and block and likely time out So our function is sending values along the channel But we need to verify that the correct message is actually being received on that message Chan so this channel syntax here is just gonna allow us to Just like pop that Message out of the channel and then we'll do a type assertion to make sure that it's actually a cart object and assert that It's equal to this cart up here So I've done this twice because I want to make sure that we can test consecutive Items and that the contents of this item is unique to the one that we defined earlier Now that we've verified our program can send values on a channel we want to work on a concurrent component That's designed to receive them. So we have this start order handler function It's again designed to continue continuously wait and handle any Messages that are received up here. So that same syntax that we use in our test For each cart that it receives on the message channel, it's gonna attempt to Place the order in this arbitrary function That's basically just going to iterate something in our order handler So if it fails, we'll log, but we'll continue on and stay in here If there's any type assertion failure, we'll log that as well The only time we're actually going to exit this program or this go-routine is if the channel itself is closed Which we can detect up there To test this asynchronous behavior We can simply start the order handler in a go-routine and send a few objects on the message chain So we start the function here Assert that the length is equal before we even start sending anything and then we send a couple messages on the message Chann you can see that this is a case object not a cart object, but we are guarding against Wrongful like type of assertions in our code So we know that there's only two carts that are processed and the length is equal to two. So this message isn't actually received Well, the message is received, but it doesn't count in our length So at this point we have two asynchronous and concurrent components of our project our subscription timer and our order handler They're both Basically small enough to fit in a single slide, which is kind of a good indicator that they're broken down and compartmentalized Which again, we said was really good So breaking things out is just going to make our lives easier in the future In a nutshell we have a working program with pseudocode that demonstrates some of those basics for writing go tests But we have yet to polish up on one of the harder concepts in go, which is race conditions and race detection Introducing that complexity kind of increases vulnerabilities such as race conditions whenever we're using concurrency such as go-routines and Channels I'm going to try and speed up a little bit again. We can use the dash race argument to detect race conditions Where Yeah, we'll just skip over that So we already have our tests written And I actually have a race condition in one of them already It was somewhat intentional. It's fine But there's a couple techniques that we can use to resolve unsafe memory access Such as blocking with weight groups blocking with channels returning a channel or using a mutex Some of these solutions are going to require me to refactor code and I don't really have time for that So we'll look at the stack trace and try to identify why it's failing Basically, this is just what it looks like it's saying on line 123 and line 138 in my go file on my go test that's what is Problematic and comparing these two lines. You can see that I'm attempting to read from the subscription cart in the main function and then in the test I'm trying to set the subscription cart and Mutate it and write to it. So that's really unsafe, especially when you have multiple concurrent processes that could access this So in this case, we're going to want to use a mutex To protect against us So the sync package, I wish I had a little bit more time to dive into the synchronization that it can offer, but it's basically intended for use by like low-level library routines Mutex stands for like mutual exclusion lock. So basically you can lock a value and unlock a value and locking it will Block any like reads and writes to that value until it's unlocked So we only need to make a couple more changes to make that test pass First we'll create we'll make sure the cart and interval variables are not exported So we can't access them outside of our package We'll add a mutual Mutual exclusion lock and then any time we want to we'll create those getters and setters and any time We want to either get the value of the cart or set the value of the cart We will lock it set it get it and then unlock it So that we'll just do that with the defer statement here. Sorry. I know we went over that a little bit fast But lo and behold synchronization protects our concurrent functions and the race detector okays our tests with just those simple changes such as a mutex so sync is just one of the many packages that We went over today and it's really really important. So I would definitely suggest going over that a little bit Sorry, I'm rambling on but I really appreciate you guys listening to me at nerd out on testing and Golang All of the examples you saw are up on my github, which is a nicky ex dove and Then a special thanks to all of these folks that contribute to open source with their gopher con or their gopher illustrations So thank you guys. I don't know if you have time for tests, but come find me out after


Just a beautiful day Gym time dunk day. What's good? Just had an amazing podcast with my boy Leo. I don't know when it's posting but you know, it was so good Boy Leo started his brand new podcast. I was his first interview. It was amazing It was just crazy how good it went. He asked me great questions. I'm really looking forward to that episode dropping I'm gonna post it on my own podcast too. It was just awesome. It was I got to say a lot of stuff I like to talk a lot. He asked great questions.  I'm really looking forward to going hard But I'm also going to do what I intended to do last session Which is focus on my one step and drop step and if it builds up to full speed I'll do that again But last time I tried to go full speed, but I couldn't because of the foot So it shouldn't be a problem today should be really good. My standing vert was great last session I'm going to keep working on that because I want that power. I want to dunk off vert I want to dunk in game super easily. So I need that power. I'm always about speed, but I need that power I do have strong power when I jump, but all of my max jumps are like full speed running so fast I want to be more controlled and I want to build that power power power. Okay. That's the plan Thanks. What's good youtube? What's good world? Haha. Yeah, okay Oh god, it's so nice. Oh, son nutrition body love dude. That's a perfect day This is foot two people dunked off his right leg one foot Easy two hands. That's a new level for me to see that. What's up, Luke?  They miss you Hi, how you doing? I don't know if it's in here, but I was just doing the eyes out I'm a little I'm a little petite Feeling good like kind of height wise, but no energy, but I'm glad I can still do that with power That's just pure power. No energy anything like that I hope today I have no energy, which is normal because three days ago when I dunked I jumped more than I have in a long time And that's why I said that so many times in the last video because That's really strenuous on my central nervous system and just using all those muscles is a huge workout So three days later to still feel fatigued is natural But uh feeling good about today feeling good to be back cooping almost kind of put back out of nowhere Feel like death That's a beautiful day. I hope you guys are enjoying your day Be grateful just to be alive.  Let's get it Enjoy your day. You know, it's just another one. We're just kind of out of you're doing it Kind of like crawling around on this planet


I am Dakar Gelbner. When you take a deep breath in and out, it activates the vagus nerve, which is this bundle of nerves that starts at the top of your spinal cord, wanders through your throat, drops into your heart and your lungs, and then moves into your digestive organs and your immune system. An active vagus nerve is associated with better health, better connections with others, and greater strength in handling stress. My lab has found that when you practice compassion, it actually activates your vagus nerve. Compassion is the feeling of concern that we have for people who are in need or who are suffering, and it really moves us to take action to lift up their welfare. Charles Darwin, the great scientist, believed that compassion was our strongest instinct, and communities that practice the most compassion thrive the best. So let's do a little bit of compassion. If you could just sit down, close your eyes, and rest your hands on your knees. Let's take a nice deep breath in, expanding our chest.   Breathing out, relax your brow and your jaw.   And now in your mind, direct these words towards this person. Breathing in, may you be filled with loving kindness. May you be safe from inner and outer danger. Breathing in, may you be well in body and mind. And finally, thinking of this person, may you be at ease and happy. Okay, you can open your eyes if you close them. The great philosopher Peter Singer wrote about the circle of care, which is the people or sentient beings that we extend our compassion to. And that one of the great tendencies in life is to strengthen our circle of care. For more on compassion and kindness, go to greatergood.berkeley.edu. Thank you.


I was starting my 25th year of life and in those days Saigon was a very tense city. Just before I arrived at a bar or in the restaurant in the city center of Saigon I had been bombed and the 30 or 40 killed. This was the characteristic of the Vietnam War. The front was nowhere and was everywhere. For us, this was a situation which was very tight because we had always to show that we were Swiss and we were Red Cross and for a long time this was a very strong argument for us. The main mandate of us was to visit six prisoners of war camps spread all over South Vietnam, the biggest being on the Phu Quoc Island off the coast of Cambodia. We had 35,000 prisoners of war, Viet Cong and North Vietnamese prisoners and we visited these camps every three to four months and made reports, made recommendations and tried to improve the living conditions of these prisoners. We went to each camp, we spoke with the prisoners and told them, you see, we have tried to do what we could. How do you perceive our action? They said amazingly, you know, for us, the material factors the improvements you obtained were not so important but you were the only, only institutions and the only individuals which came from outside to be in contact with us. We were occasionally asked to organize relief organizations when populations were, for instance, bombed out of the jungle when they were staying in temporary locations. We wrote their relief together with the South Vietnamese Red Cross. So I believe also the Vietnam War was a very good example for an institution like the ICRC to do something or to do whatever could be done under very adverse conditions. In our case, what was the biggest challenge which was also the biggest killing factor were these bombings on civilian zones in the South and North Vietnam. The new weapons which were tested Agent Orange was used to defoliate the jungle but there were many other new weapons used. We knew because we were overflying these zones sometimes the following morning they looked like lunar landscape and we said, you know, all these people who have died down there this is not normal, this should not occur, we should do something but we were blocked. I believe that if anything happens they know that we would be active on both sides according on the basis of the same criteria. I believe that Vietnam, given the geopolitical context of Southeast Asia with an expanding big nation there has come to realize that the neutrality we were always fighting for is not a fiction.