You have been here on stage before. You successfully tempered with the Wii. You successfully tempered with the PS3 and got some legal challenges over there. Some unfounded legal challenges, yes.   And well, console hacking 2016. Yes. Here we go. So I'm a lazy guy so I haven't turned on my computer yet for the slides so let me do that. Hopefully this will work. So my computer is a little bit special. It runs a lot of open source software. It runs FreeBSD. It runs, even has things like OpenSSL in there and Nginx and Kyro I think. And WebKit. It runs a lot of interesting open source software. But we all know that BSD is dying so we can make it run something a little bit more interesting and hopefully give a presentation about it. Let's see if this works. It's a good start, black screen, you know. Like it's thinking to disk and file system shutting down. There we go. And yes, I run Gen2 Linux. This is the does Wi-Fi work moment. Hopefully. NTP. Yeah, no, NTP failed. Well, that's a bit annoying, but it still works. Hello. Yeah, it takes a bit to boot. It doesn't run system D, you know. So it's saying it's a tiny bit slower, but it's saying there we go. This is the does my controller work moments. Bluetooth and the install one. Okay, it does. All right, so let's get started. So this is console hacking 2016 PS4 PC master race. I apologize for the horrible Nazi joke in the subtitle, but it's a red thing. So yeah, PC master race. Why? Well, PS4, is it a PC? Is it not a PC? But before we get started, I would like to dedicate this talk to my good friend, Ben Beyer, who we all knew was bushing. Unfortunately, he passed away in February of this year and he was a great hacker. He came to multiple congresses. One of the nicest people I've ever met. I'm sure some of you have met him would agree with that. And if it weren't for him, I wouldn't be here. So thank you. So the PS4, is it a PC? Is it not a PC? Well, it's a little bit different from previous consoles. It has actually six. It's an actually six CPU. It runs free BSD runs webkit. It doesn't have a hypervisor, unfortunately.  So there you go. And so this is different from the PS3, but it's not completely different. It does have a security processor that you can just ignore because it doesn't really secure anything. So that's good. All right. So how to own a PS4?  Duh, right? And everything runs webkit and free BSD is not exactly the most secure OS in the world, especially not with Sony customization. So this is completely boring stuff. Like what's the point of talking about webkit and free BSD exploits? Instead, this talk is going to be about something a little bit different.  What is this about? And not only that, though, before you write an exploit, you usually want to have the code you're trying to exploit. And with webkit and free BSD, you kind of do, but not the build they use and it's customized. And it's annoying to write an exploit if you don't have access to the binary. So how do you get the binary in the first place? How do you dump the code? That's an interesting step. So let's get started with step zero, blackboard to code extraction, the fun way. A long time ago, in a hackerspace far, far away, failure to follow got together after 31c3. And we looked at the PS4 motherboard and this is what we saw. So there's an ALU, a Southridge, that's a code name, by the way. Then there's a Liverpool APU, which is the main processor. It's a GPU and a CPU, which is done by AMD. And it has some RAM.  The internal disk on the PS4 is USB. Like it starts out to USB and then to USB on the Southridge. Even though it has SATA, like what? The Blu-ray drive is SATA. The Wi-Fi Bluetooth has the IO and the Ethernet is GMII. Okay, how do we attack this? Well, GDDR, what just? Oh, I have a screensaver apparently. That's great. I thought I killed that. Let me kill all the screensaver real quick. Something had to fail. It always does. I mean, of course, I can SSH into my PS4, right? So there you go. Okay, I could have sworn I fixed that. Anyway, so yeah, which one of these interfaces do you attack? Well, USB SATA is the IO, GMII, that's the raw Ethernet interface, by the way. All of these are CPU-controlled. The CPU issues, commands, and the devices reply. The devices can't really do anything. So you can't write to memory or anything like that. You can exploit USB if you hide the bug in the USB driver, but we're back to the no-code issue. So GDDR5, that'd be great. We could just write to all memory and basically own the entire thing, but it's a very high-speed bus. It's definitely exploitable. If you're making a secure system, don't assume we can't own GDR5 because we will, but it's not the path of least resistance. So we're not going to do that. However, there's a thing called PCI Express in the middle there. That's interesting. PCI is very fun for hacking, even though it might seem intimidating, because it's bus mastering that means you can DMA to memory. It's complicated and complicated things are hard to implement properly. It's robust. People think that PCI is this voodoo high-speed. No, it's not. It's high-speed, but you don't need match traces to make it work. It will run over wet string. Like, you can hotwire PCIe with pieces of wire, and it will work, at least at short distances anyway. Believe me, it's not as bad as you think. It's delay-tolerant, so you can take your time to reply.  So what can we do? Well, we have a PCIe link. It's cut the lines and plug in the Southridge to the motherboard, to a PCIe motherboard that we stick on the side. So now the Southridge is a PCI card for us, and we connect the APU to an FPGA board, which then can pretend to be a PCIe device. So we can man in the middle of this PCIe bus, and it's now times one width instead of times four because it's easier that way, but it'll negotiate. That's fine. So how do we connect the motherboard on the FPGA? There's, of course, many ways of doing this, but how many of you have done any hardware hacking, even Arduino or anything like that? Raise your hand. I think that's about a third to a half or something like that, at least. And when you hack some hardware, you melt some hardware. After you blink an LED, what is the first interface you use to talk to your hardware? Serial port. So we run PCIe over RS232 at 115 kilobaud, which makes this PCIe... I said it was delay tolerant. So it makes this PCIe 0.0002x and eventually there was a gigabit ethernet port on the FPGA, so I upgraded to that, but I only got around to doing it in one direction. So now it's PCIe 0.0002 in one direction and point point in the other direction, which has to make this one of the most asymmetric buses in the world, but it works. Like believe me, this is hilarious. You can run PCIe over serial. Also, we were asking coding, so half the bandwidth. Yeah, it's fine. It's fine. So PCIe 101, it's a reliable packet switch network. It uses thing called transaction layer packets, which are basically just packets you send. And it can be memory read and memory write, IO read, IO write, configuration read, configuration write. It can be a message signaled interrupt, which is a way of saying, hey, listen to me, by writing to an address in memory, because you can write the things, so why not write for interrupts? It has legacy interrupts, which are basically emulating the old set this wire low for interrupt and high for no interrupt thing. You can tunnel that over PCIe. And it has completions, which are basically the replies. So if you read a value from memory, the completion is what you get back with the value you tried to read. Okay, so it's PCIe, right? So we can just go wild with DMA. We can just read all memory dump. For kernel, hey, it's awesome, right? Except there's an IO and memory in the APU. But of course, the IO and memory will protect devices. It will only let you access what memory is mapped to your device. So the host has to allow you to read and write the memory. But just because there's an IO and memory doesn't mean that Sony uses it properly. So here's some pseudo code and, you know, it has a buffer on the stack. It says, please read from flash to this buffer with the correct length. Can anyone see the problem with this code? Well, it maps the buffer and it reads, and then it unmaps the buffer. But IO and memory don't just map byte food to byte bar. They map pages, and pages are 64K on the PS4. So Sony has just mapped 64K of its stack to the device. So we can just DMA straight into the stack, basically the whole stack, and take over. So now we get code execution, FreeBSD, kernel dump, and Webkit and IO slips dump, just from mapping the flash. Okay, that's step zero. So we have the code, but this is not, you know, that's not the PS4 that we did this on. It's a giant mess of wires.  But we're going to make a nice exploit. We've done that because, as I said, Webkit, FreeBSD, whatever. What comes after that? Okay, we want to do something. Of course, we're going to run Linux. And how do you go from FreeBSD to Linux? It's not a trivial process, but you use something that we call PS4K exec. So how does this work? It's, you know, it's simple, right? You just want to run Linux? Just jump to Linux, right? Well, kind of. You need to load Linux into contiguous physical, run set up boot parameters, shut down FreeBSD cleanly, hold secondary CPUs, make new page tables, disable, blah, blah, blah. A lot of random things.  Now, let's assume that you do all this magical cleanup and you get Linux into a nice state and you can, you know, jump Linux. Okay, now we jump Linux, right? It's cool. Yeah, okay. You can technically jump to Linux and it will technically run for a little bit. And then it'll stop. And you're not going to get any serial or any video or anything. What's going on here? Okay, let's talk about hardware. What is x86? x86 is a mediocre instruction set architecture by Intel. It's okay, I guess, you know. It's not great. It's okay. The PS4 is definitely x86. It's x86-64. What is a PC?  And the PS4 is definitely not a PC. Then again, that's practically Sony-level hardware fail, so it could be, but it's not. Okay, so what's going on? Well, a PC, a legacy PC, basically has an 8259 programmable interrupt controller, a 253-programmer interval timer, a UART at IO3F8, which is the standard address for a serial port. It has a PS2 keyboard controller, 842. It has an RTC, a real-time clock, with the CMOS. Everyone knows the CMOS, right? MC146818 is the chip number for that. An ISA bus. Even if you think you don't have an ISA bus, your computer has an ISA bus inside the Southridge somewhere, and it has VGA. The PS4 doesn't have any of these things. So what do we do? Well, okay, let's look a little bit at how a PC works and how a PS4 works. This is a general, simple PC system. There's an APU or an Intel Core CPU with a Southridge, Intel calls it PCH, AMD, FCH.  A PS4 kind of looks like that, right? So you'd think this can't be that. What's so hard about this?  So the PS4 has a Southridge, right? So the Southridge, the AMD standard FCH implements Intel legacy from 1981. The Marvel Aeolia, Marvel is the maker of the PS4 Southridge, implements Intel legacy from 2002. What does that mean? Ah, that's no Southbridge. That's a Marvel Armada SOC. So it's not actually a Southbridge. It was never a Southridge. It's an ARM system on a chip, CPU with everything. It's a descendant from Intel's strong ARM or X-Scale.  And what they did is they stuck a PCIe bridge on the side and said, hey, EX86, you can now use all my ARM chip. So it exposes all of its ARM peripherals to the EX86. They added some stuff they really needed for PCs and has its own RAM. Why do they do this? Well, it also runs 3B on the ARM in standby mode. And that's how they do the whole download updates in the background, get content, update, whatever. All that crap is because they have a separate OS on a separate chip running in standby mode. Okay, that's great, but it's also batched insane. Yeah. So quick recap. This is what a PCI bus number looks like. It has, sorry, a device number. So it has a bus number, which is 8 bits, a device number, which is 5 bits, and a function number, which is 3 bits.  This is what a regular Southridge looks like.   So one Southridge pretends to be multiple devices because you only have 3 bits for a function number. So you can only have up to 8 functions in one device.  If you LSPCI on a roughly unpatched Linux kernel on the PS4, you get something like this. So the Aeolia, first of all, clones itself into every PCIe device because they were too lazy to do if device equals my number, then reply, otherwise don't reply. No, they just said, oh, just reply to every single PCIe device that might query. So Linux sees the Southridge, you know, like 31 different times, which is kind of annoying because it gets really confused when it sees 31 clones of the same Southridge. And then it has 8 functions, ACPI, Ethernet, SATA, DMC, PCI Express. Okay, 8 functions, so all 3 bits. It turns out 8 functions are not enough for everybody. Function number 4, PCI Express Glue has a bridge config, MSI interrupt controller, ICC, we'll talk about that later, HPET timers, flash controller, RTC timers, 2 serial ports, I2C, all this smashed into one single PCIe device. So Linux has a minimum system requirement to run on anything. You need a timer, you need interrupts, and you need some kind of console. The PS4 has no PIT and no PIC and no standard serial, so none of the standard PC stuff is going to work here. The board has test points for any 250 standard serial in a different place. So we want Dmessage over that. Okay, fine. Linux has Erdicon, which we can point to a serial port and say, please send all your Dmessage here very early, because I really want to see what's going on. It doesn't need IRQs. You set console equals to your 8250, you set the type, the address, the speed, and you'll see it says 3200 instead of 115 kilobot. That's because their clock is different. So you set 3200, but it really means 115K. And that gets you Dmessage.  That's pretty good. Okay, we need a timer, because otherwise everything explodes. Linux supports the TSC, which is a built-in CPU timer, which is super nice and super fun. The PS4 has that. But Linux tries to calibrate it against the legacy timer, which on the PS4 doesn't exist. So that's fail. So again, the PS4 really is not a PC. So what we need to do here is define a new sub-architecture, because Linux supports this concept, says, is this not a PC? Is this a PS4? The bootloader tells Linux, hey, this is a PS4. And then Linux says, okay, I'm not going to do the old timestamp calibration. I'm going to do it for the PS4, which has a special code that we wrote that calibrates against the PS4 timer.  Okay, so now this is not a PC. This is officially not a PC anymore. Okay, now we can talk about ACPI. You might know ACPI for all its horribleness and all its evilness and all its Microsoftiness. But ACPI, most people associate with suspends and suspend and hibernate. It's not just power. It has other stuff too. So we need ACPI for PCI config, for the IOMMU, for the CPU frequency. The PS4, of course, has broken ACP tables, because, of course, it would be. So we fixed them in PS4K exec. Okay, now interrupts. We have timers. We have serial. We fixed some stuff. The PS4 does message signaled interrupts, which is what I said, the non-legacy, the nice new thing, where you just write a value. And what you do is you tell the device, when you want to interrupt, please write this value to this address. The device does that. And the CPU interrupt controller sees that, right, and says, oh, this is an interrupt, and then just fires off that interrupt into the CPU. That's great. It's super, you know, super fast and very efficient. And the value directly tells the CPU that's the interrupt vector you have to go to. Okay, let's see. That's the standard MSI way there, that your computer does MSI that way. This is how the PS4 does MSI. The AOLIA ignores the MSI config registers in the standard location. Instead, it has its own MSI controller, all stuffed into function four, which is that glue device, yeah, glue. Each function gets a shared address in memory to write to, and the top 27 bits of data, and every sub function, because you count a lot of things into one place, only gets the different five bits. And all MSIs originate from function four, so, like, this device has the fire interrupt and it goes to here, and then that device fires an interrupt.   And the eyes are missing in the front there. But, yeah. So, yeah, driver hell. Now, the devices are interdependent, and the IRC vector location is not sequential, so that's not going to work, and you need to modify all the drivers, and, like, this is really painful to develop for. So, what we ended up doing is there's a core driver that implements an interrupt controller for this thing, and then we have to make sure that loads first before the device driver, so that Linux has a mechanism for that, and we have to patch drivers. Some drivers, we patch to use these interrupts, some drivers, we raft around to use these interrupts. Unfortunately, because of the top bit thing, everything has to share one interrupt within a function. Thankfully, we can fix that with the IOMU because it can redirect interrupts, so you can say, oh, interrupt number zero goes to here, one goes to here, two goes to here. So, that's great because it's consecutive, right? Zero, one, two, three, four, five is obviously going to have the same top bits, but we have to fix the ACP table for that because it's broken. But this does work, so this gets us interrupting that function and their individual. So, look at the checklist. We have interrupts, timers, early serial, late serial with interrupts. We can get some user space.  Like, okay, what's going on there? So, it turns out that free BSD masks interrupts with an AMD proprietary register set, we have to clean that up too, and that fixes serial and all the other interrupts. This took ages to find. It's like, why interrupts on CPU zero sometimes don't run? Yeah. I ended up dumping register sets and I saw this FFFF here and not FFFF. What's that? But yeah, like tracking through the stack to find this was really annoying. All right, so we have the basics. We have like a core platform.  So, we have USB XHCI, which has three controllers in one device. Again, just, you know, let's make it insane. We have SDHCI. That's a SDIO for the Wi-Fi and the Bluetooth. Needs a non-standard config, needs quirks. Ethernet needs more hacks. It's still partially broken. It only runs at gigabit speeds. If you plug in 100 megabit switch, it just doesn't send any data. Not sure why. And then all of this works fine in the next 4.4 and then just three days ago, I think I tried to rebase on 4.9 and so we have the latest and the greatest. And everything failed and DMA didn't work and all the drivers were just throwing their hands up in the air and what's going on here? Aeolia strikes back. So, that's what, you know, the, you know, the Aeolia looks like normally. So you have, again, it's an arm sock. It's really not a device. It's like its own little system, but it maps its low two gigabytes of the address space to memory on the PC and then the PC has a window into its registers that it can use to control those devices. So, the PC can kind of play with the devices and the DMAs to the same address and that works great because it's mapped in the same place and then has its own RAM, you know, in the total address space. This works fine, but now we add an IOMMU because we needed it for the interrupts and the IOMMU inserts its own address space in between and says, okay, you can map anything to anything you want. It's great. You know, it's a page table. You can say this address goes to that address. Linux 4.4 did this. It would find some addresses at the bottom of the IOMMU address space, say, you know, page one goes to this, page two goes to that, page three goes to that and say device, you can now write to these pages and they go to this space in the x86. That works fine. It turns out Linux 4.9 or somewhere between 4.4 and 4.9 it started doing this. It would map pages from the top of the IOMMU address space and that's fine for the IOMMU, but it's not in the window in the AOLIA. So now, you know, you say, ethernet DMA to address FE something, something, something and instead of DMAing to the RAM on the PC and DMAs to the RAM on the AOLIA, which is not going to work. So, yeah, effectively the AOLIA implements 31-bit DMA, not 32-bit DMA because only the bottom half is usable.  Seriously. And this is littered all over the code in Linux so this needed more patches and it works, but, yeah, painful. Okay, devices, devices work now for something completely different. Who can tell me who this character is? That's Starshot from Space Battleship Yamato and apparently that's the code name for the PS4 graphics chip. Or at least that's one of the code names because they don't seem to be able to agree on what the code names are. Like, it's got Liverpool in some places and Starshot in other places and Fabie J in other places and we think Sony calls it Starshot and AMD calls it Liverpool, but we're not sure. We're calling it Liverpool everywhere just to avoid confusion, but yeah. Okay, what's this GPU about? Well, it's an AMD CIs generation GPU, which is spelled CI instead of SI because S was taken. It's similar to other chips in the generation, so at least it's not a batshit crazy new thing, but it does have quirks and customizations and oddities and things that don't work. What we did is we took Bonair, which is another GPU that is already supported by Linux in that generation and just added a new chip and said, okay, do all the Bonair stuff and then change things and hopefully adapt it to the PS4. So hacking on AMD drivers. Okay, well, they're open source, but AMD does not publish register docs. They publish 3D shader and command queue documentation so you get all the user space 3D rendering commands that's documented, but they don't publish all the kernel hardware register documentation. That's what you really want for hacking on drivers, so that's annoying. And you're thinking, the code is the documentation, right? Just read the Linux drivers, that's great. Well, yeah, but they're incomplete and they have magic numbers and you don't know if you need to write a new register that's not there and it really sucks to try to write a GPU driver by reading other GPU drivers with no docs. So what do we do? We're hackers, right? We Google. Every time you need information, hopefully Google will find it because Google knows everything. And any tidbit you can find in any forum or code dump somewhere, that's great. One of the things we found is we Google this little string, R8XX GPU in quotes. And you get nine results. And the second result is this place. It's a silicon kit token, okay. It's an XML file. And if we look at that, it looks like it's an XML file that contains the dump of the Bonaire GPU register documentation. But it's like broken XML and it's incomplete. It stops at one point. But what's this doing here? And what does this come from, right? So let's dig a little deeper. Okay, Google, what do you know about this website? Well, there's some random things.  Oh, sorry, Excel, like XML style sheets. And then there's a thing at the bottom there called rai.grammar.for.txt. Hmm, I wonder what that is. And it looks like it's a grammar, BNF notation description for a syntax of some kind of register documentation file. It just looks like an AMD internal format. But it's on this website. Okay. So we have these two URLs. Slash pragmatic slash bonair.xml and slash rai slash rai.grammar.for.txt. Let's try something. How about maybe pragmatic slash bonair.rai? Nah, it's a 404. Okay, it's pragmatic slash rai slash bonair.rai. Ah, bingo. So this is a full bonair or almost full bonair register documentation with like full register field descriptions, breakdowns, all the addresses. It's not 100% but it's like the vast majority. This seems to be AMD internal stuff. And I looked this guy up and apparently he worked at AMD at some point. So, but yeah, this is really, really helpful because now you know what everything means and debug registers and yeah. So I wrote a working parser for this format, not the XML. This I was apparently writing a XML parser or something like convert this thing to XML but it was all broken. Oh, he was writing PHP, by the way. But there you go.  You can take a register dump and map it to the, you know, basically documented. You can diff dumps. You can generate defines. It's very useful for AMD GPUs. And this, grossly speaking, applies to a lot of AMD GPUs. Like they share a lot of registers. So this is useful for anyone hacking on AMD GPU stuff. Over 4,000 registers are documented in the just in the main GPU address space alone. So that's great. Okay. So we have some docs. How do we get to a frame buffer? Uh, so if you, uh, you know, the user has HDMI, that's easy, right? The GPU has HDMI.  Okay, maybe it's unconnected. That's fine, right? Ah, but if you actually ask the GPU, it tells you HDMI is not connected. Display port is connected. Okay. Yeah, they have an external HDMI encoder from display port to HDMI because just putting a wire from A to B is too difficult because this is Sony. So let's put a chip that converts from protocol A to protocol B. Yeah, yeah, yeah. Yeah. Yeah. And okay, it's, uh, it's, yeah. It's a Panasonic display port to HDMI bridge. Not documented, by the way. Requires config to work. That's why it doesn't just work, even though some bridges do. And you'd think, okay, it's hooked up to the GPU I squared C bus because GPUs have in the past used these bridges and not this one in particular, but other AMD cards have had various chips that they stuck in front. And the code has support for talking to them through the GPU I squared C interface, right? Right? That's easy. Yeah, you wish. This is Sony. Enter ICC. So remember the ICC thing in the olia? It's an RPC protocol you use to send commands to an MCU that is somewhere else on the motherboard. It's a message box system. So you write some message to a memory place and then you tell it, hey, read this message and then write some message back and it tells you that's the reply. You access it via AOVIA, not via the GPU.  So now we have a dependency from the GPU driver to the AOLIA driver and two different PCI devices and two different, yeah. And okay, again, ICC, but this I squared C is a simple protocol.  It's super simple, right? Right?  And why, so only why, like why would you do this? Well, because ICC is so slow that if you actually try to do one read and one write at a time, it takes two seconds to bring up HDMI. Yeah, like, yeah. I don't even know at this point. I have no idea. And by the way, this thing has commands where you can send scripts in a script to be run when certain events happen. So yo dog, I heard you like scripts. I put scripts in your scripts so you can I squared C while you I squared C. Like, let's just go even deeper at this point, right? Because, yeah, yeah, yeah, yeah. Okay, we wrote some code for this. You need more hacks. It needs all display port lanes up. The next choice to downscale doesn't work. Memory ground with calculation is broken. Mouse cursor size is from the previous GPU generation for some reason. I guess they forgot to update that.  Ah, well, it turns out that PS4 uses a unified memory architecture. So it has a single memory pool that is shared between the X86 and the GPU. And games just put a texture in memory and say, hey, GPU, render this. And that was great. And this makes a lot of sense. And their driver uses this to the fullest extent. So there's a VRAM, you know, the legacy GPUs had a separate VRAM. And all these integrated chipsets can emulate VRAM using a chunk of system memory. And you can usually configure that in the BIOS if you have a PC that does this. And the PS4 sets it to 16 megabytes, which is actually the lowest possible setting. And, yeah, 16 megs is not enough to have more than one full HD frame buffer. So obviously that's going to explode in the next pretty badly. So what we do is we actually reconfigure the memory controller in the system to give one gigabyte of RAM to the VRAM. And we did that in VSD KXX. So it's basically doing, like, BIOS-y things. We're reconfiguring the Northbridge at this point to make this work. But it works. And with this, we can get X to start because it can allocate its frame buffer. But, okay, it's 3D time, right? Yeah, GPU acceleration doesn't quite work yet. So we got at least, you know, X. But let's talk a bit about the radio and GPU for a second.  And you do this by putting it into a ring, which is really just a structure in memory that's just a list of commands. And it goes or it wraps around, right? So that way you can queue things to be done on the GPU and then it does it on its own and you can go and do other things. So it has a graphics ring for drawing a compute ring for GPU and a DMA ring for copying things around. The commands are processed by the GPU command processor, which is really a bunch of different CPUs inside the GPU that are called F32, and they run a proprietary AMD microcode. So this is a custom architecture. Also, the rings can call out to IBs, which are indirect buffers. So you can say, basically, call this piece of memory, do this stuff there, return back to the ring. And that's actually how the user space thing does things.  This is basically how much GPUs work, but radion specifically works like, you know, but this F32 stuff. Okay. The driver complains ring zero test failed. Thankfully, it tests them. So at least, you know, it has nice diagnostic. How does the test work? It's really easy. It writes a register with a value and then it tells the GPU with a command, please write this other value to the register, runs it, and then checks to see if the register was actually written with a new value. So the write doesn't happen. It never, it's there. Thankfully, thanks to that RAI file earlier, we found some debug registers that tell you exactly what's going on inside the GPU and it shows the command processor is stuck waiting for data in the ring. So it needs more data after a knob command. Yeah. Knob is hard. Let's go stalling. So packet headers in this GPU thing has a size that is size minus two. Whoever thought that was a good idea. So a two word packet has a size of zero. Then AMD implemented a one word packet with a size of minus one. And old firmware doesn't support that and thinks, oh, it's free FFF. So I'm just going to wait for a shitload of code in the buffer, right? It turns out that Hawaii, which is another GPU in the same gen, has the same problem with old firmware. So they use a different knob packets. So there was an exception in the driver for this and we had to add ours to that. But again, getting to this point, many, many, many hours of headbanging. Yeah. Okay. We fixed that. Now it says ring three test failed. That's the SDMA ring. That's for copying things in memory and it works in the same way. It puts a value in RAM, tells the SDMA engine, hey, write a different value and checks. This time, we see the right happens, but it writes zero instead of the dead beef or whatever. Okay. So I tried this. I put two right commands in the ring, saying right to one place, what to a different place. And this time, if I saw what it did, is it wrote one to the first destination and zero to the second destination. And again, okay. It's supposed to write dead beef, which is what you see there. It says, you know, dead beef is that word with the value. It writes one. Well, there's a one there. That wasn't there before. It was a zero because it was padding, right? So yeah, it turns out they have it off by four error in the SDMA command parser and it reads from four words later than it should. Again, this took many hours of headbanging. And it was like randomly try two commands. Oh, one, one, one. Yeah. So it reads four words too late, but only in ring buffers. Indirect buffers work fine. That's good because those come from user space, so we don't have to muck with those. We can work around this because it's only used in two places in the kernel by using a fail command instead of a write command. That works fine. Again, how did they even make these mistakes? Okay. But still the GPU doesn't work. The ring test pass, but if you try to draw, you get a bunch of page faults. And it turns out that what happens is that on the PS4, you can't write the page table registers from actual commands in the GPU itself. You can write to them from the CPU directly. You can say just write memory register write. MMI or write. But you can't tell the GPU, please write to the page table register this. So the page tables don't work. The GPU can't see any memory, so everything's broken. Linux uses this. FreeBSD doesn't. It uses direct writes. And we think this is maybe a firewall somewhere in the Liverpool, some kind of security thing they added. We can directly write from the CPU, but it's like breaks the regular, like it's not asynchronous anymore, so this could break things. That's a really hacky solution. I would really like to fix this. And I'm thinking maybe the firewall is in the firmware, right? But it's proprietary and undocumented firmware. So let's look at that firmware. It's a thing. It reads microcode, right? This CP thing. It's undocumented, but we take the blocks out of FreeBSD, and that's great because we don't have to ship them. Let's dig deeper into those blocks. So how do you reverse engineer an unknown CPU architecture? That's really easy. You run an instruction and see what it did, and then just keep doing that. Thankfully, we can upload custom firmware, so it's actually really easy to just have a two-instruction firmware that does something and then writes a register to a memory location. And that's actually really easy to find. If you just first write the memory instruction, it's really easy to find in the binary, because you see GPU register offsets that stand out a bit in one column. So long story short, we wrote F32DIS, which is a disassembler for the proprietary AMD F32 microcode. I shamelessly stole the instruction syntax from ARM, so you may recognize that if you're used to ARM assembly. And this is not complete, but it can disassemble every single instruction in all the firmware in Liverpool for PFPMEC and RLC, which are five different blocks in the GPU. As far as I know, this has never been done before. All the firmware was like voodoo black magic thing that's been shipped, not even the non-AMD kernel developers know anything about this. And you can disassemble the desktop GPU stuff too, so this could be good for debugging strange GPU shenanigans and non-PS4 stuff. All right, alas, it's not in the firmware. It seems to be blocked in hardware. I found the debug register that actually says there was an access violation in the bus when you tried to write this thing, and I tried a bunch of workarounds, and I even bought an AMD APU system desktop, dumped all the registers, dipped them against the one I had on Linux, and tried setting every single value from the other GPU and hoping I find some magic bits somewhere, but no. They probably have a setting for this somewhere, but it's a C of ones and zeros, good luck finding it. It does work with the CPU write workaround though, so hey, at least we get 3D, and it's actually pretty stable, so if there's a race condition, I'm not really seeing it. So checklist, what works, what doesn't work. We have interrupts and timers, the core thing you need to run any OS. We have a serial port, we can shut down the system and reboot, and you think that's funny, but actually it goes to ICC, so again, need some interesting code there. I actually just implemented that what, four hours ago, because you're pulling the plug, it was getting old. The power button works, USB works. There's a funny story with USB, is it used not to work, and we said, fix it later, there seemed to be a special code missing, and then someone pulled a repo from the USB not working branch, and tested it and said, oh, it's working. It seems we fixed it by accident, by changing something else. The hard disk works, which is via USB. Blu-ray works, I wrote the driver for that also four hours ago, three hours ago now, yeah, something like that, and it spent 20 minutes looking for someone in the hack center that had a DVD I could stick in to try it. Apparently I'm from the past if I ask for DVDs, so yeah. But it does work, so that's good. Wi-Fi and Bluetooth works, Ethernet works except only a gigabit speeds, frame buffer works, HDMI works, it's currently hard coded to 1080p, so yeah, it does work. We can fix that by improving the encoder implementation. 3D works with the ugly register right hack, and speed if audio works, so that's good. HDMI audio doesn't work, mostly because I only got audio grossly working in general recently, and I haven't had a chance to program the encoder to support the audio stuff yet, because again, more annoying hacks there. And the real-time clock doesn't work, and if you think that's simple, well, the clock device is simple, but ever since the PlayStation 2, the way Sony has implemented real-time clocks is that instead of reading and writing the time on the clock, which is what you would think is the normal thing to do, they never write the time on the clock. Instead, they store an offset from the clock to the real-time in some kind of storage location. And there's a giant mess of registry, it's called, in the PS4, and I don't even know where it's stored. It might be on the hard drive, it might be encrypted, so basically getting the real-time clock to actually show the right time involves a pile of nonsense that I haven't had a chance to look at yet. But we have NTP, right? So it's good enough. All right, oh, and we have blinking lights, important. The power LED does some interesting things if you're on Linux, so that's good. So, the code, we can get the PS4 KXEK code on our GitHub page that has the KXEK and the hardware configuration and the bootloader Linux stuff. You can get the PS4 Linux branch, which is our fork of the kernel, based on 4.9, which is the latest public version, I think. You can get our Radeon patches, which are three, I think, really tiny patches for user space libraries, just to support this new chip. Really simple stuff, the Knop thing and a couple commands. And the RAI and F32 disting I mentioned, you can get Radeon tools at that GitHub repo. I just pushed that right before this talk. So if you're interested, there you go. And if you're going to get the RAI file, well, you probably want to run before the guys at that website realize it really should take that down. But I'm sure the internet wayback machine has it somewhere. So yeah. Okay. Well, that's everything for the story of how we got Linux running on the PS4. And you can reach us at that website or failoverfall on Twitter. Thank you. So I hope that wasn't too fast. Sorry. I had to rush through my, like, 89 slides a little bit because I really wanted to do a demo. And then again, this kind of is the demo, right? But we can try something else. So maybe I can shut this... If I can aim with my controller. This is really not meant to sound mouse. That's not right button. Come on. Yeah, I think it is. Close, close, maybe. Yes. So we have this little icon here. I wonder what happens if it works. So we have internet access. Hopefully Wi-Fi works. Actually, let me just check real quick. Because this could work really badly if we don't. Ping 8, update 8, right? Yeah, we have internet access. Okay, Wi-Fi works. Okay. Wonder what happens if we click that. It takes a while to load. This is not optimized for...  But hey, you know, it works. And now it's a real game console. And this is... There we go. Okay. So yeah, I think we can probably take some Q&A because this is a little bit slow to load. But we can try a game, maybe. Well, if you are for Q&A, I think there will be some questions. So shall we start with one from the internet? Testing, testing. Okay, hey. The internet wants to know if... Well, most of your research will be published or if stuff's going to stay private. Well, all of this... I mean, the publishing is basically the code which... And the explanation I just gave. As I said, everything's on GitHub. So all the drivers we wrote, all the... In that case, I guess also the spec is the code. If you really want to, I could write some wiki pages on this. But roughly speaking, what's in the drivers is what we found out. The really interesting bit, I think, is that there's 32 stuff from the AMD GPU stuff, and that we have a repo for. But absolutely, if you have any general questions on any particular device or any details, feel free to ask. I don't know. Again, it would be nice if we wrote a bunch of docs and everything, but it's not really a matter of not wanting to write them. It's lazy engineers not running to write documentation. But the code is at least... The things we have on GitHub is pretty clean. Okay, so someone is piling up on four. Guys, if you have questions, you see the microphones over here. Just pile up over there and I'm going to... Four, please. Just a small question. How likely is it that you upstream some of that stuff? Because, I mean... So, there's two sides to that. One side is that we need to actually get together and upstream it. The code, some of it has horrible hacks. Some of it isn't too bad. So, yeah, we want to upstream it. We have to sit down and actually do it. I think most of the custom x86-based machine stuff in the kernel is doable. The drivers are probably doable. Some people might scream at the interrupt hacks. But it's probably not terrible. And if they have a better way of doing it, I'm all ears, the other kernel devs. The Radeon stuff is white fishy because of the encoder thing that is really non-standard. And also, understandably, AMD GPU driver developers that work for AMD may want to have nothing to do with this. And in fact, I know for a fact that at least one of them doesn't. But, I mean, they can't really stop us from upstreaming things into the next kernel, right? So, I think as long as we get the code into a state where it's doable, it's fine. But most likely, I think... I think most likely the non-GPU stuff will go in first if we have a chance to do that. And of course, if you want to try upstreaming it, go ahead, it's open source, right? So... Over to microphone one, please. Hi. First, I think I should implore you to try and find Travel Hudson and control him into using your free BSDK exec implementation in Heads instead of having to run all of Linux in it as a joke. But my real question is if the reason you used Gentoo was because SystemD was yet another hurdle in getting this to run. I run Gentoo on my main machine. I run Gentoo on most of the machines I care about. I do run Arch on a few of the others, and then I live with SystemD. But the reason why I run Gentoo is, first, it's what I like and use. And second, it's super easy to use patches on Gentoo. You get those things we put on GitHub, which are just patch files. It's not really a repo because they're so easy. It's not worth cloning everything.  So it's really easy to patch packages in Gentoo. That's one of the main reasons. Yes. Number three, please. Will there be new exploits, new way to boot Linux on PS3 with modern firmware because finding one with firmware 176 is really rare. That was 405. But again, our goal is to focus on the, I just told you the story of the pre-exploit thing, because I think that's good, like hacker story, a good knowledge to try new platforms. And the next thing we're working on, the reason why we don't want to publish the exploit or really get involved in the whole exploit scene is that there's a lot of drama. It's not rocket science in that it's super custom code. This is what can free VSD. It's actually not that hard. And we know for a fact that several people have reproduced this on various firmwares. So there's no need for us to be the exploit provider. And we don't want to get into that because it's a giant drama fest as we all know anyway. So please, DIY it this time. Thanks. And what is the internet saying? Testing? Okay, the internet wants to know if you ever had fun with the VSD on the second processor. Oh, that's a very good question. And I myself haven't. I don't know if anyone else has looked at it briefly. One of the commands for rebooting will boot that CPU into free VSD. And there's probably fun to be had there. But we haven't really looked into it. And over to five, please. I was wondering if any of that stuff was applicable to the PS4. We are additional, however, it's called the new one. Sorry, have you ever tested? Sorry, say that again. The Sony brought up a new PS4. Oh, the Pro, you mean the PS4 Pro? Yes. Yeah. So Linux boots on the Pro. We got that far. GPU is broken. So we would like to get this ported to the Pro and also working. It's basically an incremental update. So it's not that hard, but the GPU needs a new definition, new chip, all that stuff. Yeah, I get it. You've found some friends here. Yeah. But yeah, as you can see, the 3D works and there you go. You will hear a buzzer. When you hear the buzzer, look down at the floor. Good. You can probably have to look up and down in this game. You have to do a physical and metal wellness exercise. Ready for another one? There is a brain thing on the wall. Well, then number three, please. I want to ask you if you want to port this radium patches to the new AMD GPU. CPU, a GPU driver because AMD now supports the Southern Ireland GPUs. Yes, that's a very good question. Actually, the first attempt we made at writing this driver was with AMD GPU. And at the time it wasn't working at all. And I was a bit concerned about its freshness at the time. And it was experimentally supporting this GPU generation. I'm told it should work. So I would like to port this, you know, move to AMD GPU. Not that we have a working implementation and we got the cleanup code much better. We know where all the knits are. I want to try again with AMD GPU and see if that works. That's a very good question because the newer gen might require the driver maybe. So, yeah. Thank you. Well, then I'm going to guess we asked the internet again. Okay. The internet asks, states that about a year ago you argued with someone on Twitter that the PS4 wasn't a PC. And now you're saying it kind of is not something. And what's about that? So again, my reason for saying it's not a PC is that it's not a PC. It's not an IBM personal computer compatible device. It's an XC6 device that happens to, you know, be structured roughly like a current PC. But if you look at the details, so many things are completely different. It really isn't a PC. Like on Linux I had to define, you know, sub arch PS4. It's an XC6, but it's not a PC. And that's actually a very important distinction because there's a lot of, you know, things you've never heard of that are XC6 but not PCs. Like for example, there's a high chance your monitor at home has an 8186 CPU in it. So, yeah. So nobody's piling at the microphones anymore. Is there one last question from the internet? Yes, there is. And the question is, well, if there was any decryption needed. No.  There's no security, like there's nothing, like stopping you from doing all this stuff. There's a sandbox in FreeBSD, but obviously you exploit around the sandbox.  There's no like integrity checking. You know, they have a security architecture, but as this tradition for Sony, you can just walk around it. So, yeah. The PS3 was notable for the fact that the PSGL break, which is a USB effectively piracy device that was released by someone that basically used to USB exploit in the kernel and only a USB exploit in the kernel to effectively enable piracy. So, when you have like a stack of security and you break one thing and you get piracy, that's a fail. This is basically the same idea, except I have no idea what you need to do to do piracy and I don't care. But, yeah, Sony doesn't really know how to architecture systems. That's it then. That's it? All right, thank you very much. Here we go, that's your applause. Thank you.


I can do? Hello. Better, thank you. Welcome to EMF 2014, we are back. Some of you may know, I'm Johnty. One of many, many organisers, we've been working on this work for over a year. It's quite a large team, you're going to see a lot of around sight. Please don't bother people randomly because you really want to thank them. They really need to get on with things a lot of the time. But at the end buy them all a drink. Everyone wearing your lanyod or wearing your hyvys. Mae sy'n rhoi ddannu'r un sydd yn ôl. O fod yw'n gorffod! Odydu iddo, mae'n ddiddordeb ar ybyddai. Go iawn, roedd yna swell. Mae nhw'n iawn i gyd yn atharni'i gynnyddio'r ddau. Roedd ymwn yn defnyddio, bod yn wneud rhywbeth o sydd gen i'n dda, ac rydyn ni'n gallu'n r adults. Mae'r rhwng haf hans o'r pobl sy'n ein hwn ar EAMF y 2012. Mae'r pobl yn y sydd i L-Wave mae'r peilio arall. y茴 yn y gyhoedi... Eich cofnod wedi'u felly hwyl. Mae chi'n pethwn i hynny wedi lici fod yn galw...反 ond e wedi rhoi 10000 o bobl. Can I also get a hand up for anyone who's come from further than America? You do not count! Has anyone come from America? You win a pint. Standing thing, whoever's come the furtherest gets a pint from me. It's just America. No one come any further than America. Maybe there's someone from New Zealand. That would be nice. Anyway we're not sure about that. Dright. Neither everything is now set up on site. Nearly everything. The lounge is still being worked on, the workshop tent I think is now fine. These stages, as you can see, do not have very good projectors. This would be a lovely video of the site taken from the air. But unfortunately these projectors are not great and these tents are very bright. When we tested this, it was a bit darker and it's not entirely good according to plan. We are working on a plan to fix this, hopefully it will be enacted by the end of the day. I can't promise that, so the first speaker's coming up, you might have a little bit of trouble. There's not much we can do about that, we're very sorry, we're working on it as quickly as we can. Hopefully people will put their slides of afterwards if you couldn't see them up here. And you'll just have to mime or something, I'm not really sure what you're going to do. Interpretive dance would be fine. After 4pm-ish, depending on how things go, you'll be able to pick up your goodie bags, or welcome bags we use the terms interchangeably, from the info tent. And they will contain your camp badge, which is the tilde Mark E, this is the improved version of the previous badge, which has a lovely screen on it, and has the camp schedule loaded on it, wirelessly connected, it's all very fancy and lovely, and it also has Tetris for when you're bored. But these will be available along with other information you need in the bag from the info tent after 4. The reason it's a little bit delayed is because they're still soldering some of the badges up, they only arrived this morning. We like to push things right to the end, and it's just in time delivery, it's completely fine. So, those are all the fun bits, there's bits that aren't as fun. The projector I've already mentioned, and we are fixing that as quickly as possible. And our workshops are really, really full. Now, I have to apologise for anyone who's been let down by the workshops signed up on email. You were all very excited and we didn't expect that. The blacksmiths came to us and said, we really want to get everyone signed up before the event, so we know how many people are going to be coming to them. And we tried to convince them that this wasn't a problem and everyone would be signing up at the event, but they asked us to do it anyway. So, we opened up email sign up, and then within 12 hours we had 400 emails and two people dealing with them, and they were not very happy. So, we had to sort of stop. That's when you all got an email saying, please stop emailing us. So, workshop sign ups are now at the info tent. I believe there are some still available, but nearly all of them are sold out, say sold out, booked out. Some of the workshops do come with an additional cost for components or whatever whoever's putting them on. Please be aware of that and make sure you've checked beforehand that you've taken cash, otherwise they're going to ask you to leave and that's a bit rubbish. I forgot what I was going to say. We are also trying to find more space for workshops to try and put more on. One of the other things we were not expecting this year was the sheer number of workshops we were going to get submitted to the CFP. We have one workshop tent and three stages that thought that would be fine. We really needed more like three workshops and one stage. That would have been just about right. That's not true. But we're trying to see if we can put them in the lounge and put them in the villages. If you are running a village and you have workshops in your village and you haven't told me already, why haven't you told me? I need to put them in the schedule. Please email content at emfcamp.org with anything that's going on in your village. We can put it in the schedule so people can come out and see you. If you're charging for them, let me know that as well. That's completely fine. If you came on site before this morning and you don't have a wristband, you need to go and get your wristband from the check-in desk before 6pm today. Otherwise we will get the telehandler and Benny will move you off-site with the telehandler. So please do that. Otherwise, as you know, someone don't do that because I want to see it happen. I've been promised it will. If you have any free time at all, which I know you all do because you're on here for a weekend and there's lots of free time, please volunteer. The entire thing is volunteer run. We need volunteers to run the workshops and volunteers in the bar. We really need volunteers to do some security which is really boring, but we do have live streams to watch. So you can kind of sit there being secure and watching live streams. So that sort of works. That's all the really boring bits, I think, and that's nearly everything I have to say. I'm really glad everyone's here and I'm really glad that we've sold out because we were really nervous that we've got so much bigger in two years' time. We thought things might go horribly wrong. We thought we might sell out. It's fine. We've sold more than enough tickets and we really appreciate everyone coming. So thank you very much to everyone who's here. Thank you for the unexpected applause. It makes it all worth it. We've worked really hard. There's been tears, there's been blood. Pretty small amounts of blood, but there has been blood. We all really appreciate it. There's not many of the volunteer and the old team in this tent at the moment. There are a few. Can you put your hands up if you want to? There's six of them. Everyone else is really busy making the event run. Find them towards the end of the event if they're in the bar by the Madrid, as I said earlier. We have to thank our sponsors. The event will run quite happily without sponsors. Nearly. The sponsors cover things like the badge. The badge doesn't come out of your ticket price. The badge is entirely sponsored so that you can't not get one. We want to make sure everyone gets the same thing. We have some sponsors covering the event. We have some sponsors covering the event. We have some sponsors covering the networking. Some sponsors who very, very helpfully stepped in to cover the larynge when we didn't have enough money. We really like them. I need to thank for the badge. I need to thank Ragworm. Ragworm in the room. These three people back there have done incredible things to make the badge happen. There is a talk on Sunday. There is a talk on Sunday which I will be drinking during. We will tell you what has happened with this badge because it almost killed all of us. We need to thank Ragworm. We need to thank Element 14. We need to thank PayPal. I need to say PayPal and Braintree because they changed it and now it's rubbish. Anyway. I need to thank Mathworks. I need to thank Seseco, Twilio, Imagination Technologies, Shadowcat Systems, Codethink, Pymeroni and LPRS. All of whom have someone on site somewhere and I'm sure you can say hi to them somewhere else. For networking and making sure we have an internet connection here which almost didn't happen but has. We need to thank Comtech, FlexOptics, Aruba, Lonat and Coloca. For the lounge we need to thank Port Cullis who have stepped in and stopped that from completely going under. For the first time we have kids childcare and proper childcare, not rubbish childcare with people who think they can do it. You can work out what their reference is to. They're located right down the end of the camping fields. They're far away.  We'll see if that works. That is open from, I'm going to get this wrong now, 10 til 8 every day I think and you can drop your kids off there, they'll be looked after and you can come and get them later on. We also have full kids workshops and they look quite fun as well. Some of them actually look better than the normal workshops. I did have someone get in touch asking if the Make a Lightsaber workshop was actually a real lightsaber. That was really good. Yeah. It is a fake one. In addition I've got a few more people to thank. We need to thank ByteMark for hosting our servers, which has been great. They've dealt with this all the time. We need to thank the CCC who have helped lent us their address space for the internet and all sorts of advice and generally just been lovely people. I need to thank IRC Cloud who Russ and I worked for. I need to thank them because Russ and I have been really shit at work for a while now. Thank you to them. I need to thank the team. The team have been amazing. We couldn't have done this without them. It's been really amazing. Now because I have a terrible memory and because this is written out very quickly in 15 minutes before the talk, have I forgotten anything that I need to say? Anyone? If anybody hasn't got their furniture for the village, it's down by stage B. Go and collect it from there. Please don't take too much. We do have a limited amount. There's more. The village tents are kites. Just as a quick thing, it is really windy here which we didn't quite expect. If you do see any tents escaping and make you run for freedom, please stop them. They can do quite a lot of damage to people and other tents. That would be useful. Quick call out to first aid as well. They're located just up by the main entrance. They'll help you with anything that's going on.  If you're going to call 999, they're going to call us back. You get a really weird loop for this. Please go to one of our staff first and don't call 999 because it just gets really awkward for everyone involved. Is there anything else I've forgotten from anyone? Will, stop looking at your phone. Have you forgotten anything? If you want to volunteer, go to the info tent and if you'd like to volunteer, I'll be useful. Anything else? When's the bar open now? You've missed out. One other thing. We've got one less food vendor than we expected. Hopefully that won't be a problem. We'll have enough coverage anyway. The van broke down and they can't make it. I don't think it'll be an issue, but we should just to make you aware of that. The barbecues are going to be outside just here. They're shared barbecues for everyone to use. You can buy charcoal from the shop. They are not welded together yet because somebody forgot their welder. They'll be up by the end of the day, I'm assured. If you like welding, go and help them. There's an arm up at the back of the tent. We have no workshop space unless it's in your tent. Maybe. If anybody does actually like welding, come and say hi and you can come and do some welding. Anything else? Can I get off stage now? We have a code of conduct this year. Please don't be mean to anyone. That's the short version. The longer version is actually quite comprehensive.  Please read it. It's in your booklet. We're not going to tolerate it. We've never had it happen. We've heard of it happening at other events. We're just not going to stick with it. Please be nice to one another. Everyone should be lovely. Anything else? The bar closes at 1am. Which is just late enough for me to get there. Right. I think we're done. Good. I'm not quite done. I've just remembered the talk that should be happening next which is high altitude ballooning on this stage they are currently stuck in traffic. They can't use a high altitude balloon to get here. I believe... Oh no, are they here? I'm not sure if that's a random applause for them or the motorways. I can't tell. Anyway, I think we are absolutely done now. Have a lovely time. If you have any problems, go to the info tent. They will help you out. They're open until I think 9pm. Otherwise, grab someone who looks important and hopefully they are. Have a lovely time and be around the camp.


Hello, and welcome to this screencast on section 10.5, the chain rule. You are probably familiar with the term chain rule as one of the derivative rules you learned in single variable calculus. For the function f of two variables, x and y, where x and y are differentiable functions of an independent variable t, we have the following statement of the chain rule, which says that the derivative of z with respect to t is equal to the partial derivative of z with respect to x times the derivative of x with respect to t plus the partial derivative of z with respect to y times the derivative of y with respect to t. Note that the chain rule here is given in Leibniz notation, which makes it easier to keep track of the variables and the dependencies. And the statement of the chain rule here is very specific to the situation given, where z depends on x and y and x and y depend on t. We will not always have such a situation, nor will we always be considering situations with variables z, x, y, and t. For instance, we could have x and y depend on more than one quantity, or z is a function of more than two variables. To keep track of the chain rule in any given situation, we're going to introduce the notion of tree diagrams. Consider the familiar situation from the previous slide, where z depends on x and y, and x and y depend on t. To construct the tree diagram, we place the dependent variable z at the top of the tree and connect it to the variables for which it depends on one level below. So since z depends on x and y, we connect it to x and y one level below z. Since x and y both depend on t, then we connect both x and y to the variable t one level below. To now represent the chain rule, we label every edge of the diagram with an appropriate derivative or partial derivative. For instance, the edge from z to x is labeled with the partial derivative of z with respect to x. The edge from y to t is labeled with the derivative of y with respect to t. To calculate an overall derivative according to the chain rule, we construct the product of the derivatives along all paths connecting the variables and then add these products together. So for example, starting at the dependent variable z, we're going to trace down one path of the tree all the way to the bottom. Right, so we're going to trace this path. As we trace this path, we pass the partial derivative of z with respect to x and the derivative of x with respect to t. This gives us this term of the chain rule as the product of all the derivatives that we passed along that trace. We're going to do the same for all the other paths and for this tree, there's only one more path and that gives us this partial derivative times this derivative, which gives us this term. And we sum up everything, all the different traces that we make down the tree.


cuz this is a zen dude show and you know y'all ho coming on to low psych we ain't about that left we about spreading positivity all over the night into the next morning can you figure out why the zen dude nation is growing on the floor bring it go ahead I'm never gonna stop what is up ladies and gentlemen welcome back to episode number eight of ask the zen dudes on today's episode Dan's at grandma is making us cookies I love my grandma she makes the best cookies dude what's the first thing I wanted to say on this oh hope you guys like that intro music it's from the official AK he is a zen dude himself and has been following us for some time and I thought we should put his intro music on the front of our bumper because the dude makes dope hip hop so then dude make dope music check him out he needs to be an intro check my sense I just pasted his stuff right there in the description as you put it on there too I did question one make them up make them up question number one dude we are not wasting any time bro but a little bit of time yeah dude how are you still like Stephen asks have you tried different strains of weed and found a nice mellow one that doesn't mess with your mellows have an awesome Sunday dudes thanks bro yeah Stephen we're always trying to stay mellow mellow mellow like Carmelo Anthony know what I mean yeah we get real alright so Stephen dude in Columbia it's a little harder because people just sell organically but if you live in a country or country you live in a state where it's legal or country where it's legal you can find a nice to Tiva blend and I would just like sample different ones and try to find maybe lower on the THC side it's gonna be helpful for you they'll be able to like you know enjoy some marijuana go jump some rope get in the zone move your body you know think about your life maybe do some introspection I think he's asking you specifically though well us personally you should you should give him a give him your give him your load up for me personally I prefer to Tiva and yeah for here in Columbia it's tough because we just get it from this guy who grows organic but once we get back to the US I'm definitely gonna find like you know a low to mid THC to Tiva that's like chill I can work with but it's not gonna pull me out and yes and my answer is we've tried many different strands of strains of weed to find the ones that work best with us and that's why we talk about the education of people with marijuana because not every strain is proper for some people however the right strain can have a huge benefit on your body and on your mind so yeah I would just say for me it's a hybrid indica sativa like when we were on Hawaii we got to try a bunch of like different like great types of weed and the one I like the most was a hybrid indica and sativa because a solely sativa for me I'm a very hyper person I have like a lot of I like a lot of hyperactivity built up inside of me so an indica is actually kind of nice for me because it mellows me out but I wouldn't do a whole indica because then I would get a little bit more lethargic and tired so yeah so hybrid for me I found to be the best Chris asks when are you dudes coming to NYC for a CDF boot camp retreat workout session your timing is on point because I'm coming to New York in May because I'm gonna be going home I'm from Pennsylvania but I'm gonna be taking the train up to New York and spending probably a week or two there and the whole goal is actually to just put on a big meetup in the middle of either probably Manhattan somewhere in Manhattan well Central Park action Central Park would be really good down by like the east side river would also be kind of cool but yeah so there is gonna be a meetup in New York comment below and let me know if you live in New York and if you want to do a zendu'd meetup palatable one time yep Mono asks how to realistically keep going when you're going through a tough time in life to make sure that you don't lose motivation focus go ahead brain man Mono you just gotta figure out what your why is man you have to ask yourself why do I want to go forward like what is it what is it that's motivating you and try to always keep that in front of you and keep it as your focus because if you don't have a compelling why like I know why would you do it like I don't do a lot of things because my why isn't strong enough and when for the things that my why is our strong enough I commit to my work hard so just connect whatever thing you want to do with your fitness to like whatever else you want to experience in life that like is your why boom great answer the only thing I would add to that is when you're going through a dude I would just put things in a perspective and realize that most of the world is going through tough times that you will never ever experience so just reminding yourself to be present and just decide to be in a good mood that's what I do a lot like I just decide that you know what no matter what's going on in my life I'm gonna choose to be happy Harry Roberts asks tips on talking to the ladies mm-hmm Harry you definitely you talk to the light house you do that you do that you welcome the girls and say that to be yourself Harry yeah that's dude that's the best advice anyone could give you just be yourself because if you're trying to be someone else and she's gonna be attracted to someone else and we go back to being the old version you should be like who's this dude so be yourself be yourself Harry I would say exactly what Bren said you have to have self-confidence man you have to have self-confidence in who you are and I guarantee you there's gonna be a lot of girls out there who were like I don't know what that Harry Roberts oh Harry Robert I want that I want I want you Harry Robert so I wonder why Harry Roberts bed looks like that's what they're thinking I wonder how they're thinking Harry wonder how Harry Robert is oh like how Harry yeah Francis asked Dan did you ever get those Jake Gyllenhaal single sides set single side Jesus single side swipe jumps and honestly I have not yet and I haven't been putting the time into practice enough to get there and that's shame on me and I still need to do that thank you for holding me accountable you the man Francis Nathan asks do you think I'm the next Muhammad Ali even though I can't stop losing my goddamn official fights yo Nathan man dude you're capable of whatever you decide you're capable of so if you need better coaching go find better coaching if you need to put more time in an effort put more time and effort in you need to improve your technique then go do it you will go as far as you decide you'll go man it's all between the ears so what do you want man do you want to be the next Muhammad Ali I want to leave you with this Muhammad Ali would not ask other people if he thought that he was going to be great Muhammad Ali believed he was going to be great you have to believe in that same way that you're gonna be great dude Colin asks to achieve the macronutrient targets I think I'm eating more than I did before is this simply eating more of the right sort of foods rather than less of the unhealthy stuff PS1 we hear about the Costa Rica trip Colin first of all the Costa Rica trip is live and we'll link it for you in the description again a sign up ask the first part of your question I guess the answer is no we want you eating the right amount of calories and macronutrients period doesn't matter what food they come from of course we do want you to put effort into getting 70% of those from whole foods but foods just means real foods like it's not made by me and meets fruits exactly so yeah man it's not about like okay I can eat a thousand calories of donuts and then I can eat a thousand calorie broccoli it's like either way you have to eat you know two thousand calories you have to get two thousand calories and so that's donuts and broccoli cool but don't just think about cutting out healthy food and not think about the total food consumption totally I totally agree Val asks there was something I wanted to ask about but I just can't remember what it was oh yes she's have a conversation I love you Val do hit workouts help with brain function I think exercise in general now I think I know exercise in general helps with brain function so yes hit workouts work as well yeah and I don't know if there's some scientific study that says like you get more you use more brain capacity we're doing hit workouts but personally I do feel that I get more into a state of flow when I'm doing hit workouts because when you're going as intense as possible you have to be very you're concentrating very hard on something so you it's easier to drop into a flow state yeah Corey asks heading the Ecuador in a week or so bring in my rope good job tips on adjusting to the altitude that's real landing keto there for a week then the mean dough there for another week gonna do the thing daily Corey yes Brandon go listen you just need to go there and do what you can do okay so usually back home can do like four rounds and you go there and you only do three rounds that's cool just let it be it's gonna be like that and you'll eventually adjust so just be patient with yourself and do the thing boom girl she asks what kind of stretches do you do and how often do you do foam rolling side note would love a stretching foam rolling video guess what dude that exists Brandon go ahead I stretch like twice a week Sunday is my big recovery day Dan you go for me I stretch like stretch stretch I mean I stretch a little bit before I dance before I jump rope I like you know throw my hands down on my legs but really what I do is I do a lot of foam rolling my hips because they get super tight and I also use a cross ball to work out muscles around my back because that gets really tight as well that I do every single day and I would highly recommend that you guys do that if you have any kind of muscular pain get a foam roller and a cross ball and the easiest way to describe what you do is you just lay on where the pain is and you press the pain out like literally where it hurts the most just lay on that and the pain will actually dissipate it's kind of a crazy thing and you'll feel much better afterward rub it out rub it out and check out our how to prevent injury video that we'll tell you that is his as well yeah LILA! hey Lila Lila the killer Lila the killer if you could live in a book TV show or movie what would it be hmm I think inception would be an interesting movie to be in and just go play around because I think consciousness is a really it's just quite it's just a huge trip for me and so to be able to play with conscious that way consciousness that way and create your own reality around you I think I might want to be in that movie I don't know yeah it's a good answer that's inception is a really good answer I think when I first read this question I kind of think that my I'll be honest with you I kind of think that this already is a movie and like I'm just I seriously I mean this I try and live my life in a way that it could be a movie like that's my biggest goal because that for me is what creates the most amount of fulfillment and happiness so I think like what I'm currently like my current life I'm like yeah I just want to keep making this more like a movie but if I had to pick an already existing movie man I'm just gonna go top of my head top of my head probably the Avengers because you know I'd like to be Iron Man I'd like to be Iron Man and just kind of have a whole skyscraper in New York City and be that intelligent so that's what I would that's what I would say I would want to be Tony anything asks how do you cope after an 11-day binge session chill take a deep breath don't make it a 12th don't make it a 12th no take a deep breath and just figure out what your calorie and macronutrient budget is for that day link below the link for our macronutrient calculator is right down below get those numbers and then plug in while you're eating that day into my fitness pal and just hit those numbers that's it don't over complicated it hit those numbers and just start snacking one day on top of the next really just apply yourself to those tactics and you're gonna break out of it you gonna break out of it you gonna break out of it you gonna you gonna make it yeah yeah the biggest thing you need to do is just like not worry that you had an 11-day binge session okay stop just stop right now I hate when people come you know hit us back and like oh but like everything's going bad so I have to keep doing this for like now dude just stop right now stop it don't make a 12-day binge session we believe in you come on Corey do you play video games currently I do not play video games but I played a lot of Madden back in the day I was a big football video game guy I was never much for a shooter a shooter game I was a shooter that's me dude I was one of those messed up kids who played Grand Theft Auto just to kill people oh yeah well I would run behind oh you did that too I did that I don't feel like every teenage kid just spent hours binging on just killing people randomly getting stars on a Grand Theft Auto it was so fun it was yeah but but it didn't make me violent I kind of don't like when people are like yeah that dude played that video game so he turned violent it's like nah dude I play some pretty violent games as a kid like football that's about the game well like yeah like oh yeah I played all fighter games I don't play video games now but I played all fighter games growing up I used to play a lot of Sega so like Sonic streets of rage you know who remember do you remember streets of rage it was beyond my time before my time brahada brahada Ben do these questions ever get answered it's fun you ask that question because I feel like this is the answer go we answer the last week we answer your question watch the last video yeah just watch these videos totally cool that you didn't know but this is where the questions get answered brahada Ben Kathy asks when did you both start jumping rope and how much time do you put into it daily what jumps cause you the most problem that's a great question mmm Kathy I started jumping a row really about two years ago I'd say a little more than two years more or less and then once I started I jumped rope in the beginning like three to five times a week and now I jump rope almost every day if not every day six days a week and the trick I had the most trouble with was a crisscross took me a while to get the crisscross done crisscross like a boss that's a tough one man I started jumping a row probably four to five years ago and how much time do I put into a daily I mean when I started every day like every day for 30 every day for 30 to 40 minutes the tricks that gave me the most trouble hmm probably crisscross definitely crisscross and consecutive double-unders like that was just really hard for me to get down like where I was doing a lot of them at once so yeah Barry what's going on my friend dude Barry actually is from a town near my hometown and we went to the same college no way was he a homie from college no dude he would have he would have totally been a homie had I known him but I think we ran in like different circles different circles but like kind of we know some of the same people yeah anyway Barry we appreciate your support brother what has been the hardest part of being an entrepreneur that's great the hardest part of being an entrepreneur is just having faith that something that's not here in the physical reality right now will be there in the future you just have to assume that like you're gonna make something that's never existed before and it's just that faith because it's so easy to be like ah the fear is taking me over and like is this really possible for my life but if you have faith and you you hold on to that I think that's what sees you through so dude it's always a mental boom dude your answer is literally my answer the hardest part is not the actual like making something or whatever it's the mindset like dude you you have to as an entrepreneur like you have to totally decide to take on this mindset that you are going to be successful and there's no like there's no plan B like you're going full into this ships burn the ships you either gonna die or this is gonna be gonna become successful or you're gonna die doing it and like I know it sounds a little bit harsh man but I really think enough people don't focus on that they focus on the strategy and tactics side if you decide you want to be an entrepreneur I think you just have to just go completely change your mindset to think I'm gonna be successful it might not be this business that I'm starting right now but like in my lifetime I'm going to be successful and like nail that tattoo that on your brain tell yourself that every single day of million times Jamie asks what are the Zen dudes thoughts on a ketogenic diet you're a ketogenic diet bro Jamie my dude I know I'm a student since you have an Amazon business that you're in the circle of listening to Tim Ferriss and like a lot of other productivity entrepreneur podcast or health podcast that talk about the ketogenic diet at the end of the day for us like it definitely can work and give you a given a result but we just don't like dieting we just want to eat the foods that we want and we've seen that through the way we eat by counting calories and macronutrients and trying to eat a lot of real whole foods that's very possible so for us it's like it's not it there's too much pain involved with dieting to actually want to do it and so that's why we don't do it we don't recommend it yeah also I think like just the simple what a diet implies that you're cutting things out and I think cutting anything out of your like saying I can't I can't eat sugar any more forever because I'm on this kind of diet or I can't have carbs because I'm on this kind of diet is not the right way to approach it and like a diet like dieting is the problem in general you should be able to eat every type of food that's out there just in the right amounts and be able to control your temptation around it rather than saying I can no longer live my life eating this food Gabriel asks dude now okay you ready you ready any specific suggestions for teams doing ZDF workouts man my biggest suggestion is just have fun with it like there shouldn't be any really intense pressure about a certain result like when you're a teenager just have fun learn to jump rope and don't stress yourself out about like having to do it every single day make it fun make it fun my dude make it fun make it fun make it fun Dylan asks hey dudes once again you guys are that's funny because I think you the shoe my question is have you ever done a mother talk mother spot sorry Spartan race warrior saying obstacle race I'm itching to do one because of the degree of difficulty that comes with it I want to reach the level of ninja you two are at Dylan I've never done one but I highly recommend that you do if you want to do it because I think things like that help you achieve a new mental level because you've never done something like that if you do it you're gonna prove yourself something at the same time make sure it's something that you want to do that you think is like least decently fun otherwise push yourself in another direction that is equally challenging but more fun totally agrees Brandon the only I was signed up for a tough mother race when I lived in Washington DC but I stopped seeing the girl that I was dating who we're gonna go on that together tough mother race date tough mother you know tough mother race sure those girls in DC you know they like to stay active Lewis ask hey Zen dudes hey Luis I have a question on rest days parentheses once a week should I keep the same macros specifically carbs for muscle repairing or what it turned into fat greetings from Mexico City dude what that makes go wow we really want to come to Mexico really want to go to Mexico like like soon maybe I should be doing July before that be crazy all right my dude is let me answer your question when you get your calorie macronutrient numbers from our calculator it asks you how many times you exercise a week and just factors that into the numbers that it gives you so if it says you're working out three to four times a week and you get your numbers baseball in that then it knows that you're gonna eat those calories macronutrients each day so don't think about eating like calorie macronutrient workout days was not workout days people do that who are really into like bodybuilding and like and just want to dial in every little body pass body fat percentage but you honestly don't need to do that just listen to the numbers every day happy to give you dude just focus more on being super intense during your workouts that's also a big one too so the food should come easier and easier as you go about it keep the same macros keep it simple like Brendan talked about just do the thing do that like Shaya asks how do you make the coffee drink which you guys drink early morning every day nice what's our coffee drink we go to put oh well two things I don't know what you're talking about other we go to cafes we live in Columbia so that the Americanos here are the best I've ever tasted anywhere in my life so we just give people to make us Americanos here yep but but I get Poco leche frio which is a little bit of cold milk because my stomach doesn't do well totally fasted with coffee I need some fat in there so I put milk just to yeah so you can do that as long as you keep it under 50 calories remember that oh also side note I've been recently getting cappuccinos in the morning which totally will break your fast technically because it's like it's a lot of milk but I've I don't mind that and I still think I'm getting some of the benefits from delaying that delaying that eating window even if I'm slightly breaking my fast does that make sense yeah absolutely because you're still only like you're very low calorie still for a large portion maybe maybe you're at 120 calories yeah so I'm playing with that and I'm doing it because it does make my stomach feel better when I have zero calories and try to drink coffee my stomach's like damn yeah now I'm gonna have diarrhea Vince axe Zendu brought us what's up man I've been listening to your podcast and it's pretty satisfying to let you let out all the things you want to say my question is what happened to the Zendu TV YouTube channel great question bro great question Zendu TV YouTube channel is a project that we have we want to create like investigative shows and just really diving topics we think are interesting it's just kind of the backburner right now because we're just so just jumping rope we're just we're really busy jumping rope jumping rope but Zendu TV will be a thing because Brandon and I just let you guys know who don't we just want to have a TV show and like a production company and just make really dope content all the time in really cool places so that's that's like our future like down the road a little bit channel straight up I hope I'm saying your name right cowlow cowlow cowlow cowlow can intermittent fasting cause an eating disorder Brandon I think if you already have a eating disorder I wouldn't do intermittent fasting just because it is kind of like an extreme thing where you're getting in this kind of rigid system around eating fasting for 16 hours and eating six for eight hours or fear a woman you know fasting for 12 hours eating for 12 hours so I would just focus more on eating whole foods moving your body and if you have issues with eating disorders in the past then work your way into being more conscious about calories and macronutrients and don't put too much pressure on the process yes intermittent fasting if you let it could cause an eating disorder and that's what a lot of people need to be careful of because guys we do not promote intermittent fasting so that you just stop eating your food and that's what can happen I have seen people say well Dan I'm losing a bunch of weight I'm intermittent fasting and I'm like bro you're eating 1200 calories a day you need to be eating 2000 so be careful of that because I have seen that happen with some people however having said that intermittent fasting is not the cause of the eating disorder it's the person who's doing it so at the end of the day just make sure to hit your calories and macros if you decide to condense that eating window in which you're hitting those calories and macros that's fine but you're in control here Amar asks what do I do if I can't move on to do another round in the workout do I take longer rests or should I stop ah Mar if you got the time man give yourself maybe a two-minute rest instead of a one-minute rest and then throw some more throw some more rounds on there bro totally agree totally agree dude totally agree Michael asked have you been on fitness magazines no yeah I have I was on playgirls yeah I was on playgirls pornography magazine it's not porn it's fitness sexes fitness you know that you know that I know that we'd like to shout out sponsors today athletic greens and cross rope athletic greens is a company that makes the protein that we take which is grass-fed non-antibiotic non so that's why we take it but pro games pro games homie it's linked in the description below athletic greens it's our supplement that we take sometimes not take sometimes we take every day yeah to help us get our greens in because it's the equivalents I believe 13 servings of fruits and vegetables some like that doesn't yeah some like that it's a lot it's a lot it's a lot of vegetables so it doesn't mean you should just take that not eat vegetables at all but when you can't get your vegetables in it's a good supplement and when you can yeah that's just extra micros for you homie and the other sponsor that we want to shout out listen ladies and gentlemen it's the best for a company on the market it's cross rope okay yo yo yo yo sitting in magazine no I don't wear jeans I wear sweats cuz all I do is jump to the road to hip to hip hop hip hip hop the fun don't stop when I listen to my hip hop and I jump rope you know I'm about to go and a load with my rope and my hip hop getting down like K-Dot yeah Kendrick just dropped a new song I heard he was calling to do out same mean things I'm not about that life no thanks hey I just want to be a nice guy yo then I stepped in I took it for the wing mother did I sin yes mother cuz this is a zen dude show and you know yo ho coming on to blow slight we ain't about that left we about spreading positivity all over the night into the next morning can you figure out why the zen dude nation is growing on the fly