without further ado because we stay on schedule. Felipe is here uh from uh from Portland uh by way of uh France from what I understand and he is going to talk to the talk to us about uh breaking into embedded devices including phones uh phones that you might have on your desk at work phones that might be I'm not even gonna get into that phones that might be around let's learn about embedded device hacking. Let's give Felipe a big round of applause. Have a good time man. Thank you. Thank you everyone. Um I'm sorry I'm really excited to be here it's like a big honor and um yeah so today we're going to talk about hardware hacking and um I'm going to share like this introduction to it and I hope um I can share some love about um hardware hacking. So uh who am I? I'm Felipe Lodrette. Um I'm a senior security researcher at McAfee um advent rate research. So I do um software and hardware and research and you can find me on Twitter at PHLol. Before joining McAfee I spent two years doing um embedded security and before that I was a C++ dev so I'm more coming from um software background actually. So you could wonder like why should you care about um hardware hacking in the first place right? Um I can see like two main reasons uh first it's like it's actually really fun and empowering to be able to um you know break open like a device that you're like having in your home at work whatever and instead of considering that it's a black box you can instead you know like start poking at it and understanding how it works. And then in a more like pragmatic way um if we consider like security in the past couple of years uh software is becoming much and much harder to like break like if you try to uh I know like point like uh whatever like um an embedded device like a web server like finding a world in a web server by itself is kind of hard but if instead you can just like open the device so there are a couple of wires and get like a shell from that that's much easier. So uh hardware hacking is great because it's also like a lot of uh low hanging foods basically. So uh today we're going to talk about uh Avaya phones um and especially like the uh night the phone picture on the right side of the slide is a 9 6 1 1 g model. Uh so Avaya is like one of the biggest uh view IP um solution provider you can find them like all over the place um covers like 90 percent of the 14 100 company and I think that's specific model of phone starting in 2006 and this is going to be end of life like at the end of the year or something. And the funny um on the bottom right of my slide there is this uh funny like um screenshot from like a firmware download page from like a really old software for these phones where um actually like you can see that um you can only download that specific firmware if you're like a DoD customer. So um basically you can see that these phones are like going to a lot of different places. So uh regarding prior art uh in hardware hacking there's like a lot of information nowadays on the internet like you can find a lot of blog posts, tutorials um so it's like the hardware hacking village that you all go go and check out uh CTF as well where you can like practice uh like kind of like the software side of stuff. And um on my slides I like a bunch of links at the end uh so you can like download the slides at later time and um also those slides uh I've like the one I've uploaded are like more text on them so it's like a more like standalone so you can read more. Um and yeah uh RedBalloon Security uh found like two assies in the same uh phone uh something like five years ago or something um and I like a talk at RSC called Stepping Ponds that I would also recommend to check out online. So uh for the next 40 minutes um so I'm going to talk about um hardware hacking and use that phone as a base to like support that conversation um and basically the idea of like the whole situation is um uh like couple of like 12 years ago maybe uh a via fork some open source code and um they never applied security patches to that. And um it turns out that like me by poking at the device I found that and it was like women like unnoticed for like 10 years that they were like actually like a CVE that got released like yeah 10 years ago for DEF CON.  But more interesting I think is uh talking about the whole process you know like how I did it um and why I did what I did it when oops sorry why I did stuff I did when I did it um you know like basically like sharing um the process so that and you know also like sharing uh you know what if the stuff I tried didn't work or also like stuff that didn't work because you know what works for one device might not work for another and having just like that mental model of what you can do and what you cannot uh help you a lot approaching like any kind of device and that's I really think why I want people to take away from that talk is more like um learning the process so that you can check out like whatever device you have. And yeah I would want to like insist that um if I can do it I think everyone in this room can do it as well. As I said I'm more from like software background and hardware hacking is still like you know like um little scary little foreign to me but it's in the end it's really not that hard if you have like a good process and you know work with patience. So uh how that whole project start um uh to be honest I was like you know working at work and we're like all these cubicles and on every desk I see this like phone like standing on the desk. I'm like wait a minute these phones run Linux they have you know network access they're like mics and stuff it's like how secure is that you know like you know you wonder like what's up with it did people consider the security of those? So um the first step before you know even like I start like stealing my co-workers phone and breaking it apart you can and that's general step is uh you can check out the FCC website usually like any device that has wifi or bluetooth has to be um on the FCC website for a compliance reason. So that's what I found um that's not my phone that's like a other model of like same brand basically and like pretty close. So you see all these cool pictures but more interesting is the um the inside of the phone uh so we can enhance it and um on the left side and right um I've highlighted like this like random debug connector I don't know what it is but that's interesting to see that's there and on the right side um there is like that like um in purpose there is that flash memory so that's usually where the file system of the phone is going to be stored and the whole point of that is just to get an idea of what to expect you know before even approaching a real target like burning hardware and stuff so that um you you know like if it's going to be a easy target or a hard target uh you know if everything was covered in black epoxy that would be harder. So another important step is like um getting like um online material like um you know um marketing brochures so you have an idea of what the phone what are the phone capabilities and the ecosystem surrounding it so that you have an idea of um you know like the attack surface and you can also find like user manuals and stuff and for instance I found that one that was pretty cool it's like a user like our advanced user manual that says oh if you want a serial connection to the phone you need that funky box and uh yeah which is interesting it teaches it tell us that yeah you can find a serial connection and you may have some special hardware for that but are we going to see that in a bit and uh after that you also want to check like forums and um so that you can find like what poor user are doing like for instance I found like the default password for the phone in uh debug interface because um and system admin so uh talking about that online and uh if you're lucky you can also download the firmware of an idea of uh what's up there and I will cover that in a minute. Okay so uh what do we do now uh now we have an idea of like what it might look inside this is a hardware hacking talk so we're going to avoid warranties and so the point is like you open the phone first um and the trick is like usually you always expect to find the same stuff inside and uh if you look at the labels of the components you will see uh and you google that uh you will find out like what it is so for instance that's my phone and um you can see in the center the CPU above it like the RAM in like uh lavender-ish color like some unpopulated like header that I marked with a question mark this I think the same header as the one we were like seeing in the picture before and uh it might be JTAG I did not have to use it so I don't know actually uh in purple we have like the AEPROM so that's uh would hold like some settings and stuff uh in dark blue in the bottom of the um of the slide is like a keyed RG45 so I was kind of confused when I was looking at that and I didn't know even the term for it and the idea is just a regular RG45 uh like Ethernet plug but with a weird uh connector so that you cannot plug the wrong stuff into it um and uh more interesting in yellow uh the UART so it's UART is kind of like the serial console and um we have like this expose like uh pads and so usually like in manufacturing or something you would have like uh Pogo pins that would like directly connect to the pads and that's uh that one is actually really interesting because we'd find like serial you would expect to find a serial console on that and on the back uh we can see the same name flash uh that we've seen in the previous picture and um yeah from that it's like where the file system would be like stored on the device so okay you have um we looked at the stuff inside and the like other thing that's interesting to look at is like test points and debug haters so and like basically all these kind of stuff that are not components and you can ask why would you expect to find them inside and a couple of reasons like one is like just usually like the dev board that the guys are going to use is the same as the production board and maybe they're just not going to solder the debug components it's the same as if you are like software and it was built with like debug symbols for instance and um also like part of the manufacturing process um you this test points and stuff might be used for like flashing the device or making sure it's flashed properly and sometimes you know like something died on um in uh in the real world and they want to like do it post more time to understand what happened and that's useful as well so what kind of stuff you want to look for in that situation you want to look at UART that we like looking at before JTAG that I'm going to cover in like in a minute and then like you can also have like random test points that chilling there like all over the all over the table all over the board and there might be like long test points like like TP1, TP2 whatever and sometimes that might be like not interesting to you but sometimes you can find interesting stuff like um you might find that like oh if you short like TP64 uh with ground it's going to reset the device which might be actually useful in some extent so uh looking at UART uh so as I said it's for like serial connection and it comes in like three or four pins the three pins would be um ground uh transmit and receive and the fourth pin would be like VCC so three volt five volt whatever and usually you don't care about that one you don't even want to plug it you don't have to and might uh yeah and um the the trick to know is um the serial console uh send data at different speed and there are only like a couple of it's called like board rate and there's only like a few uh common board rate value so if you try to plug your your system into the UART and you just see gibberish on screen the reason is you just have the board rate wrong and you could just like try to cycle through like all the possible possible one and you would find something eventually I think um JTAG is another one that's um interesting um so that's for like hardware debugging and with that you could like you know uh debug the CPU usually and execute like instruction by instruction and whatnot and it's like common many different uh shapes number of pins and whatnot so some example like on the bottom of the slide uh but actually only a couple of wires are interesting and uh at the end of the slides I like link to like a send me your article about like understanding JTAG better but basically like if you look on the top right side uh of my slides there is like a dev board and there's like a nice uh JTAG connector like plug there and on the other hand like on the bottom right it's like some uh random device I was looking at which you can see there should be a JTAG but it's never been soldered on and so I had to solder directly onto that stuff and the trick to remember here if you kind of like solder your own connector is uh you actually may have to add like extra resistors called like pull up and pull down resistors that connects like one pin to ground or one pin to like VCC and the point of that is actually I don't know the point of that but uh the reason is like if you don't have them it might not work well or it's more work most of the time it's going to be glitchy so um and if you follow the uh instruction from like for instance like a ARM specification for like what resistor you need so you might skip them but then you might waste time like understanding why it's uh weird really like fucked up sometimes so I would recommend like remember the pull up and pull down resistors. So that's just a random board that my co-worker was looking at just to show you that what I said is actually fairly generic uh on the left side we have like UART on the right side there's like some JTAG connectors uh up there is the um upright there is like the main CPU whatever that's under like a shield and usually you do that for um preventing um interferences so it's like a wifi module so that's why and the cool thing to know is usually the shield is grounded so like if you're looking for the ground on your thing like either the shield on that kind of stuff or like the casing around the ethernet port or whatever is also actually like fairly useful. So uh now that we learned how to uh look into stuff we need to know what hardware you need for um interfacing with what's inside.  So the first one uh you may want this pin or like the stickers I don't know if you're familiar with that but um actually like what's really important is like safety um so you know don't work with stuff that's plugged in the wall uh remember to unplug stuff work in like well ventilated room it's important.   I have a microscope so that's really useful for doing uh tiny soldering and the trick is I I tend to drink a lot of coffee and I shake so I would I was thinking like I wouldn't do that but actually if you um if you're careful and the trick is like really like try to um you know when you're soldering like try to hold your arm like straight and well grounded and under the microscope you can actually do a lot of stuff that's you just don't think you you wouldn't be able to use you would think you shouldn't be able to do it but you actually can. Sorry um top right side um it's what I actually use for doing tiny soldering and um it's like top right it's like flux so it's kind of um magic glue magic uh gooey stuff substance that you put on um pins to uh when you're still doing surface mount or pads so that it works really well so don't never forget flux uh bottom is um so like in the same uh size like uh copper wires like um NML wire that you use usually for making courage you know for like radio or whatever um and the trick is like the uh the wire is actually um coated in varnish so it doesn't short and you can use that to do like really tiny point to point like soldering and that's actually really useful and just next to it you have like some desoldering week so like the idea is if you put too much solder and you fuck it up uh you can put the week on top of the solder blob that you messed up with and you um put the soldering iron on top of that it's going to get really hot so you want to hold it through like the plastic casing and it's going to absorb all the like extra solder and you can clean up your stuff and then I put like a picture of like some random like uh jumper wires that are usually always useful to have and on the bottom right I have like that crazy uh heat gun uh which is like 60 bucks at Home Depot it's meant to like help paint dry or something I'm not sure but it's actually really useful for if you're on the chip if you want to remove components by just like hitting them and pulling them off so um more cool hardware um that's a multimeter um if you're not familiar with that it's you use it for um finding voltage of chip you wouldn't know like what's the actual voltage so the black goes to ground and with the red you uh with the right probably you select poking around to measure the voltage and the other really cool stuff is you can set it in continuity mode and when you have like connection between the two probes it beeps so if you do like crappy soldering job as I was describing just before you can make sure that you actually did a proper job where like everything is connected the way you think by you know making sure that both like when you probably both side it actually beeps and that's actually a fairly useful uh tool. The other one is this is a logic analyzer uh the name is like a Sally um that's a brand and the idea is the same as in software you have like ways of communicating between process like in hardware you have like ways of communicating between chips or the CPU and the chip and it's uh it's like a lot of different standards different ways of doing stuff but the logic analyzer like no lot of them and it's going to convert like a bunch of signals that come going on to wires into actual data and that's actually really useful that would be your eyes like when you're doing debugging you know and that's like a snapshot of like some spy communication uh which spies just some random protocol uh that's like an FTDI cable also come in many uh or serial cable it comes in many different shapes and stuff and the idea is just like when you want to do like UART you you plug that into your laptop and you connect the the other side to like the UART port and you get like a console it shows up at the serial port and uh as an example I have uh on the bottom right um like a Raspberry Pi that's you can actually get a serial console and you don't need like HDMI cable or anything to interface with that. Phew um the bus pirate uh also like cool device um it does um it's kind of dependent of the hardware hacking uh sorry the um the Sally the logic analyzer so instead of absorbing data it's like send data so it speaks a bunch of different protocol and you can use Python to program it so it's extremely useful if you want to like read flash program um microcontrollers and that kind of stuff. Uh more cool stuff like how about I was talking about JTAG that's the JTAG debuggers that you that's why we would plug actually like the the JTAG port on they come in many different like uh price range and whatnot so on the bottom right is a fly swatter it's like for like 30 bucks I think and it works okay and above it's like a J-Link which is more fancy and the idea is like the more expensive the stuff is the more probably the more reliable it is and it comes with like better software and why not. And on the bottom right the pink board is a JTAGulator that uh Joe Grant the same guy who made like those badges uh made and it's really awesome so if you have like that board you don't know what it is and there's like a bunch of test points and you're like I bet there is some JTAG somewhere in it but you don't know where you can just wire like all the test points to the JTAGulator and it's going to brute force like and try to find everything so it's really cool. Um another interesting important tool is flash readers so um usually flash contain uh the flash has the file system and stuff and um so you want to like interface with that and uh it comes in two flavor I guess you can do like in circuit stuff so you have like either like a little like clamp that you put on the chip or like the little rectangular one um it's a clip on um so like the Raspberry Pi is a good one for that and on the top right it's like a flash cat uh it's like useful for the rectangular one um but you can also like choose to like actually dissolder the chip and in that case you would have like some like like socket adapter so you can clip the chip on and then you plug it into your flash cat or whatever and it's going to read it and I will cover like why what's why it's important like both in a minute. And finally it's kind of the um last uh hope so let's say uh you know your flash is encrypted there is no UR there is no firmware so you can try to do like the more crazy stuff like fault injection and side channel attack so that's a chip whisperer so like more like uh kind of hard more badass device and the idea with that is um it you can do stuff like um you wire it and to the CPU and you're going to drop the power on the CPU for like a split second and it's going to like mess up some instruction and you can bypass checks and stuff that's what people use for like gaming console hacking and um that kind of stuff. So uh what do we do now um that we have an idea of what's inside the phone and um you know um the tools we can use so like I think there's three post potential track one is messing with the UR so the serial console and the one is messing with the flash and trying to recover the firmware or something and then maybe you can just do not the firmware online and the three are like valid approach and sometimes it's good to just go from one to another to another and you know whatever you find like some hurdle you can like go from one to another uh so I went for the UR first and that's what I'm greeted at by when I connect my fcdi cable to the UR port that I was showing before. So um we can see like on the left side the um the console thing it's like dot dot dot usually when you see that you can press space or escape and it's going to interrupt the booting process of the phone and in that situation uh but somehow here didn't work and uh I will cover that why in a minute and on the top right side of the slide um we can see like it read something from the EEPROM and he decided to set the console to DevNul so you don't want console to be DevNul because it that goes nowhere but that's interesting to know that uh there is some potential there and finally it says like booting Linux so it confirms that it's a Linux board but then uh there is nothing more uh after that. So that's some good thing bad things so the problems are like nothing shows up after like the Linux is as booted probably because of the DevNul situation and when I press key it's not working and uh I'm going to like come up with like why that happened. So at that point I was still like okay I cannot go anywhere so I'm going to try to recover the firmware to try to understand what's up with that. Uh multiple ways of recovering firmwares uh it's like an easy way that you can download it online and actually that worked um there is like some cool snapshot of that uh sometimes it's encrypted uh on our case it wasn't um but yeah so you can also try to sniff a firmware update uh that's really useful for IoT devices in general and you can use a network tab which is like the thing picture on top right of the slide um or you can mirror your port on your switch or whatever uh the problem with all of that is oftentimes the firmware update is going to do over like HTTPS or it might be and in case you're like stuck and then you want to go for like a more like uh hardware hacking stuff like you're going to dump a flash for instance and like even more advanced technique that I will briefly like uh talk about. So uh how do you dump a flash? There is two ways as I was mentioning just before. You can do in circuit programming and um out of circuit. So in circuit is cool because you don't have to you know like remove stuff, break stuff. But there are a couple of issues with that um the main idea is to read the flash. You need to power the flash and if you if it's like still plug into the whole system uh it when you're like Raspberry Pi for instance going to try to read the flash it's going to try to power like everything else that's connected to the same like in a bus line uh power line.  And in that case uh but it's just not going to work. Or maybe your Raspberry Pi is really strong and it's going to do it. But then the problem is the CPU start booting and it's going to talk with the flash and that's going also to mess up your stuff. So in that situation you have like two choices usually. Either you can try to like lift up the pin, the power pin so that it doesn't happen. Or you can kind of go a little more like yellow approach where um you want to keep the CPU in reset. So I've been successful with that many times where I just like start like trying to ground like random test point of stuff on the board praying that it's not going to fry anything. And usually at some point like I'm going to notice that okay when I do that the phone like light up but doesn't boot. That's good. And maybe better way is you can disorder the chip with like the big like heat gun and whatnot. You have to be careful though because if you like pull it too hard you might like bend pins and it's going to be like useless. Um but a good trick to know is like you can make a little like um tin foil a hat or whatever like around the chip. So that like when you're heating up the chip like the stuff around it doesn't like get too hot.  And uh bottom right is like I was I was I thought I would have to do that on the phone so I wanted to practice first and it just took like some random uh USB thumb drive and I tried. And actually that worked really nice but the trick is remember I also put some plugs because it helps.  A few other tricks that actually didn't work on this one but are good to know. Uh U-Boot is the most common boot loader uh not used here but it's you would find that on pretty much every other device. And the idea is you can also interrupt the boot process of that and ask nicely U-Boot hey can you load the flash in memory? And then uh U-Boot is also able to dump in a hex like in the screen um the the memory is this way you can just like write a little script to do that and recover everything on the flash which is really cool. But usually like Dev don't like letting you do that so they set they lock that approach so you cannot like press space to interrupt the boot process. But there is a trick that I also call like the YOLO approach where um instead when the idea is like U-Boot is going to load from the flash like the Linux kernel or something. But if while it's happening you start like poking with you know like a wire that's connecting to ground we start poking the NAND flash like some of the pins. It's going to mess up like what's uh U-Boot is reading.  But yeah as I said that one wouldn't work on our phone unfortunately. But uh just some cool example that I found online where like some guy actually did that and uh I underlined in uh in like purple on the bottom like the command you need to use to like load the the the flash in memory. And on the right side this like little block of text is um sorry is how uh U-Boot is panicking and be like eh I don't know how to do with your kernel and give you the the shell. So um how um so a few more like tricks for um dumping a flash so you can use JTAG so you usually have uh if JTAG works you're going to be able to um dump the RAM. So in this case it wouldn't be really like dumping the flash, dumping the firmware but in a lot of devices it's going to load the whole stuff in memory so if you can dump the memory you are like the same you're pretty good there. And kind of the last hope but that's that's one of my favorite about hardware hacking is you can do crazy uh stuff. Um an example is uh ScanLime um that person on the internet who um she had um a Wacom tablet that she wanted to dump the um the firmware off and there was like no attack surface at all. But she realized that like when you- it was like USB stuff so when you plug your USB um onto the laptop there is some USB exchange that I don't know anything of but it sends like USB descriptor and if you- if you glitch the CPU just at that time um it's going to like you know miss like a boundary check and it's going to send way more data and with that uh she was able to recover like the full flash uh full memory or something so it's really impressive but and that's what's kind of awesome about hardware hacking. Okay so uh we recovered the firmware uh in my case just by downloading online but what do we do now? We want to analyze it um the best tool for that is called Beanwalk and it's just like a basically a magic dictionary that's going to like look at your binary blob of stuff and be like hmm halfway through it looks like there is a zip file and I'm going to try to extract it. It tries maybe it works maybe it doesn't depending if it was actually a zip file or not. And it does that for everything and uh it might also find like compressed file system uh such as like SquashFS, GFS2, et cetera.  So Iron Beanwalk on so that's the firmware file I don't know it from the website and once again it's easy mode it's just a tough file and they already like split it up into boot one, boot two and GFS2 so like Beanwalk has nothing to do here really and I can just extract the whole file system and that's cool because from there I can look at what's on the phone just like that. Uh but a more interesting use case for Beanwalk is I took like a different firmware file for like another Avaya phone and this one is way more interesting because Beanwalk for like a way more lot of stuff and uh I've highlighted like you find the file system and it's able to extract it and there's also that random like MySQL stuff which I think is a false positive then you need to be aware that there will be a lot of false positive so don't blindly trust what Beanwalk says. And uh really really funny stuff that I was looking at uh if you run strings on that specific image um you will find like your boot strings so that phone is using your boot and the cool stuff I was describing before would actually work on that one. Look. So now that you have extracted the firmware file what do you want to look for? One of the most uh interesting stuff is uh finding the update mechanism so assuming you have uh encrypted firmware like if you reverse engineered that you might find the keys and how to extra to extract like subsequent uh firmware updates without having to do like any hardware stuff. And then you want to look for like secrets like passwords, certificates, and so on and so forth. Uh originally my goal was to look at the main binary so that's the thing that deal with like the VOIP stack uh that would make sense to look at that and um also you want to look at the bootloader because it may have hidden commands, explain why our console is not working, that kind of stuff. So remember the point of all of that was I wanted to get a shell on the device and um what's the process to do that? So the one I want to go for is I want to fix the serial console you know like the dev null was not cool. Um but also our approach would be like messing with your boot arguments. Um not working here obviously. Uh but you know you can specify the console you want and uh the init script can be marked as like oh I want bin batch as my init script and then you have a console. Uh and like kind of last resource stuff that I was considering as well is you could patch the firmware uh in our case the firmware is signed so uh that won't work unless you try to glitch it which really hard. But you can also try to patch the file system so the way bin work extracts the gffs to stuff you could modify it and then like repack it and uh flash it directly onto the flash. And that's usually like a foolproof method like that would work in most cases. So instead I'm going to uh reverse um the boot loader and um for that um because I wanted to understand like why dev null uh the console was set to dev null and stuff. I'm just going to cover arm in much detail and I would recommend you to check out azalea lab tutorials that cover everything. But just useful here um there is this notion in arm of literal pool which is um inside the code section you would find some little chunk of data and that's usually like used to like load in registries um absolute addresses, eminent values, magic offset that kind of stuff. And uh FYI like in case of function calls um the arguments are going to r0 to r3. The return value is r0 and when you like want to return from your call it's the return address is set in like the lr register that stands for like link register. Um and the big trick the big problem here is um we like trying to load the boot loader so it's uh it's not really half. So if you try to import an IDA IDA doesn't know what to do with it. And so you need to find the loading address for that stuff so it matches what's where the phone is loading it. A couple of tricks um sometimes you're lucky and it's going to be printed on screen. That's our case, sweet. Um but otherwise you want to look at like header file, headers in the in the boot loader file or maybe reset vector so that's stuff where like specifically for arm like the CPU needs to know uh specific addresses where like there's like a reset or like an interrupt happening and it's like fairly like um square like that's how that's being set up. So if you find that and you find like what looks like an interrupt vector handler or something you can find like oh that must be the address.   And uh QuaxLab uh what the blog post about it so I would recommend you checking it out. So as I said in our case it was easy um when it boots it prints uh the jump whatever and that's actually why you should load it. And um on the bottom right side it's like a bunch of strings um so it's a bunch of addresses that point to strings uh that's a snapshot from IDA and um we can see that um it's like all the strings like prints well so that means that it's loaded properly it's not like all messed up. Okay so now that we have the bootloader and we can like work it with IDA uh we want to search for strings uh we were looking at you know like that sentence like EEPROM red successful whatever. So we can use like cross reference in IDA to find where the string is coming from and look at the function that handles that. And uh yeah just as a FYI about like EEPROM and SPI so the EEPROM that's on the board is called a SPI EEPROM and SPI is just one of the protocol I was mentioning before. EEPROM is just like a small memory um whatever and the SPI uh protocol is just like four wires um and the IDA CPU is in command and the EEPROM like reply and uh that's a dump from like using the logic analyzer to have a look at what it would look like in real life. Uh that's the data sheet of the EEPROM uh that you can find online if you Google the the label on it um the cool thing is in uh bottom left uh of the slide it tells you like the pin out so you know how to connect stuff to it and on the right side of the slide um it tells you which uh which opcodes are for like the SPI command SPI command you would send. Like for instance if you send uh three it's going to be a read if you send two it's going to be a write. So that's uh that's something useful to know right. Let's oops sorry. Back to the um back to the IDA and uh the bootloader we can see uh the strings we were like looking for. And something extremely promising is we can see the console set to dev null but above that is like a burn towards the option and it's actually saying like oh setting console to like your zero or your one. So we want these ones to be printed. Um that's a that's a function that actually does print that string. And uh what it does is roughly uh clear some bytes on the stack and then read from EEPROM from a magic address and it's going to compare like bytes um that has just been read. And if we have the good value it's going to actually print on the console setting the console for uh dev TTY uh AM zero. So we want that to happen. And something you can ask me is like well how did you find that read EEPROM function you know? And there are like multiple ways of doing that. Uh the easy way that I've highlighted in green that screenshot is uh on the bottom right we can see some like logging that says oh EEPROM read fails. So I know that function is responsible for reading EEPROM. Uh but maybe maybe the firmware you're going to look at is not uh as verbose. In which case um as I said before like to read from an EEPROM you know you have to send three over SPI. So if you find like the SPI function and you see like sending three you know what it is. That's what I've highlighted in yellow. Um but then like how do you find the SPI function you know it's kind of a problem. Uh two ways for that. Um in my case once again it was logged so it was easy. But usually like the main trick is um you have this um something called uh mapped uh IO a memory mapped IO. It's this kind of like magic address values in the CPU that like if you write to them it's actually going to send data over like your auto-SPI or whatever. If you're lucky you're going to have a data sheet for the CPU and it specifies exactly what are those addresses. If you're unlucky um you have to guess and that's really hard. And at that time maybe you should like look for something else or just you know you're going to spend a lot of time on that. So we want to uh patch EEPROM now so we can enable the console. The plan of action is we know the read write commands. Uh we want to interface with the flash. So to do that um sorry the EEPROM. We um we need like a SPI uh device and we've seen before the best pilot is a perfect candidate for that. And we need to connect to the chip. Uh unfortunately like it's a really tiny one and so like the clip I was showing are like too big for that. You can buy online some rigs that people make. Um it's like expensive and it's like five weeks to get them. So instead I went kind of crazy and I tried to solder like um little wires onto the thing and I was like honestly that's never going to work. But it happened to actually work and I was like sweet and it's all like look like cross because it's like all the flux that remains on that. Um and that's the code for the best pilot. It's super like boilerplate uh thing and when you just write um this magic values that uh sorry I was asked to like replace them with like emojis. But uh the idea is like just after that you're greeted with console is set to uh like UR0 and now we have a lot of more stuff uh printed on console which is great.  I was like trying to press space or type like it's like asking for oh you want to look at the root what's uh it have nothing happened like uh what happened it's like just cut the trace so you can send data. We have the board so we can look at it. Um so that's just like uh zoom up on the thing and at that time I was like you know looking at the UR pad and start like with my multimeter like trying to beep out like where it goes you know it's following the continuity testing and eventually it reached some vias like on the right side and vias are that little hole that connect one side of the board to the other. So the trick is the actual like trace that connects the UR to whatever uh goes to the other side of the board and goes back and forth and it's like really like tedious and eventually I kind of lost where it was going but it was kind of under that's a mod port uh that's like the keyed uh RG45.  So I kind of went without like my YORU approach again where um I was holding like I had a plug like a jumper cable in my um the transmit of my FTDI cable and uh so I had this acquire that I could poke around and on the other side I was like pressing keys on the console trying to see if something happened and meanwhile I was poking like to the pins one by one and I think at the second one it actually worked and I got greeted by uh like a wood shell on the device. So that was cool um but the trick was um on the right side of like a proper setup where I plugged my ethernet cable into the plug. So at that time I didn't know where the keyed RG45 was so I just like send it the plastic of the ethernet cable so I could like shove it in and yeah it just works so that was nice. Uh now that we have a wood shell it's like uh a lot of debug strings printed on screen and that's annoying. So um the trick was like I want to kill the process that's like that's like floating my console. Uh there's a watchdog that's going to reboot the device if that happens so I highlighted in the corner like the magic command that you can run to kill that. It's just like randomly new stuff. Uh so now we can finally do some very research um and really what you're doing to do is like look at process running, poke around, look at open ports. Uh that's just a bunch of processes uh you see cool stuff but uh that's when it started getting interesting. I think at some point I was tired of looking in Ida and I was like I'm going to poke more and I looked at DH client and I was like huh. And the really funny part is like I run DH client and I see this like 2007 copyright. I was like whoa that's not a good thing. And if you just run DH client by itself it's like false. I was like that's weird. And if you like run it the right way they print a debug message and say oh we modified it to work it better on our device. And like huh that's interesting. So if we look back uh and we look for the specific version it turns out that ten years ago actually for DEF CON uh John Oberhide um really is like an exploit for that specific version of DH client. So I was like whoa that's that's easy. Um but we want to make sure that it's actually still vulnerable. So the um the fun was like uh done like um stack overflow like while handling DHCP options. So we can compare the original source code with the patch source code because DH client is open source. And just keep in mind that might be like mitigation like stack cookies ASLR or whatnot. So if we look at the vulnerable code on the left side we can see a mem copy like highlighted in red that's using like data.lan as a field while on the patch version that replies that by four. So the idea is like this bug is uh subnet uh it's a bug in the subnet mask handling and the subnet mask should be like four bytes. But the way the DHCP option works is that you can send up to like 255 bytes I think and it used to just don't be trust whatever you're sending which is bad. And if we look in either like the actual like function uh we can see there is no stack canary and the on the left on the right side it's it's not using that magic four so it's it's obviously like still vulnerable. So it's time to exploit stuff um working on the phone was tedious I was like I'm going to try to set up QMU um if you're not familiar with QMU it's can do like emulation of arm stuff or like many other system and can do either like user land or uh full system. User land wouldn't work here because uh DHCP is uh I'm going to try to modify an IP and stuff and that's not cool. Um but instead we can do a full system emulation and uh those tutorials tell you how to like set up a cool image for that. And we just want to set up the network properly uh because we want to interfere with DHCP and naturally um the um QMU stack would give you a DHS DHCP for yourself so you have to be stay like don't do that and yeah. So I run it in DH in QMU and after like all these effort and it's still stake fault. So it turns out that like I modified stuff that actually causing the stake fault. And um I'm not going to cover that too fast because I'm running like a little out of time but uh basically it's creating that weird socket uh it's like a name socket and uh so that's like the main application that the VOIP stack can talk with DHCP and share options. Um and uh the idea is like that's like the main application and we can see what it does. That's not super interesting. But basically on the left side it's like sending data to configure the DHCP and that start up. And on the right side I re-implemented that in Python and uh you can run that with like sockets um so socket is just a kind of like netcat in um and more like um more tools so you can create that name socket. And the idea you run the Python script on the host, you run sockets on the um QMU like binary image and it would just work. So now we can start debugging uh DHCP clients. Um you can run GDB, we can try to like actually run the Poc.  So instead I use a skype based exploit skype is a Python library that lets you like you know mess with uh network packets like a fairly low level. And um so now we want to like reach the um you know the venerable code. So um I was explaining the bug is in the uh subnet mask option and how the way DHCP works is like a tag length value like type of stuff. So you say oh this option is a uh subnet mask. Here's the length, here's the value. And obviously it was like it should be four but if you send more it's fine but for the DHCP specs. And so we can use skype for doing that. So uh that's my like a chunk of my skype script and just the payload in proper is interesting. Um basically just sending the subnet mask option with uh data that would be like a a a a b b b c c c and so on and so forth. So that uh when it crashes in GDB uh we can see like all the register we control and we know um exactly um you know if we want to affect which register which chunk of the shell code we need to change. And we can see that we're actually controlling uh PC so the program counter so that's good. We know that we have um we can exploit we have control for execution. And there are like a few more details that I'm going to skip over but um we need to scour uh to craft a shell code now. Uh unfortunately like it looks like the stack is uh you cannot execute the stack so we would have to warp it and that's tedious. But um cool stuff is like if you look forward there is like a system call in that binary that I think also have added and we do control R4. So if we put us if we can put a string in memory in the address we know it could be a valid like system command and put the address in R4 we're good. Um so if you think about it DHCP client DH client sorry we see configuration values from the server so probably there is going to be stuff that are stored somewhere and um it turns out that ASLR is disabled. Sorry I forgot to mention that. Um but then we the idea is like I just send like sending like AAAs as like a bunch of uh options and look at memory to see if I could find that string somewhere. And eventually eventually the domain option was a good one. And so um now I can show you uh a cool demo that's how I happen when you actually run the whole stuff. So uh on the um on the left side we have the phone that's booting. The phone is booting and the window uh the terminal window is like on the left side is a rogue DHCP client running on payload and on the right side is um it's a web server that will uh provide like a post exploitation kind of stuff. So we can see the packet is running. To make sure it works I was cutting uh dev random on the uh on the screen so that's why it looks like noise. It's fetching data from the um web server and I don't know if you can see but it's loading like Steve.data uh FYI like Steve is actually my uh boss that's him. And uh just then I'm uh still like I don't know if that sounds but I'm exfiltrating audio from the main speaker and you can see in VLC that when I'm speaking and clapping my hands it's actually like reading the audio on the attacker laptop. So that's pretty scary because you can turn your um phone into um you know listening device. Thank you. So um yeah and just FYI like all the tools I use are like actually already on the phone like you are like net cat you are like stuffed to like dump the audio to like STD in so you it's just like writing some shell script like bash script whatever.  You want to segregate your network um to make sure that for instance the coffee pot is not able to like send weird DHCP packets to your phone and make sure I actually patch your phone because you know it's the patch got released like I released the patch like a month ago so make sure it's actually patched. And uh if you want to think about why this kind of bug happened like the technical debt is really hard to handle and you know probably like someone had like in good face um okay um someone in good face um fork the stuff and they forget about it so um and just as a real quick um embedded systems are not black boxes so I would just encourage everyone to um look at them and uh you can do it now as well. So if you have questions uh ask me on Twitter or whatever and thank you for your time guys.


Welcome to Start Up The Storefront, presented by Aurobora. All right, welcome to the podcast on today's show. We're talking to David from Modigo Brands. Thanks for coming on. Freebo don't know. What does your company do? Well, we're a brand development company. We're primarily focused on Latin brands. We're interested not only in authentic Latin origin, but also pay attention to sustainability, healthier products. But really, you know, this is kind of an opportunity to say there are other great Hispanic foods other than Tex-Mex. Very true. It's Hispanic Heritage Month, so that's fitting. Kind of nice. Good timing. Thanks for being here. What made you want to go into the brand development company or the business, I guess? Well, our company has been in export management for about 25 years. And in that business, we've represented some really great food and beverage companies. We used to work with Jose Cuervo with their chilula hot sauce, until it sold in McCormick's for some ungodly amount. And then we worked with Grupo Modelo in Asia, represented them in Japan, for example. Took it to the number one imported beer. We're working with Novamex, which is Haritos and other brands. So we've been in the Hispanic sector for many years. And I've learned about this amazing crossover potential for Hispanic food and beverage products, both in the Hispanic sector, both retail and food service, but also crossover to a wide variety of restaurants and stores. What are some of the things that I guess you learned early on that if people are listening and they have a CPG company and they're like maybe emerging or trying to go into new markets, what would you say the thing that you took away? For example, as you say it, to me saying, OK, taking Modelo and going to Asia. Now, maybe today that sounds straightforward, but maybe at the time, that sounds unbelievably crazy. And so what goes into that? Well, that's definitely a hard business, starting from ground zero. And Corona, while it was a great beer here. Now, of course, Modelo is the number one beer in America. But at the time, Corona wasn't very strong in Asia, for example. So we kind of had to start ground zero and promote the concept of Corona as a vacation beer, a holiday beer. And the Japanese really took on to that after a while. OK. So it's a lot of brand development and then strategy around what are people really, what are they connecting it to, which I guess when I think about Corona, it's sort of like the palm tree, the beach. OK. And so at some point you go, all right, I've seen this happen enough times. I think I have a playbook or a game plan. Let me go ahead and try to roll the rock. Exactly. OK. And then what was the first thing that you went with? Well, we started Chantico agave a few years ago. And I noticed that we're only several agaves in the market. And to be honest with you, without naming names, they kind of look like shampoo bottles. And so my goal was to create something that was colorful and bright, proudly imported from Mexico, a Hispanic product. Again, a lot like Cholula in the hot sauce category that you could put on the table in a restaurant. You can use it in the kitchen. You can use it in the bar. And it goes in all kinds of retail settings. OK. So the agave number one. Agave number one. OK. And then when you start working with the brand, what is known or is like all of it off? So no logo, no branding, nothing. Now we start from ground zero. I really focused that design on the bottle, for example, and on the powder. We're also the first people in America with a Chantico powder. The idea behind it was the piña. That design is a piña, which is the heart or the pineapple of the agave plant. So obviously if you take that and you take it for a number of months, it becomes agave. Agave becomes tequila. But we take it before it's fermented and turn it into a really delicious light sweetener that's been around for 10,000 years. And let's say with this specific product. So there's a sweetener that's been used for a while. But now you're introducing it to, let's say, the American market. That's right. And so today you can say, people kind of put agave in their coffees. We see that enough. But I would say five years ago, I'd never saw that. Or maybe 10 years ago, I for sure didn't see that. And so how much consumer education is happening while you're talking to these retailers around like, hey, look. This is a thing and people use it? A lot. Because obviously people are used to traditional sweeteners, some artificial sweeteners, and some of the other sweeteners like Stevia, which has an unusual taste, I think. What's so great about this product is the taste. It's the lowest GI. It's vegan, unlike honey. It's got a really good price point. It's water soluble. So it works in cold or hot beverages. So that's what we love about it. It's really an all-around sweetener. And sweeteners, as you know, like salt, sweeteners go in everything. A lot of our trouble is to try and define usage because the usage is anywhere that sugar or honey would go. What have some of the best marketing examples of success that you've found? Is it television? Is it social media?  Well, it's definitely broadcast like this. Getting the message out to folks. Social media has changed. It's a little bit more difficult now than during COVID. What do you say that? It's crowded and it's hard to really stand up.  But it is harder now for a small brand unless you have somebody famous touting it. Really, it's a lot more difficult. Do you guys ever work on something like that? Maybe getting on some celebrities or just some, maybe use some chefs, some influential people in the food space. Exactly, without naming any names, I just met at a show, Subur in Colorado. Just this last weekend, we talked with a chef who's half Mexican, half English and he cooked for the Queen of England for a number of years. And he's very interested in using Chantico Gave in a lot of his recipes. Does the Queen of England like tacos? I don't think she does. What is the Queen of England eating? Well, I don't know why they brought him on, but obviously he provides a diverse option for the Royal Family. And he's a fascinating guy. And we're working with several other chefs, one in Colorado, one regional. And one, I won't mention her name, is on NPR National Public Radio. And we're meeting with her. So the idea is to start through our chefs, teach people how to use a Gave in some really creative ways. Okay, so let's pretend there's an entrepreneur listening to this. They have something that they make in-house. They like it, call it a Tabasco, call it their own barbecue sauce. Great, let's say you meet them and then you go, okay, here's what we do. So you take the idea and then you have a company of human beings at your location. A part of that company is dedicated to marketing it. Another part of it is probably producing it at scale, co-packing. And so how much does it take for a brand to actually not so much get on a shelf, but more so be ready to say hi, consumer. What is, how long does that take? How much money's involved in that? What does that look like? That's a good question. I think it's part magic and part practical, right? So it takes, I think something unique that might be a family story or geographical story that you can put into a format that is interesting to people. For example, for ours, we use a ketzel bird, which has a bird of a long history in Mexico as being a very important symbol of Mexico. Chantico is the Aztec goddess of home and hearth. There's a lot of thought behind who did. So put a lot of thought into the brand and the story. In terms of the practical stuff, that's not my side as much. I'm more the, let's say the creator, but the people that I work with are really good at dealing with shelf life, pricing, packaging, the ability to produce it consistently and effectively and get it to the retail shelf or the food service restaurant in an effective way. If you can't do that, it doesn't matter how beautiful the package is. Sure, sure. And then when it comes to the marketing side of it, what happens then? Well, you kind of build it with what you're doing. So we have a broker network in the West United States. So they oftentimes tell us, this is what this retailer is gonna require. So for example, we're in United supermarkets in Texas. They came to us and said, okay, first to get in, you have to do a fill, a free fill and then pay a fee. And then you're working with distributors like Khe and UNFI and they also charge you. So you have to create a really smart and a gross profit margin so that as you make sure at the end, you have a good net profit margin, which is your profit. Yeah, you mean like slotting fees? Slotting fees, yep. I never understood those. I still don't. It's just a permission to get on their shelf. I know, I don't really do anything for you. It's kind of wild. Right, but it is kind of, if you will, then there's all kinds of, there's different kinds of marketing, as you know. There's field marketing, which we're using hang tags and things right on the product. Like we used a hang tag on Monserro de Hia barbecue salsa, which was, if you don't like it, five times your money back. Oh, wow. Right, and we put that on there. We only had five people out of thousands that took that up five times your money back. What is barbecue salsa? Explain it to, is it, is it that literal or is it? It is that literal, exactly. It's barbecue salsa. We played the game with sauce because people don't know salsa is just Spanish for sauce. So it is a barbecue salsa. In other words, it's great on carnitas and all kinds of meats, chicken, pork, but it's also a full service. Like you can do dipping with the, for example, our New Mexican barbecue salsa is made with hatch green chili. It's really delicious, nice, sweet. Our Sonoran product has chiltip and peppers and that's a little spicier. So, you know, it goes on all kinds of meats and dips and marinades. Yeah, and then with the coffee, you're in the coffee business. One of my favorite things to tell me about this Avenida's coffee. I love Avenida's coffee. I've been working with our partner in Guatemala for 15 years. He's our distributor down there. They're a large farming group. Guatemala is a beautiful country. It's really sustainable the way they built the farms. And there's nine different roast areas like Antigua or Hueyue to Nango. So we're focusing on Guatemala and coffee to start. Some are bolder, some are light. Chilero blend is our blend. With the idea that if you can see on the package, it says it's basically direct source.  Those brokers add on money. Coffee's not as fresh. Our packaging facility is in Guatemala City.  How many different food products do you have under your umbrella? Just these three. Okay, and then are you stopping at these three, focusing on these three or are there more in the works? Well, you know, I would say there's a Latin brand factory in my head. I've got a lot more ideas, but you know, we have to focus as much. Again, even three products is a challenge, but I wanted to present them as a group with this idea of, like I said, innovative, healthier, sustainable products. And then capital. So right now you're doing a capital raise. Let's hear a little bit about that. Yeah, we're doing a capital raise on Start Engine, Orogo brands. And what is Start Engine? People who have never heard of that. Start Engine is a crowd funding platform where people can go if they wanna invest anywhere from $250 to upwards of, you know, whatever they choose. And it's an opportunity for us to create a group of people that we can work with as our partners, as opposed to straight equity funding. Okay, what is the investor getting in return? They get a part of the company and shares. And our goal is within at some point in time to have a monetizing event for the product, acquisition or larger investment from a company so we can scale up nationally. Yeah, and right now you're raising how much? Our goal is to raise 1.23 million. And you said you were about halfway there. We're at about almost 400,000 now. Okay, and how are you getting the word out about the raise, obviously like this, but another, how do people go ahead and like find this ability to just invest in your brand and your company? Well, there's three ways. There's our community, which is my Tio's, Tio's, you know, all my extended family lead investors that we know from our manufacturers. There's the SE Start Engine investors. The Start Engine has a large group of investors that are on the website that are looking for cool products. So they invest and then there's the public. And so I'm going out to the public meeting as many people as I can, people who are interested in Hispanic products. But I think people that are just interested in Hispanic products in terms of the opportunity for profit and the opportunity for acquisition. Yeah, it's big. We had Tia Lupita on, obviously Hector, he's been on Shark Tank. Any thoughts on going on Shark Tank? Any desire to get on there? I have somebody on my team that was on Shark Tank and for another product. And I'd absolutely love to do it. Kind of, to be honest with you, the Start Engine raise is kind of like that because I'm presenting quite often, but I'd have no problem going on there and sharing my ideas because I think, I think they'd be interested. You should apply. Yeah. We've had enough experience on that. It seems like, I mean, at a minimum, it seems from what we've been told, usually brands make like $200,000 on their website within a week of the airing. Okay. And so that's pretty significant. Obviously some make more, some make less, but it seems like the average is 200,000, but the exposure is almost certain.  It says like, it's one thing to just, someone goes to the website, orders it from their house, but it's another thing when people drive to a grocery store, find your product in a sea of aisles and then boom. Well, and I love the idea that it also points out some ideas to make the product line better. And that's important for you because when you're a creator of a product line, you believe in it with your whole art. We just spoke to, No, she was the company and he was making food paint. And so simple stuff. So it was like organic, I guess blended things, but that kids can write on like their pancakes and stuff and they can eat it. And so when I asked him like, what was the biggest challenge that he was having from a marketing perspective that he thought Shark Tank would have been it, he said, nobody understood the product. If you see it in the grocery store, it's not clear from a parent's perspective that a kid can take it out this tube right on their food and then eat it. It's also not clear that they're not gonna take out any other tube that looks like that and right on the walls. And so he saw Shark Tank as nothing more than an opportunity of like finally educating the consumer on what it actually is in a visual setting. And I was like, that's really interesting, right? He wasn't looking at it from like a Shark Tank TV perspective. He was just looking at it like biggest bang for my buck educating consumer. That's smart. And it was like, it was interesting to get that perspective. The first time I got that perspective. Yeah, it makes sense because you were just talking about barbecue salsa. What does that mean? And agave isn't as well known as I would like, especially the fact that with growing diabetes in this country, it's the lowest GI sweetener. And you can use last because you use two thirds of agave instead of a teaspoon of honey. One thing you mentioned before around partnering with your retailers, I think that's another sort of pro tip that CPG companies, some that have come on this podcast understand where it's like instead of trying to solve a problem and force the problem into the retailers, they'll start the conversation with the retailer and be like, what do you think is missing? What are you seeing as an opportunity? How much time do you spend? Or what can you tell us about any insights in that world, specifically to like the Hispanic products and how they're being viewed by these retailers today? Hispanic products are still kind of confusing for a lot of retailers. Does it sit in the Hispanic or even a lot of people would call it a Mexican food shelf as opposed to a broader Latin group? Like the International Isle or something? International Island area, the Hispanic area. And what we're saying about our products, for example, is they go in both. You know, obviously our gave goes anywhere, pancake syrup would go or honey would go or sweetener. I mean, it also goes in a Hispanic set. Same thing with barbecue salsa. That can sit right next to Heinz and whatnot and compete but it also can be in a Hispanic section. So we're actually doing a show this week in Texas, Cocina Sabrosa, which is just Hispanic food products with the idea and a lot of the big retailers like H-E-B are gonna come there so we can share Hispanic products and how they might be differentiated from traditional products. So there's really no, they don't know yet. They're still playing. They're struggling with it because I think America is struggling with what it means to be Hispanic because there's still some questions about it. You know, are you Mexican? Are you Venezuelan? Are you Cubano? Who are you? My family is Hispanic. We're from New Mexico. We've been there for 400 years. Nobody knows about us. You know, we're a small group of Hispanics. So, you know, it's hard to explain just like Asians. There's so many different kinds of people from Asia so there's not one group of Hispanics. So people tend to go, it's easy for them and that's the phrase is Mexican. So that's not a criticism, but that's just what we're knowledgeable about. Yeah, being from Peru, I mean, I don't think I've ever seen a Peruvian, and very rarely would you see a Peruvian product in something that wasn't the international section if you found it at all, frankly. You'd have to go to a special bodega or something and like here in LA, there's a bunch of them. But if you don't know, you know, it's certainly not, it's a homegrown bodega. It's not a big retailer. And Peru has a lot of amazing indigenous agriculture, you know, and so yeah, we're kind of losing sight of the broader and that's what's so exciting is that along with French and Japanese that Hispanic foods to me are this amazing cultural opportunity for all of us to learn about different Hispanic cultures. And even within Mexico, as you know, because I know you've traveled there and Southern Mexico, Zihuataneo and those areas, all kinds of amazing culinary things that you don't see in a Taco Bell or a Tex-Mex restaurant here in the States. That's very true, yeah. And I guess people's first introduction to that usually is a restaurant. So first they have it, they have it a couple of times, like Molle or something, then you fall in love with it and you're like, how can I make that? How do I buy that? And now you're all of a sudden looking for it maybe in a grocery store or just trying to make it yourself.  A lot of the restaurants are top restaurants in the world. When it comes just to go back to the brand stuff that we're talking about, when it comes to, I guess when you look at it high level from a market perspective, macro perspective, how long does it take for a CPG company in this space to be acquired? Is there a number? Like usually we see seven years to 10 years in the tech space. Is it similar in the CPG space? Is it longer? Is it more of just a revenue play? Obviously probably all of it. But what have you seen? What exits have you seen? What gets you excited about doing this and sort of you're at the end of the day, you're light at the end of the tunnel? Well, I think it's definitely possible within five to seven years, you know, historically it's been longer. But large companies, you know, large food and beverage companies are acquiring these brands, not just Hispanic brands, because it's hard for them to develop new innovative products. So they're kind of on the hunt for a lot of different CPG food and beverage products a lot quicker in their development. But you definitely have to have a proof of concept for sure. Yeah, and what is usually that number? Is there like a inflection point whether it's number of stores? It's usually sales. And I would say probably the first inflection point for a second stage investment might be in the four to five million dollar range. And a lot of investors that are interested there. And then beyond that, you know, really 20 to 30 million. Where are you guys today in terms of, let's talk about locations. Where can people buy you guys? Well, they can buy us on a lot of stores. You know, we're obviously focused in Denver. We're in a lot of stores in the Midwest. We're just getting into the Pacific Northwest in California. We've just gotten a broker here. And we got set up in Kahi warehouses here in California, Northern and Southern. We're in Texas and a number of supermarkets like United. And Publix came to us from the Southeast interested in our products. So we're looking to do a project with them starting in December, January. And what is the most surprising state to you? Where you're like, man, these people from Indiana love my barbecue salsa. Well, I think it's interesting because we are focusing on the West because there's a kind of a connection. I think the South, you know, it's interesting. There are a lot of Hispanics in the South. A number of Hispanics are there for agricultural reasons. So there is a familiarity. The Hispanic food is a little different down there. But you know, people are really open to trying new things. And if you think about it, young people, early adopters, they're very similar, whether they're from Birmingham or Denver or LA. They're all online and they're looking for cool stuff. And that's what I think we are. So let's talk about the younger generation that is on TikTok. Are you guys on TikTok? What's, how do you attract them? TikTok, yeah, is just disruption, you know. I've actually been using my daughter on TikTok. She's 12 and she's a chef. So she's been using our products and she's really cute. Her name's Jane. That's awesome. You know, again, TikTok, you know, we're doing a lot of work on investment right now too. Now, generally that's an older demographic. So you're using more Facebook orientation there, let's say. But definitely Instagram and TikTok are more directed towards young folks. Strange enough, I've relearned about LinkedIn. Okay, tell us what you've seen there. Huge social networking for business people, investors, used to be, you know, it was for getting a job many years ago. So what I always looked at it, and now it's a really, it's a way to connect with, as I said, investors and, you know, a lot of crowdfunding resources. Yeah, I'll give people a real life view into this. So when we were doing the Bendy Boy, we did a brewery. And when they were raising capital for their operations, they really didn't know any investors, frankly, when they started this whole thing. And so at some point, I was like, how are you guys getting these meetings? Because we're doing, it was during COVID. And so we ended up doing a lot of these like virtual meetings, but to an investor group. And we did like a 10 of them, maybe. I was like, how are you guys getting all these investors? And they're like, if the title of this person on LinkedIn says investor, they have a DM from me, and then I'm just following up, following up, following up. That's exactly right. And it works. Yeah. That's the thing that shocked me. You just have to go after me. You gotta stick with it, though. Yeah.  A lot of times. I have like 60 unread messages in my LinkedIn. Yeah, exactly. And I barely look at it. Yeah, it's too much. Yeah, so you get to seven or eight. And again, you still have to still jump out of the crowd for them to even open it. And a lot of them aren't right for you right this time, but that doesn't mean because when you're an entrepreneur, you're constantly looking for investment. Even after this crowdfunding is over, we're gonna start working on second stage investors. So we're gonna keep all these contact information and then keep working at it as we go. You never stop. No, it never stops, especially in CPG. You're at the seed level now. Yeah, I'm trying.  About 18 months, probably. Okay. How much will your, well, I guess we'll find out in terms of how much that, what your evaluation will be and how much you decide to raise. Yeah, it all depends. It's the bottom line of how much you're selling, where they can show a profit because people are interested in social responsibility. They're interested in health, but they're also interested in the bottom line. Yeah, I guess personally for me, everything's moving to like the better for you thing category and I think Latin food in some weird way usher is that in kind of nicely. Oh yeah. Right, when it comes to like tortillas or like the deal of pita doing the cactus chips.  Exactly. I mean, there's so many great agricultural products from Latin America that aren't even in our market except as ingredients. So, you know, I'm excited. I was telling you about Chantico. We actually use agave in our barbecue salsa and, you know, everything, we don't have any preservatives. We're non-GMO, non-gluten-free. I mean, we pay attention to everything. And on the social responsibility side, our company about 10 years ago teamed up with Trees for the Future, which is I think the number one tree planting organization in the country because they have an 80% success rate on planting trees. And so, we also have carbon-neutral shipments. And again, both for our international and our domestic US shipments. And to me, that's a big thing. Yeah. All right, let's do this. Let's go through each thing you have on the table for people listening and tell them, one, the name of it, and then two, what to eat it with. They're how to consume it. Obviously, the coffee one's obvious. I think people know that one. But let's start with the barbecue salsa. Barbecue salsa depends on which one you want. The product from Sonora, which is the pink color. Chiltepe and pepper is like saffron. It's very light, delicious pepper. Very expensive, but we found a way to access it.  So, you know, I would say this is gonna be more chicken or steak on the grill or anything that you like where you want more spice. Now, the green color is the New Mexican. Again, it's hatch green chile. It's a little sweeter. That's gonna go really well with pork. And our third variety from Honduras is gonna be more fish-oriented. And we're gonna have, it's gonna be a culantro versus a cilantro. What is a culantro? Culantro is just a variety of cilantro. It's just C-U-L-A-N-T-R-O. And then also peoncillo, which is a natural sweetener for that one. And there's a shantico raw, which is the darker one. And that's for people who really want less processed foods. So raw is cooked at a lower temperature. You know, it works like the lighter agave. It's got a little bit stronger taste, but it's people that like raw food. Okay, interesting. The powder is delicious. The agave powder. Agave powder, shantico agave powder. It's blue agave. We work with the largest producer of agave in Mexico. Quality control is top of the line. And we created an agglomerated powder that nobody else has, so it doesn't stick together. So it'll, you know, you open up a sachet, you use it just like any other sweetener, except it's lower GI, as I said. And then the wild is actually picked from the open fields in Mexico, in Guadalajara. And it's a much lighter taste. I think people like it. Some people like honey, but sometimes you want a lighter taste. And we've kind of joked around about starting a Save the Bees campaign. Because first of all, it's plant-based and it's also vegan, which is super important for people. That's actually a pretty good idea. You say you joked around about it, but the Save the Bees campaign might be a really good idea. I'm on it, man. I'm on it. On TikTok or Instagram. Right. And our new one, which you were coming out with next month, is hot agave.  It's primarily one of its great uses people are using on as pizza.  As you said, these Latin foods in different ways. That's, I can't wait to try it. See the Diavolo pizza. Yeah. It's got a little hot chili. Exactly. Hot honey. But instead of using an Italian spice, you use a Latin spice. That's a great idea. Get a savory sweet. Tell people where they can support you, where they can find you, where they can invest, if they're interested in doing that. Well, I think some of the best ways, just wherever you are, is to, for example, buy us on Amazon. All these products are on Amazon. We're also in Fair, which is similar to Amazon, but services more of the specialty market and specialty products. We're actually in What Chefs Want, which is a national food service distribution company that has 16,000 locations. So they're serving all these restaurants, so you're using us and you may not know. Yeah. You know, and we're really focusing on local Denver food service and retail to build up the brand. And then moving into these stores, some of the retail stores that I mentioned. David C. Snarros, thanks for coming on the podcast, brother. Thank you for having me, I appreciate it. Thank you so much for the support and making it to the end of the episode. If you haven't already, please leave a review and share the episode with your friends. If you never want to miss a beat on all things entrepreneurship, make sure to follow us on socials or daily content. See you next Tuesday for another great episode.


Actor Kelsey Grammer has long owned a home in upstate New York where he also opened his own brewery and in early 2023 he bought a property across the pond in Britain. While he lives away from the glitz and glamour of Hollywood these days, it wasn't always that way. Back in 2015, Kelsey and his ex-wife Camille Grammer, star of The Real Housewives of Beverly Hills, sold their former massive mansion located in the Malibu Hills. This mega home, set on five acres of land with ocean views and boasted nearly 7,000 square feet of space, with stunning features like a two-story ballroom and a kitchen designed by Wolfgang Puck himself. The estate was stunning, offering everything from a resort-sized swimming pool to horse stables, and it was also seen often in Camille's Real Housewives TV episodes. It last appeared on the market for sale at just under $20 million. In 2019, Kelsey Grammer's one-time lavish estate in Malibu came on the market for a whopping $19.95 million, which was the marital mansion he shared with ex-wife Camille Grammer. Thanks to Camille's starring role on the Bravo Reality series Real Housewives of Beverly Hills, this home was featured heavily on the show for the first two seasons. Kelsey and Camille got married in 1997 and bought the five-acre compound in the Malibu Hills a year later for $4.5 million. The five-time Emmy-winning actor known for his roles in Frasier & Cheers met Camille in 1996. Despite many years living in the stunning mansion, the couple would eventually divorce in 2001, and Camille continued to live there for about a year afterwards. Kelsey's former French country-style property was sold in 2015 for $13 million and has been on and off the market ever since. Kelsey and Camille's former residence was stunning and fully guardgated, located in the celeb-favored pocket of Sarah Retreat in Malibu. He has stayed spent a total of almost 14,000 square feet, while the main house measured in at 6,650 square feet, with seven bedrooms and 13 baths. Elsewhere on the property, there were outbuildings like a guest house, a carriage house, and multiple other structures. Once past the 24-7-Gardi gates and into the mansion, you'll find that the home is stylish in every room. Boasting spaces like a handful of living and dining rooms, and even a two-story ballroom with stunning chandelier. This space was ideal for hosting large functions and formal gathering. For more intimate occasions, there were living and family rooms throughout the home, many of which had fireplaces and outdoor access. The impressive and spacious kitchen was actually designed by Wolf, King, Puck himself, and offered wood beam ceilings, two islands, chef-grade appliances, and even a pizza oven. The kitchen flowed into an indoor-outdoor dining area too. With a mansion this big, the highlights inside were endless. Kelsey no doubt had access to a handful of activities, whether he wanted to relax in the 14-person home theater, or read a good book in the wood paneled library. There's also a wine cellar in the main mansion, while a building near the pool contained the property's home gym and a spa. The master suite that Kelsey occupied for years had a sitting room as well as a private terrace with views of the mountains and ocean, the same views that grace most of the impressive property. All of the guest suites in the mansion had their own fireplaces for a cozy stay. Aside from the handy outbuildings on the ground, there were features like a horse-riding ring and six-star horse barn, all of which were surrounded by landscape gardens and a lily pond. There was also a resort-sized free-form swimming pool along with two spas, a bathhouse and elsewhere a tennis court. The alfresco dining could be enjoyed on one of the terraces, or cooked outside in the barbecue area where there's even an outdoor fireplace. Kelsey Gramerit left the glitz and glamour of Hollywood a while ago to live in the Catskills area of Upstate New York, where he lives with his current wife, Katie, who he married in 2011 along with their children. Kelsey's Faith American Brewing Company is located in the town of Margaretville, where he owns a home. When growing up, he spent Thanksgiving at a farm in Kingston, just nearby, a time he described as the most idyllic of his life. There's little to no information about Kelsey's long-time property out here because he chooses to keep this part of his life private. We do know that Margaretville is a friendly and small village located in the Catskill Mountains of New York State. It borders on Catskill Park's 770,000 acres of preserved, protected, wild, public land.  It seems like Kelsey's current lifestyle is a lot different from his former time living in Beverly Hills. According to the actor, he had long had a vision to bring a brewery to his small town in the Catskills and founded his Faith Brewing Company in 2015. However, over the years, the actor had only spoke about it on and off, and even as recently as 2021, it was reported that it's hard to find his beers widely available. We do know that Kelsey's brewing company is Christian themed, and it's something of a labor of love for Kelsey. He said about his life out in Margaretville, We've got a lot of love here, my wife and I, she loves this place as much as I do. We raise our three children here, our family comes here, our extended family comes here all the time, and it's always a loving experience. It was always a fantastic environment in which to like, watch kids grow, and to think about what people can do, they just care enough about something. While Kelsey and his wife, Katie, love their life in New York states, most recently in early 2023, he purchased another property for his family across the pond in England, where his wife is from. It's not where you might imagine in England though, it's actually in a small town near Bristol. The actor has been a regular visitor to the North Somerset town Portishead for years. He announced that he and his wife had purchased a little place in the town and are now working on the property. In an interview with Radio Bristol, Kelsey said, I love the people of Portishead and the time we spend there.  Katie, his wife, who is 26 years, his junior is the daughter of former Bristol City footballer and Bristol rovers coach Alan Walsh. He continued on about their new property. We won't be taking up residence for a while, but yeah, we're pretty excited about it. We have a little view of Wales from the channel there. The actor has been seen regularly in Portishead, which gave its name to the 90s band, usually while he's visiting Katie's family. Although it's not known exactly where in town the property is, Kelsey said that it was close to the coast. After looking at what we know about Kelsey Grammer's properties, that'll wrap up this house tour. Before we go, answer this question for me. If you had 500 acres of land to yourself, what would you do with it? For me, I definitely think it would be a dog-sank-drait or like a farm, but let me know down in the comments what you would do and don't forget to like, subscribe, and follow me on Instagram to chat. I'm Cara the Vampire Slayer and if you want to check out another tour before you go, stay tuned for this look into the homes of Amanda Saifried also owns a house in the Catskills. Bye!