This talk was originally going to be at one but I'm happy we were happy to bump this talk up because this is still one of the quintessential topics in in security and without much ado my pleasure to introduce you to Travis Palmer and Brian Thomas. Thank you folks so as you said this is a quintessential topic in security we are actually not trying to win any awards for the length of the name of our talk but we're pretty sure we might have actually won and we're going to give you guys not just an update on kind of the current state of what DNS is but also go a bit further and actually give you guys a presentation on an attack that is doable very plausibly against some major resolvers from my understanding Google and some other folks have not really gotten back to us with a firm word of yeah totally we got this yet so very plausibly you could turn around and actually use this if you're very quick about it this is a nice stick to poke people to get actually moving so without further ado this is how you become the one in 34 million so I'm going to go through basically introduce ourselves then we're going to go right into we have to give some background on DNS a lot of people here are new to DEF CON and relatively new to the security industry I can't talk about the things that are different if I can't talk about the stuff that's originally we're going to talk about fragmentation attacks there's a bit of a history for that and then I'm going to talk trust me it's not it's a very important tangent into IPID inference and some of the more recent stuff that has come out of University of New Mexico and then we'll go right into the attack I'm not actually affiliated with the folks in University of New Mexico but they are not very helpful and where their white paper was more than enough to get this attack rolled together and then Brian who is actually a proper expert will be talking about mitigations so hello I'm Travis Travco Palmer I'm a security research engineer for Cisco Systems I am an OSCP and OSCE if that means anything to you and as I kind of implied yeah I'm not actually an expert on DNS or DNS sec I don't do this on a day-to-day basis Brian I don't think would want to call himself an expert but he's close enough and he is a principal engineer for Cisco System and has had his a lot of free time to work on FreeSB and OpenBSD so what did we actually do this is what we found was a more consistent way to poison the cache of DNS resolvers without using a man in the middle and using an existing attack that existed using fragmentation on DNS we took that attack reduced it from hundreds of iterations thousands of iterations to very plausibly one on IPv4 and for IPv6 reduced it from millions of iterations to very plausibly only one and also because of that we're basically bypassing all the current recommendations so even if you're doing everything right as people are telling you yeah we can probably still screw your domain so yes we did disclose responsibly I am my particular team was discovered this attack during a focused pentest engagement it was a couple of weeks but it's a little bit of an unusual pentest engagement to say the least and our team disclosed off to Cisco umbrella and umbrella has been disclosing this to other DNS operators ongoing both before DEF CON and will likely continue to do so after DEF CON as I'm sure some emails might come roaring in so to start as I said before we need to talk about DNS and what a better place to start than talking about the attacks because we can't really talk about all of DNS so in the early DNS in the way back when DNS was really only considered exploitable in a man in the middle scenario which it has no protections against still really doesn't have a ton of protections against but it's getting closer in any case back then ignoring other protocols the only thing that was actually keeping a normal DNS reply from being spoofed doing things off path not man in the middle was the DNS ID and the styling and the structure of the response itself it has to answer the query DNS queries and responses are very consistent so really we're back to only talking about DNS ID and again I said said before ignore other protocols there is of course the issue that this is being sent across the Internet and there is this extra level thing called a UDP which is going to have port numbers and some other things that are going to be very difficult to guess if you're trying to spoof a DNS reply and Kaminsky is for original attack basically determined that a number of implementations of resolvers that were being used were programmed to query over sequential very predictable set supports and this took out basically all their sources of entropy that were to exist to make it difficult for the attacker so again we're back to the 16 bits of DNS ID and that's really not enough to stop an attack once poison records get saved if you can try over and over and over again and if you succeed it gets saved well then you're good you're good to do this off path you need to be in the middle what do you mean what do I mean by off path have a diagram so dotted blue line here is the network path from DNS request from our target is taking and we're off path because our attacker boxes the red boxes were we're just not on the network path we can't see anything that's going on there we might have some idea of what's going on over there but we can't see or modify the traffic now for this particular case and I'm going to keep this going this diagram it's going to change a little bit but it's going to be largely the same are we have our attacker box off somewhere in the internet and then inside of the trust boundary some corporate network something that the resolvers servicing we have both our target and a puppet the puppet we don't really have to fully control it could be as simple as just some JavaScript in a web browser but what follows is a little less simple so first the attacker is going to basically figure out where the external facing IP is for the resolver this can be as simple as basically talking to ourselves making a request for domain that we control and as long as our box is also serving as a name server for the our own domain we get the IP address of the resolver the attacker then can just tell their puppet hey send a request on my behalf and since we know what it's requesting and when it's requesting we can immediately start sending responses to the resolver and since everything else is largely predictable in case the Khminsk attack even the even the destination port on UDP is predictable we can just start guessing DNS IDs because the resolver is going to make another request on behalf of our puppet we don't know what the idea is but we can start guessing and this might take a little while it's not always going to get cash sometimes the legitimate response is going to go straight through the resolver and so really we do have to find certain types of requests that aren't going to get cashed at least when not modified this can you can do this in a couple of different cases you can do this for a zero time to live domains or near or near zero it's a lot of it a lot of it is content delivery networks Akamai CDN YouTube a lot of those requests aren't going to live very long in the cash or we can start asking for things like domains that don't exist which won't get cashed unless of course you then proceed to give a response for them where you say that it is existent and then you start changing some other things like the records for where the name servers are at and other things which will of course all end up in the cash as well once it's in the cash it also comes down to a puppet but that doesn't really matter the important part is that it's saved so when our poor target machine goes out to ask for it it gets a response back that it probably shouldn't have gotten and starts reaching out immediately the internet for whatever it was trying to reach initially DNS is a is a protocol to find other things ultimately and reach out to our malicious server malicious server goes as and gives them yeah totally those binaries that you wanted yeah got them there you go and that person if that person wasn't paying attention they get farmed the program is reaching out not checking the TLC certificates and frankly anything that there's not good verification in place they get pound the end just as disturbing as it was a little bit more than a decade ago of course the Kominsky attack doesn't work anymore I'm glad we've made progress in the past decade shortly before that attack was presented even various DNS resolver applications were patched to make the UEDP ports unpredictable Dan Kominsky is a whitehead after all so he did tell everybody kind of ahead of time these say this puts up a heck of a damper on trying to do this kind of exploit because now you have a destination port to guess as well that's another 16 bits at 30 put two bits total and although you could exclude some of the ports this is about 65,000 times harder though the attack is still high-profile and scary you're not gonna be doing this in any kind of feasible scenario and I'm not going to pretend Kominsky alone made DNS sec happen that but this did help along so people scared of DNS being spoofed basically are now using DNS sec without going too far into it DNS sec is basically public key infrastructure for domain names and I can't I can the highest level of authority signs for various top-level domains die like dot org who then in turn signed the keys used by domains so for example Defcon dot org which then used to sign those keys for records individual records say where WWE dot Defcon dot org is of course it's a little more complicated than that but we really can't for a sake of time dig into it couple of things that DNS sec adds that are really important is data origin authentication basically the ability to verify the data you're receiving is coming from where it's it's saying it's coming from and also trying to make sure that data cannot be modified in transit since the records are signed by the zone owner and the zone's private key it would be really hard to mess with that because if it actually gets verified well it's the signatures are not going to line up this how cryptography works of course not everybody is actually deploying DNS sec and even if you were to you know deploy DNS sec in your domains it's not particularly helpful if the resolvers aren't doing it either at this part this is really not going too well in the last report this was made was in 2016 unless you're in Central Africa and your ISP is using Google DNS for everything you might be in a bad bit of luck in any case we need to bring up some more bad news and talk about what DNS sec doesn't actually do either delegation and glue records name server a records for IP addresses located in the authority and additional sections in DNS are not signed delegations or records that basically relay authority to another server so realistically dot com isn't going to know what servers example dot com is running so you have to delegate that authority down and I had mentioned glue records where glue records glue records are largely a fix to solve an issue of if you have your name server in your domain or there's some concern about somebody not knowing where your name server is for your domain you include that at the absolute end of the request or yeah pardon me include that the absolute very end of the response because otherwise someone who's looking for some resources and example dot com are going to ask the dot com domain okay where's example dot com it's going to tell hey you need to look over here for example dot com and okay well where's the name service for that we don't know okay ask dot com again this continues in an infinite loop so that's why they need to be this but somebody of course in here I hopefully is thinking why aren't these signed why can't we just figure out a way to sign this well your FC says so it's with actually good reason it's delegation of duty cloud child zones have a private key for thwarted records and the parents don't really can't be signing it for them but there's again there's data origin authentication you if you can trust that parent you can probably trust the children as well and there's you know large committees that are rather trustworthy running most the system of course there's also to deal with all the signatures all these signatures can make responses quite long especially if there's lots of records and as a bonus name servers can even sign the gaps between valid subdomains and sign nearby names to prove there is no sub domain in between using insect or insect three and but how bad could it be someone might ask like well if today you go ahead and ask defconn dot org for non-existent domain they're using insect and so they will respond very verbose Lee with one thousand nine hundred and twenty two bites to tell you something doesn't exist seriously this is this is our best solution you can't even send this across the internet we in one piece without expecting it to get fragmented which is something we need to talk about so Hertzberg and shulman realize that this was kind of a problem if the response is getting split up UDP has no means of dealing with fragmentation only really that's TCP and everything in DNS is running by UDP by default so it has to be fragmented the IP layer and that means it gets an IP identifier and mind you these are rather inexact diagrams but it's I'm trying to keep it simple is IP around UDP around DNS and in red is basically all the things that an attacker can't really spoof it's and that IP identifier is an IP layer three thing it's used on both IP four and IP v six in any case it's an additional factor source of entropy we have to guess it is an attacker but if you're noticing that second fragments looking a little lonely because all of those QC signing bits and the other protocols end up in that first fragment and the second fragment the only thing we have is that IP IP layer identifier now DNS actually adds even more stuff but again it ends up in the headers or the first couple of things in DNS nothing in the second fragment and it is it turns out the IP ID for IPv4 16 bits this familiar to anybody as a further bonus many resolvers when this paper was published were using global counters for IP ID and it was the default in Windows default and free BSD according to them when they published in 2012 and there was a lot of the lot of resolvers they were able to find right before they published that we're doing global IP ID so what is this attack end up looking like in that particular case well I have to add something to diagram a quick we now have an IP fragment cache on this diagram it holds up to 64 fragments that's relatively the default it holds them up so they can be reassembled when and if they all get received and if nothing else it drops them after a couple of minutes and in case the first thing we do when performing this attack is ask sometimes you only need to ask as an attacker we basically just ask the name server if the response is fragmented or it's IPv4 we get an IP ID back so we already know what IP ID the name server is using if it's using global IP ID and then we proceed doing what we were doing before puppet goes ahead and ask something on our behalf and while the name server is still thinking about responding to that we start guessing IP IDs and DNS is relatively well structured very consistent if you ask for something you're always going to get roughly the same reply sort of maybe some garbling up in the headers but again we don't care about the headers we're sending the second fragment over and over and over and over and over again and we actually have a good starting point we can we know what the IP was before and so we can go ahead and just send that sucker along and start guessing everything in kind of front of it mind you it is global so we have to do this kind of quickly other people could be talking to this name server and they probably are which is going to increase the IP ID but we can guess 64 times so at some point we're not just going to go through but we're actually going to be combined with the first fragment of the original legitimate request which contains all of the lovely headers that are signing this thing and then that's going to go right on up into the cache and then down to our puppet but again the thing we care about most is the cache because when at a later point when that poor target machine goes ahead and asks well hey where's deftcon.org they're going to get that assembled but tampered response and just like a Commencius detect before target makes a query for what was further and it gets told how to get told it lives with the attacker it's pulled some data from the attacker it died all downhill from there and pardon me my mouth is getting really dry so there's a lot of stuff you can actually do with this even with DNS Sack. DNS Sack does limit what you can do with these kind of poisoning attacks and there's only so much you can really mess with in the cache that isn't directly signed but of the four sections question answer authority and additional the ones we care about most with DNS Sack are those last two the ones that are going to contain the stuff that's likely to be unsigned and they're almost always going to make it in that second fragment and we get some options I would love to verbally explain to you all the limitations but they made a diagram for me so I'm going to use a diagram from their paper so the main ones you want to look at here because there's a couple and that a third from bottom line there is permissiver island basically the resolver is not actually doing some good resolving not checking anything yeah there's a lot of stuff you can do immediately if the resolver is not checking DNS Sack but for the other cases when everything is in line doing what it's supposed to do there's other stuff you still can still do so if they're you doing if you're using insect three which and you're using it with an opt-out that means you're not signing delegations with insect three and that means that if someone were to say I don't know replace one of your insect three records with an NS record for a subdomain yeah that subdomain exists now as far as your resolver is concerned the other thing you can do is basically something in two phases you can do NS hijacking which if you've noticed doesn't have a lot of requirements and name server blocking which again doesn't have a lot of requirements at the same time basically what you do is you poison an NS record that makes it in the cache and then you start performing server blocking on all the other name servers and it's actually really easy to do name server blocking because the only thing you have to do is ruin the responses from all those other name servers and if you can just ask what their IP ID is all you have to do is just mess it up you don't have to poison anything you don't have to have it even be correctly structured if it gets reassembled then it's the wrong way it's not going to be accepted and for most DNS software and unbound particularly two failed queries at a 15-minute interval that name that resolver marks that name server is not responsive all bad news and if that last slide wasn't concerning enough there's also the possibility the attacker could cause fragmentation by tricking the name server into believing that it needs to further fragment traffic which of course helps get a complete record into that second fragment or as many records as you want into that second fragment and this does work on IPv6 as well and there's been a couple of studies done that you can get this stuff well below what should be the minimum transmission unit any case there are some more limitations to this this you folks probably would have heard about this if had not been for some additional limitations there's a lot of name servers that aren't deploying with global counters anymore no real surprise there that's something we generally expect from the network stack of a printer windows is still as far as we know using a global counter but free BSD is now using a mostly random searchable queue system and Linux still isn't using global counters even back in 2012 Linux was using mostly per destination and we can't query something that gets cached just kind of like before a significant portion of difficulty here is finding the query that was that returns a predictable fragmented response and sometimes the options for a given target are pretty slim and maybe just the response to the any query is enough but mmm it can be dicey and there's also the future IPv6 throws a complete wrench in this 32 bits and trying to guess with IPv6 that puts you in the territory of 34 million iterations and that's that's doing it on average the actual case for exhausting the entire key space and this is assuming you know stuff isn't moving or other people are talking this name server is around 67 million you're not going to do this this is literal terabytes of traffic you'd have to send you're not doing this on IPv6 there's no way but there has been some notice this is this is kind of a bit of a thing for IPv4 and prior to our engagement with umbrella their implementation was using IPv6 whenever possible why wouldn't you and they detect IPv4 fragments and re-cour them over TCP or IPv6 if it's or ah pardon me re-cour it over IPv6 where possible and basically re-cour over TCP if that still fails or you can't just quarry over IPv6 there was also a presentation this May in at OWARC which wasn't us actually now please excuse the crayon colors I am taking directly from their slides but this is this is a rather pertinent presentation I had said before this was what we founded bypassed all recommendations yeah this is kind of the weird case because in an earlier slide in this presentation they had said drop all fragments but then here they say well the attacks only effective on IPv4 and kind of directly implied IPv6 is safe I digress in any case this is good news there's interested parties and the people who are the major operators of DNS are generally trying to lose tie up all the loose ends without any major performance impact so now we have to have that little tangent I was talking about before which might make a little more sense now so the sequential nature of IPID is largely why idle scanning exists it's what M-Map uses but as a lot of people here may know the kind of scan has some requirements for its zombies namely global counters and also they have to kind of be idle machines they need to act like a 1990s printer basically I won't explain what this particular team found in depth because we really don't have the time but in a somewhat similar manner to idle scanning a university of mexico research team developed a way to abuse the IP fragment cache and Linux machines to basically determine if something had been sent and much like idle scanning you could do this to figure out whether or not traffic was being sent on a link that you can neither see nor control and it works on fully per destination IPID counters which is what Linux had and you can measure how many and you can really just do any kind of measurement across the internet he was to say the Linux folks didn't appreciate this and they didn't really want every Linux machine on the internet being used as a zombie for an idle scan and so around the same time as this paper being published it they created some patches the first that we care about was to directly prevent measuring communication between two machines and the second was an optimization to IPID how IPIDs were handled so the first one I did say was a direct mitigation well it doesn't actually completely fix it because what it's doing is just making it harder it's basically incrementing the IPID by a random but normal between one and the number of kernel ticks since the last sent packet this kind of requires keeping track of the last kernel tick values as part of per destination counter and this map makes this operation rather extensive especially if you have a lot of things to talk to but it does work and if your interest was trying to do a long term idle scan or a long term query as to what was going on between two machines yeah this is going to make it real hard for you to do that because if you spend more than one kernel taken between your polling you have no idea what's actually coming back it's properly random the second one was an optimization before the network just act was basically keeping track of all the IPID counters for all destinations individually and storing them in a binary tree this caused a lot of performance issue for servers that talk to a lot of clients like name servers and sends every comp so I graduate knows that O of a constant is much better than O of long n they made a new way based on hashing and binning so destination addresses both IPv4 and IPv6 are hashed to match it with a bin which will contain the current IP ID counter which will then be used on the next thing that gets sent out the port now this does mean that some destination addresses are going to have to share an IP ID counter so to prevent an outsider knowing which addresses are being sharing a bin the seed for this hash function which is doing all the determination is secret and determined that boot perfect especially if you have enough bins and hash collisions are rare now that's that's a low number now if you think this is kind of low like I think this is kind of low the research at University of Mexico the researchers at University of Mexico would agree with you and so they of course immediately then proceeded to make another idle scanning using the Linux network stack so basically the way they do this is they use IPv6 to legitimately get very large source address spaces you could use IPv4 but as we know that that address space is a little bit sparse nowadays and it always sources they query the IP ID counter for their IP ID address with a fragmented ICMP6 echo request which then returns a fragmented response and they increment the IP ID counter for the target by basically spoofing a Sinect to the target and then query again basically figuring out the hard way whether or not the talking and the spoofing they're doing is in the same bin as what they're able to query on their address and then they just change the addresses if it doesn't appear to line up now why fragmented IPv6 echoes well IPv6 is a little weird because it doesn't fragment the IP ID counter unless the traffic is sent fragmented and well all the above requests as an extra stipulation have to arrive within one system tick one kernel tick of another to force that random perturbation of one to the number of kernel ticks to always be one it's funny you know the difference random number between one and one is always one not the best mitigation thankfully for most systems this timing is actually pretty low ever since jiffy's were invented that counter is about ten milliseconds a hundred Hertz it could be as low as point six six milliseconds on specialty systems that are running clocks all the way up to one thousand five hundred Hertz but that has to be a very specifically configured OS now once the collision is found you can basically just flip this triangle it's nice and start doing your idle scan basically instead of trying to figure whether or not you have a collision with the target you can start sending stuff off to another machine and see if whether it responds back to this machine that you're doing the querying for and they called this not an idle scan oh well so there is something that you folks probably have noticed at this point this is a lot of knowledge that you can get about the IP IDs in the state of IP IDs and the IP IDs for communications to other machines and hold on now why didn't the folks at the University of Mexico realize that this might have some other implications why didn't they realize that this was gonna have a huge implication on DNS and fragmentation tax well they were thinking about idle scanning I don't blame them and show of hands who knew about Hertzberg and shulman's research before today no hands yeah you can't blame them so we get to talk about something new today so basically this attack which mind you that's what I get for pressing the down arrow so we start this attack basically the same way as onus and no for those of you who are about to ask I am not going to call this an is or actually not an idle scan for various reasons but we're gonna start by finding collisions just like with onus and instead of a zombie in a target we basically have a name server and resolver now the text is a little bit small but if you noticed we're using IPv4 in there well what if they're talking over IPv6 what if we need to find collisions in IPv6 addresses like umbrella was planning on doing well no problem we'll make a my pv6 echo frags as well and what about getting our address space the entire point of using IPv6 to do this particular attack is it's got to be convenient how convenient is it trafco well it'll be us to the rescue so even the free tier of abdus vpc virtual private cloud you can get some IPv6 address space but the real question is of course you know is it big enough I haven't done it's extensive testing albeit but I'm gonna say that this is probably enough for those folks that can't count commas as fast as the slide is going to be up that is 18 and a half quintillion give or take a couple dozen quadrillion mind you statistically it only takes about 10 thousand addresses to get a 99.2 percent chance of a collision because there's only 2048 bints and this is outright better than trying to do global IP name servers IP a global IP ID for name servers because in this scenario we get protection we get that added improvement of all the other bits of binning and if there's a couple of random house on the internet 2047 of them aren't going to mess with our IP ID and also once we find this collision we can kind of sit on it and the service election is like literally any remotely recent Linux kernel if you're Linux kernels older than 2014 by the way you should probably patch there's some stuff that you need to deal with and as far as like what our target space looks like now well there's something that exists a lot more than existed in 2008 or 2012 when some of the other attacks were coming out and that's these lovely lovely public resolvers and they're really trusted in fact as I mentioned before there's entire central african nations where all the ISPs are going through Google DNS sure would be a shame but somebody did something to Google this yeah and frankly don't be wrong there's some presuppositions there's some difficulty in doing this attack you need to find the right set of targets but if you poison a resolver you're attacking everybody who's using these and as far as getting a puppet downstream well you can be the puppet to you can be both the attack server and the puppet because they're public resolvers you just enroll yourself it's free and as I kind of mentioned before this whole collision finding bit is completely detached from the attack time is on our side as an attacker and now it's actually really on our side as an attacker the hardest part and the most legwork to find this is in the collisions and although they could be found in minutes we're really in no rush we can probably stay around for weeks months or years because that key that is determining the hash doesn't change until the system reboots and I don't know if any of you know a lot about name servers they're not going to reboot a lot that's bad for the internet and we've got some interesting options as well because we can do a lot of really short duration cash attacks because we have a very good idea what the IP ID is and we can very plausibly do this in the first hit and we can also accumulate a ton of matches for multiple name servers multiple resolvers even if Google is running a hundred and one resolvers and we want to poison a hundred and one name servers we can just get the collisions all hooked up for that and then just start poisoning away and eventually we're going to get poisoned records into every last one of them if we know what the IP IDs are that we need to send it's kind of a strange workaround for load balancing and because we're doing this in a one-shot we may not even need a puppet in the case of non-public resolvers if you're trying to do this on a corporate network maybe you don't even need to have something make a request because there's a lot of stuff that happens at midnight and patched Tuesday there's a cron job or some automated client that's going to reach up I don't know if your blue teams are working midnight's but we can and maybe the public can be completely passive if we see common requests when they're being requested we know what the time to live is on those records and we can just wait till they go out of cash and have to be requested again by the resolver and so we can start actually consider poisoning things that are getting cached every single time because we can very plausibly do it the first time and maybe poison isn't even our purpose maybe what we want to do is just dots the entire world or prevent patched Tuesday from happening which would be very bad if we're an attacker in the middle of a corporate network and the windows up to windows updates aren't coming down anymore if we can do poisoning in one shot we can really mess up a response in one shot to so I've said a lot of words up to this point what is the actual final attack look like so basically we have to pick our targets and this is a little difficult to do we can at least figure out what the resolver is if we're a downstream puppet or we have some kind of way or it's in a public resolver and we also find some domains some domains that are likely to send some large responses and we have to find some servers that are actually running Linux that shouldn't actually be terribly difficult but it is some it is an important point once we return determine the resolvers public IP address for the request we can just basically evaluate the domain responses to see what or not we can actually poison anything and then we do our but we do our collision finding which should be good again until the name server actually finally restarts and then we wait puncture water drinking now right before we do an attack we can optionally trick the name server into lowering the PMTU and forcing fragmentation to exactly the number we want it to be or we could also just not and just request for non-existent domain on defconn.org there's a lot of domains that are using NSEC and NSEC 3 the responses get quite large and then we just query the name server to get the IP ID because we've already found the collision and then we do that yeah I should say we should do we do that directly before a known request is about to be sent we could do the query and then we basically send 64 smooth fragments we could send only one if we're very confident in our confidence in ourselves but I mean if we're doing this we might as well send the full payload that we could possibly send and we do that based on the known IP ID as described in fragmentation considered poisonous and I will point you to Hertzberg and Shulman's paper as to what you can do from here because it is all laid out and that paper is nice and old so now's a good time to start talking about how do you actually get around this with what's the fix Trafco well as I said before I'm not the expert so I'm going to hand it over to Brian who is closer to an expert and he can talk about some mitigations I'm Brian so much I work on the resolvers at Cisco so this is a really interesting problem and there are lots of parties involved there's the zone owner the person who owns the data there's the name server so that's the person that or that's where the data lives sometimes the same as the owner sometimes it's a registrar sometimes it's a third party and then there's the resolver that almost never lives with the data and the resolver normally lives closer to the to the client and of course is the client the person who actually wants to look up the address it's usually an address it's not always an address with DNS but address is the most interesting parts so really what we're what we're actually trying to avoid here or what what the problem is is that the attacker is attacking the resolver due to a bug in the name server and the name server is holding the data on behalf of the zone owner and the lie is getting propagated to the client so there's a whole ton of people involved here and really the the attack target is the resolver so that makes it my problem IP fragmentation has has been an issue for for years like since Kaminsky first first observed these these issues so our approach has always been well that's easy we drop fragments and DNS has a mechanism built in and where and when you actually ask a question of a name server you can say and by the way my maximum payload size is X so if you set X to something like 1410 and drop all fragments at the resolver well then the name server is going to respond if they respond with something that's less than 1410 bytes and it gets fragmented then we lose it we don't get the response and that's never been a problem because really modern day hardware really does have packet sizes of at least 1492 or whatever these numbers are so that's how we always played it and then we came to try to support DNS sec and DNS sec RFCs have lots of requirements and those requirements say that we we should deal with packet sizes of 4k and that's because DNS sec payloads are much larger and we don't really want to send everything over TCP so that seemed like a kind of sensible requirement so you have to come up with a solution to this this fragment problem we wanted to be able to turn on fragments but not full full victim to these these sorts of attacks so our first scheme came up with a as it turns out naive approach yeah so so our naive approach was was to assume that IP IDs were random because well that's what the RFCs say they are the random right and of course IPv4 has a 16 bit random number and well that's easy to attack and IPv6 has a 32 bit random number way way harder to attack IPv6 also uses PMTU to to discover what the the fragment sizes are so that's that's much more difficult to attack for the for the attacker as well so we naively assumed that IPv4 was bad IPv6 was good and we implemented something that cleverly marked packets at the at the far wall marked whether or not they used to be fragments and marked whether or not that first fragment was of a reasonable size 1280 in our case and if stuff arrived at the resolver and it was marked as a fragment at 1280 then we'd cash all of the data in the first 1280 bytes say that that's absolutely good and we cash all of the remaining data as kind of I'm not too sure about this stuff and if we ever need to use that data then we'd actually look it up again and we'd look it up over either v6 or tcp that seemed like a great idea until Travis came along and pointed out that that didn't actually work because these IDs aren't actually random so what do we do we could try to do the same thing for IPv6 but that's actually really really difficult because IPv6 fragments are hard hard to deal with at the far wall there and in fact well actually DNS already has this ability built in it's the ability that we used to use of limiting the payload size and just expecting things to go over tcp so I did some experiments and really when you run a kind of public facing resolver and we've observed that more or less slightly less than 0.1% of our traffic is tcp traffic so if you look at all the queries and for every thousand queries there's one tcp query and the other 999 are our UDP and in fact when you turn on DNS sec and you start querying all of those big juicy DNS sec payloads and you do all of this over IPv6 over UDP either IPv6 or IPv4 and actually that 0.1% goes all the way up to almost 0.5% so in fact the RFCs that say that this is a requirement to be able to handle large packets in UDP is just not true so how many people here have heard of DNS flag day so a handful of people four or five people so DNS flag day is a really big deal and it was a really big deal for me anyway and where the DNS community are basically trying to take some of the the awkwardness in DNS the the the kind of rule breaking that has been going on for years in the industry and make it not okay anymore so there was a DNS flag day last year and it was all about whether or not name servers versus resolvers supported EDNS so the way resolvers talk to name servers they'd say hey name server can you talk EDNS and it would do this by supplying some EDNS options and if it got no response from the from the name server it would send another packet and if it still didn't get a response it would try it without EDNS and if it got a response then it would kind of say okay this guy doesn't support EDNS and I'm going to continue on and there was an awful lot of kind of behavior like that going on and flag day is all about kind of saying okay well that behavior is not okay anymore if you don't if you don't behave properly then things just aren't going to work so there's going to be another flag day in 2020 and this flag day is all about fragmentation I'm not sure the entire DNS community is convinced but one of my kind of pushes for this flag day is going to be fragmentation is not okay all resolvers and all name servers don't do fragmentation so resolvers will block fragmentation coming inbound from the name servers and the name servers will just configure their UDP payloads to be a maximum of something smaller than their MTU. Measuring this on a resolver how do you tell whether you're getting poisoned well you can look for lots of echo responses you could look for IPv6 echo responses you could look for IPv4 echo responses you could graph them if you think you're getting poisoned what do you do about it like there's there's limited use in doing any of this this sort of stuff because if somebody attacks you will you just have bumps in your graph if you can't do anything about it there's no point in worrying about it you need to worry about it and make it not happen in the first place so on the name server side I mean the name server guys really have almost the same problem but in reverse I guess yeah I mean from a name server point of view really what you want to do is limit your your edns buffer sizes or your edns payload sizes and avoid fragments all together and you can do some clever things which aren't as it turns out to clever and so with the DNS IP ID 2048 bucket setup and that's all actually hashed using sip hash if you reboot your server you can receive that sip hash and you can basically give everybody new slots in the in the array but well how long does it take to reboot your server and even if you did introduce a way of kind of saying okay we see now really what are you gonna do like run that every second well why do you just choose random numbers in the first place so again yeah the answer for name service is to do the same don't send fragments and again you can you can try to avoid them guessing things at the at the name server you could tar pit icumps but really they could do exactly the same attack over DNS and you'll never be able to notice that because it'll be a drop in the ocean in terms of the number of queries that you're actually getting so like even if you graph that you're not even gonna see it and of course you can go a little bit more hardcore and you can rebuild your kernel you can make this I bi dense size which is set to the 2048 you can make that a bigger number or you could just change the code and actually make it pick a random number and for domains for the domain owner like if you're a domain owner and you don't really know what your your name service going to do and you don't really know what the resolvers in between are going to do well really your your only hope is to use DNS sec and the big flaw in DNS sec as as Travis said is is delegations delegations are fine if you're delegating to another sign zone if you're delegating to an on sign zone then actually the on sign zone is very vulnerable because your delegations aren't secured and remember that DNS sec all that tells you is whether or not something is good or not and it doesn't stop denial of services so it's not the answer to everything but it's the answer to most things these days and yeah of course if you're using DNS sec make sure that you've got reasonably size and reasonable sized keys don't don't go for the lower lower size keys yeah don't go for lower size keys and that Verisign tells you that a key that's going to be up on the internet for 120 days a 30-day pre-period and an active period of 90 days know that RSAs do get broken and if your key gets broken maybe choose a bigger key pair just an idea and I guess one of the more exciting things about DNS sec that's coming out recent and lately is that it's moving towards elliptic curve technology and elliptic curve uses way smaller hash sizes so the the signatures are well not not our sizes but signature sizes so as a result the signatures are smaller and you're less likely to actually bump into fragmentation problems anyway and yeah panic maybe panic a little all of the resources are up here if you need to screenshot this obviously the files are probably going to be uploaded somewhere on the DEF CON media server at a later point and I bet Travis this has been Brian and this has been in the talk I don't know if there are any questions so the question was regarding IP ID and whether or not we should have released a CVE for it and whether or not this has other implications oh yeah this totally has other implications this is actually rather terrifying for certain things like online gaming which are using a lot of UDP if you can get in the second fragment of these things and they're doing all their off in the front it's bad news we could release a CVE but what are what's the target for the CVE the internet I would have loved to send that to Mitru but I have a sneaking suspicion is probably not going to be accepted but I do appreciate the question it seems like a bit of a target yeah I guess it's worth mentioning as well for IP ID that the original RFCs for IP and talked about caching fragments for 30 seconds against being able to to marry them up with the original and deliver the packet that's clearly just not a good idea in this day and age and this day and age caching them for say at most 50 milliseconds would probably be way more appropriate but I haven't seen any RFCs that suggest that sort of behavior I mean it this this sort of thing could be mitigated a lot more certainly raw IP ID attacks sorry second yes absolutely so you cash it for a smaller amount of time and just fling them away way more eagerly and it will stop lots of second fragment attacks where people are dropping multiple second fragments hoping that they'll marry up on the IP ID okay thank you very much


We've been running about four years now. We're based just down the road from Angel in London But we have groups around the country and also another 15 groups internationally sister organizations There are four of us working on this full-time other people helping out volunteers, contractors and All of the local groups in the international groups who all give their time for free So really quite a big base of people who are who are trying to make this happen The issue we have today is that all money that we use is created By banks high street banks when they lend and what that means is that we have a money system That is based on debt So if we want more money then we also have to have more debt and that results in a whole series of problems Positive money is really about trying to change the financial system so it works for society and not against it We're trying to look at the way that money is created by the banking system and the effect that has on debt and poverty unaffordable housing inequality and the whole range of other social problems that we're facing at the moment I'm quite passionate about what positive money is trying to do because Number one I got caught up in the big party that was going on before 2007 everyone's investing in property borrowing too much money That happened to me And I'm now in a position where I owe Far more money than I'd like to and I think looking back. I probably shouldn't have borrowed it I shouldn't have been allowed to so I think that's something flawed in the system We think this is a big problem And we think that actually the power to create money shouldn't be privately owned money shouldn't be created as debt I think that actually the parts create money should be returned to the people essentially power to create money should be used in the Public interest and the people that create money should be the Bank of England And it should be done in a democratically Accountable manner in a transparent manner. I just feel that it's like the common denominator of so many different problems I guess it's I think one issue that if we could change it it would just automatically resolve so many different Problems that we're facing Today's really about what the individual Gets out of it themselves. How much effort they've put in is inevitably how much they've gotten out That's what I've taken you find some people who are very pessimistic Some people who don't think things are possible and then you find people who are flexible who have a can-do attitude or kind of like Oh, you know what? Oh, we can't meet on this day. We can maybe we can do this or oh, we're all geographically dispersed But hey, we can do something about it. I think it's helpful to bring people together to share ideas What's happening across the country to meet new people I've met a few people today that were very concerned about the issues raised by positive money in my area that I didn't even know existed I think it's important for a number of reasons one is to retain that momentum from supporters and perhaps develop new supporters as well To to help those understand the principles that don't already in people like me or relatively new to the concepts I think it's also important to learn lessons from one another and to think about where to go next and to use the Broad range of skills and experiences in the room today to do that to the to the best effect If only positive money could get over this one idea Money, where does it come from and then the next point of course is it's our money? And shouldn't we decide what's to be done with it? We know it's a huge change that we're trying to achieve But the thing is is that we've already got over 22,000 people on board after only going for a few years and with such a radical thing we're proposing. That's really exciting We've got groups in 19 different countries that want to try and change their own money system in their country And we're all fighting for the same cause and we can share and help each other to be able to kind of communicate What we're trying to do We should be optimistic because firstly things do need to change politicians and the people in charge running out of ideas and that there's more and more Discussion in the press and then policy circles and amongst general public about these fundamental problems with money and that needs change


Oh, let's go All right, I'm just for this I'm busting out the glasses just want to make sure I'm reading this correctly Oh, I should have probably cleaned them All right, yeah, this looks a lot nicer with my glasses on one of one Rookie patch auto Corbin Carol boom. Oh my god Look at the patch This was number nine. So if you get rookie showcase nine in the random This will be yours


Welcome to Exchange for Media. It is that time of the year when most brands are loaded with discounts and offers. However 2020 is not a usual year. This year saw one full quarter under lockdown hitting the businesses all across sectors. So do festive offers and discounts mean the same this year too. We can find out this and much more from our guest today Mr Ravi Santhanam, Chief Marketing Officer at HTFC Bank. Welcome to the show Ravi. Hi Nazia, hope you are safe and sound. Yes so far. Thanks. Wish the same for you. Thank you. So Ravi your bank has just announced several festive offers but considering that this is not a normal festive season, many people have lost their jobs. Corona is still very much around. There is fear to step out. How is your festive offer different from previous years? Two three things which are very very different for us compared to last year and looking at the current pandemic situation what we have done is we have done lot more work on online side of so that you will see a lot more online offers available from our side including a partnership with Amazon to start with and on the offline side we have gone much more wider in terms of regional partnerships. So we have 2000 plus regional partnerships that we have executed this time so that even if you have to travel you can find that offer within very nearby focus specifically on regional stores and regional offerings. Last if I remember right we might have had around 750 odd regional store offerings but this time we have just multiplied it and we are going across the country and we have more than 2000 plus regional offers already in place and in addition we will be actually adding a lot more in the next techniques. So these are the big changes that we have brought in. Also I wanted to understand if the categories are same I mean like home appliances, phones, auto clothing or the categories have changed? While the categories do stay almost whatever you just said is continuing to be here and we do see a large amount of pent up demand in these categories you just mentioned. I think mobile today for example or a laptop today for example from being a fashion device it has also become an education it has also become a work from home device for many people. So the use cases of the same electronics have changed the use cases for many people when it comes to whatever other things you said do the bills have changed like for example a lot of people are looking at dishes, a lot of people are looking at vacuum cleaners, a lot of people are looking at air filters. Now these are all things which people would have said may not be necessary but the current pandemic situation the health consciousness of people and because of which you see a lot more on the health side mobile is now a device iPad or app. On the television for example they come almost to education device at many home where people are just putting in the device and they are using the smart TV for themselves with their online classes. So we have seen work from home and work for home as four distinct categories that have emerged and we see a lot of action in those areas. That's one side of it where the traditional categories are finding new use cases. In addition what we have seen education is we have a tie up with most of the leading educational online education providers and some of these online education providers as you know if you want to go in for a two year course for your kid in say 11th and 12th these are pretty expensive. So we have BMI offers and all of these people. On the health side also we have a lot of offers on online gyms as well as medicines and other things on the health and wellness side and we'll surely very soon be announcing a very very big partnership between those four most health service providers also in terms of under health. So we see the traditional categories being there and in addition they're also adding a lot of new categories where we see because of the pandemic and because of the consumer changing behaviors and things are moving. In fact wanted to come to consumer behavior. We last spoke six months ago when the pandemic had just started. So what are your observations in the six months? What kind of unemployment has increased? A lot of people have lost jobs. Finance becomes a very very important part of you know is an important part of everybody's life. How has consumer behavior changed in these six months? See if I have to look at March then April and May. April and May was more of a shock but most of us had experienced anything of this in our lifestyle. So it was like supply just being stopped. Demand was there but supply was just getting stopped and come June, July, August we slowly understood what is this all about and we slowly started unlocking a little bit over there. Supply started coming back and what we have seen is slowly the confidence is coming back. The offline is still a little bit of a struggle for us compared to say what we saw in January but the online has picked up. But if I have to look at July it was better than June, August was better than July and September is also better than August. So which also means that the ecosystem is slowly learning to live with this and over a period of time what we have also seen is consumer behavior. At least now this what I call is the fatigue of staying at home has come. You need to be extra careful about our customers. Definitely our customers to be extra careful about their own personal safety and everything but we do see a sense of fatigue. I'm sure you're hearing that from all of your friends and colleagues and all those stuff saying that how long. So that's the reason why if you ask us what is it that we are planning to do this is also basis this kind of fatigue that we see in the market. People are getting the confidence that slowly over a period of time if they know how to handle this like for example wearing their mask and maintaining social distances they can go about many of their things which they used to do earlier. Yes maybe a travel is a little bit of a problem. Maybe dining out is a little bit of a problem but there are options of dining in. So we are seeing again dining in again travel it will take a little bit of time. We slowly started some dream shoots happening from that side also. We don't hear people wanting to go to nearby places for travel. May not be foreign travel over the next one one and a half years maybe but domestic tourism and domestic travel I expect to increase over a period of time. So from a kind of not knowing what to do all of us that situation including me as consumer you as a consumer. Now I think we are coming to learn to live with it and we do see the majority of the pain area is over and there is a clear outlook in terms of a far better outlook of what is going to happen then say compared to a me and compared to August also. I see a definite improvement in confidence coming back for the last few months. So you just mentioned that July was better than June and September like things have been improving. So but Q2 was a complete washout. So if I have to ask in terms of numbers you know what kind of recovery we see in Q1 was a washout you can say calendar year. I would not even want to hazard a guess because all these percentages are not necessary in the right way of comparing. So when you have a zero revenue in the month of say in April or May and then if you do two you can say even it is infinity and that does make no sense. So what we always do at least in HDFC Bank is we are taking January as because January is one of those 31 day months and how are we against January. So that's what is our pre-COVID level because February is the 28th month and even in March also around March 15th onwards we started getting into this kind of partial lockdown full lockdown shutting down at various places kind of stuff. So if I look at January as a parameter we are almost close to getting there. Business is almost close to pre-COVID. We see there are certain areas in every business which you will know like for example travel for example eating out. Yes obviously they are under serious stress online for example is back and more than what is there in January levels. So there are lots of positives that we see in the system and the numbers are more or less back at that level and we are very confident that going forward starting today you will see the animal spirit the festive spirit of the Indian economy will start to kick in. Actually I wanted to ask about IPL and festive season you know how much do you think will be recovered during these next 40-45 days. See the whole industry and all of us as brand marketers as consumers are all betting on this festive season for a phenomenal recovery for the loss of the quarter two or quarter one as we want to do in the financial year. Yes we had a very bad quarter as an Indian economy but we as bank actually had a phenomenal performance if you have seen our performance. This quarter is very important because I think all of us are putting our energy into it and the biggest thing that I might call it a saving grace or a blessing in disguise IPL happening at this point of time. So we are having a lot of consumers sitting at home having the time to watch this IPL and I have seen numbers I'm sure you have seen numbers in terms of the number of hours people are spending on TV as it is dramatically and working at home for example if there is a match which will happen during the weekends now I'm just going to turn my camera and have my TV on without sound then I want to watch it and I have seen the first week numbers of the IPL so I don't know whether I can quote them or not I have seen those first week numbers they are phenomenal. So cricket starved for almost six months so they have picked it up I as a consumer have been watching almost all the matches and for six months I had no cricket to watch so all of us are going to watching live sports again and that's a very big and there's a lot of it writing on it looking at the first week numbers I am feeling confident that's good for us. So this season everyone is confident and everyone is spinning hope but then considering that the vaccine is still nowhere around and cases are not reducing the next quarter again is you know I have been speaking to agency people and they're a little worried about that quarter that what happens after a festival when the idea is over would you want to talk about it I'm not too sure because you know I don't want to have a question. I am of the opinion I'm just going to repeat what my MD said yesterday he said see this glass full or glass empty so I just you know we want to see this glass half okay and we in HDFC banks strongly believe that we need to create the market also we need to create the demand also so one is to say that there is a demand in the market I'll fulfill the second is to say we will create the demand and we at HDFC banks strongly believe that we will create them. What are some of the big campaigns that HDFC is launching in this festive season? So the biggest thing is we are bringing festive treats back again last year also we started on October 1st and this year also we started on October 1st and as we talked about in the beginning compared to last year this year's treats has been and it is better. We have a lot more online play this year looking at the current situation we've added specific categories like education and a lot of other things which consumers are currently doing on the gaming. Work from home, work for home, all those categories specifically. Regional store partnerships have been and in addition we are coming in as one bank right and this is the biggest thing that which a bank like HDFC Bank can offer which differentiates us from the rest of the people who will offer needs. So if you want to buy any kind of mobile things yes we will give you a 10 to 15 percent depending on what kind of price you need and in addition if you want the remaining 85 or 90 percent to be financed I'll give you an EMI offer also on that plus on the TMI offer also there is a loan offer available. There is a specific processing fee available so we are the one we bank we are the one we company in this country where we can give you a deal on the product that you buy plus financing that purchase of that product also and there is a deal on the finance also so that's the difference and that's what we'll bring it to our customers and it is far bigger it's far better than last year and you will see the benefit of it as you log into our site and see today. Also as a marketer you have you I mean you just said you're going big on digital right this year. Sorry this year we have sorry if you talk about the media plan we have bigger media plan this year compared to last year and this year we will be doing ATL advertisements on television. So we are taking ATL route this year one is the current pandemic situation second is what we see as numbers and how television and IPL specifically and all happening at this point of time. So we will be going into IPL to advertise about our offerings and you will see us and in addition to IPL sorry in addition to IPL we are also having a partnership with what start so we are having a hundred percent of the highlights which were present partners so highlights is this and in addition we will also have other things which we have partnered with star will be in OTT we will be in the entire digital scene what I see is people are spending their time looking at screens like for example if I look at my own screen and look at my screen time I sometimes feel very angry because at the end of the day in office I never used to look at screen but looking and sitting at home it has almost been like seven hours six hours screen and then in addition TV. So what we wanted to do this year is whichever screen you are spending your time as a consumer. So whether you are on your mobile phone off screen this year because you know now that people will move out you have done so many regional partnerships do you think outdoor and radio will eventually start picking up as a market you will invest in them. As of now I have my own reservations on outdoor but in the outdoor see outdoor is one is outside in their own and outdoor is one within the shop I am putting lot of energy within the shop so in advertisements will be very big from our side and our partners are also helping us whether it is an automobile manufacturer or whether it is a TV manufacturer whether it is some of the big large format retail stores or the mom shop mom and mom stores all of them are also in partnership with us and you can see us in the point of presence big time being there in addition our own branches under 5500 branches the footprints that will also be big data will take a call in a short period of time Ravi before we wind this up you lead the marketing for such a big banking brand what kind of financial advice do you want to give to people in these times ahead of festive season I mean any any piece of advice that I am not qualified at all to give financial you're very qualified please do share some advice the only thing which I will say is the advice that I will want to definitely give this all is possible with htfcbank festive treats this festive you go and buy something for yourself for your loved ones for your family we extended family have a look at what we have to offer and get the maximum benefit from this Ravi thank you so much for speaking to us we wish you a happy festive season and we wish happy festive season to you as well as you and please do buy something in this festive season from htfcbank so I have to I have to buy all two out thank you I definitely will thanks Ravi for doing this thank you so much thank you


Hi everybody, thank you for joining in the session like the agile community. I'm really happy to be here Really like hanging out with agile people not a scrum master not scaled agile or anything I Guess we draw a lot. I draw a lot on many things but my core competency is Like a lot of uncertainty So transformation innovation and We just have a tremendous overlap, but I think I'm drawn to the whole community always because this that's just nice people focus And I just feel they're comfortable. So I'm really glad Coming back here again being able to present again And I would love you To engage a little bit. I know it's hard but I've done this presentation not this one another one last year and that was a bit It's psychologically a bit hard because like you have no You don't see anybody you don't know what the other people are thinking and so on so at some point it might get to you They got to me last time So a bit of engagement is great, right? So it's saying I understand it like anything critical Any question a thumbs up was really much appreciated more can start right now so I Also before before we go into it and you see chromatics logo down there That is my that's a partner organization based in the US and much much of what is in this presentation comes from them and Yeah, can't credit them enough So just to keep that in mind. Thank you cool So we will start with the problem space today Just talk a little bit about the issues like why do we need innovation like why do we talk even about innovation accounting? Why why do we need something different? It's kind of that's kind of the question. I guess that we want to quickly explore and so One of the main issues we're trying to solve what to make better is the business case and And Trying to engage a little bit right now. So maybe one or two if you want to punch something in here in the in the chat So who's who thinks? What are the problems with the business case in an innovation context? Like if one of you can if one of you can type something in here while I speak that will be fantastic and Well, basically, how do projects get chosen? How do potentially our way great to heart all right Thank you very much engaging your champions tangible outcome. Fantastic. So our right timeline fantastic. Can we Ask the man estimating the value is super super difficult fantastic and why is it so difficult Kind of cannot object to speed valued risk. Oh, man. The engagement is it's fantastic You might be very happy right now risk great Okay, so we see there is risk and reward goes together So there's a lot of risk and we have a lot of risk like we there's a lot of uncertainty simply and the business case pretends often to have Certainty what tries to find ways to create that make it look like it's certainly Great and we don't even know what customers want yet and do we meet their demands and To what price fantastic downside risk Fantastic, so So It specifically innovation context we have Very minimal data Right, we have mostly qualitative data They might come in this kind of form we collecting them value proposition canvas business model canvases we have Personals we use we work with So we have a lot of qualitative data and we and we're very good at we're very good at like we we're getting better and better Actually understanding customer needs forming better questionnaires Having little experiments in the beginning empathizing all these sort of things But at the end of the day all of that stuff has to be translated into something it looks like this and That's just no good because we know It doesn't really work and for once it creates a massive disconnect Between those who come up with these ideas Who create backlogs who see a chance to see a glimmer like a glimmer of hope so to say like here We have something we have a few ideas And those who have to them like the business sponsor the finance department and so on like how can they possibly really understand each other and The systems within companies as we know at the end of the day you have to calculate the number And so that's something we're the space where we thrive the space that we love the space where we try to solve more problems Really for both parties really, right? We talk a lot about empathy with our customers, but I think we should also have empathy with like the CFO and See how we can tackle the problem for me from that angle I wrote an article I've shared a link with you together with Tristan and We identified actually three distinct issues with the business case As you said they're not accurate Right and that present value like one number. We try we asked me folks like for one number five years down the track and like We don't know we don't know the tech we're using. We don't know our customer position cost We really don't know but we have to do this and so We basically forced to make things up in the sense that there's no other way to get it over the line like we have We have certain expectations We know these expectations and innovation in an uncertain context even higher because we know we get discounted right, so we get this heavily discounted because and Any business bought any any CFO knows that that's risky and that's the tool how they do it So what do we have to do in return? We have to like make up a higher Tem we have to pretend the market is larger or whatever it is to make this to get this over the line It's like the system forces us to make up to make it all look a little bit better than it really is and Lastly, it's just not testable It is testable like our eye is testable in Four or five years. We know whether or not the our eye happened, right? So that's really the problem like we want to know much earlier We want to have better predictability and we want to know much earlier whether or not something has any hope of survival or not Right does that make sense is that summed up a little bit or there any questions at this point? No How many people dropped out already not too many people are still coming in great Thank you testing hypothesis. Yeah in a sense. That is if you want It's not really phrased as a hypothesis, but right. Yeah, like the our eye is at least it sounds great So we're assuming acts whatever our eye and yes It's test it will be tested the market will test it Yeah, and we know it's anything Early validation. So what we're looking for exactly it's early validation ideally In a quantifiable like what we're trying to bridge that what we're doing quantifiably with a with a color And all the qualitative knowledge we have with it with a quantifiable model And we're looking for something that they were looking for something that is testable a lot earlier, right? and so The solution to the state for that or we believe the solution is is actually it's innovation accounting So how many how many people are vaguely familiar with like with innovation accounting? Like have used anything that they would call how many people used Can anybody just say a thumbs up or so if they have used something that would call innovation accounting? sweet one person Hands up fantastic Yeah, we're not together otherwise I would ask what it means for you and so on but maybe we skip that might be a bit the challenging but great fantastic, so to introduce the concept it was it was coined by our grease and Ever since he coined that the term gets broader and broader and broader but to bring it home It's good to look at it from this perspective of levels of innovation metrics so When we measure innovation We usually break it up into individuals like skills how teams are performing that is often very different from how projects perform Like a team can do exactly the right thing at the right time But the project doesn't necessarily need to succeed for the team to do really good work So those are different we need different ways of measuring And we're looking at the project and that's really what we're talking about today And that's traditionally what innovation accounting meant. So how can we make that a? Comparability predictability, how can we have lead indicators and all these other things? We can look at the portfolio at the funnel and we can also look at the ecosystem and look at broader broader components broader limiting factors to build the to building up the system so We are looking at the project today. So how do we get? From something it looks like this or like this or anything else like any like any tool in there To this In a sensible way and not really to this. This is not really what we want like But we want to spreadsheet. We want to talk the language of numbers ideally so this thing is way too specific like we're asking entrepreneurs or Innovators to come up with a number. We know they will never hit most likely It's underneath that Yes, just statistically like that's just most likely the case But can they be single numbers? Yeah, so basically what we do, right? Tell me the ROI Yeah, we get to that perfect. Yeah, perfect Even best case worst case scenarios we can talk about that. They're pretty they're not very good So this thing is probably like it's not very true. It's not very accurate and most of all it's not testable So what we want is actually something it's a bit like When you go to an innovator and you ask them like give me the ROI, it's a bit like going into a meteorology say hey Can you can you tell me where the where the Where the storm that is just forming And where the hurricane that is just forming it's about to hit in five days What will the meteorologist say like get out of here. I can't do that I can't tell you where exactly and in what state and at what time and with what force this thing will hit the coast They can't do that But it doesn't mean that they can't provide valuable information to make good decisions real-world decisions like most likely we should have accurate now and and that is really the best thing that we can do and We can't do that. It is basically a range It is a range of possible outcomes and the further you go into the future the further the range obviously is because You know less and less and less it becomes less predictable right But the earlier we can quantify the earlier we can make good decisions The earlier we can make good decisions the earlier we can pull the plug and the earlier we can pull the plug the more ideas You can basically test and that's how we increase the the likelihood of success Was that too much? Could I have a bit too enthusiastic or something any questions? So we're looking for something that is true like we want to tell the truth like to our best ability As accurate as possible and Ideally tested. How does that look in in our context? It looks something like that ranges We're looking for ranges. We're looking left for the most likely outcome. That's what we're trying to get to today. Yeah The thumbs up or something I'm just giving it a moment Any questions I drink something people dropping out 44 how do you get to this wonderful fantastic? Thank you Yeah, exactly by what's likely and If it's not and if there is that enough possibility That it if it's at least possible then what we can do is we can that's the basis of a meter funding approach We can put some money in to some experimentation Get some information back and narrow down this chart basically. Yeah So how much does the range help? Let's pause that question and Maybe ask that again later if that's okay. I have a feeling it might be clear So the way we we we designed this process or To to bridge to bridge the gap there between qualitative staff early research and a hurricane chart The range helps to narrow down and identify possibilities Great Sure we get to this. I will think really good on time. Hope we have some good time later on as well fantastic so Really quickly an overview and then we go into the specific into the specifics of this So we want to organize our qualitative data. We're using storyboarding journey mapping storyboarding We use that to to take out the metrics we need the lead indicators funnel metrics Then we get folks to draw this up the visual financial model So really just a tiny step further Understanding the loops of the business model kind of the things we're looking for like specifically virality Then we put that take those numbers put that into a spreadsheet for the first time and Then we define ranges for those numbers basically So it's a little bit much for now, but we just move on so storyboarding and metrics What we want to achieve is we want to have of this idea. We want to look at this business model That's really what we're trying to do. We're trying to establish a new business model and we're looking at the inputs and outputs So what are they and this is really really helpful like this is probably the most helpful to you can really use out there you say all Sorry, thank you perfect. So we're usually working like you will see two examples today one is Like an Instagram kind of app idea and the other one is usually we're using a restaurant like we're keeping a very very simple And that's kind of the ideas do the examples we go with right Okay, so in this case it doesn't really matter too much, but somebody is somewhere taking pictures The pictures are really nice, right? And you know, you see this with smiley faces and then The assumption here is that they go to an editor and they get a wonderful photo back right And then they're happy and then they share it with people and then they get more social engagement and credibility and everyone's happy, right? so This is this is fantastic because it tells us exactly the customer journey like what is our assumption here that people are actually doing They come they see an ad they download the app They pay acts for it They pay after three months. They pay for whatever it is right at a certain time They perform a certain actions exactly what we're assuming and so when we narrow down the main variables of the story then we can look at what we call the happenings metrics So How many trips do they take how many how many pictures? Whoa? That's just out of life of its own. It's a lot of numbers up there and words, but If we assume they take poor shots, then how what's the how many poor shots do they take? Do they apply filters and so on? This is in a sense what we still call vanity metrics. They don't really tell us Really tell us what are we doing? But what we can do from there and you see every step should be simple even though I'm rushing through it I'm pretty sure most of you Familiar with that and you do actually could you could you verify that for me for a second? Some of you like that that makes that makes total sense. I think right Or not like does it not make sense is my question to anybody here so far Like I do you do journey journey mapping anyway, right and so The question here. This is what we're looking for This is really the base of everything. We're looking for actionable metrics It's what we say. We need actual metrics to build a hypothesis driven financial model. So That's how you can get them. You're actually looking at rates or ratios. That's what we need to Perfect. Thank you. Yeah, if you can identify mistakes, that would be great You know always have to improve obviously just as happy Then I still assumptions. Thank you for pointing out absolute assumptions 100% I have to say that when you get entrepreneurs and teams to do this and to do this well They're thinking very hard about their assumptions It's a lot harder to to argue for absolute craziness, right? So 100% still I Usually see improvements what I would I would say Okay So how many Pictures do actually need to be improved and then in this case Yeah, at what point in our journey do they create an account at what point journey do they pay, right? This is just this is just a highlight for example So we basically made it from here to here. So we organize all this qualitative stuff And we've got our metrics out already The next part is we're not even touching a spreadsheet now Let's just go to the whiteboard. Let's go to mural or what have you and Draw this up at a high level. So from here to you The pirate metric framework We're just going through it without really mentioning it too much like it's gonna be in it's gonna be a That's basically the fundamental metric system here we're using This it's great because it's because we always look at it as a funnel, right? So we look at customer behavior in the summer. So how many of this visitors come now We went from the app forgive me for that. I should change that in the future really I have the same example in the same slide We're looking at a at a restaurant now correct Correct. They can be 100% 100% we can we can make we can actually stop there Not do this not do anything else and already Try to kill our assumptions right there And we can already look like have have a feeling have a voting have a two by two Like impact over over uncertainty business impact of one certainly two by two Just from that storyboard and prioritize risk and uncertainty 100% 100% I'm just Responding to the chat, right and In a qualitative way go out gather more information against that 100% and that's what we do and it's beautiful and I recommend everybody to just do it But we're looking for a model, right? We're looking for something where we've got 10 ideas and we need to choose and we want to go further with it and We're looking for a quantifiable so we would translate this into Visitors customers money as an output it could might as well be impact Also, yeah, the challenge is often I find like a little bit like when you have internal projects Like my examples don't necessarily translate too well So if anybody wants to jump in and see how they can translate it into like internal Organizational like in large organizations internal projects that you might be very busy with and see how we can like Rich what we're talking about here Just to turn to internal stuff Very cool, but really it's about understanding the loops and it's it's not a minor thing We will see later the loops are really what makes or break a really good business model Retention obviously right if we have folks using our product like I don't know a fidgy spinner and then they throw it away and That's it right or they can't they take one picture they go away. That's it like Sure might be okay. Might be okay in certain circumstances Specifically if you want to have yeah Retention is usually it's one of the big Leaves of growth so without retention you usually have a big problem growing anything in any sustainable way and so Then then we look at referral Insanely incredibly important that specifically when we want to have Any any hockey stick curve in our business model if you want to argue for anything remotely like that we see later It's very very important And they might refer different points, right? They might be paying money first and then refer or they might come in and then they refer to three people and then refer to one later and so on so on but these are very essential assumptions we need to talk about and This really helps to align the team to align stakeholders to to talk to each other Right and that's sitting in front of some spreadsheet and putting some numbers in here and there right? It's fantastic just for You know team cohesions, I suppose or whatever the right word is Yeah returning visitors this there's so many possibilities for loops we can subdivide the funnel a little bit different and Really understand our business model better and from there we can take what we had earlier and Put those numbers in as you said They're all assumptions for 100% But what we should have at this point at least this we thought about it really hard There needs to be there needs to be a logic. Otherwise this thing breaks. There's this right this There's a certain sanity check And we have all these actual metrics in there. I think I forgot something else Right, let's let's pause for a second Does that make sense of far did we take did we go from like a business model canvas with sticky notes and research and everything That we managed to get here in a concise way does that make sense so far as what does not make sense rather Hopefully anybody dropping off that can't be that Great, correct. They have different and that's a great point, right? Because like if we knew the business model already Then we don't need to do a sense of stuff If you know your business model you can benchmark every metric, you know the industry standard You know the costing, you know your conversion rates, you know Like how much you're gonna deviate like if it's not innovative You know the business model That's the problem. You don't know your business model and if you have to write something up for it that takes you five days Or whatever and make these huge cases and make up all the stuff and then you can't even test it at the end Then you have a very bad tool for making decisions So the question is how long will this take you if you have a trained team and You get to understanding your business model. You're not a standard Right Great Well, people do it and people are incredibly good at it But it's usually the people who are really good at telling some sort of financial story, right? You make like you don't really have much so You win in this environment if you can you know do like I call it a number novel You're writing a story there with your spreadsheet and then if you're really good writer And you can write a great text around it and make this business case and then you have really good political alliances within the company and That's that's how it works so we're looking for other ways we're looking for systems and repeatable processes that can that are better and understandable and and Can be scaled and you're right business cases Every business model is different do I forget something cannot retain Our user base. What do we do with the idea? Okay, let me try to understand this after after doing all these processes model to purchase steps If it cannot retain our user base I don't know if you if you chandize if you could clarify that might just be me I don't fully follow. I might just move on Let's just set this over your stage anyway, so we're so we came from here to here to here, right and the idea is here that Of the promises that from there to the spreadsheet. It's not so far anymore They were coming very close to the spreadsheet now and Hopefully we took everybody on the jerk Everybody who does research everybody's excited about the new idea about hey, we can deliver value here. I know people want this at least right But the financial acumen is maybe not so You know Not so high or the scared of spreadsheets like how many people scared of spreadsheets because like some folks in the organization are tremendously good and You know, you know how do you know how to navigate it? It's intimidating. So how do we how do we bridge this gap? How do we bring people together and how to make a process that works? And I and that's the idea here like now we come from this visual model that everybody understands Now we touch a spreadsheet for the first time quantified model So it's quite beautiful because all levers of growth Which is basically you can Oh, silly me. I have to jump into the actual spreadsheet. All right. That's six this me folks So I didn't give away the growth for a fiber either. Great. Good job, Elia. Cool. So Come this looks like guys Okay, cool, we have the there we go all right, so In here we can now even with this basic assumptions all blade assumptions We can start playing around we can start making some predictions and sanity checks again And in a little bit more sophisticated way, right? We can say, okay This project needs to return Needs to return a lot. We say like if what about or like we can already model each type of growth We have available we can say We can what happens if we have more customers What if they stick around for longer and what if they refer a lot folks and just checking it for my notes Are we good with time? Yeah So we can see down here We can play a bit with the numbers. So I'm just asking you one thing How do we achieve exponential growth? Like what variable here? Do I need to modify like visitors come into my store? 50% of them become currently customers eating eating lasagna I make 20 bucks out of them 50% of them come back every month in this current assumption here and 50% refer and These are the cost but let's leave out the cost in Referrals and retention what do you want me to do with either of this? What number should I put in in referral or retention? Don't say hundred ten for retention for referrals Yeah, what number should we punch in there? Nice exactly. No idea 75. Let's just have a play Fantastic and what happens? Not yet imagine the business model that great 75% refer 70 70 so next guess what do we need to achieve exponential growth? When we have a hockey stick down here, sorry for those Haven't made that clear Let's try 80 what happens when we do 80 not even linear 200 200% referral well, let's do it. Let's see. Let's go up a little bit 90 95 100 and that's linear growth. So in order to reach exponential growth, what do we need to fix here? Nothing. Oh people are dropping out Customer attention, all right, I better move on. Okay, so Look fantastic. So I guess I like your position. So okay great increase referrals Further we can have like we can have a new idea. What if we Have a different referral model and so on so we can refine the business model further We can see is there any hope and you know to be fair not everybody needs exponential growth That's not the point of the exercise yet, right? There is no like there's no need very often You know, you reach a certain point and that's fine. That's absolutely happy times But if somebody comes in and says we have this exponential great story All right, show me some basic numbers and let's send it to check this in a very quick way, right? And we can also say we can go back to our Customer journey and we can look what's wrong. What do we need to do? It's all really nicely connects And what we can do now is already We can run an experiment and we can feel it back in here. So Since it's a funnel We have early indicators for success, right? If we say we have to run Facebook ads and they're gonna bring in a thousand visitors per month and 50 of those become customers Well, try to get a thousand people for the money that you assume Onto your onto your landing page If that doesn't remotely work, there's no need to play with the rest There's no need to build anything and so on right? So this is the stuff we're talking about every day this is how we bring it from You know for what we do into numbers into a way that into a way that an into a financial model Like that's a hypothesis to a financial model But we haven't really solved the other big issue like Coming back to the beginning like it's there's still all assumptions the rest still all assumptions and If you know, it's still something we need to test and test and test and it's not it's still one number and So I'm just wondering if we should go back to the presentation here So but what we wanted to get to What's the hurricane shot wasn't it like we wanted to we wanted to do that and Whoa Always nice sailing in front of techies Doesn't make it any better. Let's just bring up the spreadsheet. So what we really want ranges Who has heard of a Monte Carlo analysis? Has anybody heard of a Monte Carlo analysis before? Cool, I expected that that's fantastic Somebody else a few more everybody has heard of it. Please say yes Great. Oh, this is fun Fantastic fantastic so great Lots of folks brilliant. I'll share I'll share an article with you later like We what we can do There's a big problem with usually with best case worst case scenarios It's usually very very bad math So we could say like hey give us the best in the worst case And we just average it out and then we know the most like like, you know We're gonna hit somewhere in the middle It's not very good math and So I just want to just want to put that out there When we continue when we talk about this now. So the beauty now is We can basically take all the stress out of predicting the future For entrepreneurs and we can take all the stress out also for business sponsors now by demanding fake accuracy I think I think we can create a lot like a lot better relationships by using ranges So you see here, it's a 95% confidence interval like that's going a little bit far for today But it's not actually saying the best case scenario It's really like I am I'm 90% confident We will be an X percent confident We will never be better than this and it will never be worse than that And we do that for each variable and then we have ranges so what we spoke about earlier and That is true That's as much as we know today right and We can talk about we have a pep and have an honest conversation about it in the team with everybody and That shows the true uncertainty So just for those who are not familiar But what you do then What you basically do is you brute force a calculation onto your business model once you've defined those ranges And you make like 30,000 calculations or something and then you pretend this business model has run at 30,000 times It looks a bit fancy back here, but just to get a quick impression So here you have the range and then you have In this case, I think we have 300 calculations or so and If the variables normal distributed it takes a bit more From the middle and a few less from the outliers and so on and that's the beauty of how making it more accurate by Not pretending that everything is linear you can Model how these variables behave variables behave together certain interdependencies and so on and what you come up with at the end is Is the hurricane check and there we are they're the meteorologists of business modeling, right? And What did we have really we often really have an idea we have a few assumptions But this is really the best thing we can do in terms of predicting the future This is the most likely outcome You know still possible pretty unlikely, but as we see this calculation here is far below the What we usually take as the basic business case So I think the one the one point I really wanted to make is that You have a lot of uncertainty and the innovation team's job It's not necessarily to come like identified a winner in the beginning, right? none of you guys should be expected to To to have good ideas like every single time, right? but Progress should look like you actually learning and Now you have the ability to run an experiment to do qualitative to get some qualitative data to do research And to come back and then narrow down the ranges and they should narrow down your hurricane chart And you should be able to each week every second week show progress and Communicate with everybody else your progress in the company That gets me really excited, right? So That's how we bridge to get There's some lessons learned and Some shameless Self-promotion We do have a course on that Yep So I will stop the presentation here. I think Maybe give me the chance to share this one link before we drop off here in case somebody wants to Do that Yep Cool. Yeah, so I hope that provided no review of our technical innovation accounting is how we bring pirate metrics together With with ranges with better math and how we you know treat UX teams well and the finance department well equally Thank you