Well hello everybody, allowing me to introduce myself in sort of simpler English than what you just heard and also the topic we're going to discuss today. I'm a lifelong learner and a problem solver, that's my background. Last 20 years has been in transformation in different forms and the last four years has been in the world of digital. So what you're going to see today is part of a larger framework for problem solving with enterprises and it puts business and IT together. It pulls the agenda right up front and builds journeys. The title is deliberate because the one thing I see is this is a journey. It's a journey that's unlikely to end anytime soon and it's changing all the time. So let's get started. So any good transformation, any good plan starts with a view of where you're going. So what we're going to do is take a peek into the future. We can't look at all scenarios but we can look at a few. And this important is to sort of acknowledge what's the impact digital is having. Because you hear a lot of conversations that I see lots and lots of presentations who people try to define digital. I'm not going to bother doing that. I'm just going to talk about the impact it's having and the consequences on enterprises and what they need to do differently. Because that's the important thing. The next fundamental question is why is this happening? So all this is going on but what's really happening? Because if you understand the answer to that question you can start to plan with a little bit more confidence. You can actually say if these things are the things that are causing disruption forget my head around those. I can plan for change. And lastly what do enterprises need to do to get started? What do they need to focus on? What do they need to think about in terms of planning and architecting change? And then how do they get it done? So here's a few examples of the future. So let's meet Claire. Claire is a working mum sort of mid-20s early 30s in the not too distant future. Something just a description of Claire. So Claire, she shops all the time apparently but she's never visited a shop. She's always online but never logs on and she optimises her ratings in real time. So the sort of ratings that are important to Claire are how likable is she, how employable is she, how knowledgeable is she and what's the rating she's had on the last 10 jobs. She's never got lost ever. She doesn't know what that feels like. She tracks her kids whereabouts and well-being wearables is commonplace these days. And she drives the car but she's never learnt to drive. Claire has five jobs at any one time and they're subject to, they're obtained through crowdsourcing. She's based on her skills, the quality of feedback she's had from other engagements and her availability. So her family and work life are perfectly balanced. And to top it all, she has a great home help. Does all the cleaning, does all the household work, does the cooking, does all the chores, 24 hours a day, it's a robot called Jane. Now some might think this is science fiction and some may argue that some of things are possible now. It's quite likely most of this is going to be possible within the next 10 years. So whilst Claire's destination digital, her journey in life is unique, technology hasn't just had a change to specific experiences. It's changed the way she lives. And this is not stereotype, by the way. I just wanted to think of something simple. So David's business trip, so David's a busy guy. His trip was planned by his artificial intelligence assistant who optimised it against cost, timing and fit with his other commitments. So he leaves his smart home, he gets into his connected car that drives him to the airport through the smart city. Once he's there, David's put three of his artificial intelligence assistants to a conference call. And on arrival in his destination, he's picked up by an autonomous taxi. On his way back, he's reminded you've got to buy something. And when he gets into his connected car that picks him up from the airport, his eating's turned on at home. So when he gets home, life's comfortable. And it automatically, his expenses are filed for him and submitted. All in a day's work. So two simple scenarios. And the important thing here is the takeaways. How enterprises approach their destination journey is therefore key. The previous presentation is actually the world, mostly the world we live in today. Preoccupied with the way organisations were making sure they work efficiently and effectively. So you don't get red scorecards. But actually, not thinking that much about the future environment that is out there. So moving from what's called inside out, you know, worrying about existing processes and structures to sensing what's going on out in the wider world. That mindset, enterprises are not just part of the future. They make it for the people who live in it. And the winners are those who, quite frankly, are connected and relevant to the likes of David and Clare. So I won't go into detail on this, but this is just an excerpt from Interbrand's top brands for the year 2016. It talks a little bit about who they are, what they're doing and some statistics, particularly around things like the reputation, the value of the firms and so on. Two interesting takeaways from this are these are actually already digital businesses. They're businesses who actually operate using platform concepts and they're very good at it. And the other one, although they started that journey by extending their services that they offer to the market. And the second one is there's an awful large investment in future digital technology in terms of the ways of working, in terms of how collaboration takes place and how things get done. So often you find with some banks recently I was talking to, we happen to be a technology company that happens to be in the retail banking sector. A bold statement really, but that's the way people are thinking now about changing the way they run their businesses. So, why is it all happening? Well, is it because of a multitude of new innovations? There's plenty of evidence of what's going on out there. There's a fantastic report if you ever get a chance to read it called, Deep Shift was commissioned by the World Economic Forum. So here are some of the things that come out that by 2025, so this is what they're classified as tipping points. So tipping points means this is going to be business as usual. This is the way the world will be. So here are some interesting ones. 10% of all humans will wear clothing that's connected to the internet. 90% of people will have access to free unlimited storage. Now, if you take that and you say, well, what did a gigabyte of data space cost me in 1970, and you compare that with today, you're talking millions of dollars versus absolutely nothing. And it's one trillion sensors connected to the internet. What do you do with all that? The first 3D car in production, but here's another one. The first 3D printed liver that gets transplanted and an artificial intelligence member of a board. So is that why it's happening? Well, the reason why it's happening is just a view is, as human beings, we just want to know more, we want to connect more, we want to share more. We have an insatiable appetite to make all these things just happen. So I think rather than having lots of people out there looking for smart things to do, people are just coming up with, we could do, like take the two examples we just talked about, how much spare time, how much free time did those individuals have if they were really tuned in to using digital the way they were? So I've got three, what I call three truths, three truths of digital disruption. Now these are useful anchors. What they provide is not so much, you know, that's your opinion. They actually provide ways of thinking about change. So the first one is that the people who consume digital services are predominantly the people who will call the shops. They will set the pace, I would like this, this would be good, these things would work. And these are not just consumers, these employees in an organisation. They're the way you run a manufacturing plant. They're the way you organise a car to be connected to lots and lots of other services. It's very, very multi-level and complex. But essentially, that's one place to focus. The second one is the move from traditional to platform type businesses. And all the examples on the brand chart that I showed you earlier are organisations that are gradually on that road. Some were born that way or claim they were, likes of Amazon and Google, but they're all on this journey. And just for those who kind of think, well, what's he talking about? There are disruptions in industries. Take the hotel sector, for instance. So traditional value chain, hotels, travel agents, travellers, replaced by hosts and travellers connected via a virtual platform called Airbnb. But you can also get disruption in a value chain. So think about devices, intelligent, or semi-intelligent devices in the manufacturing space, building cars and doing work in the world of heavy duty machinery. GE's Predix platform, fantastic idea, creates something that connects everything so that they can become more efficient and be organised in a way that in some ways, for instance, they're autonomous as well. So you're moving from a value chain structure that is where things get built, sold, shipped, consumed to an environment where consumers and producers are actually connected through a virtual platform. What's the consequence of that? Well, on the one hand, you've got a model which is, its value is completely a function of resources. How big it gets, how many things are happening in that value chain is a function of what it's worth, how much revenue it can produce and so on. So working capital is tied up in that value chain. Both physically and for the time duration it's being used in the platform business. Once the platform is in place, consumption actually drives down capital investment. The return on invested capital just goes up. So the more successful your platform concept, the more valuable your business. So it's not a function of resources anymore, it's a function of what you do and how valuable it is to the people who want it. Now with that in mind, the last one is, and those organisations who can stay ahead, both in terms of what's needed in the marketplace for them and their business, and leveraging that platform journey because it's not a one-off event, digital is not a one-off thing as we'll talk about in a short while. Those that can leverage that, so connect with it, leverage it, are those that will win. And I've classified this as asset multiplier. So effectively continue to drive down the return on invested capital impact on the organisation. So this is all about business value as well as what we're talking about. So those are the three truths. So what do enterprises need to do? Now I'm going to get a little bit now into the framework elements, some of which you'll be pleased to hear to do with how they are architect change. The first one is to understand the impact of the relevant disruptions and map the value options. We call this 4x3, so essentially how do those three disruptive effects align with a set of four sources of value? And I'll talk about what those four sources of value are. When you figure that out, you then localise the agenda, you then figure out there's lots and lots of things that could be going on, what's relevant to me, so my journey is signposted. I can see where I'm going. The second one is use that to populate a future business context, which is composable, well-organised and an extremely good platform for planning. Now the reason for that, the reason for a new business model context is quite simple. If you want to think and you want to organise yourself as a platform business, describe it. It won't be perfect and it shouldn't be perfect to start with because it will change tomorrow, but it allows you to start on the journey. And it's a model we call 5D. I'll talk about what that is in a second. So let's talk about these four sources of value first. The thing on the left is just a depiction of what those four domains are. They're transformation domains. They're things that need transforming. And the thing on the right is an illustration really that says if you really think about the fundamentals here, the economic value add is probably the best way to look at this holistically in terms of business impact and return on invested capital as we've said through the use of platform effects is one of the most important metrics to measure. The thing on the left has below it something we call, I've called there, the 21st century enterprise culture. It really is about what does your organisation need to be like. That's not something you're going to cure in one go, but it is. It is something that evolves over time as we look at. So in order really, the most important thing to move first and we just heard an extremely good case study of what are the platform capabilities and what's the culture within IT that will allow it to evolve into this new model. Platform capabilities are things like joined up data, joined up information, joined up architecture, the use of cloud, the use of analytics and so on. And also a culture that's capable of managing change at speed. I heard somebody mention the word sprint. Organisation structures that allow for that collaborative team working approach are not commonplace. The second one is, so what about the experience? Now you'll see from some of the examples I'll show you next that if I'm in a smart, this is really about places and locations. What scenarios exist? So this isn't about the incremental, the coffee shop experience that we heard earlier. It's what are all the things that could happen in that coffee shop? There's lots of things you could buy. There's lots of people you could interact with. There's lots of things you could do. Describe them for me. It is no longer a one organisation value proposition. It's a value proposition that involves an ecosystem of different players who have roles to play. The third one is changing the way that business operates of course and with the sort of digital platform, the physical IT environment and the ways of working that go with it. You have the opportunity to reinvent, reorganise, automate and apply things like robotic process automation to help that change. Last but not least, you also need, and this is an addition, an environment that is actually deliberately going after the next generation of services, the things which will move the needle for the organisation in the marketplace. So the complementary or companion to that is something we've called 5D. So starting from the premise that today's enterprise is largely reflect the internal workings of the business and have less to do with the environment that they operate in. And as this operation environment is changing, the environment around them is changing a lot and fast. This inside out view is no longer good enough. To move to a model that's sensing that digital environment and actually organise in a way that allows you to adapt as you go, that's the description here. So just quickly around the edges, the digital customer engagement domain is really where people and things interact and increasingly complex scenarios, things that we've already talked about like smart homes, smart cities, connected cars and so on. The digital service platforms are then the facilities or the capabilities required to get monetised through that scenario. So take the travel example for instance earlier, the sort of things that will be in here and exposed digitally would be ticketing, booking, expense management, loyalty cards and so on. The next one on the list is integrated business processes and this domain covers digitising existing processes and there's a big balancing act that goes on here, creating and provisioning new platform type environments that actually leverage these new capabilities efficiently in the sales environment, things like salesforce.com doing it properly is something that requires both things to exist. However, there is the IT struggle that says if business won't work with me, how can I migrate from that way of working to a new way of working, not so easy. So there's a massive cultural element to that as well. And the last one when it comes up is new value networks, you know, how opportunities like smart home and can be organised and exploited. And the last one, if I can get it to come up, it does work hopefully. Not working, it is now. Digital mustn't mean chaos. So this is where changing the mindset around architecture and adding the things on, which at the moment become almost independent islands of architecture like IoT, they get pulled together and the interface between all of those and the different moving parts on this chart are organised in a way that they kept in sync. So that's that. So the construction of the use of this to construct, how do you construct this? So here's a very simple retail food example. So we take these sort of line by line. So when you think about digital customer experiences, the food retail is now faced with a much more empowered user. And the decisions they make are obviously very different to the ones they would have made before just going into a store when they can do things online and they can get things from third parties and so on. So new interfaces to customers, new ways of connecting with customers that are done through associations. So communities that for instance are more concerned about keeping fit. What sort of things do I want to buy? Where do I buy it from? All of these influences need to be touched or at least sensed by the organisation. It can't just rely on the fact that people show up at its door to buy stuff. And of course the technology platform, the technology capabilities need to be embedded in to enabling that to happen. And here's a few others as well. So thinking about digital process transformation when you see things, you look at things on a shelf, you're passing by a store, the idea that the shelf could be a digital viewpoint that's constantly rolling with ads and so on. Personalising services in terms of business innovation and last but not least, things which connect people... Is it working? It can connect people on a personal level with the brand that they're connected with. So taking the output of that is a relatively simple overview. By the way, this is in a companion paper that goes with this topic, so you can read about it in more detail if you wish. It's something that then looks like this. And essentially this is... By the way, the health service example, which I'll put on the next chart, is a little bit more detail. And essentially what we're doing here is organising those capabilities that are required next into a structure that then allows us to plan. Here's the health example. So this is one that's a bit more detailed. So when you look at this, you've got topics like, and my eyes are not brilliant, so I'll have to... You've got topics like the service platforms are essentially the things which allow either to deliver services or consume. Different scenarios through which a patient can be seen and met and dealt with. And the connection between all of these, obviously, via a different way of integrating information. So the last question, because I'm getting at the clock here, is how do enterprises get started? The fact is that for many years, after year 2000, the primary focus was on Mode 1 type operation as the analyst put it. This is... Let's make the organisation, let's make everything inside it more efficient and effective, consolidate things, simplify things, and so on. That very strict approach to management of change, that's very strict approach to what's acceptable, and the governance that goes with it is somewhat restricting if you're thinking about agile and experimentation and so on, so-called Mode 2 operation. So are these mutually exclusive? Can that be integrated? That's the big culture question most organisations face. And what skills are required and does the firm have them? In most cases, firms don't have these structures in place. They don't have the people who know how to do these things. Multiskill collaboration is a feature of many start-up and developer organisations today. So organisations that can tap into that and build their own version of it are obviously likely to win. And also how decisions are made. What about programmes? Programmes tend to be set up, have large structures and have a start, a middle and an end. This is much more about road maps. It's about... I think you mentioned the term road map when you presented. The road map is put change on things that you both figure out and come into contact with. And techniques like minimum viable product production. How do you create a minimum viable product? What does it involve? It requires multidisciplinary teams for a start. Very close connection with the business and an appetite to get things done in weeks rather than months. And also to test and learn quickly and fail fast if you need to. So a very big practical set of questions. The question is how do you make them mainstream? Well, this piece of the framework and I'll come on to the top picture in a few minutes essentially says the answer is three elements. The portfolio of initiatives that have been flushed out of the analysis that takes a look forward and compares that with the world that exists today. And the second one then is the introduction of this into the organisation. And the most sensible way we've found to do this is either within a function that needs it today or within an experimental function that's trying to do something new. And the techniques in explore things like how to produce a minimum viable product quickly, how to create multi-skill teams, how to test an experiment. The elaborate is really taking that and connecting it with all the dependencies that exist in an organisation. So that scaling process can happen quickly and execute is clearly mainstreaming. Evolve is about taking stock of the capability maturity because whilst something like that 5D model can set an ambition it's way beyond the capabilities that most organisations have. An incrementally stepping through the journey allows the evolve to play its part in terms of raising the bar slowly. So organisationally and in terms of the digital capabilities that are required. So, a second last slide. So to get ahead, the two things that I've mentioned so far are essentially figuring out the dynamics, figuring out this four by three assessment. What's going on in your market? What's going on around the business? What are the things that I need to worry about that I genuinely need to do with moving the business from a traditional to a platform type model? The second is then that approach to experimentation, that approach to culture change, which allows what is a waterfall type approach in most organisations gets replaced by something that's much more collaborative, much more joined up with the business and what's going on in their world. The other three things on here are like stabilisers. This doesn't happen overnight. It takes time. Hybrid's going to be a factor of life for quite a while. So the three things are, first of all, the digital transformation management. Now, organisations manage change. How do they manage this type of change? How do they cope with this type of change? How do you mainstream it? It's a gradual process. So setting that up front and being deliberate about it is obviously very important. Having it buried in IT, for instance, or having it as an island in marketing doesn't make an awful lot of sense. The next one is the digital transformation neighbours. So as one steps through this journey, there are going to be things that are required, and rather than going off and having huge investments, bring them in as they're needed in a just-in-time fashion. This is both the organisational skills and the technology skills. Whereas before, an awful lot of these things were outsourced, we're now starting to see a trend where things are being brought back in because these are core skills and important to the future of the business. Last but not least is the business partner model. These platform businesses do not exist on their own. They survive and thrive on the back of partner ecosystems. Those partner ecosystems are built gradually and they become effectively the extended enterprise for your organisation. Everybody's engaged. Take the GE example, for instance, the extended enterprise for a manufacturer, for example, includes people who will have sensors in everything that they use. They're all going to be connected. There needs to be a way of organising that so that it's supporting the business mission. So in conclusion, some takeaway thoughts. The destination digital is a deliberate title. This is a journey. Navigating your way through that journey is really what this is all about. So having one to start with, having a view and a context within the environment that makes sense and being able to plan with confidence. My observation, and it's a firm I work for, our observation is most firms right now are preoccupied with the core. They're preoccupied with reorganising the capabilities within the core of the business so that they're better aligned with all these fancy things that the organisation would like to do because the good news is they've tuned into that. The bad news is if they start doing the other things first, it just makes the journey a bit harder. The culture change is the single biggest challenge and it really is behaviours and discreet capabilities that don't exist today. Another very big one is that CXOs, so people on the board, I heard some comments today, is the business involved? If business leaders are not involved in this agenda and don't know how to align the next generation IT with where they're going, they probably won't be that successful. And it's a business transformation, not an IT shift. Everything I talked about at the beginning was deliberately about people and lifestyles, not about the IT that supports them. And finally, not the best news you'll hear today, but really realistic, hybrid is going to be a fact of life for quite some time yet. So the scenario we heard about earlier is going to be around for some time to come. Thank you for your time. Thank you, Mike. If you could take a seat, we do have a few questions. I'll give you this since you're not mic'd up so you'll be able to deal with the questions. Thank you very much for that. Quite a few and again, not much time, but straight in, social question, does this future include all socio-economic classes or just a select few? Everyone, that's short and sweet. Does your model also apply to industry 4.0 use cases? Yes. I like the way that, this is great. OK, rattling through. A couple of related, but I'll pick this one. Since an ecosystem is beyond the control of any single enterprise, should enterprise architecture be changed to ecosystems architecture to support the digital business of the future? That's a great term, actually. Yes, it's a short answer.  It is essentially a business, business competence. That's just a personal opinion. OK. How did you get from the digital life scenarios like Dave's business trip to the 5D model? It seems like a big leap. It seems like a big leap. The 5D model, so the genesis of that was working, so it was essentially a combination of transformation strategies and enterprise architects, people working within the telecommunications sector over the last four or five years. So it started out a very simple idea as a sort of digital front-plane, cross-plane, back-plane type model, the sort of thing that you would expect to see in any EA's handbook. But it then became a much broader conversation about business change, and that's how it evolved. So that's a product to another of years of experience. Applying the different scenarios we've got onto it is not difficult at all, but they weren't related. Right. To start with. Have we got long-term assessments that going agile will work? They could cause a lot of technical debt that will make the organisation lose a lot of money. I'm not so sure I answered and understood the question. I interpret it as do we actually know that with everyone going agile and sprints and following that approach, do we actually know it's going to work in the long-term? I think the important thing is not to have them become setting stone. Applying agile and theory of constraints has been around for a long time. That's nothing new. Applying agile principles in collaborative team environments, multi-skill team environments is relatively new in terms of this type of activity. And it should be left to evolve. There should be some standards around it, in my view, because otherwise it will get lost. You mentioned a digital transformation management office. Does this replace the traditional IT department or is it part of it or maybe it's something quite separate in the business? No, it's an overlay. So if you think of strategy and planning, you've got annual strategies, you've got quarterly reviews, you've got monthly budget reviews, you've got weekly optics on how are we doing. The forward thinking bit tends to be a once a year, let's see where we can go, whereas now it's constant. So I think there's two parts. The overlay, the architectural thinking, the innovation agenda that goes with looking forward should be always on all the time. And it should be systematically linked to managing a portfolio as opposed to an IT line of record. So it literally is, it forces a forcing function for changing the way the business thinks. OK, and one more and then we'll go to coffee. How do I measure change in behaviours and culture? That's a long answer. The simple answer to that is are you doing the things that connect with the value chain? So in some of the models, and there's a fantastic model that goes to that 21st century organisation logo that talks about things like how are you managing sense and respond? Are you really working in collaboration with other people?  All of the things, all of those things can be broken down into outcomes that can be measured really, that you're really doing it. And you said it was part of a paper that provides more depth on this? Yes, an introductory paper that I wrote on it a few months ago. Actually my colleague and I didn't know whether he knew that I was here today. He actually published it last night on LinkedIn, so I can make that available. It's great to get some feedback. The important thing is this is something that's going to be allowed to evolve over the next couple of years and be scaled so it would just be great to get some feedback on what people think. Great, in the meantime thank you Mike for sharing your thoughts. Thank you.


Working with amazing classmates as well as professors and really building upon my skills and really learning so much about the finance industry. The whole process of working with our teams and being able to have that end product, that has been like one of my favorite experiences. Professors, they're interesting, they know how to bring the real stuff to the class. Even after slogging from eight to seven, I really wanted to come in. I've seen different faces but I'm also seeing familiar faces and I've made new relationships and friendships that I can take with me in my personal life but even in my professional career.


First, we're not going to let you deal with a hangover very well right now. We're going to start with Joe, who is going to talk about fuzzing. Yeah? Good stuff. Let's give our first speaker a big round of applause. Good job. Thank you. My name is Joe Rosner, and this is retargetable, grammar-based test case generation. So I want to start off with a story about how we got to where we are today. And ultimately, the lesson of the story is that testing parsers is really hard. About four years ago, at my current company, I started implementing a SQL runtime and parser for a handful of different dialects of SQL. And the goal of this was to evaluate SQL queries and determine whether or not they were dangerous. And in doing this, we built out mostly black box implementations of the vast majority of the SQL dialects out there. And all told, it was something like 35,000 lines of antler definition. So this is like grammar definition where you are defining what makes a language and you pass it into a program, and it will generate you a parser that can recognize that language. Most of this was done through looking at poorly written documentation. It's not documentation about how the system itself works, but documentation for a developer trying to learn how to write SQL queries and what's valid. Typically, they're incomplete. They don't have the information you need. And you're kind of fumbling around trying to figure out just what it is that's correct and what makes it correct. Things will be contradictory, and you never really know what the answer is. And in a lot of cases, these are closed source systems, DB2, Oracle, MS SQL. So you can't just go to the source code for them. Now, we used antler, but most of these tools like Bison or Yak, things like that, you will use an LR style parser or recursive descent parser, which is vastly different than what antler uses. Antler uses an all-star parser, which is a top-down parser. And so the way that the actual algorithm works and you recognize a language from this grammar is going to be different. And that leads to all sorts of weird cases where you might have two grammars that are identical in terms of definition, but they're going to be recognized differently because of the algorithm that's being used. And most of the time these lack public test cases, because they're closed source software, the vendors don't care if you can verify that what they did is correct. They just provide their software and wait for you to complain, and then they fix it maybe. So this issue that we ran into is really what gave birth to this idea and where it came from, but the security use cases for it are what vastly expanded the scope and kind of further drove the usefulness that I've found and why this has kind of come to be. So there's a few different problem areas in parsing when you're trying to verify whether your parser is correct. And we're going to talk about the three of them here. The first is overfit and underfit implementations. So the goal of a parser is to recognize a language. And the question of overfit and underfit is am I correctly accepting and correctly rejecting sentences that are part of that language? So if I have the reference implementation and it accepts something that mine doesn't, I'm now underfit. I do not recognize the same language that the reference implementation does. And if it doesn't recognize something that I do, then I'm overfit and I still don't recognize the same language that it does. And this is a problem that is specifically based, not specifically based, but it's a problem that is exacerbated by poor documentation and not having a firm understanding of what is correct and the internals of how the parser works because without understanding exactly what the algorithm is that it's going through and what it's using to determine whether or not the sentence is correct, it's really hard to replicate that same behavior and get to the point where you have something that is going to represent the same language. And so without having public grammars and without knowing about the algorithm, you're limited to reverse engineering and experimentation with the reference, trying out things, trying to discover what sort of behavior it's using and basically just trial and error until you can get to that point where you really understand, okay, this is how I need to define this section of the language and go on from there. As I've already said, there's differences in parsing algorithms. If you've ever tried to reverse engineer one of these generated parsers, it's basically impossible. It's a giant state machine built out of go-tos, essentially, and you just kind of have this state where you're constantly mutating it and it's a really bad time. You shouldn't do it. And in these different parsing algorithms, they typically have different behavior for a handful of different properties. Things like ambiguity are two different production rules going to be interpreted differently. Do you have left recursion? Are you going to continually go and repeat a specific rule and precedence? How are you expressing precedence for operators? These are all things that every single parser generator you use is going to handle differently. And so taking a grammar from one to another is going to behave very differently in all of these areas. And the worst part about all of this is that universally proving two context-free grammars are identical is undecidable. This is a problem that, for the general case, is not decidable and you cannot write an algorithm that does this thing. Now we can slim down this problem and we can take a subset of it and we can maybe do it for that, but in the general case this is not something that we can do. The next problem you worry about are tree generation flaws. So once you've recognized a sentence in a language, typically your goal is to create a parse tree or an abstract syntax tree. And basically what this does is it shows the syntactic relationship between various components or tokens of that sentence. So it has some kind of root node which is the top point of the tree and going down there this provides context for what a specific token is and how it relates to every other token in that sentence. And the whole process of building trees and working with trees is super complicated as you move from generator-generator run time to run time. Most parsers will require manual tree construction and this is something that has kind of gotten better recently in newer tools like Antler, but typically you'll have a production rule which it will try to recognize and when it does you'll then run a block of code and that code is responsible for building up a node and then inserting it into the rest of the nodes at the right point to build that representation. And this is typically the point where you're going to see things like if you recognize a number you're going to parse that into a numeric representation for the language you're in. So the number one, two, three, four gets read as decimal and converted into an integer in the language you're building this in. And we'll come into issues with that specifically shortly. This is an example of something like this. This is a LallarPop which is a parser-generator for Rust and basically on the left-hand side of the equals arrow you have the different production rules and so this is what it's trying to match and this is what it's trying to recognize and on the right side you have the action that's being called. So for each one of these going down, these are all alternate so it's going to try to match. Oh. Sorry about that. I'm trying to juggle two laptops so I can get speaker notes and it's not going so well. So these are all production rules and each one of these is an alternate so it's going to try to match one of these and everything on the left of the arrow is going to be a rule. Everything on the right is an action. So once it recognizes it, it calls the action and that is what builds up the tree as it goes down or up depending on the algorithm that's being used and whether it's top down or bottom up. The third problem is unsafe and incorrect use of validated input so once you've recognized a sentence and you think you understand what it's doing what are you doing with it? Typically recognition isn't like the only step it's typically the first step and many more steps you will take to use that information. And so most parsers will say yes this is a sentence but don't necessarily say yes this is safe or the information here is okay to use. So the only goal of parsers to ensure that a sentence is a member of that language. And so once you've recognized it you have numbers you have strings. What do you do with those strings later? And essentially what this comes down to is if you can get past the parser with something that is syntactically valid but is still something dangerous maybe down the line you copy this to a buffer somewhere it's just back to any other kind of typical vulnerability exploitation at that point smashing the stack, logic flaws and it's that simple. Your goal is to get past that point it's the gatekeeper. Reasons why this is especially bad is that a lot of defining grammars is really hard when you have complex languages so you'll have places where you might have opaque tokens where you say I need to read in a character set for instance like in a regex but I'm just going to read that in as a string anything between like two square brackets and then later on I'm going to go ahead and figure out what that actually does and this is super common you'll see it all over real parsers so you might recognize this as a sentence in the language but this small subsection of that you're not going to understand like what it's actually doing or not it has some like bad value in it so these opaque tokens are kind of rough and this ultimately is to parsers not really having a lot of semantic information about the language they're recognizing and there's just not really a good solution to this in many cases because of the tooling and because of the complexity of the languages and if you subscribe to the length set concept you know we should be working to reduce the complexity and make this an easier problem so that some of these can go away so how do we test this? it's great that we've identified the problem areas but how do we do better? how do we ensure that our parser that we're building is going to be a faithful representation of another reference implementation typically it comes down to just getting more test cases and getting to the point where you feel comfortable that your parser is recognizing the same set of sentences that another is and to do this you need a shitload of sentences like a massive amount like having 100, 1000, like 10,000 like just really isn't enough especially for large languages so writing them by hand just really isn't a viable option like there's no way you're going to have enough variety and enough targeted sentences that can cover that like it's just not realistic unless you have people just cranking these out all day for hours so something that we ended up doing was crawling the web and looking at like Stack Overflow and just pulling out really bad examples going to open source projects and grabbing them out of there and this is great because you can get a lot up until a certain point but you have no guarantee that you're exercising different areas of it you don't know really what your breadth is, how deep you're going and there's not really good code coverage tools for this so you're kind of just assuming that you have all of this coverage but at the end of the day you don't really know and even within a specific tree like the individual values for tokens just variation in that one space may have significant impact depending on what the application is doing with that value afterward so the need to generate test cases at a much higher rate is ultimately what I arrived at and so the way to do this typically is using a fuzzer and there's a few different styles of fuzzing that are pretty common today and all of these sort of had issues with the style of testing that we were trying to accomplish so I'm going to walk through why that is and where these types of fuzzing are good for and how we can get to a point where we have something that is more targeted for this and maybe be used for these more traditional types of fuzzing to augment them and make them more effective in what they do so one of the most common ones you'll see today is AFL this is like the hot fuzzer right now and basically what this does is it focuses on path exploration and code coverage to make sure that you are targeting as much of the application as possible this isn't aware of any semantics or syntax it is implemented not in a dumb bad way but it's dumb in terms of how it tries to explore so it instruments the application and watches as it goes through it it will take some kind of seed value and it mutates that and it determines whether or not that mutation allows it to explore new areas and if it does it saves that and it continues down that path and if it doesn't it throws it away as something that is not interesting and then it repeats this process billions of times over and over until you stop it basically and bugs fall out and so this can take a lot of time because it's using random mutation and it's not very targeted and its whole goal is to kind of just like explore so you might spend a lot of time in uninteresting code paths and there are some ways to help kind of guide it and cut off certain areas in some of these types of fuzzers but for the most part it's a very random process and in the case of trying to evaluate the specific types of tests that we want to is whether two parsers are implemented the same it's not immediately clear how to build a really good test harness using something like AFL to do that the way that it determines whether or not something is interesting is by hooking the sig abort and the sig kill signals and then waiting for the program to essentially like crash or terminate and so getting the semantic information that we want to about whether something is underfit or overfit or why did this test case exceed or fail and how does it compare to this other thing it's not really immediately clear how you would encode that information into something like a test harness for this and there's a great blog post showing some of the use cases where AFL actually can be used in some cases like this so basically the link at the bottom here walks through taking the string hello and passing mutations of that into a image viewer and basically getting from the point of the string hello to valid JPEGs and I have a few quotes that I want to share from this and basically just kind of showing where this is helpful and where this isn't helpful for this type of task so the first image hit after about six hours on an eight core system and this is actually pretty good and pretty surprising going from nothing but the string hello to something that is a syntactically valid JPEG is like pretty good I mean this is not crazy hardware it's something that anyone probably in this room has access to and anyone who's a real adversary trying to find bugs like this is no problem to get access to certain types of automatically executed checks with a large search space may pose an insurmountable obstacle to the fuzzer and this is ultimately like the real crux of the problem is that if you have something like this in your application where you're comparing some section of a sentence for a very specific value a tool like AFL is going to do a really poor job of figuring out what it takes to get past this check because it's using random mutation it doesn't know what it needs to do to get to the string hacked by pigs here so there's other types of fuzzers do a really good job of this and we'll talk about those in a second but this is a super common pattern you're going to see in cases like shotgun parsers, binary file formats and so things like checksums, magic numbers, offsets into files these are all things that you don't want to spend time really exercising a lot of that because you want to get past that typically to the point where something interesting will happen I mean there are cases where just the code here might be interesting but you really want to be able to get past that in practical terms this means that AFL fuzz won't have as much luck inventing ping files or non-trivial HTML documents from scratch and the reason is because these are very very syntactic and complex file formats there are aspects of it where things need to be in the right place in the right order and simply moving things around won't produce usable input and so this is one of the areas that I'm really hoping to improve by releasing this code today the next type of fuzzing we're talking about is instrumentation and solving and if you followed any of the CGC stuff a lot of the solutions that were built for that used this approach as part of their solution and like AFL these also will focus on path exploration and code coverage but they do so in a different way rather than trying random mutations they convert the problem into something that they can execute some amount of and when they get to a point where there's some kind of input or something that is non-deterministic they convert that over to a symbolic value and then you use something like a satisfiability solver to identify what it will take to move past that branching point generate me a value that will take each path and will allow me to explore those so this is how you would solve that problem of very specific strings where you need to know or check sums or anything like that where you need to know how to get past this step is it will provide something that will solve that problem for you and then generate you an input that matches that this still doesn't care about syntax or semantics it still has no real concept of what any of that is or what the language is even for that matter it's just trying to figure out how to get deeper into the program or to a point that you want in that program even with these they're still not typically easy to build test harnesses for a lot of this is still automated but you could probably hack in a lot of more interesting stuff in it if you wanted great examples of this are our Klee open source software there's like the mayhem paper which is really great and you should go read it if you haven't and then the last big type that we'll be talking about today is grammar based and this isn't a new concept it's something that has existed and there are a handful of projects out there that do it but there's a whole bunch of problems with the approaches that have been built in terms of what we were looking for so what these do is it takes a grammar file and it will generate you input that is syntactically correct based on that grammar file and some of the problems with this are that they typically provide their own grammar language so when you define a grammar you have to write it in their language and if you have ten, fifteen thousand lines of grammar for a specific language do you really want to go and translate all of that definition over to another language you're doing it by hand typically which means that you might make mistakes, you might define something that's not the same and just the semantics of that language may not translate very well to the new language for this fuzzer these are mostly targeted toward regular and context free languages and this is fine a lot of languages do fall into that category but for more complex languages things like really complex binary file formats context sensitive languages like these are just not applicable and they're not going to help you they're typically not byte oriented so you're kind of limited to a lot of text based languages and so you're limited in what you can do with these and the use cases you can have for them Mozilla's published Dharma which is their tool I believe it's primarily used for testing out recognition of things like CSS, HTML, JavaScript and ensuring that things don't break as they test our browser but these are like the main types you'll see and we're going to expand on the grammar based one shortly and last I've been talking a lot about traditional test case generation and just how you will go about actually testing and using what you generate as your input and so I've already said there's a lot of inflexibility with being able to design complex test cases and using those in a meaningful way to compare two different implementations or ask questions about how closely one matches to another or what's being done with that and you're kind of limited just to kind of does the application crash and identifying a few different things within it you also don't get a lot of flexibility in defining a feedback loop and this kind of falls into the prior discussion you can't really feed in more than like does this explore more or does this did this do something interesting and so being able to have more flexibility around that is something that's really nice and so there are many tools out there that do some parts of this but we can do better ideally we want something that's more flexible something that we can really program our own test harnesses something that we can do whatever we want we can design a test that does whatever we want for any case that we might have we want to be able to use the grammars we've already written why should we have to translate these especially by hand to something that isn't used by almost anyone and isn't going to provide any meaningful value to us other than our test generation it should be expressive enough for everything that most of the other fuzzers out there do right now so covering context free and regular but we want to support context sensitive we want to do text and binary we want the ability to kind of provide value for any kind of language that we can think about and might want to test being embeddable from any language that we might want to use we don't want to have to go out into C specifically and write this if I'm building something in Python or Ruby I want to be able to just call this directly if I have functionality around the parser generator that I'm using in that language I want to be able to do it directly in there I don't want to have to go and build a whole separate ecosystem to bring this over to somewhere else and we should focus on as much code reuse as possible you know if I have a whole bunch of components built I don't want to reinvent the wheel every single time I want to design a new test or I want to do something new with this and so the project I'm sharing today isn't the first it's not the most feature complete or even the most novel solution to this but it is more flexible and it is more extensible and it is something that can get to the point where it is most of those things so I want to share with you sin fuzz I'm releasing it today and basically this is a platform for building test generation and test harnesses and fuzzers it's organized like this at the center you have an intermediate representation which is a bunch of data structures and some helper functions that allow you to construct complex grammars outside of that you will have your various front ends for your different parser generators and combinator libraries so things like antler and regal and ebnf bison and essentially what these do is you take a grammar file with these front ends it parses them and it translates them into the intermediate representation the test harnesses are reusable components that you design and these can be as simple or as complex as you want you have the full availability of any programming language you want to use and you can design the test to do these and so you consume the intermediate representation you give it a starting rule that is the root of your tree and it will start generating new parse trees and then from the parse trees will generate you sentences based on that ideally the only piece that you as an end user will have to build is the test harness and the reason being that it likely is going to be something that's non-trivial there probably could be very reusable components and that would make the job easier for you but that is ideally the only piece that needs to be built for you antler is currently the only front end that I've written but I'm planning to expand pretty heavily out from there this is a list of a handful of the features that you get so like most combinator libraries you're going to get a lot of logical operations as well as quantification specific values for generating specific types of values right now everything is byte oriented so it has unicode support but it will output to a byte stream using the framework is something like this the core is written in Rust but the plan is to release C bindings that can be brought into basically any language but essentially the way this works is you bring in a grammar file and you read it you pass it into a generate rules function and it will take that grammar and parse that into all of the rules that are expressed by that grammar you then provide it a root node and tell to generate and then you spit out the generated string and do whatever you want with it any communication is available with this pipes files it builds cross platform and can likely target most embedded platforms which is cool so if you wanted to do anything with this pretty much anywhere you can and I have a quick demo and if you'd like a live demo you can catch me afterward and I can show you more but essentially what this does is I basically have a while loop that is really hard to read I'm sorry is any of that legible? cool I'm not allowed to plug my laptop in but I will talk you through it and if you'd like to see it I can show you after but basically the way that it works is I have a while loop here and it's calling the program and what it's doing is it's generating a string and then passing that into antlers parsing tool that you specify a grammar and a root node and it will try to recognize that language and then visualize the parse tree that it's building and in this case I'm generating dot so like graph is files and so it's basically just running through trying to generate new ones every single time that it gives me a nice little parse tree here it'll happen again and again and again it might be kind of slow so this is like you see it's a totally different tree it's bigger it has different values in it structure is different and so it's the generation is actually very fast the slow part right here is parsing it for for antler itself but it is multi-threaded it's thread safe and so you could spread this out as horizontal as you want to and put as much power on this as you want to to build test cases and that's probably pointless to keep showing because no one can read any of it so designing test harnesses this is the part that you probably will care about and so let's see a few different ways we might want to design test harnesses for something like this the first most common case is does it crash this is what most fuzzers will do and why you're probably using a fuzzer in the first place is I want to know when this crashes and identify whether this is something that's exploitable so basically the way this works is you will start the process you will generate a string of some sort and you will feed that into the application you have to listen for the SIGSEG V or SIGABORT signals and basically just watch for it to crash when it does you get core files and you investigate the crash get your right what where see if it's something that's meaningful to you so this is the most basic case and something that you will get with any other fuzzer it's pretty straightforward, pretty simple there's tons of examples out there for it you basically just register a SIGACTION handler on the UNIX side and wait for it to do one of those things now we get into the more interesting cases and these are the cases that really gave birth to this project so identifying whether a re-implementation of a grammar is overfit for a reference implementation to do this you would generate a test case using your own re-implementation grammar and you find an oracle within the reference implementation that specifies whether you're getting something that is either accepted and is valid whether there's a syntactic error or there's some kind of runtime error that's happening not because of a failure in parsing and understanding the sentence but a failure because of something happening in the runtime itself you feed the test case in and then you categorize what comes out so if it succeeds everything is good you have a sentence that's valid your parser recognizes it and the reference recognizes it if there is a failure if the failure is a syntax error in the reference implementation your parser is overfit you are recognizing things that the reference does not and that's an indication that you probably need to redefine your grammar and change it to be more like that it should kind of give you an idea of where those problems are so you might kind of guide you toward what specifically is causing that and you can manually go and change it to identify whether or not it's meaningful or modify the sentence that it spits out so that you can identify maybe why you're wrong but there's not really good tooling around doing that yet other than trying to get good error messages from the reference and parser error messages are usually pretty terrible they're kind of going to be on your own on that one if the failure is a runtime error it may or may not be a sign of being overfit or underfit this is especially common because of the opaque tokens that I spoke about basically it might recognize something that like is syntactically valid but like one of the tokens in there is just like totally batshit insane and like it recognized it because it uses an opaque token in that place it also could just be an environmental thing so like in in the case of like MySQL if you've ever written queries for it you're probably very used to these types of errors the top one is a syntactic error you see the top query here the select star from A where ID and a bunch of carats 3 and this tells you specifically this is a syntax error this did not parse and it tells you like exactly where it is so this is an example of the case where your parser if it generated this is just wrong that should never happen below that however is a runtime error where select star from A where fake column equals 3 this is a runtime error because one of the tables referenced here doesn't exist and so this is something that the parser passed and it generated something that's likely a valid sentence in that language but because of the environmental issues with what it is doing with that sentence it wasn't able to do that and so this is probably a false positive in that case and so having the oracle that can tell you about these things allows you to really cut down on false positives and in this case the next is underfit this is the other big problem that you were likely to worry about so to do this you generate a test case from the reference and this is something that you can only do if you have the reference implementation grammar so in the case of like mySQL or postgres sqlite you have that grammar definition and this is something that's open source for something like db2 oracle msql this isn't a viable option and you can't do that because you don't have the source code for that grammar but you would parse it with the re-implementation once it generated a test case from the reference and see if your parser can read it and then you categorize it if it fails then the re-implementation is underfit you have generated a sentence that is valid in the reference implementation that is not valid in yours if it succeeds then the re-implementation is likely correct for that sentence and that section of the grammar that you have used so what's ready today the code is up here it will be made un-private after this crates are available for people who want to use Rust and bindings will be released over the remainder of the year for as many languages as people are interested in and I have time to build or other people have time to build MIT licensed you can submit pull requests and I will happily accept them right now you get an antler forefront it mostly is functional there's a few little things that aren't supported yet but you can get the vast majority of many of the common types of parsers you will generate with antler what's next there's a lot the biggest area right now that's a huge issue is dealing with cycle detection and forest progression so with recursive grammars which are pretty common it does a really bad job with ensuring that it doesn't get stuck in a loop because it's essentially building a graph and like walking that so getting cycle detection in there to move forward and not get stuck is at the top of the list basically it will just run and then get a stack overflow and basically crash but for a lot of more simpler grammars like you're not going to have a problem here and this is immediately useful now like I said exposing the CAPI and language bindings it's built in Rust but this will be available I think early plans for languages right now are going to be to Java and probably go probably Python and Ruby as well in the near future the negation logic exists but it's not great the problem with negation is it's not always clear exactly what the opposite of something like one of the combinators is and so pretty much everything has negation logic right now but it could be better you don't get quite as much entropy as I'd like and so if you're using a lot of negation you might just not get as much variation context sensitive support and being able to do introspective things on the data that's being generated so if you need to generate checksums that are valid things like that having support to do that as I said it's all byte oriented right now I might be interested in building out bit level support there's a lot of wire protocols that have some of that and it would be useful for that additional front ends this is the big one also bringing support for more platforms is a real goal to make this significantly more useful and then grammar coverage information how much of this grammar have I explored and how much time have I spent in a specific area and then being able to reuse a tree and change individual tokens so I can just change the areas that are going to be dynamic in that and really really hammer sections of code if I care about that again this is the link if you're interested in the code and questions sure so how do you implement an article basically it comes down to the integration that you build in your test harness so in the case of MySQL we're literally using the MySQL library that is like the client and the error messages coming out of that will provide us the oracle and we just do comparison for what is being provided there that's something you're going to have to find if you have source code it's possible if there's something you can do to inspect the binary or just get meaningful output of it it usually comes down to just finding something in there and if you can't then you kind of just fucked yeah I don't yet the plan is to start doing some of that research and putting that together this is something I've been working on for maybe the last eight months or so and it's at the point now where it's usable in a meaningful way for us part of the problem for us was building test harnesses that gave us meaningful feedback but I do plan to start doing a valuation of that and I want to have something published probably next year to actually find out whether or not this actually is useful and provides more value than something like that anyone else cool thank you


This is the way to do it. This is the way to do it. This is the way to do it. The participation and the impact of youths in economic development in Kenya is at another level. In today's program, we have Philip Pandey, a man wearing many hats. Let's hear more. Philip, welcome to the program. How did you step into this leadership space? Many thanks for having me, Granice. It's always a pleasure speaking about the youth and especially what we do in adding our voices to this social economic development in this country, political participation and overall innovation ecosystem. My name is Philip Pandey, born and bred in this very good county of Kisumu, the most strategic city in the East African community. Where it all started, I think leadership for me is in it, in born. I started doing this at an early age in preschool and in my entire primary school, I was in leadership roles. It took a better shape in high school and more meaning in the university where before being elected into the university council as a member of the student council, initially I held a number of roles in the Catholic association, being a class representative, being a secretary of many other clubs and the project manager of the most popular club in the university there, which was the environmental club. So, that's the beginning, that's the rooting and of course, a bit of nitrance and mentorship here and there. I'm interested to understand what motivated you to pursue leadership as a career partner. I think I discovered at an early age that leadership was part and parcel of my life and so it was rather important for me to embrace than denying the fact that I needed to stand in and give more of my time and skills to my colleagues, my peers, my community. And so after my first degree in the university which was economics and sociology, also touching on lives of the people, socioeconomic issues, analyzing the context in which we lived, I decided to undertake a master's in governance and leadership because still going on pretty much at the latter stages. But yes, that I think reinforces what I do and what I practice on a daily basis. You talk more about the people, what critical area did you focus on or are you focusing on about the people you're talking about? Firstly, when I talk about the people, I talk about the society as it is and humanity as it were. But of course, with a great emphasis and focus on the young people and the forgotten bottom millions for that matter, people living with disability, women in under-resourced and under-privileged backgrounds, people living in hardships and asshole counties, for example. And those who have for many years not been included in the mainstream governance in our workings as a country, and of course in participation in economic and social activities in our country. So my people in this aspect really revolves around young people, revolves around people with disability, revolves around women and revolves around the aspect of inclusion. We're talking about gender equality, social inclusion in this time and era in the day. And how has the community received your agenda to bring social change in these different aspects? Over time, we've realized that people are conscious of the environment. People are aware that there is some sort of exclusion. People are aware that resources for young people are not as it should be. You look at our budgeting framework, you look at the amount of resources that are located. For example, in gender responsive budgets, if you look at with a critical eye, county budgets and national government budgets on spending on young people or people living with disability for that matter, you don't see it with the seriousness it should be taking. And so every day we get people embracing us, we get communities embracing our work. We've been to at least 37 of the 47 counties in this country. Oh, that's impressive. Yeah. Thank you. Trying to mentor young people, trying to build capacities of grassroots women. And we say, for example, that you don't need an education to be included. You need to be a Kenyan. You need to be a citizen of this country to share in the development of this country, to innovate, to be creative, to be hard and to be considered in terms of development and inclusion in overall governance framework. And let me take you back a bit. You've talked about Chuka University and you awarded a rare European Union champion title when serving as the youngest Chuka University Senate member. How did all this become possible? So at that time, this way back in 2014, when I was an elected student leader in the academic stockings, so direct academic affairs. And my role was to take care of the affairs that touched academic, touched discipline, touched extension and stakeholder management in that regard. And so the EU office in Nairobi was working with a number of universities. I think it could be six to eight in Kenya to mainstream certain opportunities that were there in terms of scholarships, in terms of furthering the education, post the first degree and linkages with the EU space and community as it is. So in organizing the event at Chuka University and the overall collaboration with the eight universities, I think it's in my university that they got the highest turnover and attendance of about 6,000 students participating in one day. And other than that, there were other aspects I think that they considered that led to that award. So dedication to service and just ensuring that ultimate numbers and maximum beneficiaries who are part of this program then led to that consideration, I think. Speaking of dedication to service, you have an experience in mainstreaming youth inclusion through entrepreneurship, innovation, mentoring, policy and advocacy. Why did you decide to take this part, these specific factors in youth inclusion? Well, the main exclusion factor revolves around politics and economics. Economics in the sense that even in terms of accessing political platforms, political parties and winning or running successful election campaigns, you need economic resources. Deviant behavior among young people, for example, is motivated by the fact that they are out of work or they are unemployed. And of course, we talk about unemployment overly in our context. So we realize that we can't keep on playing the rhetoric to young people of reform, come back to these, do these, be nice in your conformity to the law and order. But we need to provide real aspects of life and ensure that everybody is able to fend for themselves, that young people are able to feed themselves, that people living with disability are able to be economically viable in terms of running enterprises and businesses. So we thought that as a substitute to the song of behavior change and such like, there was a real need to talk about the economic well-being of young people. And that's where entrepreneurship as an alternative came in because unemployment is a real thing. We are talking about Kenya being in the space where the youth bulge is really threatening our fabrics. We can't really plan our resource envelopes to take care of the interest of the wide population of young people. Averagely the median age of Kenya as a country is about 19.7. So when you have that being the majority in this country, as a country and as a continent, we may not be able to provide jobs that match the demand. So entrepreneurship is an alternative. But in mainstreaming entrepreneurship, then innovation and creativity is part and parcel of young people, of the millennials, of the generation Z that we talk about. We are more inquisitive. We are more inclined towards tech and innovation. We are more inclined towards STEM and manipulating alternatives in that space to create businesses, to prototype products and to sell and launch in the market out there. So entrepreneurship and innovation provide for us a more meaning than mainstreaming employability without facilitating job creation, without facilitating startups, without facilitating people who want to employ themselves. And how much of a gap have you been able to bridge since you started this journey of youth inclusion? Very well. That becomes a tricky subject. We may not be able to quantify or basically qualify our impact. But looking at what we've been able to do, working with the initial event, the office of the president, strategic initiatives, we've been able to one go to 37 counties, like I mentioned. And in those 37 counties, we have over 100,000 young people that we've mentored and installed as grassroots change drivers. Other than that, we've opened up innovation hubs in many counties now. Specifically, if you go to Transoia today, you'll find the Convally Hub working with young people. If you go to Wajia, you'll find the Abbasuen Hub. If you go to Kisi, you'll find Tabaka Innovation Hub. So those innovation hubs work as accelerator points and safe spaces for the youth. In terms of providing direct employment, I would say that nearly half a million people have found jobs in the mainstreaming online and digital jobs through our partners that have been able to work with us. Mainstreaming innovation and ensuring that agribusiness, for example, and green jobs, ecological protection and forestry now, taking shape and employing grassroots young people in, be it a tree nursery, be it a conservation space, be it a farm, a demo farm, and actual production of food and lifting livelihoods out of poverty in the rural areas. We've worked with a number of partners, development partners, GIZ, UK Tech Hub and many others. And we've worked with other friendly organizations like the Forgotten Bottom Millions, which is an Africa-wide network for young people seeking opportunities. So we don't only provide opportunities in entrepreneurship and innovation. We also share scholarship opportunities. We share opportunities in mainstream employment. We share opportunities in collaborating with other people in development projects and anything that engages young people and ensures that we are tooling young people to productive manpower and skilled labor that is required for the 21st century jobs. At this time, we are also mainstreaming AI as part of innovation. We know, for example, that we are talking about AI taking up space in employment, but we're also saying that more jobs will be created through innovation in the AI space, artificial intelligence, and many others. So we are training people on robotic processes, automation. We are training people on business re-engineering, for example, so that you don't just go into a business or an organization like an analyst, but you can be a business engineering expert in ensuring that you add unique value to spaces that you venture into. There's so much you're talking about opportunities, scholarships. So how does that information get to get to the public through your forums? So, for example, an organization that is filling up this space is called Forgotten Bottom Millions, which picks opportunities that are available on the Internet in social media platforms and verifies and then consolidates those information and shares on WhatsApp platforms, on our websites, on Facebook pages, and on our individual timelines.  You could be in Nairobi or in Kisumu, but you're working for a U.K. or a U.S. farm remotely. And many other young people are doing that, digital jobs and digital transformation. You could, by a click of a button, find an opportunity that lands you in a German university or a Canadian university to study your masters or do a remote program in trying to advance your skills. And so that comes in handy by, you know, just having a tool and now we are developing a tool that through artificial intelligence that aggregates opportunities, especially startup opportunities. So internships, scholarships, pupillage programs, programs that are run around, graduate training programs, and aggregating them and ensuring that we see, verify, then share out with young people seeking opportunities. And could you mention some of this YouTube Facebook pages website that someone can go look into so that they can click on the opportunities? Very well. Today I'm more engaged at Corporate Career Academy as my main business now, mentoring young people and providing opportunities. So if you're looking for such, we are working with university students. We are working with out-of-school youth.  You'll find opportunities shared. Other than the Corporate Career Academy, I've mentioned the Forgotten Bottom Millions, which is a key platform, bigger than many other organizations now in sharing out verified opportunities. Go to their social media handles. You could also find opportunities in the closest hub around you. There are youth innovation hubs in nearly all counties now. But again, as a young person with a smartphone, you are not short of opportunities. There are limited opportunities online. Just search anything that you're looking for. Search for scholarships. You'll find numerous. Find search opportunities on online jobs. Look for freelancing platforms like, you know, they are many now. You look at the Google workspace. You'll find jobs. You look at FIVA. You'll find jobs. You look at Ajira digital platform. You'll find jobs and many others. So both from government side and in the development space, they are just unlimited opportunities. And of course, one thing that we're also trying to fight is young people's resentment on applying for jobs and hiring. There is a belief in this country that most of those jobs that are advertised out there are already filled. Maybe, maybe not. But when it's your time, nothing stops you. Keep applying. Keep sending out your CV on anything that comes or pops up on your screen that you think is suitable. If you qualify at least 70%, that's an opportunity that could come your way. Always keep up shooting so that when it's your day, when it's your time, sometimes luck pays than any other thing. It could be your lucky day. Your interest in grassroots youth organization, how can you describe your unique leadership towards that? Well, I think we are focused primarily on grassroots because there are quite a lot of opportunities in the capital, Nairobi. There are quite a lot of opportunities in major towns. But if you go to the villages, if you are not in the farm tilling your garden, if you're not involved in break-making, if you're not involved in all the jobs, really, then you have no opportunities. So our target has always been to go to where the action is limited and where there are less opportunities and where information asymmetry has ensured that young people are not wired or conditioned to accessing opportunities or being more aggressive. If you live in the informal settlement, for example, if you live in the Assam counties, there are no TVs there. There are some of the spaces without even internet access and people living in those spaces have hardships even buying smartphones. So perhaps the only opportunity they get is when a mentor fills their role and goes into an interactive session with them and exposes them to an opportunity. And so that's why we're also encouraging for shared spaces and safe spaces for young people. So that if I don't have a computer, if I don't have a smartphone, but I can walk into a community hub, I could walk into an innovation hub that is in my neighborhood, use a computer, collaborate with other colleagues and help in setting goals that then we keep on accounting to and being able to just ensure that everybody is taken care of. This is quite a responsibility you've taken on. What challenges have you had on this journey and how did you manoeuvre them? Well, when you fill the roles such as ours, everybody thinks that you could employ them, you could link them to an opportunity. But you tell them, OK, let's go through this process. Let me share what's available next week so you could look at. Let me share random opportunities and people do not want to follow that process. They want instant solutions like I want a job and I want you to identify what fits me so that you give it to me. So when we share opportunities or when we invite you to participate in a workshop or in a mentoring event, then show up. Because when you show up, you link up with other people. You listen to stories that you've not heard before and you realize that you need to trust the process. And where do you see yourself in the future doing this career? I'm looking forward to an opportunity to be able to ensure that young people in this country and persons living with disability are just as included as women. In the last decade, we've seen a lot of transformation in the gender space constitutionally and in terms of legal frameworks. I'm looking forward to a day when the Senate of this country, for example, can say that in the county government's operations, particularly in the county government's act, we are able to ensure that young people are part and parcel of cabinets, are part and parcel of decision making roles so that it starts from ground up. And I'm looking forward to a day when the national government through the National Assembly will be able to legislate and install legal frameworks where young people do not have to make a lot of noise to be heard, that our voices should be just as important as any other person in the space. As a leader yourself, what advice would you give a young person wanting to be a leader in the future and they're shy to get into that space? So if you want to make an impact as a young person, firstly, dedicate yourself to voluntary opportunities and paid labor and be able to be counted in terms of making a difference. The other thing is look for mentorship. You can't do it alone and you are probably navigating a space where you have less skills. You need to find a mentor that then helps you to set goals and targets and helps you to account. So you must also be thick skinned to ensure that when your peers, for example, not buying in, sometimes they say that to support the youth ecosystem, you need to use one hand to hold the critics and those who are going to be weighing you down and the other arm to do exactly what you're supposed to do. So that is part and parcel of the youth space. There is quite unlimited space for anyone that wants to venture in. This is amazing what you're doing. You have so much. And if a young person could listen and then I'm sure they'll be going so far. Dedicate yourself to opportunities, show up, find that mentor who will help you, who will lead you and who will guide you in the right path. This has been Youth in Action. See you next time. I am Nyongwese Grenis.


The president's budget proposal eliminates public media funding. Learn how you can help at lptv.org slash action. Rain, sleet or snow, the parade must go on. Haiti Clotter shows us how the world's shortest parade still draws a crowd right here in Bemidji. For 11 years, the world's shortest parade has been a Bemidji St. Patrick's Day tradition. With the weather today, it was a pretty good turnout. It's usually a lot busier, there's a lot more kids usually, but obviously with our snow coming down, it was a little slower turnout this year. The cold and snow didn't seem to bother those marching the mere 78 paces in this year's edition of the parade. Why continue to do this tradition year after year? Just for fun, just to celebrate life. The parade started at Kagan Cork and ended at Bridget's Pub. After that it was time for a dance break. I thought it was a lot of fun and glad it was short because it was cold. The parade only lasted a few minutes, but it's the world's shortest parade for a reason. In Bemidji, Haiti Clotter, Lakeland News.


I'm Eval Yaret, I come from Israel, from a company called Agile Sparks. Agile coach there working mainly with enterprises, big product development companies. And the talk I'm going to give today is going to be inspired by my experience working with these kinds of organizations, both looking at some bad things that happened during Agile implementations. Francis called it some of it unsafe. I can't agree to that name. And some good patterns that we've started to see recently implemented a different style of implementation approach. So let's start the story. Many organizations start their, or typically large organizations start their agile journey via a pilot. Whether it's a pilot that the big organization decided to do, or a group of people, Guerrilla mode decided to go agile. It worked great because in some cases, many cases, it worked great because people were focused on it. And it was a small scale experiment, whether it was driven by management or not. And then, in a lot of cases, something that might look good happens. And they see the success. The organization sees the success and says, OK, let's go for it. Let's go full scale agile across the organization. Everybody should do it. If it's so good, everybody should do it. That's what we call the mandated cross the board rollout. So if we look at it from a Kanban perspective, let's try to create a Kanban board for the change management. How many of you have seen a Kanban board before? So just to explain what we see here, because we'll look at it throughout the presentation, this is a Kanban board looking at the different groups in the organization. We're talking about enterprises here. Each group is not a team. It might be a team, but it's a certain group that can decide on its own what to deliver. It delivers a certain product or service. It might be one team or a group of teams working together. Say if you might call it an art or a group of arts. And we see here that each of those groups is now going into agile. The organization is telling all of the groups, go and do