Well, good morning everyone. I I know we're a little bit late starting. I know there's been some Transportation issues this morning, too. So we want to give a little bit of grace but we're grateful that you're here to speak with us and full disclosure I have never been to SoCAP before and Most of our panel has not been to SoCAP before either which was kind of the intention of the panel So my name is Jen Collins. I run our Opportunity Zone team at the Beck Center for Social Impact and Innovation at Georgetown we're a nonprofit that sits on Georgetown's campus and we tackle social innovation and Impact in a myriad of ways One of the ways that we do that is by focusing on what we call fair finance Which is taking a look at the systems and structures undergirding capitalism that are perpetuating inequity Opportunity zones fits within that portfolio and we've taken Pretty big stance on driving impact in opportunity zones And so I lead that work and I am grateful to be joined on the panel I'm gonna sit down because I don't want to be up at the podium but I'm grateful to be joined on the on the panel by a Bunch of folks who also have not been to SoCAP before most of them a few of them have but I want to let them introduce themselves and And then we'll talk about kind of dive into opportunity zones and what we're seeing and the unlikely partnerships that we're seeing and Across the field, so I'll let John go ahead and start. Yeah. Hey everybody. I have been to SoCAP before I know you're the one alone the lone experienced veteran So I'm John Latteria. I'm the co-founder of the economic innovation group for a bipartisan research and policy organization based in DC We helped to develop the concept behind opportunity zones and worked very closely with Legislators in the House and Senate to get it passed and since it's been passed. We've been working with really everybody We've been working on regulatory implementation. We've been working with mayors and governors on state and local implementation We've been working with investors help trying to help them understand what to do and and how to put this to use Community stakeholders. So really the whole gamut. I'm very excited to be here Jen. Thanks for inviting me Yes, so I'll go next names at least Liberto I work at an investment firm called Brown advisory actually based in Baltimore, Maryland and at the firm I lead our private market impact investments helping our clients find Investments that both have compelling risk reward profiles But also measurable social impact outcomes and as everyone in the crowd and at the conference I think knows the challenges that impact to mean something different to every client But one of the opportunities is finding those themes that resonate across a large percentage of our client fees And one of those themes that consistently comes up is this desire to invest in the communities that our clients came from so Once the opportunities on legislation came out that I think you're about to hear a little bit more about from John Pretty shortly. We really took a notice to the social impact intent behind that legislation But what we saw when managers were first coming to market And I think as the majority of individuals saw is that it was large groups raising large sums of capital to invest Nationally predominantly in real estate that wasn't serving a local community need So at the beginning of this year our firm got involved and decided it was time to create a new solution a new partnership We looked to partner with someone that had experience in community development as well as private market investing partnered with an individual that many may know named Ross Baird who is the founder and president of an impact oriented venture capital fund called village capital and we launched a joint venture called blueprint local So blueprint local is the management company of a family of qualified opportunities and funds that were really raised with a strategy Predicated on three what we think are differentiated factors one raising small place-based funds to invest in communities and led by local teams To incorporating an impact evaluation into every investment decision that we make and three Investing in a series of both real estate and operating company investments and community nodes to drive transformational change So we've raised two funds to date blueprint Texas focused on the San Antonio Austin corridor and blueprint Baltimore focused on Baltimore Maryland my hometown so it's early days, but excited to share a little bit about what we've been doing with all of you today Good morning. My name is Mike Everett. I'm president of new Ovo real estate. We're Denver based private real estate investment firm part of a larger family office in in in Denver and Arkansas and We're primarily focused on the historic adaptive reuse space and primarily not in the hotel segment We've done a little bit of retail a little bit of multifamily, but our preference is to Go buy historic buildings that no one else wants to touch and turn them into hotels and and do it in a do it in such a way and in a thoughtful intentional conscious way of Creating impact through our hiring practices through our sustainability work and and you know really looking at a 20 to 30 year plus horizon to To think about how not only can we be profitable, but how can we be most impactful in the communities that were that we're investing into? We're today we're focused. We have a we have a hotel in Baltimore as well. We had a consistent theme here Hotel in Baltimore We're doing a series of projects in Merced, California, which are Which are all Opportunity Zone? projects We're redeveloping buildings in downtown St. Louis and Cincinnati right now to convert into hotel and then we're pursuing additional Closing on additional Opportunity Zone projects in Indianapolis and Cleveland by by the end of the year our focus really has been in the last I'll say year or so as we've continued to understand and Learn more about Opportunity Zones and as the as a legislation has become clear and clear over time Our focus our pipeline focus is a hundred percent Opportunity Zone. We're Tending to start two to three different two to three unique projects a year and You know with the kind of the wide swath of opportunities that are available within the secondary markets that we prefer You know we don't traffic in New York and Washington and Seattle and Denver You know we tend to go to secondary markets We have a huge huge inventory of buildings that are that are available that could be converted to Hotel use in these in these markets that that do have great Opportunity Zone overlays to them So that's that's kind of what we're we're focused on and really trying to build over the course of the next you know five to ten years build a portfolio of of You know 20 really profitable sustainable businesses that that will will do great things in their communities Good morning, my name is David Gross. I'm an LA based real estate investor and developer And adjacent to that I have a couple of mission oriented projects that I'm focused on I Co-founded an inner-city incubator called Vector 90 In the Krimshaw district of Los Angeles We are working on replicating and scaling that to a number of other core inner cities throughout the US And I recently co-founded an initiative called our opportunity, which is a very grassroots Kind of locally oriented high-touch high-impact Movement to connect people from inner cities back to the communities where they're from by investment the OZ incentive is something that was Kind of a lightning bolt for myself and my partners who were already focused on their city investment And we've shifted our focus almost exclusively To OZs at this point and we have a number of projects in the pipeline that we're excited about I think most notably In the Krimshaw district We have Nipsey hustle tower, which we're working on right now And I think the plans for that will unveil the top of 2020 Some projects in Baltimore, Chicago, Atlanta So a number of things we're really excited about Great So I don't want to assume that the room knows the ins and outs of opportunities on legislation So John, can you kind of help level set us and give a summary of the benefit and the intention of the legislation? Sure. I just want to start by honoring the Washington Nationals. I was going to get to that Who won their first World Series game last night very exciting go-nats We'll be doing baby shark later In honor of that So how many people just by show of hands do feel like they're familiar with opportunity zones the details of the incentive? Great So in recapping what all of you should know It's a capital gains based incentive. So it's not an upfront tax credit It's not limited in that sense by the traditional limitations around community development incentives, which? Allocates a relatively small pool of capital and spreads it over a very wide set of geographies So the national impact is pretty thin in those policies The local impact can be Significant from any one project, but you just can't allocate very many of them in any given year And so one of the core design Priorities behind opportunity zones was getting past that low ceiling of the amount of resources you can unlock Especially given the fact that in a capital gains context the recovery the economic expansion that is now in its record 10th year has created a huge accumulation of new capital gains wealth in our country That could be source capital for the reinvestment and redevelopment of communities The supporting of underserved entrepreneurs the scaling of local businesses that have potential But don't typically have access to capital So the idea was to connect those dots to say we have this big pool of capital That's been untapped for community development and economic development. We have a design Limitation of previous programs that create a very low ceiling And we have the inability in previous programs to do a lot of different things through the same tool And so with opportunity zones you can have in the same community and we're seeing this in places like Erie You can have early-stage companies getting Investment in an innovation district and you can have a downtown redevelopment putting a food hall and and Hospitality and all kinds of things residential real estate you can have industrial Investment you can have manufacturing you can have broadband infrastructure all of that being financed through the same incentive And it's worth noting that's just never happened before that's this is a first in federal policy to have that Diversity of things across a wide diversity of communities being supported through the same policy So that was very much the intent behind this was to say if you're an investor you have capital gains and you're willing to reinvest Into these designated areas first you get the deferral So the time value of not paying your tax upfront but pushing that back Second you get a step-up in basis tied to longevity. So five or ten percent Depending on the the time marks you can get up to 15 percent of your original tax liability forgiven and those are both meaningful and they're they're interesting to investors, but the Big benefit comes at the very end of the timing window, and that's the 10-year plus benefit any new capital gains you acquire Or accumulate through your investment ends up being tax-free So that's a very significant potential benefit, but it's not a guaranteed benefit You have to have additional capital gains You have to be invested for 10 years or more you have to be putting your capital to risk in many cases in communities Where these are investors who would who've never taken a first look let alone a second look at a lot of the places That have been designated so it's pushing the benefit back in return for patient capital It's it's attempting to get that connectivity between where there is a large accumulation of capital and there's a large Concentration of need in our economy and it's trying to get to a diversity of different types of needs And again where a lot of other policies I think have fallen short of their potential is they've created a very narrow use case You can use this for housing you can use this for a certain type of housing Or you can use this for a very specific type of redevelopment activity And that's it and if your community doesn't fit that specific use case It's irrelevant to you and so what I'm excited about with opportunity zones what I'm hopeful about in the next year is as the market matures And as we have regulatory clarity We're gonna have more creative use cases and a wider diversity of use cases even though what we see now Which I think is already pretty remarkable in the early days But that's really the potential promise of this is scale and diversity of activity and the concentration of a diverse set of Activities in a given location that helps that entire community rise because again the needs and opportunities on communities they come in bunches It's not just housing. It's not just a lack of new businesses. It's not just a lack of kind of basic amenities grocery stores Medical clinics. It's all of those things combined in most of these communities So you have to have a tool that really supports potentially all of those different types of needs Yeah, we and we We appreciate I appreciate the the nod to holistic community development and the needs there therefore the original legislation before it kind of crossed the finish line did have some impact data transparency language in it that was removed as the as the Legislation across the finish line Which is why the Beck Center really decided to to get their voice and be able to use our influence to get Impact in the conversation and across the field in in the best ways we can What we did was we launched what's called the Opportunity Zone impact reporting framework Which is housed at oz framework org. So if anyone's interested I would encourage you to take a look I personally have a real estate development and finance background. And so we really Wanted to this to be a helpful tool to bring People who have never been into the world of impact into the world of impact and so a lot of my time effort and energy is really translating by in between You know folks like the want the the practitioners on the stage into the world of impact And we're doing that in the oz space through this Opportunity Zone impact reporting framework So I'm very grateful to have the three practitioners on this stage be framework adopters And have come up with really amazing use cases for exactly what John described in terms of holistic community development With real community engagement responsible exits considered You know a lens on on equity and and transparent behavior So I'm excited to get into some storytelling on on what they're doing in the field to be Impact Minded in this space but at least want to turn over to you because you have Kind of a whole swath of constituents and I'm curious as to You know the the dovetail between kind of financial investment Lenses and impact lenses With your clients and kind of what is the dialogue look like around that and yeah, yeah No, certainly. I think that Exactly the theme of this entire panel. We haven't had we've been very similar I think other organizations and really only over the past two years We started having those Conversations around impact in your private piece of your portfolio So I think it was a new conversation and really where we came out was if you look at the way the opportunities and legislation is written We kind of decided that even though leading with impact is the right thing to do I think the better way to describe it to clients when we do believe is it's also the right way to drive Financial returns within the confines of that legislation and it's to flush that out a little bit We think of it in kind of three ways The first is that leading with impact is really the only way to mitigate against reputational and regulatory risks that I think are very much Concentrated within this investing on the regulatory side We know that the legislation is still being interpreted today and we kind of came out as a firm of saying if we're gonna Do something and be early movers before there's even final regulatory guidance You need to be aligned with the impact in order to be best positioned for any change The second on the reputational piece I think you're gonna see a lot of groups that have been quick to put capital to work in communities They don't really know or care about you really hit long-term on trust and credibility So just full stop starting out We thought it'd be really important that from a financial perspective You need to be impact oriented within a qualified opportunities and fun to other points I mean the second is that we actually are seeing also that this impact orientation is leading to Deal level value add two quick examples that that hopefully are interesting and relevant the first would be proprietary deal flow I think you're seeing that Entrepreneurs as well as property owners that are located and intentionally in underserved communities Actually care about the integrity of the investors that are on their cap table A second would be additional city and state level financial incentives for having impact oriented projects We can get into this a little bit later But really that cross-section of collaboration with government is resulting in additional tax incentives Additional financial support through grants as well as more creative Which I think is the most interesting point more creative support through things like fast-track permitting and enabling impact-oriented groups to be the ones to Develop government owned land the third just very quick point would be our focus on being really local and intentional about community orientation I think that underserved areas are inherently challenging to be investing in and you really need the right team and a high-touch approach to create that Transformational change that we want to see so our thesis was always do this for impact Not only just because it's the right thing to do but because you can actually see how to drive financial returns in the long run Mike do you have anything to add to that conversation? Yeah, so we We're focusing on our projects and we're said California for a second Which is if you're not familiar. It's some kind of really definition of central, California. It's we say it's two hours from everything Sacramento San Jose. It's about an hour north of Fresno and you know Merced has historically been a Agriculturally driven town and big egg in a lot of ways is really what's been behind the economy in the last You know in last several decades really Recently as in about 15 years ago that University California system opened their last campus at UC Merced and so that that's now at about 9,000 students And he's really injected a new level of activity in life in the community But it's it's really it's not been enough and I think we've we've seen the need to go and help Recreate the downtown landscape in Merced and hope that that's gonna continue to leverage the what the university is gonna do In in future decades and you know, we hope to we have to work with them extensively as we continue to invest in in the community I think You know to to Elise's point around, you know, trying to try to measure our impact and trying to be You know reach a level of transparency and then reach a level of partnership within the communities that were that we're going into I think You know the biggest challenge we have had and I think what we've learned for our our future projects is is you know really being communicative up front being very open with our partners and that can mean a lot of things that could mean your your Neighbors that could mean the city that could mean, you know the operators that you bring into your Buildings in the in our case. We don't operate our hotels. We bring in a third-party operator But you know really bringing that entire set of constituents under the tent and communicating to them really clearly We're doing this and we're doing this to make money We're also doing this to really Thoughtfully impact the community over the next 20 to 30 years and and we found once we you know Once we do that and the more of more of that kind of open transparency. We have We create this huge momentum behind us and I think in candidly in in Merced You know that that transparency for us has been built over time. We probably weren't as good at it Three or four years ago as we are now but as we you know as we as we Really tell the story of you know our intentions are our desire and really need Based on our opportunity zone is as Dave mentioned we're gonna be in here for we're gonna be in here for 10 15 20 years We've got a big You know, we've got a you know huge incentive to invest for the for the future. So you know being open really kind of getting into the getting into the The nuts and bolts of what this community needs and why it needs it and how we can help supplement the The local economy and how we invest and you know how we really how we really Plan for the future five ten fifteen years out is is really important Thanks Mike and and to that kind of theme of collaborative behavior needed especially with the high touch points that Elise so eloquently described David, I am curious as to You're doing a lot of the really high touch very difficult Very time-intensive work Within these within these neighborhoods So curious as to if you're seeing kind of if you have any best practices or stories to share about Things that have gone well and initiatives that have gone well or not Sure, I'll approach it from an impressionistic level first When we first opened Our first location in the Crenshaw district for this co-working concept Vector 90 we went into it thinking we're going to scale fairly quickly We thought we'd have you know four to six locations opened in the first 18 to 24 months and almost immediately it was clear that wasn't going to happen because you know a lot of the Engagement a lot of the impact to make impact it had to be very high touch it had to be kind of person-to-person business-to-business Engagement and the things that we were doing they just didn't scale because it was idiosyncratic issues that we were dealing with With our membership base and for the first year I was like this isn't going to scale I don't know how this is going to how this is going to translate to rolling it out into a national model and till you know we You know by going through the process we found that taking the time and really digging in the channel and focusing on individuals and Neighborhood impact it resonated and it like what we were doing and why we were doing it It did spread and then we started We started to receive this national support and buy-in to this Crenshaw based model So in a very indirect way, you know focusing at the hyper-local level Is giving us the momentum to now think to now plan on expanding? But yeah, I'll be honest it wasn't it wasn't planned in the beginning We thought we were going to open a door and we were going to find something that worked in the first six to nine months and then Wash rinse and repeat and that was not the way but but going through that I do believe that you have to As as Elise said you have to find a way to really drive Real impact and that's the only way you'll be embraced and people will Will accept you into a community But I think the way forward is going to be some barbell model where you have initiatives that are focused on the grass root very direct Impact partnering with Institutional caliber caliber institutions or organizations that have the manpower the processes and the systems to identify those things that have worked and then help scale them So You know I'll give you a couple of a couple of anecdotes We were launching so we've partnered with some national Organizations so Guggenheim Is now a partner on the financial literacy side? So we were in JP Morgan. So we were kicking off You know an open online Foundations of investing course and we decided to we were like how do we get people excited about being engaged and being involved in this so We were like, you know, we're gonna do a giveaway where we To incentivize people to open their first investment account We're going to see their first trades with the hundred dollars, right? So we were we had ten thousand dollars earmarked for this We announced the the campaign and I think we hit that hundred new investor target in less than 24 hours and You know, we receive we were inundated with hundreds of emails initially like hey I've always wanted to invest but always thought it was intimidating. I was afraid. I want to be a part of this And so then it was kind of a dual-sided thing We had more and more people saying I want to contribute to what you're doing I want to fund someone else's account and obviously we had you know tons more people saying I want to open an account I want to be involved and so now it's become this kind of virtuous cycle where we you've it was our first time actually raising Donations for a philanthropic cause and we're at a hundred thousand dollars now. That's been raised for this and it's going So that speaks to a broader point that everyone that we've engaged with or tried to help or work with in some way I've been I've been taken by how much they in turn wanted to help or give to someone else That's great, and and you're giving them a platform and a pathway to do it to do it Which I think is is really important and helpful To the field John, I'm curious as to what you're seeing as behavior that is Quote-unquote working some interesting or innovative models that you've seen With pretend, you know with the potential to scale It there's so much that's happening that that has Real promise and that I think breaks the mold for the way again these policies have traditionally had impact So I think about a couple of things what one of the common themes you find in opportunities own communities is Places that should have assets but but instead they're being they're behaving like liabilities And another way to say that is you have government owned vacant land in Primaries of a city that no one even really knows who owns it or who's accountable for it, but it's just not doing anything It's not creating a tax base. It's not Generating any kind of draw for people to come into a city to spend their money It's not creating any kind of housing or commercial activity It's just sitting there and when you go throughout opportunities own communities over and over again You find that kind of that that dynamic where there's been historically just very little accountability Nobody knows I've had governors and mayors tell me we didn't even know what the city owned or what the state owned What was on our balance sheet? So how are you supposed to have a thriving community if some of the core assets core property core buildings? In places that could be prime corridors of commerce and housing and and activity if the people who own it if the sector That owns it doesn't even know it owns it or it has no accountability to do something with it And that's a common theme another common theme is you have Communities that were built for a much larger population that have suffered depopulation have had industrial Displacement and now you have the infrastructure and the bones of a community that was once thriving But nobody with any ideas or capital to put those things to new use So one thing we hear about is a lot of old factories and mills being put to new use Actually, this kind of gets to your to your example of taking property and adaptive reuse That's happening in a lot of places. It's happening in kind of emerging industries You have vertical farming you have salmon farming and old paper mills in the northeast up to taking these abandoned mills That were once the centerpiece of the town's economy and are now putting them to use for an industry That's really an emerging industry with huge demand of the US that can help to revive these communities in the process You have connectivity funds that are set up to close a digital divide and go into places where major carriers Don't lay infrastructure and therefore the local residents don't have access to high-speed internet local businesses Can't create a thriving online business You have these kind of basic amenities that are missing and in some cases they scale very well nationally Because if you can do it in one of these places you can do it in dozens or hundreds of places along the same model So we're seeing a lot of that type of creativity in each of those cases There's really strong engagement with the specific needs of the community and this is where I really want to underscore what Elise said this is smart investing to go in and really adapt your strategy to the to the Potentiality and the local assets and needs of a community and to get buy-in early really does make everything else easier It de-risks your investment And it it helps you fine tune a model that then can scale in other places because once you have that track record That's transferable to the next community in the next community in the next community So we're seeing a lot of that type of malleability of the incentive being put to use but I do want underscores It's very early So some of the most creative stuff that we know about is not public. There's been no press release It's not going to be public for two or three or six months But those are the areas because we're getting into that first real wave of the market everything to date has been essentially pre-market activity The regs are not done investors are still learning about this every day people who could be either major contributors to a fund or fund managers or Local eligible businesses or public sector stakeholders today still learning about this almost two years into it being law So I think 2020 is the year we're gonna really find out what the scale and potential of the market looks like And we're gonna see some of those creative models emerge that then create a lot of copycat in a positive way a lot of copycat behavior in the second And third tier Markets so that's what I'm excited about and I think a lot of it really does get away from what has traditionally been We're to date been a very heavy real estate and traditional real estate focus You're gonna start to see that boundary get pushed way further out I think as as the momentum and the maturity continues Yeah, and at least I would like to hand it over to you on that on the I think You and and and your initiatives and blueprint are are really looking at a collaborative approach between both bricks and mortar And the businesses that fill it as your approach to opportunity zone So can you talk a little bit about that and what that looks like in real life as you as you pursue the strategy? Yeah, certainly I think it switches your focus of traditional investing from transactional deal flow of this deal vis-a-vis this one to How do you look at a place and say what's gonna make this area thrive almost as if you were buying an Index fund on pick your small area of a city It's just a little bit of a different way of thinking about investing and when you start thinking about the growth of that area as a whole Over time it's less about again Just my what is my one investment gonna look like in a model over ten years It's how do I bring in different stakeholders from a bunch of different groups? And we often think of kind of four groups that are just core and necessary and need to be involved in a city You have your investors your developers that are providing capital and deal expertise You have your community stakeholders that really need to be involved in idea generation particularly for Greenfield real estate development I think it's really core of going to a community first before you even create a plan You also have government talked a little bit about how they've been involved and then you also have this nonprofit piece as well Which we found to be really interesting This is a tax incentive for tax paying investors But seeing the abundance of non-profit endowment foundations that are getting really excited about potentially putting capital to work as a mission related Investment because they really care about an area and see this as a way to scale their investment in both the real estate and operating company side I think is something exciting and it's bringing together people around a table that Traditionally weren't getting around the table and having that conversation. I think if it's done in the right way That's really one of the most interesting ways that that collaboration can push a place forward on a number of different levels Not just one project. That's making money You have to loop in community banks think about small business lending This is an equity incentive not a debt incentive, but debt plays a really important piece, too It just makes you think more holistically about the entire community And I think that'll be important on over the long term absolutely I'm interested in the in the non-tax paying entities the the world of philanthropy We've seen some national philanthropies really step up and and help toward drive impact in this space We've seen Kresge give for sauce guarantees to To community capital management and archteris. We've seen Rockefeller support local, you know TA at the local level We're doing some work with the Lumina Foundation focused on the role of higher education Institutions and how they can really act as anchor institutions within opportunity zones, which I think can happen in a whole myriad of ways Oftentimes they can be conveners. They can be a trusted community intermediary There's actually an HBCU historical black colleges and universities OZ fund that has bet that has been started and they're doing some really interesting Work within within opportunity zones and driving impact So it's it is an interesting thing and a really helpful thing to for these non-tax paying entities to really help be Help catalyze impact and in these neighborhoods John are you seeing it in any other ways we have philanthropy we have higher ed Are you seeing the non-tax paying entities any other any any other groups that are really catalyzing? OZ investments. I would just want to go back to kind of the premise there I because I agree that they can be so critical This is one area though for me that's been a disappointment so far because I think the the philanthropic sector in general has been at scale I think missing in action There's some you have two years of a market being shaped That's the prime window for philanthropy to have stepped in and said we're going to really help to that 22 million you mentioned Kresge is fantastic. Why is that not 220 million or two billion of lost guarantee is not invested capital either It's just a backstop So and if they do their jobs, right? They're picking funds that are never going to need the loss guarantee But it helps to de-risk for the private investors to really get them off the sidelines And part of the reason this is so important gen is that the the narrative that opportunity zones has unleashed a torrent of Capital is wrong. It is hard to raise money right now It is very very hard if you there's just an article this week in the Wall Street Journal About how most fund managers are out there probably at 10 or 15 or 20 percent of their their raise goal and that's pretty pervasive There are some funds that closed very quickly had phenomenal success. Those are impact funds in many cases So it's not universally true But as a marketplace this is still new and it's still high risk and perceived as high risk by most investors You could put their capital to use the incentive is very powerful But on its own it's not going to change behavior without all the other stakeholders including those non-tax paying stakeholders that have a That have an obvious role to play in shaping the marketplace unless they step up So what we're seeing with the loss guarantee for example is that those funds are going to be phenomenally successful I think as a result of having that backstop and the credibility that comes with that having been blessed by By kresge and having that as a tool to be able to to shake loose more capital from investors But I'm just sad that two years in we don't have more of that type of very basic. It's not this is not rocket science It's just very basic engagement from philanthropy on an issue by the way that they're supposed to care about on an issue That they're they're literally capitalized that they have a huge tax incentive tax benefit to To do this very type of work and here we have this transformational window We're two years in we don't have the kind of scale that we should have So yes, we're seeing some of it. I don't think we're seeing nearly enough of it We're also seeing another example of kind of anchor institutions and in some cases non non taxpayers that are helping to lead the way is hospital systems In in these communities. It's just fantastic Using using their foothold in a community to go into affordable housing Developments for their local residents and doing doing things that help to anchor and catalyze the whole market of activity We're seeing this in Cleveland. We're seeing this in several different areas So I'm I'm hugely optimistic that once you get to that type of stakeholder ship from the from the those anchors Many of whom again are non-tax paying and are directly incented by this but see the overlay with their mission That's where you really start to get to the creative potential and the scale potential that today has been pretty elusive No, I think that that's important candor. So we we appreciate it And I agree on the on the it's Ed's and meds, right? So higher ed educational systems can be huge catalysts for for impact behavior in this space The medical systems can as well We're seeing it in Philly actually to where we have an investor. We put together an opportunity zone investor council We have an investor who is looking at his whole his whole he's a celebrity chef And so his whole mission is really access to healthy foods who's partnered up with an entrepreneur accelerator in Philadelphia and they're really looking at food as As wellness and so they've teamed up with the local children's hospital to scale out some of their ideas and get some cross-pollinated programming going between and it really just speaks to that holistic neighborhood development They're doing a mix you steal in a zone in Philly Where they're going to be helping accelerate Food and beverage entrepreneurs and catering and things like that Predominantly people of color in West Philly. So we're starting to see that holistic neighborhood development and the anchor institutions playing a really vital Part in scaling out some of the ideas within the local communities because oftentimes they're the largest Workforce provider largest employer And can really act as a community intermediary I'm curious on the prepped from the practitioners on the panel We've talked to mostly about optimism, but appreciate John's lead on the candor for you know, where he wants to see philanthropy Step up. What are your headwinds that that you're experiencing and where would you like to see some? You know additional support or partnership along the way Mike you're you want to take that? well first I think to underline what John said it's it's it is early and And I think you know we live in this world of Wanting needing instant feedback and results and gratification what not and we you know we read you know We read these articles about well opportunity zones haven't transformed the entire country therefore must have been a failure well, you know, this was rolled out two years ago and By nature of these you know opportunity zones are generally meant to create Development a new activity you don't go buy an existing building and sit on it That's an opportunities on project. This is something that's transformational And so we need to be patient to John's point and think you know we keep in mind This is the real benefit comes from an economics standpoint comes 10 years down the road So this isn't a this isn't a short-term, you know quick flip kind of a kind of a situation I think gent your to your question. I think what we've seen work well and not work well Across different communities markets. We're looking at To put it to put it simply the cities and the local leadership who understand the power of this legislation The ones that really get it have put resources and not not talking, you know tens and Millions of dollars and lots of lots of you know put a big staff behind this But the communities that have put an opportunity zone Zarr and the mayor's office or in the economic development department of a particular city They're the ones who are really out Figure out how to attract the capital how to attract the non-tax paying entities who could who could Supplement what's happening from an oz investment standpoint We see they're the ones that winning. They're the ones that are winning when we go candidly with with with Merced, California Which is against smaller towns so they're not a ton of resources to go put behind marketing themselves as an oz destination, but smaller town whether They just haven't grasped Haven't had the time energy resources to go figure out How do we fit in this world of oz and what who can we attract and what what can we do in this in our city? That's gonna make you know, it's gonna make a stand out over time and That to us has been kind of the have and have nots and and you know as we see hopefully over time as we see Cities get you know more organized around this and really bring like at least I'd bring the stakeholders together It needs it requires a city requires the local private developers it requires other you know community community involvement that's really gonna it's gonna Help elevate the the profile of that of that particular city That's where it's gonna be successful in our opinion and where we've seen the disconnect or the maybe the lack of momentum has been Has been due to just a lack of understanding education and you know We're really having a clear vision as to how to maximize the benefit of this Yeah, we've been really inspired by I think Alex is here Alex Fox part see here. Yep, so Alex loves the state of Alabama and has done has gotten his hustle on and has really ignited a lot of collaborative behavior and Just really I'm telling the story of the great state of Alabama and there's Going to be kind of overmarket Over opportunity zone market investment in that state for sure given just his drive and ability to really advocate For this state. We're seeing similar things in Baltimore You know Ben Siegel is there as the Baltimore Opportunity Zone coordinator So the human capital and capacity at the local level and those mayor tubs in Stockton is doing an amazing job of of Telling the story of of his city and so we are seeing these these real heroes of their of their homes Mayor David David Holt in Oklahoma City I mean I want to invest in okc because that man loves his city so much and has done a lot to Really outreach and tell the story and put some some resourcing to helping investors, you know find deals and And have community engagement mean community is part of the capital stack Which is what we would really like to see as our framework is being implemented So I think that's just a really important point Dave and at least is there anything anybody else you want to kind of call to step up in this space that you'd like to See more of our headwinds that you want to you think would be helpful to talk about Well, just because you mentioned it I do think that something that I'm hopeful to see in the future as evolution of thought around that point around community wealth I mean there's a lot of low-hanging fruit as far as creating local jobs and supporting local businesses I think a lot of impact-oriented groups ourselves included are being forefront thinkers on I do think though There's going to be this next step of evolution on how do you actually help the individuals the community stakeholders create wealth Alongside and we're seeing some interesting talk and contemplating some interesting talking on this end But I mentioned it just I think it's a really easy step is just making sure you talk with the local community before you make your Investment plan and I can go a long way and kind of spreading the opportunity for everyone involved some other interesting models I mean empowering local community land banks to acquire land before you do develop the developments They can participate in any appreciation and land values I think is really interesting then even some models around equity ownership So crowdfunding and making sure even if it's a small piece that individuals in a community could own a piece of what you're doing I think these are all really interesting ideas and to echo also some of John's points of just what's going to come in the future I hope that that's something that becomes more creative over time and that you see more of and actually I think it's a place where philanthropy can also play a role in bridging some of those capital funding gaps, so David any thought yeah, you know from my perspective It's so early on and the in the evolution of this market space that it's hard for me when I engage with people to talk about Tangible successes or failures But when I engage at the local government or the community level I met with a lot of skepticism because all they know about the legislation and the impact of it was what they read in the popular press You know and so I genuinely believe that you know emotion is The currency that incentivizes change and so for this to work You know, I think people that are kind of at the forefront of this gonna they're gonna have to find the narratives That really demonstrate and share, you know, how impactful this can be and show successes to effectuate that that Till this is that buy-in at the community level that we all keep talking about it So important to really drive impact and change. That's great. I think we have some handheld mics We have a few minutes left. I just want to open up to the floor for Q&A Do we have a micro? Yeah, we have a Thank you everyone. I'm Catherine Chen with RBC wealth management. I have a question about timing I know we're early on but my understanding is that the maximum amount of the capital gain deferral and the benefit If it's not funded by the end of this year, you know, obviously, you know every year you wait There's a portion that gets reduced So I just want to reconcile that with the concept of somehow losing some of the maximum benefit And then your comment of there's even more more capital to come what Disadvantage or, you know, sort of sort of haircut you think that will matter in terms of people Being willing to allocate money of knowing that they're gonna lose some of the capital gains maximum benefit Do you want to take that John or Mike? Yeah, I would a great question and that's exactly right There's some of some of the upfront benefit does go away if you're not if you haven't placed the capital this year into into a fund Is a we we talk about this every week So I think our our view on this is that that that upfront benefit is kind of the bonus But the real from an economic standpoint the real benefit is the is the capital gain Is that the forgiveness of the capital gain on the new on the new investment that that goes in so, you know, and you you know, you do need to wait ten years to To realize that and so by by definition You're you know, you have to look at these things as long-term investments again that that the upfront capital gain It's it's it's a nice pop. It does add It does add to our return profile But we're you know, we really look at it as the the long game is the is the is the ten-year capital gain waiver Hi, I'm Oscar Perry a bellow senior economics correspondent at next city. I'm really interested in this community wealth Thread that came up at the end here. So The deferral the step step on basis that applies to capital gains income But I'm starting to hear discussions around the the ten-year benefit should be accessible to any investors in the funds or the Projects even if they're not investing capital gains income up front and I'm hearing some discussion around the IRS doesn't think that right now But maybe it could John you design you help design this What's your take on like that that ten-year benefit? Is that should that be accessible to someone investing? $100 into an opportunity zone fund So it's a good question. Let's talk about what the word should means there In a in a perfect world. Yes, and the decision to actually narrow that Was one to reduce the potential cost profile of an already pretty cheap provision of the tax of the tax code tax bill so So the so yeah, I'd say in terms of original vision and what what we would hope for as a as a mechanism to move capital You've got one series of benefits tied to capital gains reinvested and you could have a second stream of If you don't have capital gains to reinvest, but you want to support these communities You want to make these long-term investments then you're getting the tenure you're getting access to the tenure benefit right now That's not the way it's being interpreted. That's not the way it's been It's it's been codified But I think these are the kind of things that as Congress looks to iterate which tax policy is always iterative They look to iterate this and say where can we get additional? Engagement in the policy and how can we create new avenues for wealth building within the community itself? That would be one way absolutely to do that But I just want to note there's trade-offs everywhere and so one of the critiques is that this is too big of an incentive one of the Critiques is this is too little of an incentive There's some kind of happy medium we have to reach and I think with this first iteration. That's where We need to it's part of the reason why data collection is also so important Is it over time? we want to understand as close to real time as we can how this is moving and You apply that knowledge to improving and iterating the policy Over time there's things that Congress can do now To take care of that. There's also next generation opportunities on 2.0 because this is just a fraction of the country's it's a large fraction But it's just a fraction of the eligible communities So if you can imagine a world in which they expand that these are the kind of things that I'd want them to Consider because I think that's where you get to much. It's not just a uptake question It's also ease of implementation without the capital gains timing considerations and questions It's actually much easier to deploy capital and and to time that for fund managers and for investors alike So I think I think of this as kind of the pilot concept and then there's plenty of ways you could expand that and set the dial differently over time. It's a good question I think we have time for one more. I see a hand up over here Lisa Richter of our capital. Thank you I'm glad you got into the timing question because one of the things that I hear in challenged communities is that is they just haven't had the time to get organized for this and It almost seems as though they're we need that 2.0 or the ability to get on that trajectory sooner rather than later The question I wanted to ask is The if so much Emphasis is focused on the capital gain generated within the 10-year investment period the new investment That seems to carry automatic incentives for things that have always been very challenging in community revitalization strategies i.e. displacement and I'm just wondering to what extent John and others Do you would you give comfort that the structure of the program itself has some mitigation for that risk not Complementary philanthropy, but the program itself. Thanks. I Don't want to monopolize this so I'll just I'll answer very briefly I think the answer is yes the nature of the incentive is build new capacity or improve existing capacities substantially So you can't just buy a building Kick out the residents claim the tax benefit after 10 years after the thing appreciates You have to in a housing context for example build new capacity or improve Existing capacity so for example what we've seen is affordable housing developers who have existing assets within opportunity zones They go back to their their portfolio and say where can we add new units to substantially improve these previous things? So you're adding hundreds or thousands of new units to those those types of buildings So there's some basic guardrails that keep that because of the nature of the way the incentive is delivered That are additive But I think we often skip the really important part of this conversation when it comes to displacement and Attuning to the community's needs state and local policymakers have an enormous toolkit That they can deploy and they should deploy to make sure that this is being shaped Towards their local residents in most opportunities own communities Displacement from gentrification is simply not the concern and I think it's really important to distinguish The percentage that do is small, but it's meaningful But for the most part you're talking about communities that have depopulated that have tons of vacant housing and available stock That desperately need an uptake and uplift of their housing values and housing stock It's just a different playbook. It's a different set of concerns It's not that one is more important than the other so you have to write size What is the concern that we should be prioritizing to the communities in question? And I think too much of the not just with opportunities No, it's too much of the broad conversation about community revitalization is dominated by Washington, DC, New York City San Francisco and those those conditions do not exist in Akron, Ohio or in South Bend, Indiana That's just not the reality So you need a you need to understand what those realities are and attune to that and that's where local leadership is so important Because they control zoning they control permitting they control fast-tracking and all kinds of other tools that they can use to shape the incentive They also have other incentives themselves to skew towards affordable housing or Workforce housing or other things that make it easier to avoid displacement So there's a lot of room for that kind of creativity and it's it's not a fade-to-complete that it's going to be used a specific way That's where the the local ground-up side is so important And to that point, you know, there's 8,764 zones Urban Institute did a study that less than five percent have risk of gentrification gentrification meaning forced migration, so it's a very small percentage And we need to be mindful of the other 95 percent and and what's going on there both are important, but Just to put it in context. It's a fairly small percentage where there's some some sort of forced migration risk I want to thank my panelists. I want to thank John for for really helping push this legislation that set up to talk about baby shark. No, oh Well, do you want to do it? I didn't want to embarrass you, but I did I did tell him that ask him to to Yeah, Alex is egging you on. Are there any nationals fans in the audience? Are there any baby shark fans in the audience? Oh You've got it on tee up Oh So everybody should be the first time that's happened with an opportunities on No, thank you But thank you for for pending legislation that the the conversation swell around it. It's really precious It's it's an opportunity to drive impact across the country and it's setting some tables where innovation happens under circumstances like these So we appreciate everybody everybody's hard work and thanks for being with us this morning


The first step to being made whole is recognizing that there is in fact a whole that needs to be filled.  This isn't to say that there's something wrong with you, that the rest of the world is talking behind your back wondering when you're going to get it all together. This is to say that the whole world sees your honesty and honestly responds with me too. Because we are all in need of wholeness and we will be made whole by one another.


So, hello everyone, welcome to this video. In this video today we will talk about Dependence Monkey and this video is from series how to beat Python's PIP and more precisely we will talk about Dependence Monkey and how Dependence Monkey is checking Python dependencies of a TensorFlow stack. So if we take a look at TensorFlow, we can pick one specific version to simplify things a bit, then we can have TensorFlow in version 2.1.0 installed from the PyPI, the Python package index and we can see 23 dependencies. These 23 dependencies are in a form of dependence name and version range specification that needs to be satisfied in order to install TensorFlow. You can also see some environment markers that means that these dependencies can be installed conditionally, for example based on Python interpreter version. Okay, so we have these 23 dependencies. Let's create a dependency graph. In this case we have just one let's say direct dependency that is TensorFlow in the specific version that we mentioned and these 23 undirected dependencies that we saw earlier. We can create a node in this dependency graph that directly corresponds to dependency name and version range specification that needs to be satisfied for all the dependencies in the dependency graph when installed specific package. So this version range specification is resolved based on resolver and that basically checks what dependencies are available, for example on PyPI or any other Python package index. And I did that. So for example, if we take a look at ABSLPi, we can see some additional nodes that correspond directly to specific versions that satisfy the version range specification for ABSLPi as mentioned in TensorFlow dependency listing. The same can be applied for other dependencies basically for all these 23 direct dependencies of TensorFlow. Okay, these are just direct dependencies and to inspect the wall dependency graph we need to take a look also at transitive dependencies of TensorFlow, meaning dependencies of direct dependencies. I checked just few, I provided a link to a GitHub repository where there is stated the wall dependency graph in case you are interested. And here you can see that, for example, at Opt-Ansum depends on NumPy, PsiPy also depends on NumPy, but different version range specifications in comparison to the dependency that is stated for TensorFlow in version 2.1.0. Which version of NumPy is installed? Well, that depends on the actual resolution process. If we install these dependencies using pip, pipen, .3, then the latest possible NumPy will be picked. If we create some smarter picking, like which NumPy versions work very well with our TensorFlow release, then we can gain some higher quality software in comparison to the software that has, let's say, the latest, but not the greatest, dependencies. This, which NumPy dependencies will be, or which NumPy version will be installed, that also depends on other packages that you have inside your dependency listing. If you are using TensorFlow as a library in your application, you can use, for example, different version of PsiPy or something else that includes a different version of NumPy, and then the resolution process can create a software stack that is formed with a different NumPy version than stated here. Also, this depends on time. When you install or when you resolve dependencies, then it depends on which versions of NumPy can end in your software. Okay, and this all can be applied also to other direct dependencies, so let's take a look at it. If we check TensorFlow 2.1.0 as a wall, we can see NumPy, that is, a dependency of H5, Py, OptanZoom, PsiPy, Keras processing, TensorBoard, and possibly others, but I think I stated all. You can find the listing in the GitHub Gist, and if you take a look at direct dependencies of TensorFlow, we can construct something like 3 multiplied by 10 to the power of 13 possible combinations of packages in different versions that can be installed. Of course, this version varies based on the actual resolution process, and if you include also transitive dependencies, things get more complicated. So, you can see that NumPy is introduced into TensorFlow stack by different dependencies, a direct or transitive ones, and if there is an issue in NumPy, then there can be some effect or some bug in the software, and you can break the application, it can misbehave, and the overall quality of software that is shipped is, let's say degraded, or if you install proper version of NumPy, then the quality can grow. We can also touch overpinning, underpinning in these cases, but that's a different story. We will talk about it in the upcoming articles. If we take a look at the TensorFlow dependency graph and check how many nodes are there, you can find something like 17 million nodes in the dependency graph, and this number can grow over time, so you can, there are new releases of NumPy that can satisfy version range specifications in the dependency graph, and in that case, dependency graph grows. It's worth mentioning here that this number corresponds to a number of packages in specific versions coming from a specific Python package index. So it's, for example, a node in this dependency graph, in this case it's NumPy in version 1.7 coming from the Python package index. So you can see that's pretty huge dependency graph. If we go more into the theoretical direction, let's say that we have an application that depends on two packages. One is called SimpleLeap and the other one is AnotherLeap. We consider these dependencies as direct dependencies of our application, and also these dependencies do not have any transitive dependencies that would be introduced into the software stack. Then we can explore the state space that is created out of these two packages. So let's say we have SimpleLeap and UnderLeap, and we are just checking combinations of different versions for SimpleLeap and UnderLeap. Then we create some scoring function that will explore how the software behaves when it comes to its quality. You can imagine quality being how performance the software is. This is just a brief introduction, and you can find more information in the linked article. This is just to give context what we are doing with the dependency monkey. So as stated, we have SimpleLeap and UnderLeap, and we have some scoring function that tells us how good the software is when we change versions of SimpleLeap and UnderLeap. We are in a discrete phase, and here you can see these purple dots that correspond to the final score that is computed by the scoring function. It's not that intuitive, so let's try to interpolate this function, and in this case, we get this surface that corresponds to the interpolated discrete function for scoring these combinations. And as you can see in this artificial example, our software can have different aspects, different quality depending on versions that we installed. Okay, now let's say that we pin SimpleLeap to some specific version, and what we will do, we will change some versions of UnderLeap, or we will change, or we will check what versions of UnderLeap can be installed. In that case, we are basically mapping these two-dimensional space where x-axis corresponds to versions of UnderLeap and y-axis is the score. This function tells us how the software behaves when we have SimpleLeap in a specific version, and we are changing these versions, or possibly, for example, builds of AnotherLeap. Then this scoring function can look like this as an example. Okay, this was a more theoretical background, and let's take a look at a resolution process that can create different combinations and satisfy version range specification and resolve dependencies considering the dependency graph. So our ultimate goal will be to resolve TensorFlow software stacks with different NumPy versions, but still respecting version range specifications on NumPy, so we still satisfy version range specifications that are put by different libraries such as TensorFlow, SciPy, and others that use NumPy. So we will create a reconfigurable resolver. This resolver is created of pipeline units, and these pipeline units basically form a programmable interface to the resolver, and you can think of a pipeline unit being something like do not submit already or do not check already seen software stack. So if we resolve some software stack multiple times, that can happen based on dependency graph resolution, consider just first resolution and do not create multiple duplicates. Then another pipeline unit can be in all versions except for some specific library, in case it will be NumPy. We can do this also on the direct dependencies, but in that case we are directly saying that these dependencies have to be present in the dependency graph, but it's not something that needs to be true. Like if we've been, for example, TensorFlow in a specific version inside TensorFlow stack for all the releases of TensorFlow, TensorFlow does not need to be present in the dependency graph, meaning the dependency graph can look like different because there are no restrictions or emerging range specifications that would be introduced by TensorFlow package. And you can think of any other pipeline units. These units are basically some programmable interface, and they do one thing, and they do it well, and when grouped into a pipeline configuration, then the pipeline can resolve software stacks that have certain aspects and are suitable for our needs based on what we want to see. So, for example, we want to see different versions of NumPy in a TensorFlow stack. Okay, if we take a look at dependency monkey as a component, we already know the pipeline. So, in green, these green rectangles are pipeline units. They are grouped into a pipeline configuration, and this pipeline configuration is used by dependency monkey and resolver that can resolve software specs that are respecting Python packaging requirements or Python packaging guidelines and can resolve pinned-down software stacks with certain aspects, with certain quality. The input to dependency monkey is a vector. This vector can be formed out of direct dependencies. These are required, so stating what needs to be resolved. In our case, it will be TensorFlow in version 2.1.0. Then information about software environment, hardware environment, meaning where the software will be run, which operating system, on which hardware, what CPU, all these things that we want to consider when we want to run some specific jobs that verify quality of software. Another input can be library usage. In other words, API calls to different libraries, like TensorFlow, so if you are using convolutional network or convolutional layer, you can form or act differently in pipeline configuration or pipeline units. And then there is another parameter, this decision type that also parametrizes dependency monkey in some way, we will not go into details. Then all these things in the input vector are taken. The pipeline configuration is created, this pipeline unit can have parameters that state how these pipeline units can act. And the resolver and dependency monkey itself uses knowledge base to resolve software stacks. So one of knowledge that it uses is how dependencies are structured. So if you are following our series, you know that pipeline dependencies can be resolved using solver. And it basically states what dependencies correspond to which version or any specifications of packages. So that's one of the knowledge that we have and is required to actually resolve software stacks. Another knowledge can be information for pipeline units, so pipeline units can act differently. So if we do not want to resolve same software stacks across multiple dependency monkey rounds, we can code it into a pipeline unit that will query the knowledge base and ask for already seen software stacks in the previous rounds. So we do not need to inspect or check or resolve software stacks multiple times and things like that. Dependence monkey is a component that is living inside a cluster. So it can be run inside a cluster. In that case, the interface to trigger a dependency monkey run is an API server. So we can directly go to an API endpoint, pass in all the parameters, and dependency monkey will be run. That's just another interface. In the demo that I will do, there will be used dependency monkey locally, so it will be run from command line, but that's just another interface how to interact with dependency monkey. Once dependency monkey resolves software stacks, software stacks are sent to Amun to check inspection, to run inspections, and these inspections verify software stack quality. So if you are following our articles, you know that these inspections check how the application behaves when there is a metrics multiplication done, what is the overall performance or how the application behaves when convolutional layers are created, or it can be done on model level, not on operation level. So Amun creates inspections, and these inspections create, or the output of the inspections is a document which can be subsequently reviewed, how the software behaves, where there are any build failures, where there are any runtime failures, and based on this review create, again, knowledge for dependency monkey, and later for resolving high quality software, as we will talk in the upcoming articles. Okay, so let's dive into a demo. And we have switched to terminal. As stated in the presentation, terminal is just one interface to interact with dependency monkey. I will type directly commands to interact with dependency monkey, but this all can be accomplished also inside a cluster where API endpoints would be used, and these API endpoints accept JSON, and that JSON is subsequently transformed to parameters that are used by dependency monkey. Okay, so let's take a look at first command that we will trigger. So in this case, we will run dependency monkey. The implementation lives in TOTS advisor, because it's reusing resolver that is used in advices. We will talk about this in upcoming articles. Dependency monkey accepts PIP file. So if you take a look at PIP file, it's under PIP file as produced by PIP end. You can see that we are requesting TensorFlow in version 2.1.0 running Python 3.6. So that's PIP file. Then there is a recommendation type. Again, I'm not going into details, but that's just another parameter that is passed to advisor. Then we are using predictor. This predictor guides resolver how to resolve dependencies inside dependency graph. So package combinations is one implementation of predictor. This predictor accepts parameters. If you take a look at these parameters, they state what package combinations should be resolved. So in this case, we are resolving TensorFlow in version 2.1.0 with different combinations of NumPy. Then another parameter is account that says how many softwares text we want to resolve. Then another parameter is called AMOON context. If you take a look at it, it's the JSON file that is sent to AMOON. This is used to describe how the runtime and build time environment should look like and how the build process should look like. It also states script that will be used to verify that the application runs correctly. In this case, we are using UBI-8 Python 3.6 that is compatible with REL-8 Python 3.6. You can see run requests and build requests. So that's about AMOON context. Another parameter is seed. This seed is initializing random generator so dependency monkey can run in a completely random manner or if you explicitly say seed, then the random generator is initialized so multiple dependency monkey runs produce the same results. So I put it in there just to have similar output across multiple runs. Then another parameter is runtime environment. Let's take a look at it. So runtime environment states for resolver that we are using REL in version 8 running Python 3.6 and platform is x8664, so Linux. This is input to resolver as dependencies that are installed can depend on the environment. When we are running Python 3.6, we can end up with different dependencies that are installed given environment markers that are supplied to packages in various parts of the dependency graph. So this needs to be done. Okay, then we have pipeline configuration. So let's take a look at it. It states what pipeline units should be present inside pipeline configuration. The implementation of advisor implements multiple pipeline units and these pipeline units can be of different types. For example, boots, thieves, steps, strides, and reps. They serve their own purpose and we will talk about them in the upcoming videos. So for us, the most interesting ones are strides. So here you can see multiple strides called one version stride and the limit that each package should be present in a specific version or in one version that is resolved. So in this case, we have ABSL Pi, GAST, Google Pasta, and all these packages in one specific version. And then I commented out one pipeline unit that says one version of NumPi. That's something we don't want. We want to resolve multiple versions of NumPi for multiple combinations that can occur inside a TensorFlow stack. Okay, so this is pipeline configuration. Let's take a look at what we have. Another parameter report output that just says where the report of dependency monkey run should be stored. So it's a JSON file that captures all the relevant inputs and the outputs produced by dependency monkey. And then there is output. In the first run, we will use output directory. In the second run, we will submit the first stacks to Amun. So Amun really builds them and verifies that they are running correctly. Refer to the previous article from Python Beats, how to beat Python's PIP series to get more information about this. Okay, so let's try to run this command and what we should see, we should see the dependency monkey starts and it starts resolving TensorFlow stacks. As you can see, there were produced 10 stacks in six seconds, something more than six seconds. And these results were stored inside output directory. So let's take a look at them. These outputs are PIP file log files. So standard PIP and files. And as you can see, they differ with NumPy version. This is all considering the dependency graph and version rank specification, as stated for all the dependencies inside TensorFlow stack. So for example, that SciPy TensorFlow and all these that were stated in the presentation. Okay, so now let's try to submit them to Amun. So we will remove commented out URL to service and let's run it. As you will see, it will be slightly slower because there is networking overheads. But if we take a look at it, now we should see in Argo workflows new inspections that are running. So each inspection creates image stream, then creates build config build. So the application is built inside an open shift cluster and subsequently there are run inspection drops that verify that application runs correctly. So again, refer to the last article about Amun to get more info about this. Okay, so that would be the first commands that we run, our first run of dependency monkey. And now let's take a look at second commands that we will take a look at. So again, we are running dependency monkey with same PIP file with same decision type, but we switched predictor to random walk. This predictor can randomly walk the dependency graph and randomly resolve some software stacks that are valid TensorFlow software stacks. We will produce 500 software stacks. We will use same configuration for Amun, same seed for the resolver and the random generator, same runtime environment, same report output, and we will adjust pipeline configuration. So if you recall, we had specific wraps, sorry strides that removed or prevented from creating different combinations of packages that we didn't want. In this case, we removed them, so we are not limiting to some specific combination of packages. Okay, so let's give it a try. Again, first, we will run it for results inside an output directory that is called Out2. And in a second, we should be able to see that software stacks are resolved. As you can see, they are resolved much faster because the resolver uses pre-computed information about dependency and dependency graph. This was already mentioned in one of articles from previous How to Beat Python's Beep series. Okay, so as you can see, we generated 500 TensorFlow software stacks in 30 seconds. That is something like, sorry, in 16 seconds, and it's something like 30 seconds, 30 stacks per second. As done before, again, we can submit these dependency monkey run to Amun. So Amun again verifies that randomly sampled software stacks TensorFlow built correctly and that they run correctly. So we can do that. Again, it will be slightly slower because there's networking overhead to submit these results. And as you will see inside Argo workflows or Argo workflow, there are created new ones. So why do we do this? We basically randomly sampled the state space of all the possible TensorFlow software stacks that can be resolved. And by doing so, we can spot issues that can occur when resolving some packages inside dependency graph. So an example can be resolution process when you install TensorFlow together with other packages like Flask. So Flask introduces its own dependencies that can be shared with dependencies of TensorFlow and they can affect how the resolution process looks like and how the resulting software stack looks like, meaning different versions of packages. Okay, so by doing this random sampling, we found issues. An example can be an issue in TensorFlow 2.1 when specific URL lip was installed together with TensorFlow 2.1. We got this import error when we were trying to import TensorFlow. So here we can see, we tried to import TensorFlow but we observed import error when there was import range from package 6. This issue was caused by bundled 6 that is shipped together with URL lip 3 and the initialization of the importing logic in URL lip 3 collided with 6 package imports. So we observed this import error. That means that TensorFlow when installed with this URL lip will never work, will never start. So this is one observation that we have. You can find all the files on GitHub that are used in this demo and in this presentation. I will link them in the description down below. So that was our demo. I hope you liked it. Everything you saw is made in a project that is called project.tot. Project.tot is a project in AICoE, AI Center of Excellence that is a team inside Office of the CTO, the Red Heads Office of the CTO. You can find us on homepage.totstation.ninja. All the source code lives inside a GitHub organization that is called totstation. We have also Twitter handle. So if you are interested in updates, any progress we make or any quality that we conduct, feel free to subscribe to our Twitter handle. And we have also YouTube channel. So that's the YouTube channel where this video is published. So feel free to click on the link and feel free to subscribe to be updated when we have anything new. This way I would like to thank you for your attention and I hope we will see each other next time in the upcoming articles about project.tot and about how to beat Python's PIP articles. So thank you and have a nice day.