It markets means at least a couple things, and I'm going to drill down a little bit into this as well. But on the one hand, it has to do with passive versus active investing, index funds versus stock pickers, if you will. It also has to do with high-frequency trading and algorithms. And maybe the common denominator behind all of this is technology, because if there wasn't technology, we wouldn't be having this conversation. It would still be people, guys with pieces of paper, and it was all guys back then, writing things down and placing orders. And for better or for worse, I'm old enough to remember those days back on the exchanges. And to my right, Adina runs an exchange that has been at the very fore of technology. The NASDAQ was fintech before fintech was cool, right? I think that's the way to look at it. So Adina, why don't I start off by asking you just sort of what your idea of automated markets is and what that means. Very kind of open-ended question. Just get some top-line thoughts. Sure. Well, I do think we have to start by saying that the markets have been and will always be a combination of humans and machines. And I think that decision-making around investment decisions, whether they're long-term or short-term, will still culminate and start with the person, making a fundamental choice. Now, it could be a technical choice. It could be a fundamental decision around the future of earnings of a company. It could be a strategy that they're undertaking in terms of delivering an outcome. And even if they are therefore then turning it into an index or turning it into an algorithm, there is still a human at the beginning of all of those decision chains. And so I'm a believer that anytime that the markets will swing too far towards automation, first of all, I think humans will have the opportunity to come in and arbitrage that with their judgment and with their wisdom. I also end with their strategies and other things. But I also believe that technology, as you said, is this kind of inevitability that's come into the market. It's democratized markets over time. So millions of investors have much better access to markets than they ever did before. They also have better access to information, better access to strategies, better access to investment vehicles, much more choice. And it's a lot cheaper for them to trade and to invest. However, you then also get a level of complexity that comes with technology as well. So I think that's part of the mystery that you mentioned, that the fact that you have all these different elements to the markets today, whether it's index investing, active investing, alternative investing, algorithmic investing, all of those culminating in the markets through technology, I think makes it so that it's harder for an investor to kind of understand the whole. But it is and it's important for them to do it. But they should also always recognize that there are humans in that chain throughout the markets today. Nice to hear that humans still matter. Absolutely. David, let me switch over to you and ask you about your particular institution. And big bank in Canada, how? Largest bank. Largest bank, right? Let's use a superlative when we can. Largest bank, yeah. Go ahead. OK, good. Forgive me. Forgive me. And I want to ask you, how does this debate or tension, if you will, impact your business? So what part of your business would this relate to, even? There's when you think about the topic that we're discussing today, there's the electronic side of things. And then there's the liquidity volatility aspect of markets around passive and active. You know, we're invested heavily in algorithms and algo trading and trying to execute for our clients in the best way possible. We launched a product called AIDEN that focuses on block trading to help our asset management clients and our large funds trade effectively and get the best execution. So certainly AI is helping execute, helping manage the complexity in markets. And we've invested in that. I think one of the other distortions that we're seeing is liquidity. So when you have the amount of flow that you see into passive funds, you've got intraday liquidity challenges. 25% of the trading occurs at the closing auction. And that is the rebalancing that you see in passive funds. So intraday liquidity has certainly changed. And then what we saw over December was certainly a liquidity challenge and a number of issuers. We saw so many clients come into us for liquidity to take positions where, and we see that as a function of a lot of float going into passive and trading that we had to supply liquidity that we hadn't seen before. So there's a liquidity distortion that's happening on the passive active side as another dimension to this whole challenge. I want to ask you about being able to oversee your oversight and being able to understand all this stuff. Because this is not directly related. But say JP Morgan and Jamie Dimon, who I think is a terrific banker and is a really smart guy. But even he didn't see the London Whale. Now that's not an active passive thing. But it sort of speaks to perhaps hidden parts of the bank and very complex strategies. How do you as a chief executive make sure that you understand everything that's going on like that? Well, you rely on your team. But it certainly will be a part of the governance of any organization from the board through the management team. You set up a governance structure that allows you to sit down with your chief risk officer with your head of capital markets, with the trading side of capital markets or the investment side of capital markets. And when you do business reviews and you triangulate as a CEO, it's always hard to get it from one source. So as a CEO, you're constantly triangulating across your organization, outside your organization, trying to paint a picture. So for me, that comes through regular business reviews from functions through to businesses and then certainly interacting outside the organization. You pick up the themes.  Every CEO has to set up that structure to triangulate upon the key issues. I'm realizing that Bill and Ron on the outsides are the two longer-term investors. I don't know if we did that on purpose or not. Keep them away from the center, the too much activity. Ron, let me go over to you. I mean, as a longer-term investor, real estate and those sorts of securities, if you will, or investments, I should say, how does this debate even affect what you do for a living? Well, we do take the long haul. We have to. We have about $190 billion that we invest around the world in real estate, private equity, public equity, all public markets. And so from our perspective, what we find is technology is the common link through all of this. And technology and what is happening to technology these days will only further leverage what is taking place. But from the trading floor, where we utilize algorithms for trading, all the way through to the companies that we own directly, we're using technology. We're using AI to drive ultimately better decisions. And we're using it in real estate. We're using it actually in real estate, for example, in our shopping malls, where we have to ultimately understand the customer better, what their needs are. And so to us, it's really about managing the technology and managing the risks of technology. And it's even raising some very important questions as a fiduciary for us. In the public trading area, for example, when we're using algorithms more and more, the so-called black box stuff, we're starting to get questions where people say, how can you actually fulfill your fiduciary duties by turning it over to a black box of some kind? How do you know that in these algorithms that some of them being self-learning, how do you actually know that you can do something with it? Because once upon a day, if you measured PE and moving averages, you could actually figure out why you invested and why you followed something. And so this technology story, its complexity, is not an insignificant issue that CEOs all have to deal with. And for us, when it comes to the fiduciary side, we have to be very, very mindful that we can't push it so far that we turn our brains over. And I think Adina's points are bang on. There's always a human process at the very beginning of this. There has to be. Judgment has to ultimately be exercised in all of this. The machines and the language and the decision rules and the algorithms are nothing more than something that is repeatable over time. It's repeatable, and what we end up doing is putting our trust in the very things that create those decisions and that they are repeatable and that they do add value somewhere along the way. And of course, we're utilizing them in terms of efficiency, cost reduction, opportunity sets, making our companies better, understanding arbitrage opportunities that are hard to find just by doing a bit of math on the back of an envelope. So for us, it's everywhere. And we really have to, I think the biggest challenge for us is making sure that we are fulfilling our fiduciary duty, leveraging it, and really leveraging the deep insights that it's capable of bringing. And that's what frankly we're all really after, I think. Right, go ahead, jump in. I agree with everything you just said. We've actually invested in a great Canadian company based in Toronto. All Canada today. It is, called Quandel, which is an alternative data provider. So it basically does allow for, we basically work with issuer, our issuer clients and other organizations to find information that could be useful and informative to you and your strategies that you're implementing to help you look at some level of predictive capabilities to help you understand and get deeper insights into company performance or industry performance or trends that are what might impact certain industries or companies over time or asset classes, including real estate and commodities. So, and that's something that we believe at the end of the day, again, it's a matter of making better decisions, but always understanding the underlying math. And it is so important that we never let go of that underlying math as us understanding how it's impacting and how the outcome has been generated, but allowing that, allowing the algorithm to a couple with the data, a couple with your knowledge to make it so that you can make smarter investments. It's interesting how the conversation has sort of shifted a bit to risk management, right? Which I think is very much at the core of this.  That's what she told me. Yeah, but you also have a window into this world more directly because you're on the board of BlackRock. And I know BlackRock has also products and services that speak to this risk management when it comes to algorithms and such. So I'm wondering what your take on passive versus active and its impact on legacy institutions, for instance. Well, great, thanks, Andy. I think there's sort of two mega trends taking hold right now and have been working for the past decade or so, we'll continue, which is the move towards passive investing. I mean, in the U.S. equity market, 10 years ago it was about 9% passive, it's about 44% passive today, roughly. And Adina may have some other data on that too, but it's a big shift in that way.  And related to that, a lot of the active management shifted to the private markets. So you've had a real pickup in private alternative assets, private equity, et cetera. And that's where a lot of the active management has gone, active corporate ownership has gone. But so I think that will continue. I think investors are basically moving away from active public management towards those two poles, private markets, and passive. But I think the biggest concern I have when you touch on risk, and I'd love to hear the rest of the group's comments on it, is this fragmentation of liquidity, because it used to be liquidity was centered in the NASDAQ and some other exchanges around the world. And now when we talk at BlackRock, we use 70 liquidity centers, 70 market makers on a daily basis to basically execute our volumes. And it works fine. And virtually all of them are technology-enabled, all of them are many are algorithmic, many are electronic market makers. But the thing that's so different is what happens in a period of stress? They at least have no affirmative obligation to make markets. And as David said, during closing periods, during periods of stress, they're absent. And it was different when the major banks and the exchanges, who have a more affirmative responsibility to keep markets moving, these folks will withdraw immediately. And I think it's a big deal, Dean and I were talking. The asset management industry is growing. So you have more assets, you have more trading volumes, but it's way more fragmented. And I don't think we really have a great handle on the risk that's inherent in that. And we haven't really been tested in a significant way. I want to hear what you have to say about that, Dean. But I want to pick David's brain on that first. I mean, you were talking about that as well. And is it a scarier world with that diffusion? It is. Certainly, we experienced that in December. You're seeing that so much float is captured within the passive industry. You said 44% of assets, they tend to rebalance at the end of the day, and they have to trade in the direction the market's going. So there's less float to take the opposite position in the market. So not only do you have a liquidity challenge, but you exacerbate volatility as you have to trade towards the market moves. So you have got this block of float that's moving in the same direction. So absolutely, you start to see liquidity challenges. And we're going to have to stress for that. We're going to have to look and work with markets, makers. We're going to have to work with regulators to talk about the amount of float and the liquidity in the marketplace. Otherwise, you're going to see distortions in the marketplace. So that's certainly one impact. The other impact is governance and oversight of corporations. And how do you manage all the concentration of passive ownership in three large organizations? So I mean, that's probably we'll get to that later. That's certainly a third issue around passive versus active in a concentration within passive. Right. Liquidity challenge sounds like it could be something you do at a bar, but it's real. Yeah, it's real. It's very real. But David's honest, something we should just touch on is 15% of virtually every company S&P 500 is owned by three firms. And those three firms are? BlackRock, Vanguard, and State Street, the big index players. And in many cases, you add on the rest of the passive, you're talking 30%, 40% passive ownership for these large cap names. And again, I don't think we fully understand what does that mean. Is it encouraging activism and short-termism? Because of the fact that they're not going to react. I mean, they're not going to react to a business plan put forward by an activist because it's not their job. It's interesting, I was talking to Jack Bogel, who, of course, just passed away last week, I believe. And he was the founder of Vanguard and a huge advocate for passive investing and indexing and for index passive, right? Right, exactly. And I asked him, what would happen if the entire world went to passive investing? And he said, oh, it'll never happen. And I said, but what would happen? And he said, chaos. And I was like, well, when you hear 44%, you know, it's actually getting there. Yeah, so I think actually, when we look at the US stocks, first of all, about 10% of the trading occurs on the close. It's actually from anywhere from seven to 10. So I think it might be more exaggerated in the Canadian markets. I think that when it comes to passive ownership, we do a lot of work around that. We give our clients something called passive IQ, which allows them to see all of the funds that they are in, all the passive funds and ETFs that they're in. And I would say it's somewhere in the range of 20 to 25%. It's a little closer to where it is in US equities, but the 44% also includes fixed income. And there's a lot of money flowing into passive fixed income. And that's where I think you might have some significant liquidity breaks because there aren't as many organized markets. And as you know, bonds, core bonds in particular are inherently a liquid. And then you don't really have as many active owners taking positions, you don't have as many banks taking, using their balance sheet to manage those positions. So if there is what I would call a one way trade in that space, I think that we're gonna have more of a challenge in terms of who's gonna stand up and manage that if there is a significant move from the passives into or out of certain bonds or even treasuries. But I think with regard to the equities world, I agree it is a fragmented world. We would love for it to be less fragmented. So I think that the rule base that we operate under allows for a lot of competition, not only amongst exchanges, but between exchanges and our own clients in terms of broker dealers have single dealer platforms in addition to exchanges. So there are four exchange groups that operate 13 exchanges in the US. We trade all US equities, NASAC trades, equities that are listed on New York and New York trades equities that are listed on NASAC. We all trade on each other's equities, but it is a very competitive world that has been through regulatory fiat. And the challenge that large buy side firms have today is managing in their liquidity in that very crowded field, knowing where to put their liquidity. They don't wanna show their hand too much, but at the same time they have to spread their flow across 30 to 40 different venues in order to get things done. And it's a very complicated world that requires algorithms and requires a lot more technology and a lot more trading savvy and more potentially AI over time to help manage in that fragmented world. We'd love to re-aggregate liquidity. That would be wonderful for us. But I think that today we have to decide at the end of the day, what's just the best way to use technology to operate in the environment we're in and to find a way for large institutions to manage their flow in this automated space? Well, let me ask you a metaphysical question, which is maybe the most salient question and I hinted at this earlier, which is, what does this mean for the ordinary average person, Adina, hearing all this stuff? So I would say the first thing I would say, and we say this over and over again, if you are an average retail investor, you've never had it better, okay? So a super... Because of cost? Costs are down 90%. I'm sorry, costs are down about 75%, at least even in some index funds now, you're paying zero fees, right? And zero commissions on certain online brokers. So costs are down anywhere from 70 to 100%. Spreads are down 90% in the last 10 years with the competition has brought a lot tighter spreads. And you have a plethora of choices on how to get your order flow executed, especially if you're executing 100, 200, 500, even a thousand shares, you are gonna get an instantaneous execution either at or better than the inside price on the market. So it is a wonderful world for retail and they have a lot of choices. They have index choices, they have active choices. The one thing that they are held back from though is private equity, right? And that's where if you're in a pension fund, you may get some access to private equity, but if you're an average person, you're not getting access to some great private companies. And that's where I do believe still that we have, that's one disadvantage that I'd like to see change over time. Well, that's where blockchain comes in because we're experimenting with taking an asset and bringing it into smaller pieces and registering that in a decentralized ledger called blockchain. So you can take an asset or even a company and create a unit in a decentralized blockchain and then sell that into the marketplace. The other common I'll make is around the individual small investors around liquidity, the structure of an ETF, which is a very profitable product for you and all the listings that you get as an exchange is inherently it redeems in kind versus in cash for mutual funds. So because of that, exchange-traded funds are more liquid and therefore create an opportunity, a liquidity opportunity for smaller retail investors. So lower costs, more liquid instrument, more accessible instrument, and then I thought blockchain I think is an opportunity in the future to take assets that are less liquid and make them liquid. And I think that's an exciting part of this. Yeah, that's sort of a very exciting part of it, including real estate assets. I know that they're so... Yeah, exactly. And I think to Dean's comment, I think we're going to actually see alternatives come into the retail market. You're at the very leading edge of that happening right now. I mean, it has been traditionally an institutional and high net worth product, but there's some, I mean, one interesting company, iCapital, is sort of a technology platform that allows a retail investor to buy into private equity and build portfolios for the first time. So you're beginning to see it. And venture as well. And actually, we're working with iCapital and other large warehouses to allow for liquidity and private equity funds for the first time so that you have organized liquidity events through auctions, which if you do it on a regular basis, can create the opportunity for more retail to come in. And so we've actually been partnering with them and others to do that. Ron, let me ask you, go ahead, jump in. No, no, carry on. Okay, I want to ask you, these guys, I heard them mention the R word, I think both of them regulation. And I'm wondering if you have any thoughts on that with regard to this discussion. Yeah, we're a bit of a lightning rod, to be honest. When you look around the world today and you think the top 50 sovereign wealth funds, pension funds globally aggregate well into the trillions, I know the top 10 in Canada is 1.6 trillion. And they're investing in across all markets, all around the world. And a lot of it's going directly and into private markets as well. And the regulators previously spent a lot of time focusing on the banks post-2008, obviously, and they're heavily regulated. This world of sovereign wealth and pension plans has been a light touch regulator pretty much. But given the concentration of capital that is now flowing into this world and the kinds of things that you'll find, the Singaporeans and the Canadians and different- The Norwegians and- Norwegians and Adi and other people coming out of the Middle East. The regulatory environment is starting to shift on us and we're starting to see it and they're stepping in. And what it always comes down to is issues around, A, is leverage employed. And ultimately, and very rapidly, I propose this conversation around liquidity, the liquidity conversations that are starting to take place are not unlike the banks because with the kind of liquidity that's in the market and when you show up with margin calls and other issues, it's starting to become a really, really big issue because the end user now, it's not just centered in the banks, it's the end users, it's the large asset managers that have to show up, depending on what instruments they're using to get their exposure. And liquidity is a pretty big deal. So our regulatory environment is changing. Of course, in the marketplace, we're seeing regulation start to impact trading. We do notice that liquidity becomes, particularly under stress conditions. It is not an insignificant issue. And of course, if there's gaps in the market under stress conditions or lack of liquidity that's in the marketplace, you're gonna start seeing that in the form of differentiated margin calls from what we otherwise might have seen five, six, seven years ago. So it's a complicated area. And as algorithmic trading and decision-making and actual, taking large blocks and peeling them out little bits and pieces at a time in terms of trading to drive cost effectiveness, this is all going to come together to create a world that is quite complicated. Once we start putting technology behind a lot of it, there's no way back. We're not moving back from that. It's going to happen. But you can bet that regulatory imposition, regulatory things are going to come along and start to impact this more and more. Start putting blockchain in it, other issues that are happening and the world catches up to what that really potentially could be all about. It's going to be a significant, complex problem to have to govern and manage. And a regulator is going to have to be a very, very critical component of that going forward.  We already do. LCR ratio. We already do. Well, it's interesting. And again, it sounds like it's something, unfortunately, that might be precipitated by a crisis. I'm talking about incremental regulation because imposing regulation without a crisis for better or for worse and probably for worse is actually pretty tough for the regulators to do. I mean, look what happens in the wake of crises each time. There's another way in which may or may not be appropriate, but it seems to happen that way. I want to go over to you, Bill, and ask because pick up on something at Adina said, which is that it's never been better for retail investors, the world that we're living in. But they see from a cost standpoint, certainly, but maybe from a risk standpoint, I'm not so sure. I mean, we see things like the flash crash of 2010 and other many things like that happening and people get scared. I'm just wondering what your thoughts are on that. I'm where Adina is. I think it's never been a better time because of the range of products that the customer, the other client has access to has never been better, never been cheaper, never been more liquid. So that I think is absolutely true. I'd just make a quick corollary point. It's true for institutional portfolios too. Institutions have never had a better time to build portfolios, passive products, ETFs, alternatives. They really can shape their risk profile and their return profile from the ways they could not do a long time ago. So all that I think is very, very positive for billing portfolios. I think the thing that is the one last thing that needs to be solved for, I think the retail investor is distribution channels. A lot of the channels are still high cost and add very little value. I mean, much of the advisor channel, I think it's still a high cost, low value ad channel, for example, for the retail investor. When behind it sit products that I think are actually quite efficient. So... Well, those are the margins of the business, right? Yeah, but I think they're under assault. We used to say that question, where are the customers' yachts? Maybe in this new environment, the customers are actually able to buy the yachts. Well, there's a lot more electronic connection to the customer. They've got a lot more electronic choices to do it. And I think that friction is gonna start to, the distribution friction is gonna start to come out. The manager cost, the manager fees, that's already been squeezed. The transaction fees, that's been squeezed. But parts of the distribution channel for the retail investor is still quite high. And I think that's the next thing that's gonna get disrupted. And you're seeing it already. I mean, you're seeing lots of, because I said direct electronic channels between the asset manager and the client. All right. Just to follow on that, last week we announced that we're merging our ETF business with BlackRock Canada. So we're creating one RBC... Did you know that? I hope you knew that. I see. It's called RBC ICRS. I like Dave say that. The news to me. Very exciting. And one, we needed more liquidity in their ETF product. For two, we had to give access to BlackRock into ourselves, obviously, for more, bigger product shelf, but better distribution through our branch channels. And our advisor channel. And then we had to manage the cost structure of the business. Yeah, exactly. And all the rebalancing, all the trading, therefore for cost reasons, for liquidity reasons, for customer choice and distribution. RBC ICRS was formed as the largest ETF company in Canada now. Now this sounds like more concentration though. I mean, is this something people should be concerned about? It's a scale game. It's a scale game where the largest asset manager in Canada, active asset manager in Canada, BlackRock is one of the largest in the world, and we felt we had to bring those two businesses together, given the scale required to compete in the ETF. I think they, exactly. It's technology and scale that are coming together. Yeah, exactly. You have to have both now. Yeah. Well, that's on the one hand. And then on the other hand, you have this. So it's more concentrated at this level, but more diffused at the exchange level. Right. So the first thing I would say, I just wanted to go back to this advisor point. Yeah, sure. So I do think the role of advisors is fundamentally changing, but I don't think it'll necessarily disappear. I just want to say, I think that, I think that they're fundamentally changing into more like long-term life advisors, right? So how do you allocate your assets over the long-term to manage for your life events, to manage through? And not every person in the world has a full understanding of how to interact with the markets and how to make those choices by themselves. And so I do believe that, again, those humans and those advisors can really help manage people through different life events and asset allocation decisions. However, individual stock decisions or individual fund decisions and other things, that at that point, then it's likely that these large-scaled advisory firms and wealth management firms will use technology to make those individual choices and to allow for them to pick which of those funds within the universe of funds in this strategy will work for them. So again, it's going to be a combination of that advisor giving longer-term advice and the algorithms and the, I would say, that the technology providing those shorter-term decisions around where to put your assets in the short-term. So I believe that you're right that the role of the advisor has to change. The distribution channels continue to be an area that will be disrupted, but I also believe that there will always be humans to help other humans make these longer-term decisions. In terms of the markets, though, the markets, so the rules that came into place in the United States that diffuse the markets or create a competition around the markets that occurred in 2006. So we've been operating in this environment for 13 years. This is not a new thing for us. It has resulted in a lot of technological advancement, us always having to get better at what we do, providing all of our clients better service, better capabilities, frankly, faster trading, more resilient trading, resolving the issues that led up to the flash crash and making sure we don't have those types of things again. Those are the things that resiliency that we've been baked into the system that I think actually are beneficial from the fact that we have to compete.  I mean, of course I would, but I actually think it's, competition is good and healthy, but sometimes it can get a little bit too far where the client ends up not benefiting from that level of competition. Well, speaking of competition, there was an announcement I guess a couple weeks ago that a group of securities firms are gonna start another new exchange. Another one. And this, of course, what is it? Brad Casayama started IEX a number of years ago and then these guys, I think it includes Citadel and some Wall Street firms. So what is your take on this? I mean, does the world need another exchange? Adina? Well, this particular one is only on the sell side. So it only has sell side firms. I think that in terms of, so first of all, this is the sixth time our sell side clients have chosen to try to create competition. So it's nothing new to us. And as we said, we have about 35% of the trading in the US equities today occurs off exchange altogether. And it incurs in firms like Citadel and Vertu and some of the other banks that are involved in this. They are facing new disclosure obligations in terms of how they manage order flow from their clients. The SEC is establishing new disclosure obligations to confirm that they are not conflicted in their broker obligations and fiduciary obligations to their clients. So the exchange may be an attempt for them to manage that conflict more successfully in terms of some of the internalized flow that they may move onto this exchange. The other thing is that we, our view is always that if there's a new competitor first of all, obviously has an opportunity to make us better, but we are extremely equipped to manage that. And I do think competition has gotten harder. New entrants have gotten, have a harder road to hoe than they did before because we have gotten to be a better, we are better than we were 10 years ago, right? So, and the experience for our clients is better than it was 10 years ago. So we will always be, you know, we're always happy to manage to new competition. They will also remain our largest clients, always. Those firms are our largest clients and we treat them that way. And that's always been the relationship we've had with them. You know, the gaps, I know Brad pretty well. He's an XRBC and he's the subject of Michael Lewis's Flash Boys. Right, good context. Thank you. Michael's book. The two problems that he is trying to address through IEX and through the algos is one, the whole maker-taker model with the broker-dealers, right? So data for large rebates for trade flow and he identified that as an issue. And then the second issue that he's trying to address is latency arbitrage that, you know, certain exchanges provide. So, you know, through his algos, he has got a speed bump that doesn't allow you to front-run or to have a latency arbitrage opportunity in your electronic trading. So I think, you know, that's where he's coming at it. I don't know if you're more familiar with it. Yeah, no, I think he has some of the issues. He's about a 2.5% market share though. So it hasn't, I mean, maybe he's happy with that. It's tough to launch, as you said. You know, a number of tried to launch, right?  And they're tough decisions to switch out of large exchanges. Yeah, I think that it's also, I think he came up with something very novel to try in the market. Most of his flow continues to be essentially what we call dark, meaning it's not a lit exchange. So about 2.5% market share, of which less than a percent is actually a lit exchange and the rest is done more through the midpoint markets. But the fact of the matter is 2.5% in the US markets is frankly a lot of volume. So he can be a pretty good business between 1.5% market share. With that size and he's meeting a niche need. I think that we try to focus on meeting the broader needs and we also have midpoint capabilities and other things to try to create, meet both the niche needs and the broad-based needs. And so it's a good competition. Great. We are gonna throw it open to questions in a little bit. Just wait, hold it for one second. Good, I'm glad we have some people chomping at the bit. That's good stuff. We'll get to you first then for sure. I wanna ask just Ron though a question. And that is, so active versus passive, right? Here and there. And then you've got short-term perspective, long-term perspective on investing. Does one match up with the other better? We break those. I'm gonna break those into the four pieces if we could. Yeah. On the active front, active to us is literally buying companies.  One is we buy them privately and own them, sit on their boards, change out management if we have to. And we do that for real estate. We do it for private equity. We do it for infrastructure, airports that we own, all that stuff. That to us is active management when people get in and roll up their sleeves and do all the hard work to make companies better, faster, quicker. And so that's one part of it. That is actually where humans get deeply involved in making things better. And that is the value creation proposition. In the world of indices and passive holdings of pure beta exposure, we don't spend a lot of time trying to beat the index from that perspective. Where we feel we have an edge, we will get out there and try and improve. But that typically, I can tell you in these markets, 99% of the time it is coming out of our private holdings where we have that capacity to pull that off. Where we feel we can do it in a more publicly traded kind of way other than getting our beta and our S&P exposure, S&P 500 exposure, is we've probably got about 12 billion invested in various hedge funds where there's lots of arbitrage and trading and bridge watering and all kinds of stuff going through. That's a good verb. I've never heard of that. So from that perspective, it's a, we look at that. And to us, it really does come down to we have to deliver a certain return profile, a particular risk profile. And a lot of our liabilities as well, which are very long duration liabilities, our teachers, 300,000 teachers in Canada, they live now, particularly the women because 75% of them are women, they live until they're 92 years of age. They actually collect pensions for 32 years while they only teach for 26 years. And so we've got to make sure that the money is there. And so through all of these sorts of things from our perspective, we've got to get the exposure, but we also know where we have an edge and where we don't have an edge. And that becomes a critically important conversation to us for us to have every single day. That's fascinating. If guys pick up on it. Sure. Go ahead, Bill. Ron's point, Andy. I couldn't agree more. I think, just to put a point on it, active management, which you defined as being engaged with the company, company building has shifted from public markets to private markets. And that's where the alpha has gone. I mean, in days past, good, high quality public active managers put people on the board, knew the company intimately, held them for five or 10 years, and that world has just shifted. It's gone to the private market. And institutional investors, and I think in the future, retail investors will want to capture some of that alpha that comes from company building, but it's happening in a different place. The public market investors are the passive investors. There's three big companies, for instance. It's becoming more passive and less active, but the active hasn't gone away. And Ron's point, it's just moved to the private market. Right. It suffers from short term. Where the passive, it's gone to private, but they're still active, but the kind of active going on in public passive markets is shifted. This is where, for those of you that read Larry Fink's letter the other day, this is exactly where the active part of it is gone. It's around what are you doing about climate change? What are you doing to make a better society? What kind of purpose does your company have? How are you benefiting the communities that your public market operates within? There's still a lot of engagement that's really gaining momentum around that particular part of the world, but in terms of operating companies and making companies better at that letter level, it's in the private space. And I would say it's not like there are no active managers left in the world. Right, right. There's still thousands of them, and there's still... 80% Yeah. And there's activist investors as well. There's still an enormous market. And the way we look at it is we say, well there's maybe $70 trillion in investable assets today, and there's probably gonna grow to 100 trillion anywhere between the next five and 10 years. So it's a growing pie. However, these alternative strategies, whether they are alternatives, true alternatives, or passive, are getting the... They're getting a lion's share of the increase and they're getting a lion's share on the inflows. It does not mean that there will not be active public investors out there. I think what it means though is that it's gonna be a much more competitive space for them. So scale's gonna matter and performance is gonna matter. And those are the two things that I think are gonna define the success of active managers going forward. Sometimes you're gonna get some niche active firms that are small and have these niche strategies and they do an extremely good job and their performance is excellent. And they will continue to gather assets. I think the large-scaled players where they can have economies of scale, they can bring in technologies, they can be more sophisticated, they know how to manage the markets, and they can have performance. I will tell you, they will continue to gather assets. There is a large middle though that where they're gonna have to find a way to differentiate themselves over time. I just, I'm sorry, we're just on fee compression. The actives are getting pushed hard on fees and they're losing talent. So it's a culling of the herd in a way. Yeah, and there'll be some that will gain scale and survive, absolutely, but it'll just be a smaller number. I wanna get to our audience. Can I jump in on one thing? One thing. I worry about the distortive and the inefficient allocation of capital as passive grows, and you're allocating based on index and your scale within the index, are you rewarding poor performance? It's always relative to index instead of an unabsorbed basis. Right, so are you misalloying capital? Does it make it harder for that smaller, mid-sized, public company to attract capital? Yeah. Well, I'd love to see some studies on that. I mean, I don't know if there are any. But that would be the trend, you think, as passive grows, it could reward poor performance. Let me ask, Adina, just one quick question. That's not actually really completely related to this, but you're here, so I gotta ask you, which is about the government shutdown in the United States and whether that's having an impact on the IPO business if you're seeing that. Sure, well, I'll just take a little context. The United States had about 250 IPOs last year, and I am proud to say that now is like 186 of them, so we are, we are really happy about that. Market share, market share. But at the same time, we had 22% higher number of index or of IPO applications at the end of 18 as we had at the end of 17. So there is a lot of demand for companies wanting to tap the public markets. So we were excited, I think what created a dampening effect on that was certainly the volatility of the fourth quarter. So some people were either thinking of going public in the fourth quarter and pushed to the first quarter or thinking of going public in the first quarter, maybe decided to push to the second quarter. But when you can't actually bring any IPOs to market in the month of January, then you're starting to squeeze essentially the pipeline into a shorter time horizon. And what we'd like to provide is for a nice spread out environment for companies to be able to tap the public markets, have their own day in the sun, be able to know that they will have access to investors. Investors will be ready to make that investment decision. And if you have fewer months in which to be able to do that, it means that the IPO environment could become more challenged. So it has been an issue because we have not really been able to bring companies public this month. And once their financials go stale in mid-February, then they might have to wait a month or two before they tap the public market. So it could be a full quarter. So a cascading effect. So not just during the shutdown itself, but has an implication beyond the shutdown potentially. It would because it recognized that once the SEC reopens, they're gonna have a huge backlog. And they're gonna have to get their way out of a backlog of applications. And they do real work. I just wanna say, we do partner with the SEC. They do really important work to assess these companies and whether the companies are just probably disclosing risk, their financials, their governance. We rely on that. And we then look at it as to whether or not they're meeting our listing standards. But it is a partnership. So when our partner's not there, it means that we can't do our job either. You certainly saw that in the Q4 results of JP Morgan and Morgan Stanley in the city, right? Very, very slow primary origination. Pretty good secondary equity trading and good hedging activity on derivatives kind of drove the day, but the primary origination business was very, very slow. All right, you've been waiting patiently. Now is your time. Thank you so much. Thank you. I actually only came to see Ms. Friedman live, but you have inspired me to ask a question. I am a tech pioneer in energy blockchain industry. And my question does not relate to crypto investing as in from a speculative currency investment, but what is more of a utility token and hence more analogous to a private equity investment? Have you had discussions with regulators and do you as an exchange or you as investors, how do you see the development of that option for investors? So registered, essentially utility, but registered to securities being traded, invested in in a more significant way. Well, I will answer first, but I think you had some really great perspective on that too. First of all, I'm pretty excited about the notion that the blockchain and the idea of these digital assets could become a real part of the market. And, but I also agree that the registration of those securities or whatever they are, so if it's a real estate asset, it may not be a security, but if the proper registration of those and defining the regulatory environment around those digital assets will be an important first step to making sure you're delivering something that has fairness to it for investors. But I do believe that the blockchain does create the opportunity for primary issuance in addition to secondary trading in these new digital assets. And the key is to put the right regulatory framework in place. And so we are working with actually members of the World Federation of Exchanges, other some of our partners around that. We've been invested, we've invested in a company called Symbion which is a smart contract company, technology company that allows for the primary issuance of these types of assets. We've been looking at the nasaic private market as a means for us to be able to create and distribute them. And so it is something that we're very focused on, but that regulatory framework is an important next step. And that I would say is still very early. The SEC is they're trying to define their role in that space. David, do you want to weigh in? I think when you're putting your money into an asset, it's the credibility of the person managing the asset. So you can use decentralized blockchain registry to record who owns the asset and transactions in the asset. But at the end of the day, if you're managing that asset, reputation and brand are really important. These ICO coin offers, where's your money going? I guess you have a decentralized registration of what you own. But at the end of the day, someone has got your money and they have to redeem it. And there's no disclosure. And there's no disclosure. So regulation has to wrap its arms around that, but certainly buyer beware to start. So I think you'll see big brands and big asset managers pave the way using their brands to do that. Bill, you had a point? Yeah, I think it's actually, so what Nina just said, I think it's going to be less relevant for the securities market, which are actually highly efficient, and for the alternative market. Because if you look at things like LP interests, loan documentation, very inefficient back-offices, very expensive to transfer ownership and record keeping, and you can really see the blockchain playing a major role. When we're transferring an LP interest from one investor to another, it's a very cumbersome process. And if you could do that through the blockchain, it could really make that efficient. So I'm looking for it more on leveraged loans, private credit. Private real estate. Real estate. Another area. Private equity. Private equity. And they wanted to decentralize and fraction ownership and leisure craft. And therefore you could have... Sign me up. Yeah. Again, we're back to the yachts. Liquid, right? Exactly. Because that's how society's moving, right? Rent-o. Part of ownership, temper ownership. And a two-way price discovery around that, right? So which we're very happy about. It's gotten more complicated. Question over here, right in the middle? Wait for a microphone and then over to you. Yes. There's a microphone going this way. Please. Thank you. Good morning. Assuming that algorithm are not emotional, will technology increase volatility or and reduce financial instability? Or just the opposite? That's a great fundamental question. I like that. And we didn't really get to that. You want to start with that one? Yeah. Go ahead. Go ahead. I'd love to hear your point of view first. Yeah. Okay, so the algorithm is objective. It takes the emotion out of investing. Often that is the Achilles' heel of investing, is selling at the highs or selling at the lows and buying at the highs. So that objectivity is fantastic. Provided what's buried inside that algorithm is in fact repeatable and it can continue to be trusted. If I take a glass of water and drop it on the floor a thousand times, chances are we can figure out what acceleration due to gravity is. But when you're in the middle of markets that are open loop nonlinear feedback systems being driven by all sorts of different timeframe groups of people, an algorithm that might work today is something that, I'm not talking process algorithms, I'm talking trading algorithms. Those trading algorithms inevitably have to be updated. In fact, I'll never forget sitting with, I don't know how many PhDs quant types building algorithms thinking that they'd ultimately find perpetual money making machines and every six months they're going back and redefining their algorithms because they didn't take something into account. So I think we have to be a little careful. There are some people that have sort of really found some interesting nuggets of information all comes back to the same story. An algorithm is nothing more than an abstraction in this model sense. It has found some piece of information that you can hang on to and the real question becomes how stable and how dependable is that information going forward through time. And in the event that it's no longer stable and that it does lead to mistrating or triggers certain buy and sell signals that no longer are relevant because the markets change, the structure's changed, then I think potentially, and we've seen this before, it can lead to, if it's big enough, instability in bits and pieces of trading. It can do that. Long term capital being an example where people thought they figured out an algorithm that was flawless and they said this model has a one in 500 years chance of blowing up but they were off by about 475 years, right? So the answer to your question is maybe I guess. That's it, right? Or yes, yeah. Yeah, and the other thing that happens with algorithms is behavior changes around them and therefore they have to retune because the behavior around them changes because it becomes, that algorithm has to change. I would say this, herd mentality can occur in whether it's humans making the decisions or whether it's machines making the decisions. And I think that when I look at volatility, there's actually a very good article in Bloomberg a few weeks ago that showed different periods of instability in the markets and how much volatility there was, whether it was in 1987 or in the 70s, obviously in the internet bubble and then in other periods of time. And actually what happened in the fourth quarter was not nearly as high in terms of the volatility in the markets and today as you've seen in these other kind of stress periods. So people were feeling a lot of stress and there was a herd mentality, kind of in periods of the day or in certain securities but that was just as much a human emotion around a divergence of views as to whether or not the economy's gonna continue to grow, whether or not the trade war will become real or not, what's gonna happen with interest rates and the changes in monetary policy. All of those things coming together to create a divergence of views as to the future of the economy, which then creates volatility because it's actually humans making judgments but then potentially coding them into algorithms.  I guess that's the best way to look at it. Question over here. It's a nice segue, Kastin Jona, European Commission. The previous panel I was in the sentence and it was a panelist and not the least said uninhibited capital flows are bad. We know that now. What was that? I'm sorry, say that again please. That was the quote of the panelist on this panel. Which was, I'm sorry. IMF. But what was the quote? What was the quote? That was a quote, yeah. What was the quote? What was it? We didn't hear it. Uninhibited capital flows are bad. Now we discussed and you discussed very much how to optimize capital flows and how technology can help you on this. What I would have expected actually is to hear also how technology can help to have impact and to connect this capital flow to what we're discussing here, which is the 17 sustainable development goals and how that could be brought together. Would you have a comment on that? I mean, I would say absolutely. It's actually an excellent question because we have not even touched on that point here today. We've talked much more about the technical aspects of the market. I think that when we look at impact, it's all about going back to the use of data. So if you, and unfortunately, I think a lot of what I would call impact oriented data is hard to find in a unified and uniform way to make it so you have enough iterations of it that you can actually use technology beneficially to drive an outcome, which I mean is every company might report their ESG results differently. And it's hard for an algorithm to take disparate information that's not uniform and not unified in the way that it's presented and try to find a conclusion out of it through technology alone. So it does, I think still, I think ESG investment and impact investment is still very much an element of judgment taking a lot of disparate information, collected in different ways in, I would say unstructured formats, and then talking to people. When I was talking to a woman yesterday who does a lot of impact investing, she was fascinating, but it was all about meeting management and having a conversation with management and understanding their intent and understanding where they're taking their business and how they're doing business. And that's still very much a dialogue. That's not something that technology can uncover all on its own. So I don't believe that technology is yet there to really help drive to impact investing it. But that's my personal opinion, but I'm not an investor. I think exactly, that's exactly, we come back to your opening statement, which is it starts with the human side of it. We engage with lots and lots of companies around their climate issues, their carbon footprint, diversity, women on boards. There's a ton and ton of things which we believe ultimately makes things better. Technology is starting to enable some of that, but it is the human interface that makes the biggest impact at this point in time. And that goes all the way through to companies disclosing, you mentioned SDG, and all the things that can come out of that. I think those are all very, very important issues that we do raise. But when we get with them, giving some sense of carbon footprint, what they're doing, what the big risks are to the organization is really what the conversations are always about. These are big issues and they're hard to get a handle on. And quite frankly, calculating carbon footprint in a lot of companies is not a simple thing to do as you all well know. And so technology is enabling many aspects of this thing. And I think there's enough asset owners that are committed to moving it forward. Absolutely, and we are, but that technology is gonna come after the human framework of it all has begun. You wanted to follow up very quickly? I just have a policy makers come in. This is where I come from. Where we have the debate in Europe now about sustainable finance action plan, which includes a taxonomy, which would generate exactly the data that technology could use then to also direct the flows or keep them going. So it's a broader partnership, we have to discuss that. Yeah, and we own the exchanges in the Nordics and we've actually set aside a set of standards that we've given out to all of our companies that they can voluntarily choose to disclose that does create that taxonomy. So it's important. I would just say we've made a lot of progress with machine learning and breaking down the bias. So we have an algorithms that read 7,000 periodicals every day and have different correlations to companies performance in ESG than star portfolio managers get. So I think you have to put the other way around and say data drives correlations and sometimes don't let the human overwrite happen. So I think machine learning is going to completely change our assumptions. I'm really glad that you asked that question because it really sort of did relate it to the wider purpose of why we're all here at WEF and it also spins it forward. So you're talking about why isn't this connected to this? I think Adina and Ron and David et al explain why. And I think it also speaks to maybe a great opportunity. So in other words, people looking for businesses and companies to start up and endeavors. I mean, this sounds like a really wide open field essentially. So I think that that's a really great place for us to end which is kind of a hopeful forward spinning way of understanding this somewhat arcane subject matter. And I think these guys really did an awesome job explaining it and helping sort of delve into all the different parts of it. So with that, we're going to wrap things up. And please join me in thanking Ron, Adina, Dave, and Bill. You guys were awesome. Really great. Thank you so much. Thank you all very much for watching. Thank you. It's great.


Hello and welcome to another quick tutorial on SD25 NFC channel. Today we'll be talking about the use of E-Design Suite. To go to E-Design Suite, simply go to the search box www.se.com and put in E-Design Suite and search for it. You click on E-Design Suite and start your design. Now we're going to do RF design and then we actually added a two-new tap beside the NFC design tap here. You can start with NFC tuning circuit for today. Normally this is an application that you can download and run as a standalone application on your computer. But now we put it on the network so that it can be run through the browser. It makes it a lot easier so you don't have to install the application every time. So here it gives you the matching network of the SD25 R3911B in differential configuration. Clicking on here you can scroll to different configuration. For example, you want the design using 3916 differential with auto antenna tuning. And here is the diagram that shows all the necessary components. And here you can also see that you can change the antenna metric and its characteristic so that you can arrive at different values for the capacitors. You can also try something like single-and application here. You can see now you only use RF01 and RFI1 which is input and output of the SD25 R3916. So here you go. Use these tools on the browser instead of using it as a standalone application. Thank you.


Good morning everybody it's my absolute pleasure to welcome you to the first webinar in our project changing research methods in covert 19 method suited or adapted to research in times of physical and social restrictions. We're going to be running for a couple of hours. And and if you are on social media and would like to tweet then we're using the hashtag covert ncrm as written on this front slide. My name is Rob Meckin. I'm a presidential fellow at the University of Manchester in the School of Social Sciences. And I've been on this project with two colleagues, Professor Melanie nine at the University of Southampton and Dr Andy coverdale also at the University of Southampton and they'll be introducing themselves. Slightly soon. So in terms of what we're going to do today. First of all, we'll have an introduction, an introductory and overview from the from myself and Andy. And then I'm going to invite a team, the in touch team and they're going to talk about adaptations and sensory material methods in covert 19. And then after five minutes of Q&A, I'll invite Poppy Gerard Abbott from the University of Edinburgh and she'll talk about transferring online value in covert 19 methodologies. Then we'll have a 10 minute break and then Melanie will take up a session on expressive methods and he will talk about survey methods and then I'll finish with secondary and secondary data analysis and then we'll have about 15 minutes of Q&A and then a brief next steps and closing. Obviously the timings are slightly out now that we've started a few minutes late but it gives you a rough idea of how we're going to go. So how we're running the meeting is we're running it as a webinar so all participants are muted in terms of your microphones and videos, partly because we're recording and the speakers have their videos and microphones enabled. Excuse me. And they've given consent for recording the session so any communication if you could use the chat function Mel and Andy are going to be surveying and responding to questions on the on the chat and in the Q&A sessions, they can check through what's come up and and we can put questions to the speakers at that time. So this is a part of NCRM we're affiliated with the National Center for Research Methods which is comprehensive training in research methods and NCRM is a UK wide organization funded by the Economic and Social Research Council, and it delivers resources and a comprehensive program of online and face to face training, including core and advanced quantitative qualitative digital creative and mixed research methods. I officially started at NCRM in January 2020, just as NCRM was funded for five more years in its fourth phase so it now ends in 2024. And my proposal, and my interest still is to understand interdisciplinary collaboration infrastructure and innovation in the context of methods both in the social and natural sciences. So why, why have we got this project and what's the webinar, what's the position of the webinar we're, we're obviously in a very difficult period with social and physical restrictions because of the COVID-19 pandemic, and the ways that that's unfolding and the kinds of societal and social changes that are happening, they're affecting our methods. And so a lot of the methods and practices that researchers have been doing and are comfortable with needed adapting need changing in order to keep research moving ongoing and keep progressing. There's been a huge amount of activity. In terms of blogs and support and crowdfunded documents and I'm sure lots of people will be aware for instance of Deborah Lupton's crowdsourced document which has been very useful to lots of people. So the background to the to the project and what we were interested in doing is is synthesizing some of that information and some of that support and providing a platform for sharing. So our project was had two main aims which were to synthesize the evidence available to the research community on how social research methods have been adapted for or may work in the pandemic conditions. To engage the research community within and beyond the Academy in learning and sharing positive methodological responses to and possibilities within the constraints of COVID-19 measures when conducting social research and just moving my head out of the way there. And so the project design. We, sorry, in terms of objectives we had, we wanted to stimulate sharing of the challenges and adaptations in terms of methods to conduct a rapid evidence review and a synthesis of emerging grey literature, and to produce and disseminate some guidance materials. And we had two work streams. The first was around the published evidence that was emerging, which is led by Melanie and a lot of the work was conducted by. Andy. So we have a rapid evidence review and a great literature review in that work stream. And in the engagement side, which is the side I was leading. We were looking at sharing experiences and resources through a series of workshops. And those two sides we were meeting every couple of weeks. And that's what we were, what we were doing and what we were finding in the different work streams. Mel and Andy came to, to all, all of the workshops between them. And so with them out of that we're making reports and co produced resources so lots of things were shared by participants in the workshops and guidance materials and these webinars as well, which this is the first as a second one in a couple of weeks. So I'd like to hand over to Andy briefly so he can just talk quickly about the rapid evidence review. Yeah, good morning there. Thanks, Rob. Good morning everyone so just a quick intro for me just to start with so I'm Andy Coverdale. I'm a research fellow at the Education School at the University of Southampton, and where I'm primarily doing participatory research with people with learning disabilities. And my main role on this project as as Rob mentioned was working with Mel on the the rapid evidence review which to talk about now. Next slide. Thanks, Rob. So, so yeah, developing our sort of protocol around this lots of explorative work and piloting of methods. You can see a summary of our searching for the literature that I won't go through all of that but you can have a look at some of the sort of the search terms that we used. I think the key thing with with rapid evidence reviews is that it's important to obviously adopt a very systematic and rigorous approach but also be prepared to make concessions and be flexible within the time constraints which in our case has been a six month project so for example I mean we set out search literature on comparable sort of crises like SARS and Ebola but that we found that wasn't particularly relevant. Thought about forwards and backwards in sort of citation searching that wasn't particularly effective in our 2020 sort of time span. You know we focused instead on like the hand searching of selected journals so it said it was very much about as I say that rigor but also being flexible around that as well. So on the next slide. I mean I won't go through these but you can see sort of the stages of filtering down the papers using inclusion exclusion criteria to get to those highly relevant and informative papers. We did reserve others that can help with the contextual information sort of help us map the literature more broadly and in terms of numbers I mean we're talking about roughly 800 papers or papers down to about 50 or 60 which is sort of ongoing so yeah and those with those final selected papers we did develop quite a rigorous quality criteria for evaluating them for the for the synthesis which is ongoing. We're still obviously doing that now to the final stages of the project. There was clear challenges with COVID and obviously how researchers have gone about creating solutions but we also found COVID has presented researchers with an impetus to create new research and adapt projects so interesting to obviously look at that dynamic and obviously we were very interested in how effective these examples have been as well. And then the final slide Rob. Thanks. So with a lot of the COVID related published literature is yet to come. But we did have examples of things being fast tracked, you know, and there's some examples I won't go through and you can see three examples there. Obviously we're evident from call for papers that there's more special issues to come this year but obviously the published literature is yet to catch up in many ways with obviously the typical literature sort of publishing cycle. And so lots more to come obviously this this year with with that. So it was important that we also tapped into the great literature. Obviously great opportunity to get things out fast. Rob's already mentioned Debra Lupton's crowdsource documents which was, which was widely distributed earlier in the year some really good blogs as well that we've come across that we've used as resources and obviously the great literature has informed and pointed us to some of the published literature to come. And yeah, and I would just add to again what Rob said you know throughout I think reviewing the literature the published and the great literature has has informed and been informed by the workshops and you know the wonderful people we've had on our workshops. And so the two strands the two work workflows have been very much interconnected. Thanks ever so much Andy that's that's great and I'll just give a little overview of the engagement work that we were doing so we ran eight online workshops from December to December last year. And these were on interviewing. That was particularly something that ESRC were interested the funder were interested in making sure that we got some guidance up quickly. And working with groups with additional challenges and participatory and deliberative methods we had a session on ethics and creative and sensory methods and then online ethnographic methods surveying longitudinal methods and finally we decided to have one on the 19 data and secondary data. And our aims were around knowledge exchange so we were inviting people who had had experience of adapting their methods in in the pandemic, either in an open project that have been running on, or they've been planning and working out how they were going to do a project on in the tangible conditions, and so they were offering support and sharing the different ways they found through the issues. And one of the key things that we developed was using a metaphor to explore people's experiences and using that as the foundation to kind of discuss and and talk about what people have been involved in and excuse me. And that's a sort of adapted a method from the steps center in Sussex, which is called rivers of life, where people represent their personal history as a as a river, and their engagement with the environment, and we turned this a little bit into the rivers of research. And people were participants were asked to create an image and think about their last year of their research experience, and to represent that in terms of a river and we had lots of lovely images and ways that people had taken that we offered a bit of guidance around kinds of features of waterways but people, you know, took it and played with it a lot in their own ways so we had lots of people using rivers as their own lives people traveling down rivers. People traveling up rivers we had dams and will pools and boats and all sorts, and, and different kinds of representations so by row drawings and collage and all sorts so it was a really nice way to to explore and it became turned into what had been a nice exercise became kind of the core of the, or a significant part of the workshops, which you know we had between five and 18 people at in the different workshops. And here's an image which was taken behind at the University of Surrey was gave us permission to share. Just to give you an indication of the kinds of things this is from her view as a PGR director and seeing PhD students navigating through the, through the, the river of the last year. So just the top line from the engagement really is there are lots of expertise and people are keen to share and discuss, and there are many researchers and similar situations who, whose expertise you can draw on, and, and seek out. We found that I'm partly because of the, the metaphorical method rivers kind of draw attention to pace and timing and speed. So that came out quite strongly in terms of how the long lockdown had affected methods and that the uncertainties that we're all facing. I need new research contingencies in terms of research design and how we're understanding how to how to deal with these kinds of uncertainties.