Welcome. I am Alison Johnstone MSP, the Presiding Officer of the Scottish Parliament and Chair of Scotland's Futures Forum, and I would like to welcome you all to this online edition of the Festival of Politics 2021 in partnership with Scotland's Futures Forum. This evening's event is in conversation with Professor Suzanne Simard. We are pleased that so many people are able to join us online this evening, and I look forward to hearing comments and questions from our audience as we get into our discussion. I am delighted to be joined tonight by Professor Suzanne Simard, who is author of Finding the Mother Tree. She is Professor of Forest Ecology in the University of British Columbia's Faculty of Forestry and has earned a global reputation for her research on tree connectivity and communication and its impact on the health and biodiversity of forests. Today, Professor Simard's work is taking a scientific orthodoxy and has inspired countless researchers, writers and filmmakers, as well as generating millions of views of her TED talks. Despite Professor Simard's groundbreaking discoveries, she was initially dismissed by the male-dominated scientific establishment of the day. It would be years until the world took her ideas seriously. In 1997, a landmark paper in the journal Nature coined the term Wood-wide Web to describe her work, marking the dawn of a new era of ecological awareness. Her latest book, Finding the Mother Tree, has been described as a scientific memoir as gripping as any HBO drama. Welcome, Professor Suzanne Simard. There will be an opportunity for our online audience to put questions and views to the Professor throughout the event. If you would like to make a contribution, please enter your question or comment into the question and answer box. Be grateful if you could state your first name and where you are from and will aim to get through as many as possible. I am going to begin with a question directly to the Professor. I understand that you hail from a family who are some of the original settlers in British Columbia. Can you imagine a life without having grown up alongside trees? I can't actually. It's true. I grew up in what we call the inland rainforests of British Columbia, which are these amazing forests, huge trees, cedars and hemlocks. They're iconic forests. I've always had them around me. I have had, on occasion, the opportunity to travel to different countries. When I was in my 20s, I went back as a backpack tourist to Asia. I just couldn't wait to get back home. After a while away, I missed the mountains. I missed the forests. I guess everybody would miss their own home. It was the trees and forests and mountains that, when I finally got back home, I felt so much more at peace. They're in my blood and bones and DNA, and I can't imagine not having grown up among them. For those who are not familiar with your revolutionary work, could you give a brief summation of how you managed to change the world's views of trees? I know that's quite a big question. It's quite humbling. I don't often think of myself that way, but it started out just like everybody trying to work in the area that you're interested in. Of course, forests were, as we mentioned, just part of who I was. I followed in my grandfather's footsteps and great grandfather, who were horse loggers. I became a forester. As a young girl, of course, I couldn't do the heavy, dangerous work. This wasn't part of the picture. I did get involved in the field of civil culture, which is how to grow trees. I observed that we were taking our iconic old-growth forests and converting them into very simplified plantations. I was seeing that these were not very healthy forests. They weren't diverse. They were really at risk of infections by diseases and insects. That's what got me going on trying to figure what we were doing wrong. I felt like we were really unraveling these forests through our forest practices. When I was seeing these pathogens spreading through the trees, I started looking below ground at what we might have disconnected. That's when I discovered in following some of the amazing research already done in the UK in laboratories in the meadows that plants could be connected by mycorrhizas. Mycorrhizas are these fungi that are basically obligate symbionts with almost all plants around the world. Certainly all trees, all tree species depend on them, where the tree provides photosynthate and energy to the fungus which grows through the soil and picks up nutrients in water and delivers back to the tree so the tree can meet its needs for nutrients. In discovering that these trees could actually be connected by these fungi, that really showed to me that when we were converting our old forests to plantations or even just deforesting, that we were really disrupting this network and that provided avenues for these infections, these infestations. It was really degrading the integrity of the forest. That's how it got started. It was from a truly heartful place of being very disturbed by what was going on in our forests. I wish I could say that things are different now, but we still have these same issues of clear cutting old forests, planting them to simplified plantations and disrupting these networks in fairly major ways. Our forests are still not as resilient as they could be. That's how it got started. After many more years since that first nature paper came out, my lab has been full of graduate students who have expanded on the story of what these networks do and how important are they and how do they help forests recover. I'm sure we'll talk about some of that as we go along. I'm heartened to hear that you mentioned that your lab has expanded and that you're working with graduate students who have obviously been inspired by your work. Clearly, as I mentioned in introduction, you faced a lot of scepticism in that male-dominated scientific community when you began this work. Did you ever lose faith in that period? We're still not seeing the change that you would hope to see. Do you ever despair that the world is just not waking up to the ecological reality? I mean it's taken a long time for sure and it's been a big struggle, but now I feel like there are so many things converging on that people are really opening up their minds and accepting these findings. Things like we're in a climate crisis. We're in a pandemic. People are going to be migrating to get to and from these different crises that are going on. I see that in my own country. We're already undergoing climate migrations in small pockets. I think that reality has really hit home. I've read that almost 80 per cent of people on the earth have experienced in their own personal way some aspect of climate change. Of course, we've all been exposed to the pandemic. People are waking up because they need to. We're in a crisis and looking for solutions. There are solutions. With regard to those solutions, how do we proactively heal forests from human impact? That's the big question, but we know so much. I think I want to send this hopeful message out there that we do have a lot of good practices that we can apply to help mitigate these problems. For example, I can give you three ideas. One of them is that in large parts of the world, whether it's the west coast of North America or the boreal forest across Canada, Russia and Scandinavian countries, or the tropical forests across South America and Indonesia and so on, those places, there's still a lot of old growth forests left. Well, I should say there's still intact old growth forests left. I'm going to backtrack it. There's not as much as we've had. We've lost about a third of those forests worldwide from land use change and other practices, but we still have a substantial amount left. The first thing that we need to do is protect those forests because those forests, intact forests are actually absorb or store about 70 to 80% of our terrestrial carbon. If we keep these forests intact and then restore our remaining forests, scientists are saying that these kind of practices can actually absorb about 35% of greenhouse gas emissions. It really is in our favor to really focus on this. Of course, the other 65% comes from things like mitigating fossil fuel consumption, decarbonizing our energy sector as quickly as we can. Of course, cement production we also know is a major issue. Those things aside, in forests, conserving old growth forests or old primary forests is the first thing that we should do because there's no amount of planting trees, which I know that people really want to do, but planting trees is not going to add up to the same mitigation that's provided by saving what we have still remaining because these old forests store a ton of carbon, tons and tons of carbon. The second thing that we can do is to go into forests that are damaged and restore them. That does involve tree planting. It also involves favouring the advanced growth of older trees, letting them grow out through their whole lifespan instead of cutting them down in 50 or 60 years, but just actually letting them grow old so that they can accumulate carbon in the ecosystems. That's the second thing. The third thing that we can do is that on areas that were forested at one time that could support forests, we can convert those damaged ecosystems back to forests. That's called aforestation. Really it's like making a new forest. I think that there's definitely room for that around the world and I think that's the hope with these plants for planting a trillion trees is to try to get some of those places back into forests. The last thing I'm going to mention is that when we are going to still need wood in the future is when we do harvest trees, do it sustainably to make sure we leave the legacies in the forest so that they can be bootstrapped into the next generation. We need the gene banks, for example, of the old trees. We need to leave these old trees behind and then take out a few trees as we need them, but this is a major shift away from the current practices, which are predominantly to clear cut forests and plant them to monocultures. Thank you. So many points there that I'd like to pick up on, but I'm going to a question from Stu and Stu asks, how does one get into silver culture? Do you need more people specialising in this skill? And he'd also like to know how soon can he start? Thank you, Stu. That's a great question and I encourage you with all my heart to go and do that. We don't have enough people working in silver culture and really this is, you've hit on one of the major solutions, I think, and that is that people over time over the last 100 years have moved away from the land and into cities and that means that we have less knowledge and fewer people looking and knowing the land. And really, we need to understand, you know, we need to understand the places that we're in. For example, as a civil culturalist growing trees, you would need to know, you know, what are the native species there or what species can you introduce there if there are no native species that are left? And how do you manage them? And how do you keep them adaptive and still resilient as climate is changing? Because that change is going to be continuous and episodic as well. There's going to be episodes where it's more severe than other times. And that's going to require healing of the land and to heal the land, we have to have eyes on the ground. We have to have real practitioners out there. And so that was where the civil culturalists comes in. And the civil culturalists, it's a position that is, you know, needs to be more and more highly valued where you really are using all of your mind, all your understanding to bring to bear on this global issue of how do we keep our planet green and continuing to take up carbon. And just one more comment I'll make is that, you know, over the last, I've watched over the last 20 years or more that there's been fewer and fewer civil culturalists, there's fewer and fewer people on the land as we move towards a more digital world. And we also move more towards genetic or quick solutions. I shouldn't say genetic is a quick solution, but there's been more of a focus on that than in ecology. And so I encourage anybody who's listening, especially that you're interested in this too, to pursue this because we need you. And there are forestry schools around the world that train in this area and you can easily find them. I think there's a forestry school in Aberdeen in Scotland, actually, if you were to stay home. But certainly I work at UBC, which has an excellent forestry faculty as well. And I think these faculties have undergone a huge changes because of the, you know, of the, I don't know, I would say this shift, this cultural scientific shift has happened over the last 20 years, but it's going to shift back again as our needs become more evident and we need more experts like you're going to be. I hope in the future. Thank you. I think that was a very empowering and inspirational call there for those who'd like to get involved. You know, I just can't think of yet more useful work. I'm sure you really appreciated your response. I have a question from Vivian. Can the fungi so essential to intact forests be replaced or introduced? And how quickly can anything even resembling a functioning forest be built up? That's a great question. It depends on where you're starting from. But let's say you're in Scotland. I've been to your beautiful country a couple of times and I love it. You know, I think that at one time Scotland and you've got remnants of these forests was forested. And then there was a sort of a huge deforestation of events or events over time of conversion to agricultural land. And in that sort of conversion, you do have a wholesale shift in the below ground fungal communities. At least we've learned that from our forests in British Columbia where forests are converted to or we lose the sort of like the foothold of the forest on the land. So what we find is that in a forest, for example, that might have let's just say a baseline of 100 fungal species all different with their different niches in a forest. That's about on average what there is in our native forests here. When you convert them to say a grassland or a simplified community, the first thing that you do is you lose a lot of those native native fungal species that are associated with the trees that were there. A lot of them are very specific to tree species or at least specific to genera or families of trees. And heathlands or grasslands form a whole different suite of mycorrhizal fungi, these helper fungi than trees. They tend to be what we call our busketer mycorrhizes, which don't really associate with trees unless there are some trees that do like the yew tree and the willows and cottonwoods, cedars, cyprus, they form our busketer mycorrhizes, but the pines and birches and oaks all form this other group called ecto mycorrhizes. So if the conversion has happened over like for a long period of time, you would have to bring those ecto mycorrhizes back into the community to get trees established again. And there is a long history through the world of bringing trees into these kinds of communities or even migrating things like Douglas fir has been migrated over to New Zealand and Australia and Chile and Scotland I think. And what's happened is that at first these migrations failed because the mycorrhiza community wasn't there to help them along. And so once people figured out that they needed the soil inoculum to go along with it, they started introducing the inoculum along with it, and that really allowed these trees to get going. So that is, we have a good history that we can do that, right? We can introduce inoculum and you can purchase inoculum. You can also bring it from an intact forest nearby. For example, I know in Scotland you have places where there's still forests left and you can actually re inoculate areas that you're trying to a forest or a re forest that have been in fallow for a long period of time. I've successfully done this in lots and lots of experiments myself in trying to restore grasslands or even restoring old mine spills where we've brought in migrated or transferred soil from intact forests and really the seedlings get going and they do really well. Thank you. I have another question from Ian in Creef this time. When reforesting, it would seem that diversity may be the key to combating the numerous pathogens that are attacking our trees. Could we plant no more than 20 per cent of any single species in a reforested area with maybe the young trees of each species being sourced from different areas? That's Ian's question. Yeah, Ian, that's a great question. Really, this kind of harkens back to Stu's question about knowing your land and knowing what species belong where. I think I would be hesitant to put a number on the proportion, like 20 per cent, it depends on what belongs there to begin with, what's well adapted to that location. For example, in some of our forests in Canada, we have huge areas of land that are predominantly lodgepole pine with some aspen and spruce mixed in, and it comprises naturally about a high proportion of the community over 50 per cent. To say, oh, we're only going to have 20 per cent lodgepole pine, it would actually not be good for those ecosystems because that's the species that's well adapted. So you really have to know your land, what belongs there, the kinds of disturbances that are coming along, but certainly having a diversity of species, what naturally fits is what you would aim for. So understanding the natural patterns that were out there prior to or in similar areas, and then you're absolutely right that that diversity is absolutely essential. Once you match that properly, it's absolutely essential for keeping forests healthy and resilient and resistant to infections and infestations. So there's a couple of ways I can describe this, but one is if you have a diversity of species, let's say you have 10 species instead of just one of trees, then if there is a specific insect that is specifically attacks or infests that one species that maybe you've planted over 100 per cent of the area, if that insect or pathogen takes off or becomes epidemic, then it will take out that one whole forest if it's only comprised of one species. If there's 10, and that's only comprising 10 per cent of the population or of the community, then you'll still have all those other trees left that are not susceptible to that particular pest or herbivore or pathogen. Another thing is diversity, this works in so many ways, but diversity also increases resilience or resistance of individual trees to infection because they're healthier and they're growing better because they have a community of companion trees and plants. So in ecological theory, we call this niche complementarity, where species complement each other. You know that some species will have the ability, for example, to fix nitrogen and that improves the nutrition of all the trees around them. When the nutrition of the trees is high, they tend to have more resistance to diseases, just like we humans do as well. When we're healthy, we don't get flus and we don't get sick as much. It's the same thing in forests. So that diversity is really, really important in keeping the health of the forest up. I have got more things on this, but I think I'll wait for the next question and try to build that in. Thank you very much and the next question comes from Mark. It is, how widely are your ideas about good forest management now accepted? Is there still a significant challenge in getting those ideas accepted? What are any barriers and are there any areas or countries of best practice that you would highlight? Yeah, so many great questions. I would say where I'm from in Canada and I would venture that this is true around the world just because we've moved economically towards the concentration of ownership of wealth in the hands of a few. And that's true in forestry. So, for example, where I'm from in British Columbia, we used to have many, many mills and small companies and they've merged into five major companies now. And so they basically have jurisdiction over huge amounts of land or resources. They have licenses to harvest huge areas of forest. And because it's economically driven, instead of ecologically driven, they tend to go for the cheapest, most expedient way to reforest, to log for one and then to reforest. So the logging by and large is still clear cut logging where they don't leave big old trees behind, which is what my research shows that you should be keeping old trees so that they have a diversity of fungi associated with them, that they connect the forest, that they form the nucleus around which forest can recover. And so the cutting patterns haven't changed. And then how we reforest, we are still trying to reduce diversity in forests by trying to concentrate resources on those individual trees in the monoculture or in the planted forest with the idea that all these other plants are competitors rather than part of a community. And so I would say, you know, no, things haven't changed. In fact, in some ways they've gotten worse as we've gone down this economic model of concentration of wealth and ownership. And we really do need to diversify that just like in, you know, economics and ecology, both need diversification so that we can have a diversity of ideas and practices put in place that will make a more diverse landscape. So, yeah, it's, there's a long ways to go, but it's hopeful because, you know, more and more people are understanding how crucial it is that we do transform forestry practices that we do need to, you know, conserve and restore and an a forest that we need to figure out better ways of leaving the legacies behind an ecosystem so that they can become healthy forests in the future so that there are good gene banks. There are good, there's good soil left. There's still a lot of carbon and nutrients in the soil. We know now we really need to focus on doing this. And it's seeping into public awareness. And so I think that once that awareness is there and people are feeling the crunch and the pressure from climate change that they'll pressure their governments to try these different methods that are much more holistic and ecologically sound. And I can't remember the other parts of the question. Yeah, I think that was a very fulsome response. Thank you very much. I'm just wondering, can you tell us more about your current work on whether trees can distinguish us as human beings? What do trees know? Yeah, trees know a lot. Holy. So, you know, trees actually are evolutionarily much older than us. And we, you know, we all kind of evolve from the same basic genetic material back, you know, millions of years ago, four million years ago. And so a lot of the, you know, the genetic makeup of us comes comes from the genetic makeup of our predecessors of our ancestors, which includes the trees. And so because of that legacy, because of that gene pool, we share a lot of things in common. And so let me just talk about some of those things. So one thing, you know, you know how humans were very social creatures, right? We live in communities and we specialize. We have, you know, we have politicians and we have bakers and we have school teachers and, you know, the whole complex system that makes a country or a community work. It's the same thing in forests that, you know, trees growing communities as well. They're very social. They don't grow by themselves. They're a consortium of many species, many individuals, and even across different kingdoms of organisms. For example, the symbiosis between fungi and trees is the community is that symphony of organisms that really makes a forest healthy and vibrant. Yeah. So, and so what do trees do? How do they do this? Well, one of the things they do is they connect below ground through mycorrhizal fungi. And I wouldn't say it's not just the fungi. They're connected and have pathways or networks with many other creatures. It's just that the fungi are really cool because we can actually see these networks. We can see the trees linked together. We can make maps of these networks. But bacteria also do this. They form biofilms and they connect individuals through biofilms right along the mycorrhizal networks, for example. And through those networks, the microbial networks, the fungal networks, is a lot of communication going on. So trees will transmit resources back and forth between them according to who needs more. So they'll transmit carbon and nitrogen and water and phosphorus. And then our later studies are they also transmit information about things like identity. And I think that gets to your question about perception of identity. Who's around me? So the trees perceive their neighbors. They know who is related and who is not in the plant world. We've done lots of experiments to show this. And they respond differently than neighbors respond according to the identity of their neighbor. And they send messages back and forth. And one of the things we found out is that old trees will favor their kin if they recognize kin that seeds from their own cone crops, for example. As for people, certainly when we do experiments like we purposely injure a tree or shade a tree or pull its needles off, trees go into an instant response. They change their biochemistry almost instantly. They start to go through all kinds of hormonal responses. If it's an injury, like if I pull the leaf off of a tree or a needle, it will trigger these defense responses in the trees. They'll start to make enzymes and monoturpings to defend themselves against this. And what we found is that these trees through their mycorrhizal networks also transmit this information about defense to their neighbors. And then the neighbors upregulate their own defense machinery and they defend themselves. And so, yes, the fact that we can measure this when we injure a tree tells us that, yes, they, of course, are responding and are to us. They have a way of perceiving us just like they do when, for example, a squirrel clips off a cone or anything that might feed on the bark or rip off needles. Or just any animal or organism that does that also elicits these kinds of responses. And so, yes, so they can distinguish whether, for example, a caterpillar eats a leaf or a person pulls off the leaf. They have a different degree and amount and type of biochemical response to that. So, yes, it's intriguing, it's interesting, it's not something that I've directly sought out to say, can they perceive us? But to our experimentation in different ways, I understand that they are able to distinguish. That's incredible, you know, you're saying they'll know if it's a squirrel or if it's a person. Just that level of awareness. So it must be hugely traumatic if we see large-scale logging of an ancient forest. You know, we've seen some documentaries on TV here in recent weeks just showing what's happening with regards to clearing sites for meat production and so on. We're seeing almost straight lines, right angles of ancient forest beside nothing at the moment. That must be very, very traumatic for those forests. Is that what you're telling us? It is. It's traumatic. Of course, by clear-cutting trees, you're killing the trees. There's no two ways about it. Once you take off the photosynthetic crown, cut the tree down, the root system dies within us very, very quickly. It's basically you've killed the tree. Of course, the tree is going to respond to its own death. So, yes, it is traumatic. The other thing that's traumatic about it is that there are ripple effects all through the ecosystem. So it's not just the tree that goes, but all of the symbionts, the fungi, the bacteria, the soil food web also turns over at changes. So it'll change from what it was in the forest, which would be very diverse and highly tightly coordinated in a complex system to kind of a more weedy kind of environment. Where a lot of weedy species come in, fungi that are easy to grow on grasses or thistles or whatever the invasive plants might be. So the whole soil food web changes over. Along with that goes a change in the soil carbon pools, nitrogen pools, the water. So, for example, just to go through an explanation of what happens, let's say you have a forest, cut down all the trees. Because you no longer have a crown that's photosynthesizing anymore, you know, the canopy. It's also not transpiring anymore because photosynthesis goes along with transpiration. That's the emission of water into the atmosphere. And so the area, or at least the canopy will be dried out because there's no transpiration. So what happens is that water stays in the soil and the water table rises up. So when the water table rises up, you can get flooding. The other thing that happens is when the root systems die back that the pores in the soil, you know, the little holes in the soil. That's where all the bugs like live in and they get air in the soil. That's where soil water is regulated. Those collapse in a lot of cases. And so in the collapsing soil and the rising water table, this can result in runoff. And so you can get erosion in those cases. The other thing that happens is that, you know, you take the trees off and those trees are turned into different forest products like toilet paper or paper or all parts sometimes into long-term wood products like houses. But the other part that's affected is what's below ground. And so as this water table is rising up, the soil is also heating up because there's more sunlight coming onto the forest floor or onto the ground. And so decomposition also speeds up. So you start to get, you know, these soil organisms will start because it's warmer. Their life cycle speed up. And so they start to decompose the organic material that's in the soil. And this causes even more emissions of CO2 and the reduction of the carbon stocks in the soil. And what we have found just speaking of carbon stocks, which of course we're all very interested in because of climate change and greenhouse gases. What we found in our forests in British Columbia is that when they clear cut, we actually lose a lot of that soil carbon just from, you know, just from cutting down the forest. A lot of that forest soil gets moved around or disturbed and it increases decomposition. And we found that we are losing between 60% and 80% of forest floor carbon right along with clear cutting. And so the consequences are huge, right? So it affects the hydrology, as I mentioned. It affects carbon pools and it affects the nutrient availability as well to the regrowing forest. So yeah, there's lots of downstream ripple effects from cutting down a forest. Yeah, thank you very much indeed. I think that this sort of leads us on to another question from Ian and Creef who's asking and they answer me, seem obvious, but I'd be very grateful for your comprehensive views. Should we shift, therefore, from clear felling to continuous cover felling, ie thinning or small group felling? Yeah, I agree with that. But in general, that's right. But one of the cautionary principles is not to do the same thing everywhere. So we need a variety in the cover that we leave on the landscape and it needs to be matched with local site conditions. And so certainly what we're doing, I have a big project in British Columbia that crosses a climate gradient that's the size of Denmark. And we're trying different ways of group selection or group felling and leaving groups of trees.  So if it's a really moderate, more sort of very rich environment with like high rainfall, it's nice and warm, then you can actually leave fewer trees just to have the same effect is in a more poor ecosystem that's colder where you would want to leave groups of trees to protect the new seedlings coming up. So the message there is don't do the same thing everywhere but make sure you're matching it to the site. Conditions, another cautionary principle about continuous cover forestry. Again, like we don't want to do the same thing everywhere in some places, it's not going to match up very well. So first, just to give an example, continuous cover forestry means that you have canopy that's still intact. And in some areas, you actually need more open spaces for some species to regenerate. So things like pines, pines need more space. And so if you want to encourage pine growth or anything that's shade intolerant, you need to create bigger gaps than if you are growing back shade tolerant species like in my case in my world it would be cedar trees or maybe in Europe it would be beach trees. You would want to leave a more closed canopy for regeneration of those shade tolerant species. And so you can work with canopy cover to guide the development of the next forest and the species composition, which is really sensitive to the amount of shade that is being cast over them as a consequence of your cutting patterns. Thank you. I have another question from Stu who is asking, what sort of understanding of some of the concepts you've been discussing was already known to indigenous cultures in Canada or elsewhere? Is this something you look at? It is definitely something I look at and so this idea of connection, which is what really I think people really, we all feel that this is intuitively makes sense and of course scientifically we know that this exists now. We've used western scientific tools to understand these solid connections, these internet below ground, but in talking to Aboriginal people and even in some of the early writings in oral history there was an understanding already that these fungal networks existed in the soil and that they supported, they were the foundation of the forest. And these fungal networks were treated with a great deal of respect, as were the trees.  We're of the same genetic origins and whereas in the western world we've tended to separate ourselves from nature and think that we're different, but we're actually not different. We're just, you know, an interdependent with the whole ecosystem and that is really is the worldview of many of indigenous cultures worldwide. And so, yeah, that understanding is well known. It's in language. It's in ancient languages of words that describe this kind of connection, what it means. The English language doesn't include so much of that, but in the cultures around where I live, every single one of the hundreds of languages here have words for this idea that we're all connected together. And I continue to work. I work quite closely with Aboriginal people, indigenous people in Canada to try to reinvigorate our forests based on this idea of connection rather than disconnection, which is what we've been doing industrially for quite a long time now. Can I just ask—obviously we've been speaking about your book, Finding the Mother Tree—could you perhaps elaborate on the importance of the mother tree's role in looking after the forest? Yeah, yeah, I can.  So it really is about the role, the role of the elder of the mother tree in the forest that's so important.  Well, they actually are the biggest and oldest trees in forests, or if your forest is all of the same age, it would be just simply the biggest trees. And the reason that they're so important is because they have, you know, they have big crowns, a big canopy, they photosynthesize a lot compared to smaller neighbours. A lot of that energy ends up, or at least, you know, between 30% and 80% of that energy, depending on the ecosystem, ends up being transmitted into the soil and into the roots and into the mycorrhizal networks. And so they are the big connectors, they've got more roots, they've got more points of connection, and they connect to many of the other trees in the forest. In making a map of our networks in British Columbia, we found that, you know, mother trees can be connected to 80% of the other trees in the forest. So they really are the nucleus. And then, through our experimentation, we found that these old trees will, in forming these connections, they provide like a great platform for new seedlings to get going on.  The other thing that we found is that these old trees will transmit nutrients and water and carbon directly into seedlings to help them, give them a head start, to subsidize them until they have enough leaf area themselves to photosynthesize and support themselves.  They all are perceiving this from each other. And there's another important point that I wanted to make about the mother tree. So the mother tree really is mothering, in a sense, the new generations of seedlings coming up. And these old trees again also recognize which ones are their own kin, which is brand new research coming out of, you know, labs in North America that trees recognize kin and they actually can support their kin through transmission of carbon to the mycorrhizal networks and to increase the nutrition of their own offspring. And, of course, that fits in with their ideas of natural selection, you know, that they would favor their own genes so they can carry on to the next generations. And so, yeah, so that's what led us to calling them mother trees is just because of that, you know, that nurturing aspect of the old trees in the forest and how important these elders are in their role in doing that. Thank you very much. That was very illuminating. Can I just ask on forestry? I'm just interested in your views as to the gendered nature of forestry as a profession. Yeah, what's happening out there on the ground? Yeah, I mean, it's changed dramatically over the span of my lifetime when I started in forestry. Of course, I described how I came from a forestry family, which of course was male dominated. My grandmother's were not out there chopping down trees. They were at home looking after the kids and milking the cows and, you know, keeping the home stable while the men went out and logged.  And so, of course, their viewpoints are what shaped those things, the science and then the practice. And so when when women started coming into the field, which, you know, I entered in about 1980, there are very few of us. Most of the research, you know, very little have been done by women. And so that perspective was missing. And I think that now that there are more girls coming into forestry, there is as many girls in forestry now as boys. And that's a good thing because then there's a diversity of understanding and how we look at forests. So, for example, this idea of connection and collaboration, you know, that was very much a that's a very much a female point of view. And it took female females to do, you know, to carry that research forward. Whereas before, you know, it was in the practices till still today are heavily dominated by the idea that trees only compete with each other. And that shaped forestry practices in dramatic ways and competition was very much like the thinking of men and how forests were structured. They didn't really think about this collaborative social aspect of forests. And so growing, you know, plantations so that the trees are spaced apart uniformly that they grow by themselves and compete for light and water. And that's, you know, it's all about the individual. Now we've changed that idea to know that, you know, that actually forests thrive in their social communities. But it really did took it took a lot of women in the field to change that perspective. But and it's not trivial, you know, that that perspective has not been there for so long. And because, yeah, like I said, like forestry practices are completely shaped by the idea that competition reigns supreme. How we plant trees, how we space them, how we weed them out, how we reduce the diversity in plantations to try to concentrate resources in those individual trees that will be the next crop. All of that is based on this idea that competition is what matters the most. And now we know that's not true. So that it's only one aspect of the many ways that trees interact and communicate with each other. So it's a good thing that, you know, there's a diversity of people working in the field, you know, that more women are getting involved. And more, and, you know, a diversity of cultures as well. You know, the idea of oneness and collaboration was long, you know, as we talked about before embedded in indigenous worldviews. We need to get that back. We need to incorporate those viewpoints into our practices now, because those are sustainable practices. And now I'm remembering one of the questions back was, are there examples in the world of where we can see where this works and there are. Two examples come to mind. One is the Menominee Forest in northern United States. I think it's in Wisconsin where the Menominee Nation has done selective harvesting of their forests for 100 years or more.  It's still a thriving productive forest. Another example is the Wildwood Forest on the east side of Vancouver Island. Another selectively harvested forest that is producing even more basal area or wood than it was before. So it definitely can be done to use these perspectives of connection of society, of leaving the elders of more continuous canopy cover or using partial retention. These ideas work and we've got lots of evidence around the world. Thank you. As someone who's obviously very happy working in forests, I'd like to ask how you feel about your work inspiring the world of film, such as the film Avatar, which apparently borrowed aspects of your work. And now we hear of the Amy Adams biopic of your life. Are you comfortable with that world of entertainment? I suppose it's got real power to carry these messages far and wide. It sure does. So far, so good, right? Yes, there was Avatar. Although, you know, back when they made that movie, I was not really, I wasn't directly consulted on it, but they read my papers and so that did influence them in that way. And, you know, James, or sorry, Richard Powers book, The Overstory, you know, with the main character, one of the main characters, Patricia Westerford was based partially on my character. And all of that matters, right? It raises awareness in the public when it's in popular culture like that, in popular media, films, books. You know, people are writing symphonies, people are writing operas, people are doing art using these concepts, and that all expands our awareness in the world of these connections. And, but when it comes to Amy Adams and the biopic, I don't know, you know, I'm a little bit nervous about it because who knows what that's going to turn out with, turn out like, and hopefully my hope is what it will do is that it will even expose even more, you know, what we're doing wrong and what we can do right to help people move along so that we can be a more healing kind of planet instead of, you know, exploiting. Thank you. That was a very interesting response and I'll certainly await the biopic with interest. You've spoken there about healing and some of your writing has been very personal. You've spoken about the forest as a, and you've also spoken about, you know, the forest being all about connection and relationship.  And I wonder if you'd be willing to expand on, yeah, just on that idea of why that was so helpful for you at that point in time. Yeah, you know, from growing up in forests, it was always, the forest was always a place of sanctuary for me and always, you know, that's where I, you know, I feel good in forests. And it's not just me, people all around the world. It's well documented now that the healing aspect of being in a forest, forest bathing or, you know, and it actually, there's, you know, evidence of our, you know, how we change when we're in a forest, how we calm down, how we calm down. We, we have like positive hormonal responses to forests. There are even, you know, you know, the soil organisms in the soil, the things that we smell. A lot of that is the cycle of the nutrients that's going on in the forest floor. We smell that and we feel good. It's life, right? It brings life to us. And so certainly, you know, when I was going through chemotherapy for breast cancer, I went to the forest as much as I possibly could. And one of the, one of the, the drugs in the chemotherapy was pachlataxol and pachlataxol is a derivative of the U tree. And I know you have U trees in the UK in Scotland that are, can be a thousand or more years old. These are ancient trees and they produce taxol or the trade name pachlataxol as a defense, a defense enzyme themselves. And so it protects them against disease and infections. And this was actually known by Aboriginal people in North America for a long time and used for its medicinal properties and tinctures and ointments and bathing to heal, you know, I'm not sure, heal illness. And so I would, when I knew this, right?  And, you know, one of the things I also did is I made a promise to myself that I would try to give back. And so I've got a new graduate student, Eva, who is Eva Snyder, who is working with us. And the idea is that U trees live in communities as well, right? They have neighbours. They're connected to other neighbours in my world. They're maples and cedar trees and the ferns and the understory plants. And the idea that the community influences the health of the U tree and that it influences the chemistry of those U trees and the pachlataxol that they produce. And so we're trying to figure out whether or not there are certain community aspects that enhance the ability of the U tree to produce this amazing chemical that we have used and benefited from as human beings as well. So that's her work and that's pretty exciting. It certainly is and thank you for your answer there. I think we would all agree that you have given back in so many ways and continue to do so. We're sort of, we're beginning to run out of time just now. I think I've been really struck this evening by, well, I think you're, would you say it was a fair summation that your number one call really is about saving what we have. We are, we are, we are focused I suppose because we've got to the stage of having lost so many trees that we have a real focus at the moment on planting. But I think it's fair to say that we simply cannot bear or contemplate the loss of any more of our great forests and we really have to protect what we have as a, you know, if there was one message that you'd like, you know, this is the festival of politics. If there was one message that you'd like to reach politicians and parliamentarians worldwide, what would that be? That would be it. I mean, it's too, it's great to invest in trees and planting trees and it gives people a sense of agency and healing the planet and that's a good thing. But the first and foremost thing that we need to do in addition to decarbonising our energy sector, the first thing we need to do in forests is to protect the old forest that we still have. Because, you know, when you cut down those forests as climate is changing, we're never going to get them back, right? Because climate is changing so quickly that the forests won't come back the way that they were before. They're going to be different species compositions. There's going to be a much less carbon capital in them. We're going to lose so much biodiversity and carbon in the cutting down of these forests and we can't afford to lose any more. We're in a biodiversity crisis and in a climate crisis and those things are tightly entwined. They feedback on each other. So yes, we need to save these old forests and so how do you do that? Well, you know, there's all kinds of mechanisms that are coming up, market mechanisms that we really need to start really pushing and that is to make it profitable to save these forests instead of turning them into logs. So how do you do that? Well, you start valuing those other things that forests provide to water, air and so on. But one of the things that's easy for us to count that actually will take care of a lot of these other things is carbon. And so I think it's an easy logical mechanism that we can really move on very quickly and that is to buy carbon credits. Use whatever mechanism is in place to raise the price of carbon so that it matches the value of these ecosystems and then start buying them up so that these forests can be left intact. And then the cultures that are living in them and a lot of the intact forests are in Aboriginal cultures around the world. They map directly on each other is to empower the people with enough funding, which you can get through the carbon credits to actually start stewarding their forests like they did for thousands of years. They can look after the forests in the way they've known to do for long, long, like for millennia. So to me saving these forests is first and foremost and there are market mechanisms that are emerging and we need to push those forward as quickly as we can so that we can make sure that these remaining forests are left intact. Thank you. That seems like a good point to end on what we need to do and how we might go about it. Can I just begin to draw this evening's proceedings to a close? I'd like to thank you all for your contributions. I'd like to thank everyone who's joined us online. Thank you so much for your questions and for making such a positive contribution to this excellent in conversation event, but I'd particularly like to thank Professor Susan Suzanne Simard for giving us your time this evening and taking part. Can I take this opportunity to remind everyone watching that the festival continues for one more day with panels on resilient cities and business innovation and solutions to sustainability in big brains to big solutions and the role of culture in good health and wellbeing as well as discussions on mental health and the inequalities created by Covid-19. Thank you so much for joining us this evening and I do hope that you'll be able to join us to model too, but thank you again. Thank you all for joining us.


So, let us just recap what we did in the last class. We introduced this notion of strongly convex function in the last class and then we derived some of this properties right. And then we started talking about how to use different strongly convex functions. Two candidates we discussed last time was Euclidean regularizer and the entropy regularizer. Euclidean regularizer was strongly convex in which norm? It was in L2 norm and the entropy one was strongly convex in L1 norm. So, we will see how to use these two regularizers. And we have also discussed that that the gradient being bounded the norm of gradient being bounded is in a way in a sense equivalent to the Lipschitz net property of the convex functions right. Now, continuing from our last class at the end we had shown that if I am going to use a strongly convex regularizer how my difference in the functions look like. So, we are always interested in the regret bounds will be of the form f t of w minus f t at w t plus 1 right something of that sort and we are interested in bounding that. So, let us rewrite that and try to complete the proof we discussed last time. So, we wanted to argue that f t of w t minus f t of w t plus 1 is upper bounded by L t square by sigma. So, what is L t here? Yeah, this is the Lipschitz constant for function f t and what was sigma? It is the regularizer parameter it is sigma strongly convex that sigma parameter. So, we say we are saying that now, if I am going to use my follow the regularized reader the regularizing being my sigma strongly convex function and each of my functions f t is Lipschitz constant is Lipschitz with constant L t then if this w t's are computed based on my follow the regularized reader then we are saying that this bound holds good ok. So, now, let us see why this is so. So, I made a formal statement of this in the last class, but I am just I am just stating what is that we wanted to show in that claim. So, now it is why this is true. So, for all t let us define f t of omega to be summation of. So, this is the function that follow the regularized reader trying to minimize in each round right I am just calling that function f t. All these functions f i's are they are assumed to be convex and also Lipschitz and this r w is assumed to be what strongly convex strongly convex with parameter sigma. So, anyway r w is anyway convex and this is all the convex. So, this this f of f t for capital F t is all convex function, but further we are saying that this r is sigma strongly convex. Will this make this whole function also sigma strongly convex? So, you can verify this that is indeed true like if I add two convex functions it is going to remain convex function, but if I add a strongly convex function to another convex function the sum becomes a strongly convex function and it continues to. So, this entire thing is going to be sigma strongly convex ok. Now, let us with this definition of capital F t let us work out what happens. So, by our rule of follow the regularized reader what is f t is going to be computed as this is like arg min of f t of omega omega right. This is how my follow the regularized reader is going to work. Now, since this guy f t is regular is a sigma strongly convex function I can write it as f of t w t plus 1 is upon bounded by f of t w t sigma by 2 norm of t plus 1 whole square. So, now, why this result is true? So, why this result is true? So, what I am doing here basically yeah. So, f t w t here is what w t is the minimizer of f t w that is my definition right. And now, I am applying I know that since f t is a less convex function or here is sigma strongly convex function it need to have this lower bound, but we had also something some some linear term here right. What was that? That was the gradient of f t of w t and something like w t minus w t plus 1 or maybe it was 2 plus 1 w t ok, but now that w t is the minimizer of f t d w it was getting nullified right. So, we had we had given this as one of the properties of my sigma sigma strongly convex function. So, one this is hold. So, here I am fixing w t and now this is another w t plus 1 ok, this is the minimizer. So, if this is the case I can also write it as now I will do the same thing at f t plus 1. Now, w t plus 1 is the minimizer of capital F t plus 1 ok. And then this has to this will be plus sigma by 2 norm of w t plus 1 ok fine. I have these two equations now. So, let us add these two together. So, if you add them. So, while adding I am also trying to do some simplification w t plus 1 and this is greater than or equals to sigma times norm of w t plus w t plus 1 square this correct like if I. So, right hand side by adding this and this they are the same I get this term and the other thing I have simply. So, f t plus 1 this corresponds to this and f t w t plus 1 corresponds to this and this and this term I have just simply taken on the left hand side. So, now, go back to this definition of f t function here capital F t function. So, f t plus 1 involves sum till t whereas, f t involves sum till t minus 1 right and both of them computed at the same point w t. What is this difference is going to be? It is going to be f t w t ok. And what about this difference? This is now computed at the point w t plus 1 for f t plus 1 and f t and this is going to be what minus f of t w t plus 1 and this is still sigma norm of w t minus w t plus 1 whole square right. So, this so far I have only used the property of this function f and the sigma strong convexity of this. Now, I also want to bring in the Lipschitz properties of my f t functions ok. By the Lipschitz property of my f t function how can I bound this? This is upper bounded as L t times norm of w t minus w t plus 1 whole square is this correct? This is just I am applying my definition of Lipschitz minus of my f t function. So, note that all I am doing for some norm right now I have not specified any norm here it could be L 1, L 2 whatever ok. So, if I am saying that my R w is strongly convex with respect to some norm this L t that is the Lipschitz is also with respect to the same norm ok. Now, with this I am going to now I have a lower bound on this through this and I have an upper sorry I have an upper bound through this and I have a lower bound on this. So, if I am going to compare these two things. So, does this Lipschitz is little square here just check I think there is no square here right for Lipschitz it is just a norm ok. So, if I do this now what I am going to get from this if I now this is a lower bound this is an upper bound. So, if I now going to compare these two things through this I am going to get upper bounded by my L t times norm of W t plus W t plus 1 right. So, using this lower bound on upper bound I have this relation, but now just plug in this relation back here and that is what you wanted to show that is F t of at omega t W t minus F t at omega t plus 1 that difference I have I am just use it. So, this will using this I have whatever I want yeah. So, if modulus is there, but if you remove modulus also the bound should hold right what is the Lipschitzness absolute value of this should be upper bounded by this. Now, from the left side if you remove the absolute value this can be only smaller right. So, the upper bound still holds ok fine. So, with this lemma what we are finally, able to show is let me write it as a theorem now ok is this clear why this is true what we did. So, earlier we have already a result which says that regret is is upper bounded by R of u minus R of W 1 plus summation F of W t minus F of W t plus 1. We had this results right earlier this is how we bounded the regret. Now, we are saying that so, then this is simply follow the this is the bound for the follow the regularized reader with my regularizing function R and what was W 1 here? What was W 1? W 1 is basically the whatever your algorithm found in round 1 right that was obtained by minimizing this R function. So, that is why this R W 1 I am simply writing as minimization over R of omega. In the first round you do not have this summation is empty you are only minimizing the regularizing function ok. So, that is why result of R of W 1 I am writing. Now, for this part we have just demonstrated it to be this ok. So, every term here so, this is for each term right ok. So, I had to make one more thing here. So, here what we are basically saying is this is like L t square by sigma. So, what is now L here? So, this is let us say they are all F t's are all L L L Lipschitz they have the same Lipschitz constant. In this case what this will turn out to be this is going to turn out to be simply N L square by sigma and that is what we have written here, but what we really would. So, suppose this L t's are not the same and they are going to be different in each round let us say this is L t square by sigma. I could I have just multiply N and N here. So, here either I can assume that my Lipschitz constant L is same in for all the functions in which case L t will be replaced by L and this is simply N L square by sigma. Other possibility I can think is ok this is t equals to 1 to N. Instead of assuming that all the functions are the same Lipschitz constant what I can assume is the average value of the Lipschitz constants. This is the average value of the Lipschitz constant right square average value of the squared value of the Lipschitz constant I can say that ok that will be some L square and in this case again I will get the same N L square sigma bar ok. So, what we are saying is as long as all your functions are convex with the same Lipschitz constant L then this bond holds. If you are going to use some regularizer some sigma strongly convex regularizer with sigma here then this holds or let them be all f t this f t functions be convex and each f t is L t Lipschitz constant and the same bound holds provided I interpret this L square as the average value of all the L t squares fine ok fine. So, now we have this nice simplified version on the regret bound provided my I use follow the regularized leader with a strongly convex function sigma in this case sigma strongly convex function. So, now, let us go back and work out if I use different different regularizers which we discussed like our Euclidean distance and entropy distance what is the bound we are going to get here ok.   So, this guy is is it a strongly convex function we discussed this last time right this is a strongly convex function with what sigma value. So, I have it is then sigma also here eta also here without eta it was if I have this is one strongly convex right. So, again what was the definitely what was the property that we used to check strong convexity the Hessian based condition we had right it was del square R of omega x into x if this is going to be greater than 0.2 square. So, with respect to this norm then this guy this this function R is a sigma strongly convex function. Now, can you check me now I am already norm has been defined for you to be l 2 norm. So, what is this? So, if you take this function and try to compute this condition for what sigma it holds 1 by eta. So, let us say this is then 1 by eta strongly convex and with respect to what norm it is true with l 2 norm and what about this? So, for this we made some more assumption right this u i's are. So, each of this u i we also defined. So, this is true for any u this Euclidean distance or this Euclidean regularizer was defined for all u, but then we defined when we defined this entropy regularizer we said that this is coming from a set S where all my w all my components are positive u and also. So, this is what we called probability syncs simplex which you did not like and we call it as probability space right we all this u as it is coming from this space. This function is it again strongly convex and now see when we said this thing right this was nothing, but l 1 norm. So, I do not know we computed this last time or not you can verify that this one if I am going to take this quantity to be 1 this is going to be 1 strongly convex with respect to l 1 norm this is a function right with respect to the l 1 norm and one can also show that if instead of this if you take it to be some b number this can be shown to be like 1 by b strongly convex or like not exactly you can take it to be less than or equals to b 1 ok. So, sorry like earlier may be equals to 1 it is anything less than or equals to 1 or less than or equals to b ok. So, now what is going to be this bound look like if I am going to apply these regularizers ok. So, can you now compute if I take my Euclidean regularizer can you come to work out what is this going to be.


The call was sounded by Kiflin Karoo, who was the guest speaker at the recent opening of Library Week Observances, held at the National Archives in Viji. Karoo is the executive director and founder of Helen's Daughters, a non-profit organization which focuses on the economic advancement of rural women. A native of Fawasaw Babano, Karoo as an only child, found solace in reading, utilizing her community library from an early age. Despite the rise in modern technology, Karoo, who once worked as a library assistant overseas, thinks libraries still have a place in building young lives and by extension communities. She has challenged local libraries in St. Lucia to find ways to revitalize the interest of people young and old in libraries and to make it more inclusive and welcoming to both urban and rural communities. Libraries have allowed me to see past my reality and in many ways to manifest my future. I didn't just see Fawasaw, I saw Africa, I saw Europe, I saw other Caribbean countries, I learned about people, I explored new cultures and this is why libraries are so pivotal in our society. Libraries should play an essential role in local communities, not only in providing access to information and resources but supporting literacy, education and lifelong learning and most importantly they should serve as a community gathering space. Libraries and information units across the island is this week celebrating National Library Week. Under the global theme, there is more to the story, a theme which seeks to dispel the notion that libraries are no longer relevant. Google cannot and will not replace libraries. Google cannot shut our doors. Librarians are trained to serve and will continue to make libraries relevant. The National Archives, which is responsible for collection and preservation of original documents of historical value in St. Lucia, is also playing an integral role in the week's observances. Archivist of the National Archives, Margot Thomas says, despite the advancements in technology which has drew many from the traditional ways of reading and seeking information, there is still a place for libraries. Just give me a test to my favorite authors through a book. My authors like Thomas Hardy and Derrick Walker, Jeffrey Archer, Rick Wayne, Doris Lesson, Pulis Book, Barack Obama, give me reading time. Good. And as I know it today, with a real book held in real hands and savoring the touch and feel of. The Library and Information Week is an annual celebration of libraries and the services they provide to their communities. Events are being held across the country to raise awareness of the value of libraries and encourage people to use them. From the Communications Unit of the Ministry of Education, Sustainable Development, Innovation, Science, Technology, and Vocational Training, I am Chris Satney, reporting.


Are we good to go? Well, welcome to the session on the introduction to writing spiders and web agents. My challenge here today is that I've got, what, 45, 50 minutes to introduce the idea of writing agents and spiders. And so what I'm going to do here is I've got three main goals I want to accomplish here in the session. One of them is I want to convince you that writing these things is a worthwhile activity. Even if you don't do it professionally and there are a lot of opportunities to do this kind of stuff professionally, it's just a really fun thing to do. I know a lot of people who maybe don't code as much as they'd like at work and they get home and they got this need to code, you know.  You can have a lot more fun writing spiders and web agents because once these things start moving and they just become kind of organic, especially when they interface with some very non-web kind of things like, you know, cell phones and printers and fax machines and stuff. It can get very fun. The other goal I have is I want to get you guys thinking about ideas for spiders and web agents that you want to go off and design on your own. So I'm going to give some suggestions for ways of coming up with ideas. You know, it's easy to write a spread or an agent. The hard part is having a really great one that does something really cool. So hopefully I can kind of give you some ideas on how to come up with some ideas. The other thing I'm going to do is provide you with some basic technical skills. And pretty much all you really need to do these things is basic technical skills. There's so many great software available now that you can download to do wonderful things. And there's some great programming languages. So it's taken a lot of the mystery out of doing this. So a lot of it is just being introduced to how to do this kind of stuff. So I'm going to make some suggestions for programming languages, things that I've used in the past, some downloading techniques, some parsing methods, things that I've discovered over the last, you know, three, four years. And ways of making fault tolerant designs. You've got to keep in mind that unless you're spidering your own site or your own server or a known server, these things are going to change at the whim of whoever is the webmaster. So I'll give you some ideas for writing fault tolerant spiders and web agents. Real quick bio. My name is Mike Schrank. I am a freelance internet strategist and developer. I'm based in Minneapolis, but I work all over the place. The nice thing about being based in Minneapolis is that you only like two, two and a half hours from anywhere in the country. The bad thing about being in Minneapolis is that you're two, two and a half hours from anything in the country.  They're not the only thing I do, but I'm doing more and more of them as time goes on. People are seeing that you can do some really cool things. For example, I have one client that's basing their entire business on what I'm doing. And I've got others that see that they can get a real competitive advantage over their competitors by being able to know things before anybody else knows it and then being able to act on it really, really quickly. So there's some real powerful things you can do here that make a lot of business sense, in addition to being a lot of fun to write. Okay, what's the connection between spiders and agents and hacking? The way I do it, it's the classic hacking activity if you belong to the MIT School of Hacking.  So it's kind of the MacGyver approach to web development. You're taking things from all different places and doing really bizarre things. You can also do some things that are useful, security things that you can monitor logs, you can look for unauthorized servers on a network, you can test system integrity, you can do all kinds of cool stuff like that. Okay, what is an agent? Well, basically here, if you can think of a browser as being a very general purpose agent. And it's a general purpose in the fact that basically you just point it someplace and it goes there. Okay? If that's all you do at the internet, you're really missing out on a lot of stuff. And I'll show you some examples of some stuff here in a bit. All agents work on a client-server relationship. So you've got, in this case, a browser sends out a request to some web server. It gets a response back. And generally the initial response is some headers, HTTP headers that are sent back. We'll talk more about those later. And also some HTML. Now a client must ask for everything. So in other words, if you went to the old DEF CON page, there are nine separate images that are on that page in addition to the HTML. So after it receives the HTML page, it's going to send out requests for those images and all those images come back. What do spiders and agents do? Like I said earlier, this is not difficult stuff. They download data, they parse and filter data, and then they take some kind of action. And in all the examples I'm going to show you pretty much, they all do the exact same thing. What they download and what kind of action they take is entirely up to you. When you're thinking about agents and spiders, don't limit yourself to spiders. Spiders are just a general purpose, excuse me, they are a special purpose agent that acts specifically on links. So they're going to grab a web page, look for all the links, and then they'll grab a web page, look for information on that page, then gather up all the links, and then do the same thing in all those links. So don't limit yourself to spidering, that's only a tiny, tiny portion of what you can do with these things. Try to instead think about providing web services. Now if you think about web services, you can still employ spiders if you want to in your services, but I think it gives you a little better framework to stand on when you're thinking about what kind of things you might want to do. Why write a spider or an agent? Like I said earlier, browsers are kind of dumb, you point them in one direction, they'll go off and they'll do it and then they'll stop and wait for the next instruction. Well with an agent or with a spider, you can automatically surf the web, which is very nice. You can also filter out information that you're not interested in. You can also download and repackage information for later non-web devices. So in other words, you could write a little deal that goes off to your favorite web page, downloads it and does clipping and stores it in a manner that would be good for a PDA, or you can send stuff off to printers or fax machines. The other reason you want to do agents and spiders is especially if you're not interested in images, you can download stuff really, really quickly. So speed is a big, big advantage. Okay, where do project ideas come from? Think about areas where you can bridge things. In other words, I've got a bunch of stuff there, everything from HTTP servers to phones to webcams to printers, instant messaging, fax machines. Think of instances where you would want to combine those types of things. Another thing to think about when you're looking for coming up with an idea for writing a spider and agents, look for things that you can automate. For example, there's a web page called www.whois.net that is run by network solutions. And the really cool thing about this page is that if you're looking for, if you want to take out a domain name, instead of going to register.com or something and just typing in the name, oh, taken, how about this one? No, it's taken. It's not very effective. What you can do with whois.net is you can type in a word or a series of characters and it'll return all the domain names with that series of characters as a subset of the full name. So if you're looking, if you want to do a website, for example, on food or something, you can type in food. It'll return all the domain names that have the word food in it. It's pretty useful. The thing that makes it not quite so useful is that it only returns, I think, 10 or 15 names at a time. And as you can imagine, this might return, you know, it's like they're going next, next, next, and there's thousands of these things, right? So one little spider I wrote is I go out and I spider this page and I just type in the word that I'm looking for as a subset of a domain name and basically it just returns all of them much, much faster. Another example or another thing to think about when you want to automate stuff is think about things that you can perform periodically. I talked a little bit earlier about doing things with PDAs. Let's say, for example, you work for a corporation that has people in the field all over the country. You could set it up so that at night the field representative's computers would go out and download some information from your corporate website and then this agent could transform this stuff and put it where it needs to go so that when the salesperson or whoever it is in the morning there's a hot sink on their PDA, they've got all that information in their PDA. So that's an example of something that you'd want to perform periodically. Okay, let's look at some ideas here.  So it may or may not be a good idea. I guess it depends on your mom. So what would you do? You would download pages from CNN. You'd parse, look for stories about you, and that's really a matter of a simple string comparison, really. I mean, it's not hard. Then the action is you would email your mom. You can either email a link to the story or you could email the actual story. Okay, create a service that provides real-time weather data. In fact, I know several people who are doing this. This is actually a useful thing. Download pages from the National Weather Service. Parse out weather information and forecasts, and then make the stuff available via some type of either a SERP or some other web protocol. So basically it's just a service sitting there that people can access. Track the web technology used by each of the Fortune 500 companies. Okay, how would you do this? You would download the HTTP headers from a set of web pages that you have stashed away in a database. You'd parse out the server information, and then you would store the stuff in a database and run comparisons and do whatever you want to do with it. You guys remember the cartoon? There were two dogs sitting on the street, one of them said to the other one, the nice thing about the internet is no one knows you're a dog. That's not always the case. I actually met my dog on the internet. There's Jackson right now. He's showing the proper technique for standing on a 17. If he knew I was talking about him, he'd kill me. Anyway, no, I actually met my dog on the internet. A lot of animal shelters and rescue groups will have websites where there are post-dogs that are available for adoption. It's kind of like a canine dating service, and that's where I met my dog. That made me think, you know, if you wanted to, you could periodically run a server, excuse me, run a spider, to check on a Jack Russell or any other kind of dog that's available at your local shelter. And again, it's a simple download, parse, filter, take action. You download pages from your local animal shelters and dog rescue organizations. You'd filter out information about Jack Russell terriers, and then you can send yourself an email when a dog is found. Number five, scout all the top employment websites for jobs that match your skills and automatically send the contact person a resume. Good idea, I don't know. I don't know if I do this one, but it's the kind of thing you can do. Again, download, you download pages from employment websites, parse and filter, jobs that fit your skills and gather employee email addresses, and the action you take to automatically send the resume to appropriate employers. Number six, write a web agent that returns the best price on a dozen roses delivered in Chicago. This is actually something you could sell if you could do a service like this. In fact, there are some like this out there. My Simon, I think, comes to mind, and I think CNET does something like this for computer hardware. So what you would do, again, you download delivery pages or flower delivery pages, check for best prices in Chicago or wherever it is you're looking, and then you'd print the names of suppliers, offerings, and prices. Okay, you're starting to see some similarity here. These are vastly different kinds of ideas of things you can do. The process is very, very similar. I had a camera in your medicine cabinet. When the door opens, take a digital picture and post it on a website. www.medicinecabinet.com, I don't... This might also be an idea for a business, I'm not sure. So here what you would do basically is you'd have some kind of a trigger mechanism, and you can interface any kind of hardware you want to an agent, right? Anything, even a medicine cabinet. So you'd have some kind of a switch with the door opens, bam, a picture's taken, you really don't have to parse anything, but it's pretty easy to post the stuff on a website then. Okay, automatically analyze server logs for anomalies and send to a printer when found. So here instead of going out on the internet, you could conceivably load a local file, a file on your own network. I suppose even a log file you find on somebody else's network. You could parse out anomalies and then you could print out things that don't look right so you can look at them later. So I did an email the fact service. This is what I actually did. I started writing spudders and agents back about four years ago when I was working. I was a consultant over at Medutronic and was involved with a lot of telemedicine kind of stuff. And this is what I actually did. I set it up so that if you sent an email to a specific email address, it would take the contents of the body and it would fax it to the number that you put in the subject line. So how do you do that? When you load, you go out and you get the emails that are sent to a specific address. You parse the message in the phone number and then you fax the message to the phone numbers. It's embarrassing or easy. This idea I love. Write a new front end to somebody else's online catalog. The idea is not as strange as it sounds. For example, if you have a client that receives a royalty on things sold on another website, people might want to make the whole thing look like it's theirs, right? And if they have a relationship with this other company, they have everywhere to do that. So this is not such a far-fetched thing to do. So here you would somehow gather the contents of a form that would normally appear on this other website. And you gather the variables. They all come from databases or cookies or stuff. And you make sure that your variables and methods match the form that your target is. And then you basically submit the stuff to the target site. This idea I like, but I'm hesitant to do it. Start a digital clipping service at databases, information and images about both your client and their competition by spattering web FTP and new servers along with DNS records and new domain name applications. I'll tell you why I'm not doing this a little bit later. But basically what you do is you download company data from various sources. You parse out everything that's applicable about your client or your competitor. I suppose if you were really smart about this, both the client and the competitor would be your clients, right? So save work. And then the action you would take is you somehow post all this stuff on a central database with a centralized web server so that they could go and access their data. Okay, those are some ideas, okay? You see they all have very similar process. What they do though can be very, very different. I've written agents in a number of languages. I first started doing things in Java again back when I was doing telemedicine things. And I was doing a primary in Java because I wanted to have a graphical user interface. That was a nice way of doing it. I did some things in Perl. Perl is very powerful for this kind of stuff. I did some stuff in Tickle-TK. In fact, I actually wrote an article. It's the March 2000 edition of Web Techniques, which I think now changed their name to New Architect. If you're interested, I'll have a reference to this in all URLs that I'm using at the end of the presentation. But I did an article about writing intelligent web agents in Tickle-TK. For about the last two and a half years, I've been writing almost everything in PHP. I love the language. I've never had to debug the interpreter, which is a very nice thing. And it just runs so reliably. And it's very similar to Perl. But I believe the syntax is much simpler than Perl. So if you're going to download files with PHP, how do you do it? Well, if you're doing a local file, you use the method I'm showing in number one there. But basically, you just create a file handle to whatever it is you want to access. If you want to do a web page that's number two, it's the same thing. You just open a link to a URL. Then you've got a handle that you use to reference that URL. PHP also allows you to do FTP if you need to do FTP. I think that I tell you I wasn't going to show you any code. My intent was to not show you any code, but here's some code. Actually, here's a partial listing. If you're interested in looking at this, I'm not going to walk through it. This is on the CD-ROM, I believe. Basically, this is an example of doing socket programming in PHP. And this particular example goes out and it performs a who is function. It looks like this. I had a client who we were talking about some stuff and about doing some stuff like this. And he told me that the people who run the who is servers, they watch how many hits they get from particular IP addresses, and they only allow so many in a 24-hour period. Anybody else here about this? Do you know what the number is? Thousands? Good. Okay, launching your agent. I tend to do everything from a command line, even though I write this stuff in PHP. A lot of people don't write PHP programs this way. They'll write pull programs that they launch from a command line. You can also do it from PHP. If you're going to do it, this gives you some advantages. For one, you don't need to run a web server. So if you're doing stuff in PHP and you're doing it as a module with a patchy or something, I tend to not like doing that because I might be fooling myself, but I feel safer not having a server on the system. So there's also a standalone version of PHP that you can see in the first line here. PHP also has a separate command line version, which you should really use if you're going to do this, because in addition to not having a lot of the webby kind of stuff that you would normally have, you can also have separate configuration files and all kinds of cool stuff. It's also a smaller footprint, I believe. You can run these things as a scheduled service, so they can run like every hour, every six hours. You can have them run at midnight. You can have these things run on startup when people log in. You can run and have them loop continuously, which is a fine thing to do. In fact, you can have lots of these things running continuously on the same machine, and you can get some incredible efficiencies by doing that. Because most of these spiders and web agents just send out a request someplace, and they spend almost all their time waiting for a reply. So if you can run them simultaneously, if you have a need to run them simultaneously, it's a worthwhile thing. Okay, I showed you how to download stuff using PHP. I tend not to do that. What I use instead for downloading stuff is a really great program called Curl. Can I see how many people here are familiar with Curl? Excellent, excellent. Curl is a command line tool for getting and sending files using a URL syntax. If you're interested in getting a copy, there's a couple of places where you can get it. I was thinking about putting it on the CD-ROM, but they've got binaries you can download for like 50 different platforms, and I didn't see the need to do that. Again, these URLs will appear at the end of the presentation, too. Curl supports a ton of internet protocols. And it's open source, but it's not GPL, which means basically there's no license whatsoever. My understanding, and if I'm misspeaking here, or somebody please raise a hand, this stuff is completely free software, so you can do anything you want with it. Don't confuse Curl with the small URL with Curl, the client-side programming language, which is a product that the people at www.curl.com are selling. There's also a copy of the Curl manual on the CD-ROM, so if you're interested in looking at all the features, it's got tons of features. I'm going to just show you a few of them here. To use Curl, you just type Curl, and then your target, and it'll take everything it receives and direct it to your standard output device, which in most cases is going to be your screen. You can also redirect the contents to a file. You can also use the number three there. You can use the O-switch, so you can set the output to a specific file. The way I tend to use Curl is I use it inside of PHP, where I will do things. I'll actually run like a little shell to execute Curl within PHP. The nice thing here, if you look, it'll take the entire contents of the default HTML page from this fictitious website and load it all into this variable, so I don't even have to create a file or anything. It's just all in that variable. Once it's in that variable, it's very convenient and very easy to parse out. If you're just writing PHP for a web application or something, never do this. You don't run command line stuff inside your web applications. Again, maybe I'm fooling myself. I'm feeling safe in doing this because there's no server attached to it. Here's an example of one of the options you can have with Curl. In this example, I'm downloading the headers for the New York Times website. There's all kinds of cool stuff. I can see what kind of server they're losing. If you're thinking back to the one idea where you monitor the internet technology that's used by the top companies, this is basically how you would do it. You can also see there's a cookie that's being set. Curl also allows you to send cookies back to servers, which is very nice. This was an HTML page, but you can download anything with Curl. It doesn't have to be HTML. It can be images or flash or whatever. You've got some nice ways of dealing with password protected sites. You basically just set up your URL, you set up your user and password, and then you use the U switch in Curl to pass the username and password to whichever site you're using that uses standard authentication. The other thing that it's got that's really slick is it allows you to download sites that are encrypted. And all you have to do to do that is add an S to your protocol right here. And then it automatically decrypts it and it will throw everything into this variable called page data, which is pretty slick. Something you need to remember when you're writing agents and you're going up to servers, you're going to leave a log in their log file that shows that somebody's been there. And if you don't do anything with Curl, I think basically it just says Curl in the version number. But you can make it look like anything you want. For example,