Ок, привет всем! Меня зовут Алексей И сегодня я вам расскажу about Postgres query statistics Какие проблемы мы можем с ними поделать? Я покажу примеры queries И что это значит И что делать с ними Также здесь есть подлинк с презентацией, вы можете доноточить и попробовать queries на вашей installation И вещи, о которых я не буду говорить Я не буду говорить about query planner statistics Postgres activity statistics and query planner statistics are different topics Also, I will not talk about utils and tools that works with stats only psquare and plaintext queries I want to go out of this talk with a few take home messages I want to convince that the postgres query stats is useful and solves many problems By default, postgres stats are provided in plaintext form and may seem the stat is to be inconvenient to use but this isn't true I will start from simple queries examples and you can see that the stat isn't scary Examining these queries examples I will show which stats should be used in particular cases So, let's start What is postgres from system administrators point of view If we use system activity tools such as stop or similar we will see some processes We can estimate we can see how many resources they consume CPU, memory, swap, discord, network, utilization but nothing more If we want to understand what is going on inside the postgres we should use special tools and details and these tools provide an interface to postgres stats Postgres is a complex piece of software that is comprised of many subsystems Let's have a look at this picture and try to answer the question where postgres spends its time and here we have two answers First is the service and client queries and second answer is the background operations Let's review both cases and begin from top Imagine that we have a client application that works with the database For any client postgres creates dedicated process which we call a backend The application sends queries to the backend The third query plans them and executes Next running a query requires reading or writing some data to tables and indexes This is being done using the shared buffers If it's a simple select query data is read from disk into the shared buffers and the result is returned to the client If the data modifying query is such a update, insert or delete changes must be committed and all data changes are written to write a head log In this way postgres works with clients Next we have background operations that are invisible to the clients These are query logging flashing dirty data to disks auto vacuum processing write a head log logical decoding or sending to streaming send buys As we can see postgres isn't a simple thing and if a problem comes up in some part of postgres it may influence on other postgres parts And here postgres starts will help to track postgres subsystem activity estimate its effectiveness and suggest some optimizations but there are a few problems with stats First problem there are too many stats in postgres For example in the current release there are over 100 items and number of items directly depends on number of databases tables and indexes in the whole instance The second problem is that the stats are provided as permanently incremented counters and you can see billion values in stats and these values say nothing to you It's very hard to track changes in very large values Third problem is that there is no history for stats There is no simple ways to see stats of 5, 15 or 30 minutes ago You can use third party tools which allow to solve this issue but postgres doesn't have tools for that Next problem is the absence of building tools for working with stats You can use only sql, psql and self-written sql queries Yes, it may be flexible but also too hard for beginners And this problem entails the next problem and a lot of monitoring tools for postgres are available and they have different sets of features Often you need to use two or three tools since one tool would have features that another tool doesn't Well, a simple way to not suffer from these problems is to know what is provided by stats and how to use it as is It's important to be able to use stats directly from postgres without any special tools It requires basic sql skills for writing queries Stats are based on counters on shared memory and every counter is associated with a specific event and increments when receiving it occurs Special functions are used for extracting values from these counters and there are many such functions and to make them usage easier functions are organized into views and by default there are about 20 different views You can create your own views with your own set of functions In addition to building functions we can use stats from additional extensions There are official and unofficial extensions and official extensions are collected in the country package and can be installed from official postgres repo Unofficial extensions is not supported by postgres team shipped without any warranties and should be used at your own risk On this slide we have stats functions and views that provides various stats for database objects Not to get lost amidst these names we can use previous diagram Here you can see which stats functions or views are associated with postgres subsystems Now I will talk about the most interesting views and explain how to use them and what help they can provide to us First of all there is PGSTART database that shows information for all databases in the current postgres instance There is a summary information about number of tuples that were written in the database number of commits and rollbacks information about temp files, recovery conflicts deadlocks and others So how to use this information and using this information we can estimate cache hit ratio Cache hit ratio is effectiveness matter for shared memory which is known as shared buffers It's obvious that reading from memory is faster than reading from disks and when with cache hit ratio we can get an idea whether we have enough shared buffers Using the blocks hit and blocks read we can determine cache hit ratio in persons Really this value should be close to 100% Values below 90% say that the amount of shared buffers isn't enough for current workload and postgres is forced to read data from disks and evict hot data from cache Now that we can see low cache hit ratio immediately after postgres restart this case is tracking anomalies such as rollbacks, recovery conflicts deadlocks and temp files By anomaly I mean an unwanted event happening in the database and not bugs As with cache hit ratio we can calculate commit ratio using transaction commit and transaction rollback values If the number of rollbacks is too high you should direct your attention and investigate why too many operations and with rollbacks Anyway it's bad behavior when application does something change data or spends time and in the end rollbacks its results And on this slide I put caution in values and if you see such values you should be worried It's also important to track recovery conflicts and deadlocks which cause and eliminate them Recovery conflicts are related to streaming replication and this means that the queries, variance and back has been canceled Detailed information about conflicts can be found in server logs or in PgStar database conflicts view Deadlocks are unpleasant events too and detailed information about them can be found in server logs For detailed information about log tracking you should enable the log log weights option in PostgreSQL Conf It helps to get a clearer picture of when deadlocks occur and a little bit about temp files Temp files are always written to disks and queries that use temp files become slower You should track queries that creates huge temp files and optimize them if it's possible Temp files option is enabled You can find detailed information in server logs or use PgStar Statements extension About the latter I will talk in a few minutes Next important view is PgStar BGWriter With this view we can estimate effectiveness of checkpointer and BGWriter These are the ground processes that perform very important tasks First task is a checkpointing This is a regular operation related to data synchronization between shared buffers and data files and creating checkpoints, special records and writer head log Second task is a permanent writing of dirty data to disks so that the backend handling user queries seldom or never need to wait when writing takes place Moreover, BGWriter helps to understand checkpointer process to reduce amount of work for checkpoints For controlling efficiency of these processes you can use PgStar BGWriter view There you find information about execution checkpoints their times, spend time and buffer area Endless start from checkpoints As I mentioned earlier Checkpointing is a process of synchronizing all dirty data from shared buffers to data files on disks and creating special records in writer head log when sync is done There are two types of checkpoints Required checkpoints and checkpoints by timeout Checkpoints by timeout occurred with specified time intervals and Required checkpoints takes place when Postgres writes specific amount of writer head log Checkpoints by timeout are to be preferred to require checkpoints and require checkpoints take place when database is under heavy write load that is when huge amount of data should be written to disks When a checkpoint occurs it creates additional load and can cause performance to drop severely and the pg start bidger writer helps to determine how many checkpoints are being executed and what is the type If required checkpoints values are too large we can increase the amount of writer head log that is required to trigger a checkpoint Postgres query conf has several options for controlling checkpoints These are minimal size checkpoints timeout checkpoints completion target checkpoints segments Last option is actual for Postgres versions before 9.5 In 9.5 release checkpoints segments were replaced with minimal size and maxful size Thus increasing checkpoint timeout or maxful size we spread checkpoint execution in time and reduce load that is created by checkpoints Next with this start we can estimate bidger writer effectiveness The main goal of bidger writer is to write out dirty pages from shared buffers to disks and it is writing pages by batches and the size of a batch isn't constant If the batch is big and too many pages have been written bidger writer is forced to pause and this is required to reduce the impact on IO So we can use buffers clean and max written clean values to tune amount of work for bidger writer If storage system is fast and we can add extra work for bidger writer with increasing max pages and the multiplier parameters or reduce these values and increase delay when we want to reduce the impact on performance from bidger writer Also Special attention should be paid to buffers backend of sync This item shows how many Syncs is calls are called by backends As you know Sync is a heaviest call and if each backend will do things, this will negatively influence on performance To this end backends try to delegate all things to the checkpoint or process It has sync queue and its size depends on the size of shared buffers In the worst case when sync queue is full backends is forced to do things by itself It's a very unpleasant situation and it means that there is a serious problem with storage So if you see values more than zero it means your storage is overutilized And next view is the replication that shows information about connected streaming replication in standby Here you can find information about remote host parameters and group of locations values Vacations are a very important piece of information and that helps to check for problems with standby The main goal of this view is replication lock tracking Replication lock is the situation Standby server is behind the upstream and has no time to replay all write ahead logs received from upstream Every location describes a specific position in write ahead log and here we have 4 positions These are sending from upstream to remote host writing, flashing and applying logs on remote host Thus calculating difference positions we can determine bottleneck and know that the ideal situation when all positions are well And replication lock can be caused by several problems such as overloaded network or errors on network storage or CPU Network becomes a problem in case when upstream needs to send too much data to standby over network with low throughput Overutilized storage when postgres has no time to write flash or replay received data And third very rare case is when CPU is bottleneck Problem here is that recovery process that replace lock is single core and in rare cases it can utilize core almost to 100 persons For example when lock contains small interval of times our host has very fast storage subsystem And below show queries examples The first one is tracking write ahead logs position in human readable decimal format If we measure these values with some interval of time and compare them we can get an idea how much write ahead logs are generated by our server For example for replication lock tracking in bytes and in seconds Here we use additional functions pg current transaction location xlog location pg current exact replay time stamp pg xlog location div And a few words about lagging seconds This query can run only on standby server and has one drawback If your upstream server query show growing lag and in fact it's wrong Here is another example of query which I used very often Here I'll calculate difference between each position and the total lag in the end Next Next view is the pg start all tables The chose starts for all tables in the current database This view includes starts about users and system tables Each database has its own view with its tables With this view we can get a picture of tables usage There is info about tables access, sequential and index scans and how many tuples were returned by these scans What operations around tables and how much tuples are affected Also information about life and that tuples helps to understand when auto vacuum should start for this table and of course auto vacuum and analyze counters and time of the last execution and there is too much information that it don't paste at all And the simple thing that we can see here are the sequential access tables sequential scan takes a table and reads it from start to position when the query condition is satisfied With big tables it can take a long time It's important to see how many tuples were returned by one sequential scan For example in LTP workload if one scan returns more than one or ten thousand rows you should check query, explain it rewrite it or probably create new index Also with this view we can estimate table size Of course we can use meta comments but they are not so flexible as your own queries For example we can use ordering, grouping, filtration and various conditions For example show the large tables or tables with indexes or tables with large indexes Pay attention Here we use additional functions PG size pretty and table relation size Next place where these stats can be useful is checking for tables write activity Write operations are heavy operations and if it's possible to make them more lightweight it should be done Update operation in fact consists of two operations delete and insert When tuple is updated all tuple is marked as removed and new version of tuple is inserted Additionally index entries related to these tables also should be updated Thus tuple updating is a demanding operation and for reducing write overhead hot updates were introduced in Postgres 8.3 This avoids index updates in case when non-indexed values are updated For viewing stats two fields are available and tap up and tap hot update Using them we can determine hot updates profit Thus we need to answer questions how to increase number of hot updates For increasing number of hot updates we can play with fill factor option What is fill factor? This is a page packing first ratio and can vary from 10 to 100 persons It works in the following way When new tuple are inserted in the table and the new page is allocated this page is filled by new tuples according to fill factor If fill factor is below 100% new page will have free space that can be used later for update operations New version of tuple can be written in the free space and in this way hot updates will occur more frequently and it's positive fact for performance But there is a drawback in fill factor The lower fill factor you use the more space tables will take Another thing which can be useful is viewing list of tables that should be vacant Unfortunately Postgres doesn't offer a simple way to show this information So we will use slightly more complex way To do this you need an additional function current settings that shows specified Postgres setting But how does Postgres know that the table should be vacant When AutoVacuum worker starts it builds list of tables that should be vacant Analyze it or both It will process tables that have a number of tuples larger than a threshold The threshold is based on the number of tuples in the table and following settings Vacuum scale factor and vacuum threshold Tables to analyze are determined in a similar way but instead of the tuples number of changed tuples since last analyze is used Thus we can write a query that shows this information If you count rows in this list we can get something like AutoVacuum QSize But know that these parameters can be rewritten for particular tables with alter table command And this query doesn't account for it But our colleagues from Aveda company implemented a plugin that accounts for this specific settings The query used there can be found in this URL Fortunately in Postgres 9.6 StatsCollector will have dedicated functions that allows to track AutoVacuum activity and progress Next view is the PgStartAllIndexes This view is similar to the previous one and describes indexes stats It helps to estimate how actively indexes are used Here we see index names and how many tuples are returned Practical usage of this view is searching for rarely used or unused indexes unused indexes in this database are very bad they take space and amplify write operations because write operations concern not only tables but indexes too Also indexes are vacuumed during tables AutoVacuum Basically we need to check database for unused indexes and remove them For that we should find all unused indexes with zero index scans Found indexes should be remembered and removed But be careful here indexes that are not used on master can be actively used on send buys Be sure that the indexes are not used on all hosts Also I recommend to pay attention to the following URLs There you can find useful and extended examples for unused queries searching Next and very useful view is the pgStatActivity It contains information about connected clients and queries that run at this moment Each line in this view corresponds to a Postgres backend In it we have various information such as backend pit remote host information start time of backend transaction or query backend states current or last executed query The most simple thing we can get from this view is the information about overall database activity For example number of used connection It can be useful to track when number of connections is close to the limit With this query we can see how many connections in persons are open to the database Also this view can be used for tracking how many and what users are connected to the databases It's useful when we need to know who hijacks the data and what users It's useful when we need to know who hijacked all the connections And in this example CronRoll is the bad boy Another way to use this view is checking for long transaction and queries Here we need the following columns backend start, transaction start, query start and state change For calculating the age of query or transaction we need to use an additional function clock timestamp which shows current time Calculating the difference between what values we can get the age of a transaction or a running query And what should we do with query or transactions that take too long Remember them, terminate and optimize queries or applications that executes them In LTP workload long transactions are unwanted and you should avoid them Problem are bad transactions that are opened and then get stalled in an idle state This can happen in a situation when application opens the transaction with begin command does something inside it and leaves the transaction without commit or rollback It's an unwanted situation because in this case the table and the indexes bloat This may affect performance negatively For searching for bad transactions we need to use the state column Check for transactions that are in the idle or aborted state If you see many idle transactions with age of dozens minutes and more you should terminate them as soon as possible Next, your application should be optimized to do the transaction processing in a proper way Next things that can be tracked with pg-style activity are waiting clients Pay attention to the waiting column In a normal situation all values here should be false If there is a true value it is bad It means that the backend is blocked by another transaction or query and waiting for the blocking transaction to end While it's waiting the backend does nothing useful From the user's point of view the database seems to hang And what to do in these situations Waiting transactions or statements should be canceled or terminated Application code that runs blocked statements should be optimized To see the details you should enable the log-loc-waits option And when it's on, additional info will be written to the PostgreSQL logs Also pg-logs view can be used to perform an immediate investigation when a log happens And to continue with blocking queries a few words about pg-logs view This view shows information about current logs Using queries from these URLs you can understand which queries blocked one another And when blocking statements are found you can cancel them and the block statements can continue their work In this example there is blocked update operation and it waits for alter table But alter table goes into idle and transaction state Thus, canceling alter table will solve the problem and update can continue Yet another useful stats are provided by pg-stat statements module That is included into the official country package Before using pg-stat statements you should load a library and create an extension in a database When all conditions are satisfied and the extension is created stats are gathered permanently and can be viewed in the pg-stat statements view This view contains one role for each decision query user ID or database ID Also there are other useful stats related to these statements number of calls, total rows that were returned timings and resource consumption There are higher stats how many blocks are read from shared buffers or disks how many buffers are written or distorted Stats about I.O. related to local memory and temporary files and finally timing for I.O. operations There are many ways to use these stats For example, we can calculate average query time for a Postgres instance It's a database's common health metric which can be used to track database's response time Accurate this average time will go up Also we can analyze I.O. stats For example, to find statements that write data to the shared buffers more frequently than others or statements that created too much temporary data or use a lot of local memory Another way to use pg-stat statements is to make queries reports Model has functions for researching stats and it possible to gather stats for a period of time and make them into the reports This way of query reporting is more convenient than log parsing It's fast and doesn't consume much CPU or your resources A common way in this case is to calculate summary stats for all statements We can calculate the stats for particular statements and determine the ratio in the summary stats In this example the select statement has a total execution time of over 15 hours that constitutes 14% of the summary time Using the total time block read time and block write time cons we can calculate CPU or block IO time usage Thus we can make a report with top problem queries and optimize them Well, there are a lot of stats and unfortunately I have no time to describe all of them I will just briefly say a few words about other useful stats Besides stats that I have mentioned so far there are stats for tables and indexes buffer IO PgStats IO all tables and PgStats IO all indexes Also there is a view which describes functions usage stats PgStats user functions There you find the number of calls and execution time Also there are many functions for counting sizes of databases objects All functions can be listed with DF metacomment in Pesquale Official country package hasn't only PgStats statements but also two other interesting modules First is the PgStatTaple that allows to estimate the bloat in indexes and tables But be aware estimation time directly depends on tables or indexes size and consumes extra IO Second module is a PgBufferCache that allows to inspect the shared buffers It shows how many free or dirty buffers are there and their usage But using PgBufferCache functions can cause performance impact, which is caused by shared buffer scheme Besides official country package there are many third party modules for example PgFincor or PgStatkaCache They are unofficial modules and they are not in the country package PgFincor includes functions that helps to perform low level inspection for operation systems PgCache and determine which tables and indexes are in the PgCache Another module PgStatkaCache allows to determine CPU and disk utilization by square statements This is possible using geter usage system call that is involved before and after executing statements This module doesn't have serious performance overhead, but to use it you should Postgres 9.4 and PgStat statements enabled in your database Well, that is it and I want to sum up a few take home messages First, as you saw statistics can be very useful It allows to keep an eye on your database find and prevent different kind of problems and well next Second, stats in Postgres isn't difficult or complicated Third, stat can help you to answer the questions related to performance issues or problems in the database And last and important try to experiment with stat write new queries change them, optimize try to construct ideal queries for your Postgres Stealth information can be found at these URLs that I highly recommend and that is all Thanks for your attention and if you have any questions feel free to ask them and my colleague Lya will help me if I can problems with communications That is it Maybe you speak louder You had it read when that number was one thousand Yes, read number one thousand this is okay Can you help me? I take tuples in red if more than thousand everything is bad If you use OLTP or gloat we can see queries that return tuples and if query return too many tuples and it consumes more time if more tuples then more time and in our experience we consume we use this limit in one thousand tuples ask a red mark for any questions yep


and I'm delighted today to be joined by Dr Ashley Jackson. Ashley is co-director of the Center for the Study of Armed Groups at the overseas development institute ODI. As someone who has a lot of experience working in the humanitarian development sector in various countries around the world but has also done a PhD in war studies. I have the honor of being Ashley's supervisor and I got quite a bit of insight into the research that's gone into this book and the really impressive field work that she conducted in Afghanistan with civilians, members of the Taliban, hundreds of interviews in depth, the interviews as well and a very in-depth field work. So I'm delighted that Ashley is here today to talk to us about her book Negotiating a Survival and she's going to talk about the idea behind the book, the ideas of the book and also I think talk about the timeliness of the book and how it helps us understand what's happening today. After Ashley's spoken to us we're then going to be joined by Professor Matz Badal, Professor of Security Development here in the Department of War Studies too. He was actually second supervisor and Matz is going to just raise some questions, offer some reflections on Ashley's talk and Matz has also got quite a bit of experience interested in Afghanistan. So after that we will then go straight to you the audience for some questions. Now the way we're going to do with questions is you'll need to put them into the Q&A box. If you look at the bottom of your screen you should have a little Q&A box, so that's different to the chat box. If you type the questions in there I will try and get through as many as I can. I will post those to Ashley and my groups on together and we'll do that after Matz and Ashley have made their comments. So without further ado, Ashley thank you very much for joining us. It's great to have you back in the department again. Over to you. Thanks Garen. So you know this book is about really as Karen sort of talked about it's about two interrelated things. The first is life under Taliban rule and how people have navigated and survived the Taliban's return. All this research was of course done before August when the Taliban was still an insurgency, an insurgency that controlled much of the country but an insurgency nonetheless. And as Karen said I spent two years doing hundreds of interviews, traveling to different provinces, villages, in the country, talking to civilians, anyone who would talk to me about their experiences with the Taliban, as well as Taliban commanders and fighters, shadow governance officials, things like tax collectors and education monitors, just to understand what life was like under the Taliban. And I started this work in 2017 when most of the world had kind of forgotten about Afghanistan. I'd slipped from the front pages, there were a few international forces left and you didn't really have a sense of what was going on in the country if you were outside of the country. In fact you could find reports about the Taliban's shadow state in the Afghan media, but not really in the international media and had really scaled down NGO and UN presence as well. And that's just sort of to set the scene for how I went about this and what I found out. And in the book begins with a one-man story of how he's sort of survived, survived and negotiated life under the Taliban.  And I meet him at a point where the Taliban has just taken over his village effectively. And so they govern pretty much everything. And when I meet Haji Aman, he insists that life is better under the Taliban. And when I push further, it emerges that life is better because there's less violence. But there's a lot more problems for Haji Aman to deal with now that the Taliban have taken over. Because he's a village elder, people expect him to solve all these problems that directly or indirectly relate to the Taliban's takeover. The Taliban has stopped aid work that is happening in the village. They're taxing farmers more than they used to before they took over. They're doing all sorts of things. They're planting IEDs in populated places. They shut down the schools. And people are very upset. But they can't do anything, of course, because they're afraid of the Taliban. And they do expect that the elders like Aman will somehow deal with the Taliban. Now, at this point, when the Taliban takes over, Aman doesn't have much interaction with the Taliban. He knows some of them. He knows some of the people who've joined them. But he has to make linkages to who the key decision makers are in the village and who's actually going to help fix the problems, who has to say. And then when he finally meets them, he raises a number of these issues and the Taliban shuts him down on every single one. But he senses that at least on the schools, on reopening the schools in the village and the area, it's not a firm now. It's a no. But if he comes up with a plan, if he comes up with a proposition to the Taliban, he might be able to get them reopened. So that's what he pursues. He goes back, he tries to think of a plan to reopen the schools. But it's not as simple as going to the Taliban saying, you've got to reopen schools. It's really complicated. A lot of the schools, when they were shut down because of fighting were damaged, they were looted, some of them down to the window panes, they don't have books. They definitely don't have teachers, either the teachers are sitting at home or they fled the area. And the problem is the teachers are employed by the government Ministry of Education. And what Aman has to do then is convince the government to reopen the schools and hire new teachers. But he also at the same time would have to convince the Taliban to approve. Of course, they'd want to vet these teachers that are working in their areas. And so it's a delicate multi step plan that Aman has to come up with. And he has to work out all these kinds of moving pieces on all sides. Of course, he also has to convince parents to send their kids back to school and that it's safe. So he goes back to the Taliban and he says, look, I understand you don't want these NGO projects to go forward. I understand that you're not going to lighten up on the taxes, but can you reopen schools? I have a plan. And the Taliban's first reaction is to be annoyed. You know, I've already talked about this. We say we're not going to open the schools. But Aman does something interesting. He says, look, people are very, very upset. They don't like what you're doing here. They're too afraid to tell you, but I'll tell you what the community is thinking. And I can't predict that if you don't act responsibly, if you don't give them something, I can't predict what they'll do. And so when I'm interviewing him, he sort of smiles as he says this, because what he's doing is he's leveraging the community to get what he wants or to get what the community wants. And he gets away with it. It works. The Taliban has no choice after Aman has come to them with a veiled threat, effectively, but also a plan in which, you know, he's negotiated with the government, parents, convince some teachers to work, etc., etc. They have to give him something. They realize that they have to give him something in order to keep the community happy. And, you know, this is not a unique story. I interviewed scores and scores of people who did similar things in their communities, who had other kinds of negotiations, NGOs, businessmen, truck drivers, people whose relatives have been kidnapped or had been sitting in a Taliban jail.  Because this is what the Taliban has been doing for years. This is how they made a comeback. This is how they survived and how they endured was by negotiating coercively, violently, but negotiating with civilians. And when I first went to Afghanistan as an aid worker in 2009, this pattern was already kind of starting. This is not new. This is a decade old, decade and a half old of negotiations. 2009 was right before a big military surge, which was supposed to route the Taliban.  And even then, NGOs were talking about, quote unquote, using community acceptance to remain present in insecure or volatile areas. There were all these euphemisms around what was going on. But these were sort of the nascent kinds of engagements that Amman and others were doing on behalf of aid agencies so that they could continue working in villages as the Taliban started to reemerge. But the problem is, or the problem when I set out to do this research was that we knew so little about how these deals were struck. Nobody really wanted to talk about it. The stakes were high. The consequences of talking to you or dealing with or cutting deals with the Taliban were severe. So we knew very little about how these deals were negotiated, renegotiated and contested. But if you went and talked to people like I did, you very quickly understood that they shared information, they strategized that there was a whole culture of how to engage with the Taliban. The problem was is that it wasn't really incorporated into the way that we saw the war. It was an essential part of the way the war played out, at least from the Taliban's perspective. But I think in part because everyone was so focused on violence, on violent incidences, on big offenses, on IEDs, on the Taliban's spectacular attacks. This kind of course of negotiation that enabled the Taliban to survive and thrive was overlooked. And I think that was one of the many, many mistakes of the intervention. And that's sort of what the second part of the book is about. And that's how the Taliban returned to power. I mean, this is the other side of the coin, right? The Taliban built a strategy around this kind of engagement, around the combination of violence and negotiation to gain a foothold in village after village and expand right throughout the country. Now, I completed proofs for this book back in April, and that was before the Taliban made a sweeping advance through the north, where they took about 200 districts. And then they finally captured, of course, the capital of Kabul in August. The book came out in September.  Did this analysis actually hold up to how things unfolded when so many people were surprised when there was so much lamenting that, you know, how could this have happened? And so when I went back through, actually, it turns out very much of this tells the story of exactly how this happened. You know, if I were given the chance to edit it, again, I honestly wouldn't change very much. There's a typo on page 135, but other than that, it really does sort of leave off before the Taliban's takeover, but it traces the path. And at least 200 of the districts of the Taliban took in that offensive through the north in May, at least half of those 200 districts. It wasn't, it wasn't, you know, a really military offensive. It was a negotiated surrender. The Taliban used negotiation to take a lot of terrain. Now, a lot of this terrain, they already de facto controlled or they controlled 80 to 90 percent of these districts that were taken. I mean, maybe you would have one or two government buildings in the district center. This is the case in several of the districts that I visited that, you know, quote unquote fell during that period. But it was it was a Taliban area. You couldn't do anything without without the Taliban say so. And that includes government officials still living in those districts. They kind of had to negotiate the Taliban presence as well. But the Taliban controlled the terrain, the behavior, the movement, the everyday conduct of the lives of tens of millions of Afghans. And I think that's something that, you know, when we talk about color-coded 2D maps or if anyone's familiar with these maps that the Long War Journal or SEBAR produced, which sort of showed government control, Taliban control and contested areas. I mean, we've all known for a very long time those of us who've been engaged and trying to understand the conflict that those maps were dangerously deceptive and misleading because it's not really about flag planting and zero-sum notions of control. It's about these overlapping layers of often coercive influence, but also about providing incentives about, in the Taliban's case, Sharia courts, which even a year ago extended into the Kabul suburbs in areas that were a short drive from the presidential palace. You would have a Taliban court. So it's that kind of strategy that we failed to understand.  It was a failure to pay attention to what was really going on, to the ways in which Afghans themselves were experiencing the war. And I think it just sort of shows how outdated some of our notions are about how modern wars from Afghanistan to Somalia to Yemen are fought in one. But of course, the Taliban itself kind of incorporated iteratively this strategy over the course of a decade. And the third chapter of the book really focuses on the Taliban. The rest focus very much on civilians. But they combine, and they learn, they learn by doing, they learn by successive sort of experiences of trying different things in different places over a decade and a half. They developed this strategy where they combine violence with persuasion, with incentives like allowing schools, such as in Amman's case, or Sharia courts and the services going forward that they provided. And they also use social capital as well, the networks and relationships and emotional ties that allow them to gather intelligence, but also to negotiate with people. You know, I interviewed village elders and even government officials and security officials who had brothers, cousins, uncles, and the insurgency. The Taliban for the most part isn't from outside. There are commanders and fighters and were commanders and fighters and shadow governors from the actual communities. And that allowed the Taliban a foothold, a way to engage with the population, but it also allowed the community a way to pull on those emotional ties. You know, you should provide the schools, you shouldn't plant IEDs and the bizarre, et cetera, et cetera. So civilians use all of the same kinds of elements to negotiate with the Taliban. They don't have the kind of capacity for violence, for example, that the Taliban has, but they can withhold or grant their compliance to the Taliban, just like Haji Amman did, with that subtle threat of I can't control what will happen. On the other hand, he could deliver kind of the collective compliance of the community. He could manage the discontent of the community for the Taliban. And people leverage this ability, these offers of help to the Taliban, whether it's to inform for them, to act as intermediaries with aid agencies, whatever it is, they use creative forms of leverage to get protection from the violence that the Taliban would otherwise inflict or to get benefits either for their community and their family or for themselves. And, you know, we see this play out across society, you know, NGOs, trucking firms, these entities can offer doctors, health clinics can offer things that the Taliban wants, whether it's medical services, whether it's bribes, whether it's taxes, whether it's aid projects, and people very quickly learn how to, again, negotiate with this, to use this to their advantage. And the Taliban also, particularly after 2014, after they really gain a significant territorial foothold, they realize that they need, they need these entities, they need to provide schools, they need aid projects, they need businesses to keep flowing because they're preparing to take over, they have more of a a state-in-waiting like posture. They're building this sort of shadow state. As hollow as it might be in some respects, it's the infrastructure and the performative aspects of the state which start to influence their behavior where they really try and get UN agencies to work in their areas and those kinds of things. But I don't want to go on too long because you can, of course, read the book and I'm interested to hear what Matt has to say and any other questions, but that's effectively what the book is about. The centrality and the complexity of the Taliban's relationship with civilians in the past two decades of war in Afghanistan and how, despite such a focus on counterinsurgency, stabilization, hearts and minds, how deeply misunderstood and overlooked the civilian perspective has been. But I'll leave it there and I should also, before I pause to hand over to Matt, thank you and I should thank them not only for being here today but they've been here since day one of this journey being my thesis supervisor, having to navigate the ethical clearance for this research, you can imagine, wasn't very easy. I'm providing advice on exactly how to do this all along the way but with that I'll pause there and back to Kieran and Matt. Matt, would you like to come straight in? Sure, thank you very much. Well, thank you so much for this opportunity to, I think, open the discussion more than anything else and as you've just heard now, I was actually very privileged to be able to act as a second supervisor on this project. It started as a PhD and has now become a book and the first sort of general point I simply wanted to make is the extraordinary importance of this work as an example of research that tries to look at the relationship between the Taliban and the civilian population. When I say the importance of it as an example, there are many other cases, many other contemporary conflict with that particular dimension has also been neglected, as you say, and has been overlooked, leaving us really in the dark when it comes to understanding the dynamics on the ground and I think that is incredibly valuable and of course I think that was brought out and I'd like to talk a little bit about this when we come to the dramatic collapse of the regime in August. I think by any measure it was in many ways an extraordinarily intelligent failure not to see and predict the nature of that collapse and the speed of it. Now many will say that we saw it coming but not the speed and the nature of it and it's fascinating as you say and it's certainly a great credit to your research that you didn't actually have to go back and change your manuscript much afterwards because your analysis of relations between the civilian population and Taliban up to that point made this particular outcome likely. But I would like to ask you about the nature of that relationship a little bit more in greater detail. I mean you are right, it is a pioneering work in that respect but there are others of course in this debate which is now developing around the nature of the collapse and so on and so forth. Others have emphasised that and probably gone further than you, you might correct me here, that the Taliban for all that we know about it as a movement gradually developed and certainly in the areas it controlled a degree of local legitimacy and even some public appeal. I'm quoting here people like Anna Tom Leven and Gilles Doronsoro. I think you emphasised in your presentation more the element of violence and negotiations and simply stepping into that particular vacuum. But I wonder whether you'd care to comment a little bit on that view that in actual fact it had an appeal because it was actually compensating for central government and deficiencies and that to some extent it responded to popular demand for certain kinds of public services. And the failure to see and understand that was one reason why people didn't understand that essentially the takeover was largely one of in many cases negotiating a transfer. The second sort of big question I suppose building on that I wanted to ask you, you have emphasised the closeness of that relationship over time and the degree of sort of penetration, I mean uneven of Taliban in certain areas of people's life. I wonder what that means about the future and where we are going now. It's a very lively debate as you know at the moment in the United Nations in many capitals about how we should relate, how we should engage with the Taliban and how aid of various forms humanitarian and others should be channeled through to them. I wonder whether the fact that they have established governance structures, whether that opens certain possibilities which might not have been there in the future and how you see that developing in the months ahead. We have the renewal of Yunama's mandate for example coming up in March and I think there is a lot of debate in capitals about the most meaningful way of engaging it. Does this history that you have outlined for us provide us with a very different starting point from earlier. So those are the two broad questions I wanted to start off the discussion but I wanted just to congratulate you again and also encourage people to to buy and read the book because it does provide a very different part of the story one which as I suggest you know the events of August and illustrate have been overlooked and neglected as you suggest. So I leave it with that Ashley. Thanks Max for those really thoughtful comments and incredibly difficult questions. I'll do my best to answer them just starting with legitimacy and public appeal. I really just like the word legitimacy just as much as I like the word support when people talk about you know support for the Taliban these kinds of things and I say that from a place of having started this project unconsciously kind of using those those frameworks but learning the hard way just how inappropriate they were and how they weren't the right language to reflect the civilian experience. I don't think civilians thought about legitimacy in the way that that we do of course they didn't think about it in the way that scholars do but first of all when you talk to someone it's very likely in a war zone if you ask them how they feel about one of the combatants they're going to give you a false preference on the first ask and you'll be very lucky if you get down to some sense of how they really feel which is when you do incredibly complex and not always a coherent as it wouldn't be because emotions are involved because fear is involved because often there's no good option which I think is how most people I met felt they didn't like the government they didn't like the Taliban they were exhausted they had been born into a war a lot of the people I met were obviously yeah they didn't know a time when there wasn't a conflict after 2001 there was a sort of brief reprieve but it almost felt like a re-traumatizing false start to many of them and so I think what we have to our starting place has to be that people are exhausted that they probably don't see any option as ideal and they may do with what they can and they also think about the short term because because the present is so volatile it's very difficult to think about the future right to think about what happens next I mean towards the end of my research it was clear in certain areas that people were expecting expecting the Taliban to take over and not expecting peace negotiations by the way to to work out terribly well which of course they were right but so they were poising themselves for that they were strategizing for that but a lot of them couldn't think beyond next week or etc in it so this idea of support I think it needs to be rethought I talk about compliance there are other scholars and our Jonah others who use different terminology but there has been a lot made about okay the government was so bad that the Taliban came in with these services and these these things and they they won popular support they they weren't corrupt or they were able to fight justice with these sharia courts when the the afghan government courts were notoriously complicated corrupt and inept and there's some truth to that I've done really extensive work on the Taliban courts trying to look at how they work trying to talk to people about their experiences and that's an interesting example in the fact that you know the Taliban's justice is very simple it's still political there's still a degree of corruption um but it's easy for people to understand it's more reliable but it's again like a lowest common denominator equation it's it's the best that they can get um and I think we we underestimate that that that you have a situation in which people are so exhausted and so desperate that you know they just they want what you or I would want in that situation they want safety security uh and they want to be left alone to live their lives and Taliban administration is inapt you know beyond courts they they aren't made of bureaucrats they're the opposite of Ashraf Ghanian ex-world bank official they they don't know how to run things they know how to wage a military campaign but I think what we're saying now is is there severe limitations in what they can provide people and also at this research I should say people were very very clear on that right they didn't they would talk about how even the Taliban's version of Islam was an ignorant version of Islam you know they didn't really know anything they certainly didn't know about schools or all those kinds of things so I think we have to question those narratives of the Taliban people winning because they were somehow better or more legitimate or they were able to provide more than the state I think it's more complicated um unengaging with the Taliban now and whether or not aid is a means to do that I I think aid is a means to keep uh thousands tens of thousands hundreds of thousands of people from starving to death at this point like that is that the humanitarian imperative of um a looming famine has to be how how aid is framed I think what we've seen is that the Taliban what I've written about is they're really good at negotiating they they drive an incredibly hard bargain that's what the book uh documents is that you know this is central to their military strategy their political strategy what we've seen in Doha they gave almost nothing up and got a US withdrawal um so thinking you can sort of buy them off it's not going to work right it's absolutely not going to work you may be able to persuade them through kind of multi-pronged diplomatic strategy to do certain things um but they're very unyielding so to think that for example um you can get them to embrace women's rights if you if you get in an affidavit that's you know that's an overly simplistic argument but that's not going to happen what we're seeing work on the ground though with girl schools is local level quiet local level negotiations province by province district by district you're seeing more girl schools reopen it isn't an official Taliban policy at the top there's this weird ambiguity where they told girls to stay home etc but you're seeing things on the ground be negotiated and I think that's how it works that's how it will kind of unfold the international community strategy then has to be based on okay what is going to enable different contingents and constituencies in different parts of the country best negotiate with this new government and that's that's what it's about that's what it's about the empowering afghans themselves in incredibly impossible difficult circumstances to figure out the way forward and to negotiate it as they have been now for for many many years um but I'll I'll leave it there for for the moment thanks actually for the uh for the excellent questions and comments as well we've got quite a lot of questions already in the q&a box um as I mentioned if you if you have a question um just pop it in the q&a box and we'll we'll try to get through as many as we can um I'll start from from the top perhaps we'll start with one um I might bunch some more together later on um so the first question from Daniel could you elaborate on why does he believe the dynamic that you've described um has remained unknown in the white discourse around afghanistan and foreign assistance more generally where do you locate that failure I don't know I mean there's a lot it it surfaces in different ways right there's a huge uh literature industry around humanitarian access negotiations right and that's what part of this kind of overlaps to um this dynamic exists in those in humanitarian diplomacy this is how aid agencies have been operating for years in in places from biafra to afghanistan in the in the eighties and nineties to you know you name it um so it exists but I think the way we conceptualize conflict and the way you know certain post 2001 narratives around counterterrorism really um shut down the ability to see things clearly on the ground to engage with the taliban to engage with civilians directly to understand what kind of deals they were striking I think you've had a bunkerization of diplomats and others and afghanistan which prevents them from actually getting the knowledge that they need and the insights that they need to really understand the situation instead they're they're caught they were caught in sort of embassy green zone echo chambers and that isn't to slam the people who are in those jobs who are often very intelligent and committed but it's the infrastructure of how we engage in these places that we build barriers that we put people on sanctions lists that they can never get off you know that fit it puts blinders on and they're also discursive blinders right you can't I'm sitting in in Nairobi now trying to um learn more about how you know al shabbat and how these dynamics might be more different uh or similar to afghanistan and you have less space to talk about engagement with al shabbat they've been around for decades how are you ever going to understand why they keep coming back and why everything you have done so far has failed to solve the problem that they present if you don't engage with them if you don't listen to them I think that's part of it as well is that particularly in the post 9-11 era we've stopped talking to our enemies um there's rarely even sort of back tunnel kind of dialogues in the way that there might have been during the cold war let's say or that there was you know with the IRA or others there are other historical examples where I think we've moved we've moved backwards and what it's meant is that there's no way out of these wars other than either keep fighting somehow uh or withdraw and see the ground as they've seen happen in afghanistan uh and there has to be a better way honestly um so yeah I think there are many dimensions to that to that problem there are there are quite a few questions about responses and a few questions also in more specific detail on the on the argument you make in the book I'll start with the um maybe the kind of the responses one so Antonio asks a question actually there's two parts to this question the first is um why do you think the US and afghan authorities um have failed to tackle the less spectacular elements of the Taliban interventions in local settings so for example you know um imposing order there's so much academic attention on governance failures yet they've not addressed that which I guess is partly what you've just been talking about the second question this is a separate question um he's asking about methodology how did you overcome the logistical and security challenges in digging to using rural Afghanistan I want to combine that with um part of the question um from Nicholas Barker as well who asks um could you talk more about the ethics involved in conducting the research how you dealt with issues like informed consent and managing the risks to interviewees and to yourself I think that's similar to the methods question that Antonio was asking yeah maybe I'll I'll start with um the governance values I mean there's a whole industry around subnational governance, state lesion all these kinds of things we've just talked about where the one would try to understand this those dynamics and address them right but that's not really what happened on the ground in Afghanistan what happened was the USID gave millions to um contracting firms who had burn rates and had to spend money on ridiculous things that money um fueled corruption fueled then insecurity created a war economy that perpetuated insecurity and also disenfranchised and alienated actions from their own governments because things didn't get better they saw the corruption that that aid that was meant to help them instead fueled they saw the government getting weaker and more corrupt and they saw the violence getting worse and they saw themselves unable to protect their families unable to um access the kind of futures that they hoped for for themselves and their their children and unable to sort of envision a future in which you know the ones that they were kind of promised after 2001 um so why do I think that they were unable to tackle it was because they didn't try in the right way I think there were those myself included who were around at the time of the surge or you know when these questions were front and center as part of the counter insurgency campaign who were pleading for different solutions calling out that these would only make things worse um AstroCirche others have written about this wrote about it at the time everyone's all this coming so I don't know I think there's part of this which lies in incentivizing with hundreds of millions of dollars in industry you know of consulting firms in London and DC and elsewhere and contractors and all these kinds of things and indeed NGOs and UN agencies to play into these narratives which misdiagnosed the problem um and and fueling it with cash right I think it's I'm pretty cynical about all of this I think if you had one percent of the money you probably would have been better off I think a lot of Afghans don't find that they they feel they benefited from all of these interventions for quote on quote about stabilizing and providing them with services because at the end of the day they don't have many of those services they don't have many of those benefits um so it's both incompetence lack of accountability um a range of factors again uh but I keep going for a very long time talking um I guess on methodology and ethics um obviously Kiran and and that's where they're all the way through the the standard uh university ethics process but everyone knows it's more complicated on the ground when you're sitting with someone when you're kind of trying to figure out how to do this safely I would say I moved back to Afghanistan and I spent you know a year and a half pretty solidly there and then many trips after that going back and the first three or four months we're just figuring out how to talk to people safely and you only really do that by trial and error informed consent doesn't come from you know a tick box it doesn't come from handing someone a form right we all still have to kind of do that but it doesn't it actually doesn't address the ethical dimensions and we all also know that so you have to be able to read the room for me that was also about working with people who I developed really strong relationships with who are my partners they were not just translators or fixers or whatever you might call them they were collaborators who we had a responsibility to each other to try and understand the situation and respect each other's point of view on it and listen to each other and and talk to others to try and understand the best way to go about things but it was also about a lot of it was about again these relationships about nonverbal cues about understanding what someone was trying to tell you when what they were saying was something different which I think is often common in Afghanistan and had I not worked there for a decade I probably wouldn't have picked up on a lot of the sort of ethical cues or discomfort cues or you know the sort of for lack of a better term double speak that was that was going on and I'm sure that I still miss things right so I think ethics in that sense was very iterative and it was very situational to try and understand you know was it appropriate to meet someone outside an area where they were coming from this Taliban control yes often it was better to meet them outside would it be appropriate to talk to a woman in certain circumstances that you know would be different from talking to someone else it was also about you know selecting different interlocutors again a lot of it was about endlessly calling people for information and trying to triangulate and trying to learn from the experiences of others and when I say others I mean mostly Afghan journalists because they were the ones out there and were incredibly generous with their time and their advice to me about how to do this safely and respectfully so yeah a lot of it was just learning by doing but also having a degree of humility and listening to people who were smarter than I left about all of it thank you there are a few questions about kind of the implementation or policy responses I'll come to those in a little while I want to first get to the questions that are talking about kind of the nature of hierarchies and the local variations so there's the first part of Nick was his question was to what extent were Taliban strategies for dealing with civilians directed from the top or did lower level commanders have the freedom to adapt and improvise on the ground according to their own interpretation of what would work or what was necessary and then Ivan Yashawa asks well as well congratulations on your book and he asks about the point about social capital do you observe some variability in relationships between Taliban and civilians depending on the level of social capital they enjoy in different locales so you know in places where they recruit combatants versus places where they don't recruit so I guess to some extent a question about local variation and local autonomy yeah those are both really good questions and I'll try and answer on this concisely if I possibly can so when it comes to the top down versus bottom up I think we saw with the Taliban insurgency from roughly 2006 onwards when I start to get a little bit organized is this process of you know commanders at the bottom are it's really disorganized right there are these little like fighting groups and there is this leadership in Pakistan it's really disconnected they're all trying to figure things out in real time and you have fighting groups on the ground making decisions making policy and the Taliban leadership at a certain point seeing what's happening and saying okay we have to institute hierarchy but when you do that as things are moving along it's necessarily top down and bottom up and I think that's what we've seen with Taliban policy making another researcher Ramatullah Miri and I write about this about this top down bottom up policy making in a report for USIP because that's what we found was like okay how do we explain the fact that it's not totally decentralized and it's not totally centralized that there is this give and take across the country with different different nodes and different layers of complexity but when it comes to this engagement with civilians at a certain point what the Taliban leadership understands is it is absolutely central to their strategy to their political and military objectives that there's a degree of autonomy at the local level to cater to civilian preferences and desires and to keep the civilian population who is very pressured and very coarsed by the Taliban but to keep them on side to keep from pressing them too hard and also to keep local commanders happy right Afghanistan is an incredibly diverse country even amongst the Taliban there is a diversity of opinions about things like girls education or about you know the role that elders should play in local decision making and you have to accommodate that you can't push these communities so hard that they rebel I mean this is the insurgent dilemma you know how much can you squeeze them without alienating them and pushing them to the other side and I think the whole Taliban structure and we're even seeing it play out now that they're in government is that there is okay this acknowledgement of okay we formulate policies at the top that are broad and vague enough to encapsulate a range of practices at the local level but over time as the insurgency went on especially with things like taxation or things like education you saw the the leadership trying to narrow the parameters of that local variation so that's that's kind of the dynamic but when it came to striking these local deals local commanders had a huge degree of autonomy about how they how they dealt with civilian populations just in so far as they were able to continue to win basically they were able to continue to survive the Taliban leadership wasn't hearing about massive discontents then then they they had the ability to do that now when it comes to the second question social capital one of the one of the big conclusions of the book was that for civilians social capital was everything especially low in civilians I mean NGO's aid organizations they had things to offer the Taliban community elders could deliver on like compliance of the community but when it came to like ordinary people their connections to the Taliban were everything so it was not only about you know where you're related to a Taliban commander's how did you cultivate links across all of all of the people you had to negotiate with you know the the Taliban commander's on this side the Taliban commander's on that side of you government officials you know how could you leverage business connections people you went to school with your uncle your wife's uncle someone you used to work with how could you leverage this network in which you embedded in where ethnicity tribe work all of these things that we would call networking really in our lives how could you leverage that to to navigate a safe way for yourself your family your community so that's yeah as nebulous as that is that is one of the main conclusions of the book is that social capital is essential there's a question by jury which is linked to that so i'm going to throw it in um you can you can come to it perhaps um along with these other questions or not you've kind of answered it to some extent which is isn't the core of your book really what factors determine when the Taliban may be more yielding to community preferences versus when they respond violently to pressure from community so um kind of probably should have tapped it on with with advanced question um i wanted to ask you Tina's question as well who thanks you for the interesting presentation and discussion how do you think this kind of important research could be better disseminated to policymakers so that they could incorporate it more in their policies um in the slight risk of overburdening you with questions i've also asked Sophie's question on there because it's also about the implications of your research so Sophie says i wanted to ask whether you see the Taliban is different from other insurgent groups is there something unique in that approach to negotiating or what more can we learn about insurgency and counterinsurgency from the experience of the Taliban um those are again three really tough good questions we'll try to be concise um in terms of jury's questions it's very good one um a lot of things is the short answer i think the the last chapter before the conclusion picks apart all the different sort of um temporal and other aspects i think one is where an insurgency is at in their evolution and that evolution as Demsher Jory will know well is not linear right you can sort of have a really organized insurgency on the ascendancy that gets not knocked back disintegrates has less capacity to negotiate etc i think there's consensus in the literature people study this and i agree when insurgencies are disorganized they lack command and control they lack these structures to negotiate they lean back on violence and that's what we really saw with the Taliban and the third chapter in the book goes through their use of violence pretty extensively because i think this is a very important question to to dissect because violence is not only about the levels of violence i think you know the levels of Taliban violence over 2020 were eye-wateringly high yet they were using that violence in much more strategic controlled intentional ways than they were in 2006 and 2010 even in 2014 so you have to look at the types of violence you have to look at sort of targeted assassinations you have to look at the use and placement of ieds you have to look at the way they treat prisoners for example those kinds of things violence can also be a form of negotiation it can be part of that large larger equation in which this halavan is trying to force the community to do something it is a form of leverage when it is more strategic so i think we have to even break down violence into understanding you know decoding what different types of violence mean at different points of time but in terms of when the Taliban responded violently to like pressure from the community in Afghanistan i think it's different from other contexts in that you didn't have a lot of like uprising spontaneously or otherwise against the Taliban especially not many that um were not externally supported or organized you had militias and things like that supported by the americans and others but they're only really a handful of instances i can think of where the Taliban had to crack down on it where the community kind of pressured them so much the Taliban felt that they had to you know it was more this give and take a dialogue and even where the community revolted there were several instances with Taliban took that seriously and then replaced commanders who were abusive for example um so i think the Taliban might be an outlier in that respect that you you had this language of negotiation and communication with violence in the background of course um but yeah that that was a working a working modus operandi i guess we can say um when it comes to how to better disseminate honestly i supported and funded this work by writing policy reports i've been an advisor to the uk parliament and two ambassadors on my hozzad and come with aid agencies and un agencies and i have done interviews and tweeted and done everything i can to try and use this to influence people's thinking and framing of the of the problems in real time as much as an kind of academic policy wonk like myself can do that adeptly i've at least tried i've written foreign policy articles i've written new times editorials but it doesn't matter like it doesn't matter if people don't want to actually see it if they are incentivized not to see it if there are discursive barriers from even talking about something that challenges the dominant military foreign policy narrative you're just going to be screaming into the void i mean occasionally you'll do a briefing where people think that what you have to say you know it is that second narrative you know they have their official work narrative and then you say something that earns your respect because you say what everyone in that room knows or is thinking or the smarter people in that room know or are thinking but it doesn't seem to change policy um i don't really know i mean i think the one thing that that does sometimes change policy is confronting it in the media or calling it out directly um i think you know i i subsidize this work by doing a policy report thanks to the Danish government who funded my work on a Taliban shadow state and i guess that policy report at least where where i was sitting in call at that time made a lot of people really uncomfortable that their aid money was going to schools the Taliban was uncontrolling and this was not i mean this was not like headline everyone knew this was happening but to put it out in the public domain and take that second narrative and put it in a space where it's unavoidable can sometimes be useful but i actually don't think i've had very much success impacting policy for those reasons um and i you did kind of overburden me so i forgot um what the the third question was um it was about lessons about whether the Taliban is is unique or yeah generalizable and about counter insurgency as well yeah i can't really say i'd like to offer an opinion i think towards the end of the book i try to offer an opinion that hey some of this could apply to Somalia or it might play out differently if you were to apply a similar theory to what's happened over the past um eight to ten years in Syria but i really really dislike when people opportunistically use afghanistan in that way when they haven't spent the years on the ground talking to people and then make conclusions about oh this theory is surely what's happening there i didn't want to be that academic um i really wanted to engage deeply on one context and allow others who work on democratic republic of congo or work on Somalia or Syria or wherever to if they found parallels to use that to help them find ways of explaining things i think you know i really react strongly to um a lot of academic books that use a cross-section of case studies where people haven't done the deep work and i just think it's unfair and i think i think that's ethically irresponsible and i draw a lot of inspiration from scholars like jute for weijan for example who i watch to talk of her is that um one of the the participants here don't know what if i go and organize where she really talks about hey you know i know this context and i know my limitations and i know what i can actually add to the debate and it's not by overstretching myself to make comparisons about what's going on with the mind i being similar to what's going on with the taladan or even another another group and in the same country and so i would just respectfully leave that to people who know those contexts better i hope i hope there's something useful for non-afghanistan uh readers and i think i think there might be because i draw inspiration from people like for weijan from others and and certainly their work is influenced by but i wouldn't want to force it we have um we have a few questions kind of more focused on the the more recent developments or the aftermath before we get to those i'm just asking one from henrik who's talking about third party interventions um empowering in quotation marks local communities you know for prt's or through NGO interventions and how did those influence local communities relationships with the taladan for example did it increase in awareness of ihl by the locals did that lead to better or worse treatment by the taladan uh i think you know one of the dangers of writing about how ordinary civilians negotiate with the taladan is that you risk um people misinterpreting that to mean that they have an extraordinary power to influence the taladan that's not what the book at all says i think there have been a lot of strategies in afghanistan political and aid development strategies access strategies that have put civilians in the middle in ways that have been unrealistic and borderline unethical um people are doing the best they can negotiating for what they can engaging with the talaban as they can and many people don't they flee i mean as we've seen after august but as we've seen certainly in the years leading up to that they um join the government side they refuse they avoid engagement as for as long as possible but of course no one can kind of engage uh or sorry and avoid avoid engaging with an insurgency on the ascendancy for forever i mean it's just impossible they they want to control your life and your behavior and at some point you have to find a way to navigate that so you know i guess i'm skeptical like there have been community mobilizers or and elders this example of a man or other people i talk about in the book who do extraordinary things but putting that burden on people and those kinds of like um cookie cutter or lack of a better term interventions uh that were meant to quote-unquote empower communities i think in the face of violence uncertainty and an international community which was always going to leave and people kind of always knew that and a talaban that was always going to be there and people kind of always knew that they were they're very limited in the impact that they could actually have well the question from from kaiser um i'll follow that by more from jasmine kaiser asked how flexible is the talaban in terms of its ideology is it willing to stretch it to gain more power and then jasmine um asks why do you think that the power the talaban didn't really comprehend fully comprehend the challenge that they faced by taking control of the country for example the bureaucratic challenges the government's challenges which he says is a british afghan i have first hand contact with people on the ground and there seems to be a kind of a limbo at the moment with those who previously had private and public sector jobs and they're waiting for government this governance decisions to be taken yeah um well when it comes to ideology the talaban is very flexible they're very pragmatic which is really ironic when you think of the image of a hard line sort of talaban driven by their their vision of Islam well it turns out uh actually that can be very flexible when it contradicts with uh their military and political objectives you know and i i think we've seen that play out with the way that they have developed a taxation system of an insurgency using the idea of the caught snusher and ways that um even to ordinary afghans and villages i i wouldn't i wouldn't render a judgment about what's islamic and not but what ordinary afghans told me was like you know that's not everyone knows that's not that's the talaban justifying taxation as islam because they need money for the war effort you know so people understand this but there's also the role that the ulema and you know the ordinary village mullah or whoever plays in the talaban is not this sort of um removed a political role it is very much part of the structure and the talaban's vision of islam and its ideology is very much formed around its battlefield objectives and i recently did a paper um with again uh the the score i mentioned before ronald's willemiri on talaban narratives around al qaeda and their narratives around al qaeda like their narratives around a lot of things you know whatever helps us to win the war whatever helps us to to continue to control things to continue to fight is that is islamic that is right you know there's no contradiction there um and this this mantra of of things is is interesting because it covers up a lot of differences of opinion it covers up a lot of tensions but ultimately i think you have a very um flexible talaban ideology when it comes to what they themselves perceive as something that um threatens their survival expansion ascendancy that doesn't mean that they're going to be flexible on other things right the things that one might want them to be like women's rights like all those kinds of things so i i qualify that flexibility by saying it's it's on their own on their own terms but i think you find that with with a lot of a lot of insurgencies the second question is really interesting and you know i share your observations with friends on the ground is sort of looking and watching the talaban i don't think well we know for a fact that they did not expect to the country to go to them so quickly this the what seems to be here is this big sweeping offensive in may was geared at securing a government capitulation um they figured that the government would eventually come to them with some kind of deal whether it was to remove ghani and form some other kind of power sharing deal we don't really know what was ideal but we do know that that's what some of the talaban leadership was expecting that's been documented that heard that as well in in conversations with people um close to or in fact in the the political commission so they did that and then they gained enormous funds of territory they didn't realize i think how open the door they were pushing on was um that is sort of like a doggie door they kind of slipped i think you know i had that so much of the country went to them then also um led to the more hardline military commanders who had been arguing for no peace talks and a military take over all along it it gave them greater legitimacy and credibility and i think there were many of them and we saw this in different parts of the offensive i won't go like into the weeds here but we saw commanders acting of their own deletion really going for certain small cities and things like this which they then couldn't control there was backward forward movement rape obey but then what happened with the cities is you saw the taladan lining up to sort of choke them and to prepare to besiege them and yet they just gave way the aking government never really sent the kinds of reinforcements never had a clear strategy as to how to how to fend off this taladan offensive and then of course the taladans around Kabul ashut khanifis the taladans in doha say to the americans are you going to provide security are you going to you know or we are waiting we're waiting up and they were you know in the city limits and the americans declined to do so and then the taliban took to Kabul so i think and there are definitely those within the taladan movement who understood that this was an absolute disaster i mean how could they not they were not prepared to take an entire country and insurgencies the rare few that game power militarily or even through peace talks they have time to prepare somehow they have time through especially if there's political dialogue if they have sort of capacity building you saw this in what is now south sudan you have this time it doesn't necessarily mean that things will go smoothly that they will ultimately have the capacity or good judgment to run a country that's how they didn't have any of that they were kind of moving from one day to the next and have not and still do not have the time to think about these big picture issues that they also do not have the capacity they just don't have the people and many of the people who they would need who they hoped to co-opt i think rather naively have fled the country so what we're seeing now i think is a taliban leadership that is really grappling with the the catastrophe the catastrophe of their success basically um yeah that's was really interesting also quite depressing in a lot of respects um at least for the short term um part of what you were saying links back how to do to the last question that we have which is about you're talking about how you know there wasn't the resistance that they were planning for they were expecting it um is that in to some extent connected to this the negotiations that we're taking place on the local level so daniel had asked this question earlier in the wake of the afghan government collapse do you see any additional political or security implications of the widespread bargaining around service delivery to be documented for example do you see the connections developed around service delivery bargains having facilitated the negotiated surrender of the afghan government at a sub-national level during the 10 days leading up to august so it's interesting applying your kind of you know the focus of your research on on on that itself absolutely i think you know bargaining around service delivery was also bargaining around governance was bargaining around who who will control and govern our lives and that laid the groundwork but also created these relationships and those social capital networks and you know it reconfigured people's consciousnesses about who really had power and who really had control over their lives regardless of whether or not there was still government presence regardless of whether or not there would have been a map that said that area was under political government control it was really the taliban um but i think also what you you have is is effectively a situation in which the the taliban negotiated and coercively negotiated and pressured communities into a situation in which um they told these communities year after year that they were coming back you know even even during the surge people knew that the taliban would kind of come back um certainly this happened after major offenses after 2014 even in the trump era people knew that the taliban would come back but what happened with high level political negotiations and negotiations that started in 2018 in doha was you had a gradual process of confirmation of all that at the political level what you had was a recognition of the taliban already having kind of the authority already being a political force to be reckoned with and an understanding that their return to power power sharing absolute power whatever would be the way that this was going in the medium term but on top of that the way that these uh negotiations unfolded the fact that the afghan government was excluded the fact that in response to these bilateral negotiations that excluded them that were between the us and the taliban it fractured it didn't solidify you had a disintegration of the political alliances that underpinned the afghan republic um and you really saw the psychological effect of the us sitting down with the taliban and indeed signing a deal without even really genuinely getting the buy-in of the national government that they were supporting um you saw the psychological effect of that and I think ultimately that also influenced um the nature of the reactions or non-reaction to the taliban advance that people just fell back that soldiers fled into shijikistan or they um struck deals I mean many many did fight back let's I'm not going to underestimate the fact that there were bloody bloody battles but certainly in many cases people understood and have prepared themselves for that moment and and that you know why would you why would you fight the inevitable you wouldn't and I think that's that's really what happened thank you very much for for sharing this insight with us I mean it's I learned something every time every time we have these kind of conversations or I hear you talk so the detail the level of knowledge you've got from from from your research is is really useful and maybe like this to the last very very tough question um where can people find your book who is it published by where is it where is it available and which is it going to cost them um it is available from Hearst Publishers you can get it on Amazon you can get it on bookshot you can get it you know wherever whatever books are sold sadly there's not a kindle edition quite yet but I'm told that's coming in a week or two um well I mean it's always possible that Amazon is lying to me but it seems to think there's a kindle edition there's also okay that's great that's new to me yeah I check it out there's also a very affordable hardback which I have 23 pounds for the hardback so um snap them up while they're there um ask me thank you so much genuinely just really really valuable research and um also you know it's the it's the honesty with which you've done the research and reflected I think the views using the ground that I find really really useful in this um and obviously more timely than perhaps even you would have imagined at the time of doing this research but I think it shows you know just the quality of the research that you've done that um you only have that one typo on page whatever it is well that that I know about I mean yeah if we find any more we'll we'll send them to you yeah Matt's um thank you so much Matt for for the brilliant um questions and reflections as well um hugely appreciate it um thank you everyone for your questions and um the recording will be available on the War Studies YouTube channel um relatively soon and we'll share that online so thank you everyone for joining and thank you again Ashley thank you so much


Hi, I'm Ravi Kallon, Parliamentary Secretary for Sport and Multiculturalism. Welcome to the BC Human Rights Commission Public Engagement website. I've been asked by the Attorney General to consult with British Columbians and make recommendations on the re-establishment of the BC Human Rights Commission. Currently, British Columbia is the only province in Canada without a Human Rights Commission. That's about to change. I believe that taking proactive steps to address discrimination and systematic inequalities will make life better for all British Columbians. You each have a perspective. Your stories and ideas will help shape the role, functions and priorities of the Commission. I invite you to join the online discussion before November 17 to help create lasting changes for British Columbians. New questions will be posted regularly, so come back often. In addition, I invite interested groups to submit their written comments to us directly, and I look forward to meeting with British Columbians in person and throughout the province to gather all viewpoints on human rights. By the end of December, the feedback we have gathered will be summarized in my report to the Attorney General. I hope you will take the time to view our site and join in the public discussion. I look forward to hearing from each of you and joining in the discussion myself. Thank you.