Good morning and welcome to the 21st meeting of the Public Audit and Post Legislative Scrutiny Committee in 2018. We have apologies today from our convener, Jenny Marra, and from Willie Coffey. At the outset, I can ask everyone in the public gallery to either switch off their electronic devices or switch them to silent mode so that they do not affect the committee's work. Looking at the agenda, we need to take a decision on taking business in private, so do members agree to take items 4 and 5 in private? Thank you, that's agreed. I would like to welcome our witnesses today to look at the Children and Young People's Mental Health report. First of all, Caroline Gardner, Auditor General for Scotland, good morning. Claire Sweeney, Audit Director, Performance and Best Value, good morning, and Lee Johnson, Senior Manager, Performance and Best Value for Audit Scotland, good morning. I would like to invite the Auditor General for Scotland to make a brief opening statement Thank you, convener. This report looks at children and young people's mental health services across the country, and it finds that services are under significant pressure. Demand is increasing. Over the last five years, the number of referrals to specialist services has increased by 22 per cent, and children and young people are also waiting longer for treatment, with more than a quarter of those who started treatment in the last year waiting more than 18 weeks. The Government's mental health strategy focuses on early intervention, but we found that in practice this is limited. The current system is geared towards specialist care and responding to crisis rather than identifying and helping young people early. Access to early intervention services, like school counselling, varies across Scotland. The system is also complex and fragmented, making it difficult for children and young people to get the support that they need when they need it. Accessing the right services needs to be easier for children and young people, for their parents and carers, and the professionals who work with them. We did find examples of good practice and projects that are aimed at improving services. The challenge is how we sustain those improvements in the longer term, especially when projects often rely on short-term funding. Data on spending, performance and outcomes for children and young people is limited. We don't know with any accuracy exactly how much is spent on mental health services for children and young people or what impact that spending has. The information that we have indicates that it is a small proportion of overall mental health spending. Without a clearer picture of what is happening across all four tiers of mental health services, it will be hard to make the improvements that are needed. That will require the Scottish Government, NHS boards, councils, integration authorities and voluntary organisations to work together with children and young people to bring about a step change in how support is provided. Convener, as always, my colleagues and I will do our best to answer the committee's questions. Thank you. We have many. I would like to start with Colin Beattie, please. Thank you, convener. Order general, one thing that jumps out in this report, which you touched on in your opening remarks, is the issue around data. I can't even remember how many reports now that you've raised this as an issue. Obviously without data you don't know if you're getting the correct outcomes, you don't know if the money is being spent in the right place. It really is fundamental. Are those responsible for providing those services actually responded to the recommendations around data in your report? I'm pleased to say that the Government has accepted the findings and our recommendations from this report on publication, which is obviously an important step. The chair of the task force, which the Government has commissioned and which is reporting jointly to the Government and COSLA, has made this one of her early priorities in the first report that she published just a couple of weeks ago. There is an acceptance of the need. Obviously the hard work that's now required is to collect that data and make good use of it. Just sort of spreading this a little bit, you're looking across the public sector now. As I said before, we've seen so many of these reports with comments on lack of data. Is there generally an improvement overall? I think that we see pockets of improvement. If you're asking me for an overall picture, I'd say that we still don't have the data that we need, particularly in developing policy areas, like providing in this case more early intervention and preventative services in health and social care more generally, what's happening in community services and primary care? We tend to be very good at collecting lots of data in more traditional services, like hospitals, than more flexible services, which are often those that are going to be preventative. I think that it's one of the blockages to making a reality of the Government's outcomes approach, is the data that would let them plan and then track progress over time. Would it be correct to say that the majority of these data disconnects come when it's local councils collecting data on one side and Government collecting data on the other, and somehow it doesn't come together? That doesn't help, but I don't think it's the whole story. If we look at the integration of health and social care, there are certainly gaps in social care and gaps about the way health services and social care work together, but there are also important gaps in what we know about community health services and primary health services. I seem to vaguely remember that the Government put some sort of task force together on data several years ago. Am I completely— I don't—it doesn't ring a bell as an initiative in itself. We have seen lots of initiatives around things like the integration of health and social care, the wellbeing of young people more generally through the early years commitments, but I think that that's a question for Government rather than for us. One of the big things—again, you touched on this—is the question of the financial reporting as to how much is actually being spent on this particular issue. It's hard to understand why that information isn't available at least on a local basis. Is it simply that it's not being harvested nationally, but it's actually being reported locally? Is that the issue? You'll see on Exhibit 9 that we try and pull together what information is available on spending on these services across Scotland, and the numbers are so variable as to just not be credible. I'll ask Claire to talk you through some of the reasons we found for that. We had a really, really hard time with this report in terms of getting a clear picture about how the resources were being used. The bit that we could see to a certain degree was how much was being spent in health, but that data itself was very limited. So, what we've presented to you in the report is as much as we could gather nationally, it's really not good. There are key gaps in how the information is collected locally and also how it's reported nationally and publicly. Some of the key areas we've highlighted at Exhibit 9 so there are really inconsistencies in terms of how organisations work out how much is spent, what's included, what's excluded. We found some key gaps in that. In some areas of Scotland, the information on how much is spent on community services just doesn't feature. So, the information is not by any means comprehensive, but we've given you as much as we were able to collect with those heavy caveats that actually there are some significant gaps in that information. I mean, I'm looking at your paragraph 50 here where you say that CAMHS workforce has increased by 11 per cent between 2014-2018. That's a fairly big increase. How does that feed through into the results? It's not clear from your report that there's a direct correlation between the increase in the headcount and what the outcomes are. You're right that it's not clear and I think that there's a couple of reasons. One is that one of the findings is that the number of referrals increased by 22 per cent over a five-year period. Although there has been that increase in the workforce, the level of demand is also increasing and increasing more quickly. Just as importantly, one of the data gaps is about outcomes, it's about what difference these services are making for children and young people, ideally in helping to address their problems early and set them back into thriving and being able to benefit from their education as they grow up, or getting locked into a cycle of struggling with their mental health in ways that limit their potential for the future. We just don't know enough about that. Again, it's something that I think Dame Denise Coyer, the chair of the task force, is very keen to fill. Surely the local council, which is mainly concerned with CAMHS, must have some data on outcomes. There must have something to justify the headcount increases and so forth. I think that it's not quite right that it is the councils that are mainly responsible for the child and adolescent mental health services. Again, I'll ask Claire to talk you through that. Overall, what we saw was our lack of clarity about how the whole system worked as a system. We were looking for connections between different services and what we saw were in some areas quite a siloed approach, so we would see particular specialties focused on certain needs of children. We would see very broad services trying to support children in a whole range of different ways, through local authorities, charities and the private sector. All of those good initiatives to help to try and support children were absolutely clear that this can only be addressed by a range of different organisations working together more effectively. However, we did see data gaps and problems throughout the system in terms of how the money is accounted for but also critically in terms of what difference any of that makes to children. We've made a series of recommendations in the report that those things need to be sharpened. There needs to be a much clearer sense of what interventions work, where the money should be targeted and monitoring what difference any of that makes to children. We've got stories throughout the report from the children and young people that we spoke to in carrying out the work. That told us about how frustrating it was to repeat their stories to different professionals, to be unclear about what services they could access when they really needed help, and that came from their families, too. We see it as a problem across the whole system rather than just particular parts of it. Who should be doing the assessment of the outcomes? Anybody who is providing services and support for children should be thinking about what difference their service is making, what impact that is having on children locally. As the Auditor General said, the bit that we can see more clearly is data on waiting times. It does not tell a great story, but it gives us a picture of what is happening. In terms of measuring outcomes, what difference the services make, there is a real gap there. We also highlight in the report that it is not seen as a priority in all areas of Scotland. The integration authorities have a key role to play here in improving the line of sight and the priority that is given to those services across Scotland. It is not feeling that way, and it is certainly from the evidence that we saw at the moment that it is a priority everywhere. You are highlighting a concern that each individual organisation that is involved should be assessing its own outcomes, but, as you can appreciate, probably everybody will have a different criteria that might apply. We could end up with data that is not of much use anyway, even if there was somebody bringing it all together. Who brings it together? It links very clearly to the national performance framework. That is right that it is difficult to its challenge and across the whole public sector to get good information on outcomes. If it needs to be very locally responsive to need and different, and there needs to be variety, we recognise that that is true, there still needs to be something that brings that together. Again, the simple question that we would ask is how do you know that the millions that are being spent on these services are making a difference? How do you tell that that is happening locally? At the moment, we do not see that thread throughout the way everybody is working. We see a system that is under a significant amount of pressure. Lots of efforts go into harnessing the views of children and young people, a real policy priority around the mental health of children and young people, but we are not seeing that translated on the ground. During our fieldwork, we saw examples of measuring outcomes locally. What we are trying to say is that we have no idea at a national level the outcomes that are being achieved. Therefore, where do you direct funding and what do you spend the funding on? The Scottish Government is working on developing a number of quality indicators for mental health across six different quality dimensions. The issue is that we understand that boards will have choose which ones they want to measure. Therefore, again, that will make benchmarking very difficult. Thank you, Colin. Can I just follow up on that pointly, Johnston, if you do not mind? In terms of the sharing of learning, it is slightly different from the data and the financials, but on page 21 of your report, you talk about NHS Airshire and Arran, and they seem to have continued to improve performance. You say that they have continued to meet the 18-week standard consistently throughout this period. What are they doing that is so significantly better than some of those that you reference on page 19? How is that learning either being shared or going to be shared either as a result of this report or in general? Airshire and Arran are taking a whole-systems approach. They are working towards multi-agency collaborations, so they are seconding teaching staff into the CAMHS service, for example, and vice-versa CAMHS staff sitting within schools. They are using their data to understand the challenges that they are facing to go on to pilot different initiatives that will address those challenges. We have to be cautious, because what they are struggling with is how they maintain those different pilot initiatives with short-term funding. The other thing to acknowledge is that different areas have different needs and different challenges, so going forward, other areas could learn from Airshire and Arran, but they need to look at the situation in their area and decide what different pilot initiatives would suit them. There are a number of things going on where we will see good practice being shared. I was at a conference yesterday in the Youth Commission where they are, for example, and I know that they will make a number of recommendations and share ideas on good practice. I think that the task force will, hopefully, progress that work in terms of sharing good practice. I am sure that we will come back on the nature of funding. If I might just press you on what is something of a local issue, because the contrast between Airshire and Arran that we have looked at, you talk on page 19 about NHS Grampian and NHS Tayside, which are, obviously, of particular concern to me in a representative capacity. NHS Tayside seems to have a 21-week wait time and NHS Tayside at 18 weeks, which is a significant waiting period. This is for first appointment, isn't it? This is where someone has an identified need to get into the system as quickly as possible, presumably. What are NHS Grampian and NHS Tayside doing to either address those wait times and or learn from the likes of Airshire and Arran to improve that performance? Do you know? I think that, later in our report, we refer to some good work that is going on in Grampian. I think that we have to be cautious about the waiting time figures. We have outlined the data that we had available, but we know from having looked at it more in depth, different areas measure waiting times in different ways in terms of what counts as treatment starting. We find sometimes that it is assessment that they go to and then could possibly go on to another waiting list to wait for the treatment that is decided that they need to be given. As well, the differences are to do with workforce capacity, as I said, issues with the way that data is collected and monitored. Changes to the referral process as well, we found that there were issues around that, about the referral criteria changing and therefore the number of young people that were seen changed and fluctuated over a period of time. Thank you. Ian Gray. Thanks, convener. Auditor General, you bring lots of performance audit reports to the committee covering a wide range of services and projects. Some of them are good, and later in the agenda, we will look at one that is good. Some are bad and some are downright damming. I wonder where you would place that report in that spectrum. I think that this one does highlight a real problem. We know that for young children, for young people, dealing with mental health problems early in their lives is difficult in its own right, distressing for them and distressing for their families. We know from the evidence that it can make a real difference to how well they are able to thrive for the rest of their lives. If they get the help and support that they need early on, a relatively minor problem can be nipped in the bud, they can get back into full-time education, they can continue to build relationships, to build their confidence, their ability to flourish as people. If they don't, they can get into a cycle of depression, anxiety, doing less well at school, being less likely to fulfil their potential once they leave school. For us, that is why this is so important and why the failings that we have identified in the Government's accepted matter. I think that for the team doing the work, hearing the stories of young people and the difficulties that they were encountering in getting some of the help that they needed. For really quite small reasons, often teachers not knowing what help was available or how to refer them, referrals being made, so they didn't meet the referral criteria for the service that they were being sent to. They are things that should be quite straightforward to fix and yet they were having a real impact on young people's lives, and that is why that matters to us. The team-identified problems that are exceptional go beyond the usual difficulties that we find in public bodies working together or outcomes or data. That is something more than they are used to dealing with. It was a combination of what ought to be relatively straightforward interventions for lots of the young people that we are talking about here being difficult to access in practice, and we have some of the pathways that are described in the report. Lee, do you want to talk a little bit about that? Yes, I think that it is about the barriers. I think that there are several things. The lack of early intervention and prevention services, so yes, the referrals to the CAMHS specialist service are increasing greatly. Although we don't have the evidence, because the data is not there to understand what the demand for the lower-level services is, we imagine that some of those young people probably do not need the specialist services, therefore, with better services in place at those lower tier 1 and 2 levels. As we also suggest in the report, it feels like the four-tier system or approach to the service delivery feels slightly not fit for purpose any more, and there is a need to re-look at that and look at how we provide a more person-centred service to save children and young people bouncing between the different tiers and making sure that there are lower-level services to prevent people being referred to CAMHS or for their condition to deteriorate further. Is it fair to say that the report could be summarised as saying that we cannot go on like this? Something has to change. We say in the report that the step change is required, and I think that both the Government and the chair of the task force recognise that. The challenge now is to make a reality of the changes that are required. On that challenge, the report also says that it is not clear how the Government's mental health strategy will address those issues and improve outcomes. Why do you feel that to be the case? If you look at the strategy, there are 40 actions in there, and 15 of those relate to children and young people. I think that what we found was that a lot of the actions within the strategy were very much focused on trying to understand how the system is working and the challenges that it is facing, rather than on the outlining action that was going to be taken and the outcomes that it wanted to achieve. The Government has said that it will develop a framework that will measure progress and outcomes, but there is no timescale for that work. Obviously, it delivered the progress report against the strategy a couple of days ago, and if you look at the progress, there are things happening. Again, it focuses on things such as the youth commission and the task force. Of course, it will look at the challenges and figure out what is going on and what needs to change. I think that that is why we feel that. Since the report was considered and was published, a couple of things have happened. One is the annual report on the strategy. The other is the programme for government, where there were some announcements around counsellors in schools and some additional funding. My question is whether the strategy, with those iterations, is still inadequate to the challenge that the report identifies? I think that we need to see the outcome of the task force and the youth commission and see the recommendations that they have made. Obviously, we say in our report that we would like to see the task force consider the recommendations that we have made in our report. The funding for more school nurses and more school counsellors will start to make a difference, but, as we say in our report, it is also about looking at the way organisations are working together and to work in a more joined-up and collaborative way. That still remains to be demonstrated by the plan to do that. That is local and national bodies working together to look at that and take that forward. Thanks, Ian. Alex Neil. Can I focus on the demand? Obviously, the most worrying aspect is that nobody seems to have a handle on the level of demand. More importantly, not just the numbers, but what is behind the numbers, what is the demand for, what kind of services do people need? Obviously, historically, as you have said, there is a total lack of sufficient data on qualitative data as well as quantitative data on demand. How quickly do you think that gap can be rectified satisfactorily? We mentioned a case study in the report in Grampian, where some mapping was carried out to really get to the root to understand levels of demand, levels of need and then start to think about what the services should look like to respond to that. It can be done, is the message. The other issue that I would say that we have not touched on so far is that we did see a lot of very committed professionals who want to get this right. There is a lot of frustration within the system. We also highlighted in the report that there are some particular groups of young children and young people who are more affected than others. That is well known. There is lots of research available around that. Again, we would like to see that more clearly recognised and targeted. Exhibit 1 in the report sets out in much more detail the children who are more likely to have a mental health problem in Scotland. For example, children who are looked after through the care system. Children who are living in more deprived areas across Scotland. We did not see enough activity to target those children to help to give them the support that they need. There is more to do, but it can be done. A case study in the report can show how that can happen. There is a close link between children with faces and children with mental health problems. The mapping exercise in Grampian should that not as quickly as possible be done—filling the data gap on a permanent basis is a slightly longer term exercise by the time that it is set up in IT systems and all the rest of it, and we know the problems associated with them. Should that mapping exercise done in Grampian not be replicated throughout the entire country? I think that it should. As we are talking about what has happened here in Grampian, for me it reminds me very strongly of what we have said on a number of occasions about genuinely transforming care for older people. If you are going to do that, you need to know what need and demand looks like on the whole. It is a relatively small number of people who require the most intensive support. In this case, you start in schools, in early learning centres, in nurseries, identifying who are the children who appear to have challenging behaviour or whose parents are struggling for a range of reasons. You build that up from localities to the health board and gradually to a national picture. It also helps you not to treat the data collection as a separate thing. As you are identifying those children, you are starting to understand what help can be provided in a nursery or in a school and what does require a referral on to a specialist service. I think that local intervention is always the best starting place. Can I ask Caroline the auditor general? Obviously, we know historically, to some extent, the data that is available is historical data, but it seems to me that it is a fast-changing world in terms of children's mental health and the requirements. If you take, for example, just an issue like autism, we are much better at identifying early on autism. I am not saying that we are perfect far from it, but it is a lot better than it was even 10 years ago in identifying children at an early age who are showing signs of possible autism. Is there anyone looking at the trend, the changes in the nature of childhood mental health and adolescent mental health? I will kick off and I am sure that colleagues may want to add in. First of all, you are right, we do not know what is causing this increasing demand. There are two broad theories. One is that life is genuinely more stressful for children and young people with things like social media, playing a part. The other is that the reduction in stigma and greater awareness is making it easier for young people to come forward. Nobody really knows how far those two things are the case and what else might be happening. I was encouraged to see the proposal put forward by Dame Denise Clare in her preliminary report that suggested moving away from the current four-tier approach that aims to cover everybody to something that is much more focused, first of all, on children with relatively mild levels of need who can be helped in schools, who need specialist services, and then explicitly focusing on children with neurological problems like autism, ADHD, aspergers syndrome, and thirdly, fourthly, children who are at risk because of deprivation, because of adverse childhood experiences. It seems to me, without obviously second-guessing Dame Denise's expertise, that that is likely to give you a better way of understanding what is happening than treating all mental health problems as though they were the same thing. You would also give you a better understanding of what resources you need to put in place and what kind of expertise you need to be with place when and where as well. We mentioned in the report the Thrive model that is being used in some areas in England, and that is exactly what that model does. It looks at the different needs, the different support that children and young people need, and it starts to map the resources against that. I think, again, to come back to the Grampian example, one of the reasons that is attractive is because it gets away from a more siloed approach of, you know, you are a specialist in X, therefore that is what you treat children with, that condition, and then almost there is sometimes a lack of connection to things like general practice, schools, education. What was good in Grampian was seeing that sense of a whole system coming together to start to think about the shared responsibility for those children in that local area. That is what we would like to see more of. Presumably, the Government has announced basically a 6 per cent per annum on average increase in mental health resources over the next few years. Presumably, that additional money should be focused in, first of all, in developing something like Thrive throughout Scotland, so that you do get a better use of resources and more targeted in earlier intervention and all the good things that we have been discussing, but also that the resources actually go to where they would be most effective. In the report, we see that it is really important to understand the levels of need, what makes a difference before you start to think about then how you spend that resource. That is absolutely critical. It can only be done with the children and young people involved, as well as the folk who are providing those good services locally, so, yes, there is a need to understand it all. One of the key issues that we have here is that there is a whole section on resourcing that says lots of good stuff about money being put into various things. It sounds like from what the committee is hearing that no one has really worked out where the most effective interventions are and at which of the four stages those should be made. Also, is there an issue that you are alluding to about non-recurrent funding in terms of how, particularly, third sector organisations are able to budget and say that we will be able to deliver the service effectively in the future? Can you just tell us something about that? You have just summed up the end of the key messages of the report, where we say that transformation will only happen if there is a clear review of what works, a plan for how the system needs to change and a move away from relying on short-term and isolated initiatives. Now, it is easy to say that much harder to do it. Clare, would you like to pick up what that might be in practice? Absolutely, and I guess in a practical sense what we saw examples of were initiatives and projects, say by the voluntary sector being introduced because there was a pot of money, and then it wasn't clear how that would be, but if it had worked, would it be mainstreamed in any way? What was the shared learning around that? And if it didn't work, where was the decision making to say actually, for good reason, we're not continuing with that. Whilst we saw a lot of commitment to the overall idea of supporting children and mental health services, we saw some resource against that. What we didn't see was a sense of the system learning together and working as one coherent whole. The messages in the report from children and young people about it feeling fragmented, we saw that through the way that money is countered, the way that performance is measured and the way that there is a focus at the moment on short-term initiatives. Just sticking with the demand area that Alex Neil was talking about, can you tell us more about the benefits of early intervention and prevention, so the early stages of the four, if you like? What you see is the positive outcomes that they can bring, and specifically moving on to the resources that Claire Sweeney was talking about. Is it as obvious as to say that extra investment at tier one and tier two logically reduces demand on tier three and four, such that those who require stages three and four actually get a better service and the money is better allocated? It's a bit of a long be the focus around health and social care services that that idea, that a focus on prevention and early intervention is a good thing and will have a positive effect on reducing demand when people will be less likely to get into a crisis situation. We were very careful about our language around that issue, the team talked a lot about, so we see in the report that the service is more focused on crisis and specialist need. We're not saying that that's a bad thing, of course, I'll always be children who need that kind of support, but we're saying actually to the exclusion of prevention and early intervention, that's not a good thing. The shift needs to happen, there needs to be more of a shift, there needs to be a clearer picture about what works and a greater commitment to that early intervention and prevention that people know makes a difference. We also think that one of the factors underlying the increasing number of referrals to specialist mental health services and the increasing number of rejected referrals is young people who could be well supported by lower level services closer to their home in their school, but because that service isn't available, they're being referred sort of up the chain to more intensive services that really aren't the best ones for them being rejected and clogging up the system there, so the system's under more pressure, the young person's not getting the help they need, and we're not breaking out of that cycle because we don't yet have the school councillors trained teachers to spot a problem early and know who to refer it to and a system that can respond in the best way for an individual child's needs. I'll just press you. We're going to come back to the rejected referrals in two seconds, but if you wouldn't mind developing that point, Auditor General. What do you see having written this report as the key barriers to the early intervention that Claire Sweeney is saying is vital? I think that there's a number of things, and the team knows more about it than I do, but very briefly, I think that there's something about having the services available in the first place, making sure that people like teachers and GPs who are in contact with young people every day have some training in that general level of mental health and know what services are available and making sure that the system works smoothly, so it's easy to make a referral once for the right information to go for that referral to be assessed and picked up where it's the right service. Again, it comes back to looking at the system as a whole rather than having separate bits of it working in isolation. Bill Bowen, can I just echo what Claire Sweeney said that we should give due credit to the professionals who are working hard to deliver the services? We're asking questions here about the system as such, and we move now to rejected referrals if I can just ask some questions on that. You speak on your key facts section that there's been a 24 per cent increase in the number of referrals that are rejected since 2013 and 2014, and further on in paragraph 25 you give some reasons why this occurred, including the young person did not meet the criteria for treatment, a lack of tier 1 or 2 services for children experiencing less severe mental health problems, and that the referral does not give enough information. If I can ask a couple of questions around this, one that you say that national data is not being collected on reasons for rejection, are you aware of local data being collected? Is there any evidence to show that NHS boards, which are under the most pressure perhaps either because of the number of referrals they receive or the level of service provision they have, apply the criteria for rejection more strictly compared to NHS boards who are under less pressure, if you follow what I mean. I'm pleased that you don't feel that you have to be careful with your language. Locally there probably is some data collection. I think that we found that throughout our fieldwork that there are local examples of collecting data, but it's just at that national level that we have no idea what the trend is in terms of rejected referrals, what the reasons are for those rejected referrals. I think that it comes back to the lack of data, although the SAMH recently published a report that the Government commissioned looking at rejected referrals and what happened to young people after their referrals were rejected, and they do make a number of recommendations around that. However, to understand better, we need to come back to the data. Is that the answer to my first question or my second question? Sorry, repeat your second question. The second question was, are there hotspots in health boards that are perhaps under more pressure because of more referrals or less services available? Do they reject more people because they know that they can't treat them for whatever reason? Again, we simply don't know that because we don't understand the reasons behind the rejected referrals. It's not collected on a national level so that we don't understand the reasons for the rejected referrals. As I said, we've outlined in our report what we think it is that they don't meet the criteria. We did hear of children and young people who would have benefited from the lower-level early intervention prevention services and because they aren't available locally, they are being referred up to CAMHS, so they are rejected. I think that it can be to do with, yes, that the referral doesn't have enough information. I was at a conference yesterday, I was presenting the findings from the report, and there was an academic from the Highlands University there, and she'd done it. It was about four years ago, and she'd looked at the reasons for rejection particularly. It was a range of different—it was more likely that children and young people with behavioural issues had been rejected. Referrals from teachers, for example, were more likely to be rejected, but that's a small-scale study. We need to understand what's going on at a national level, and that, again, comes back to the collection of the data. Did she give any reason just out of interest why teachers' referrals might be rejected? She hypothesised that it was perhaps the language that they were using because they weren't a clinician. That's why knowing the language to use would perhaps make the referral more successful, but she didn't know the absolute reason. It was just a follow-up on that. We're talking about local strategies and what might be failing around data collection and provision of services, which lead to the inappropriate referrals. I just didn't want to lose sight of paragraph 70. In paragraph 70 of the report, it says that local mental health and wellbeing strategies focus on adults. It's the problem here that, at a local level, there just isn't a strategy for children and young people's mental health services. It's not that it doesn't gather data or that it's not working very well. There just isn't a strategy there, or the strategy is for adults. There's certainly a sense that we've got of the level of priority that it had in certain areas, and there's a link potentially to our message around waiting times, where we say in the report that the waiting times have been a focus. That's not to say that we shouldn't be concerned and know more about how long children are waiting to get services, but we've already highlighted the problems with the data, which made it hard to answer some of the previous questions. Our sense is that it just does need to be a greater priority. Lots of really committed good people working in the system and some clear evidence about where the problems are, but a lack of pulling together to make sure that those needs are responded to and that the money is there to make that happen. We definitely see this as an issue that all a number of organisations need to work together very closely, including the Scottish Government and COSLA, to address. That surely is a massive disconnect between stated national priorities and local priorities on the ground, isn't it? We say in the report that we see a commitment in terms of policy around this issue, absolutely. It's very clear that people recognise that. It's an important thing in Scotland, but what we didn't see was that translating into practice in all areas for shuwa, and that's why some of the recommendations speak to that point. There's a related point that I also don't want us to lose sight of further up that page in paragraph 68. I think that lots of the policy focus has been on the importance of integration authorities in getting this oversight of what's happening for children and young people as a whole, but we found that only 11 of the 31 integration authorities across Scotland have got responsibility for both children's mental health services and social work mental health services. They're the ones who have the best place to do that. The other 20 clearly will find it more difficult. A few further questions, if I may, Auditor General. Just coming back to Bill Bowman's issue of rejected referrals, page 18, there are some reasons given for why the referrals are rejected. Presumably what I'm hearing from your answers earlier is that there's no data. We give three reasons why referrals are rejected. The child or young person doesn't meet the criteria for treatment, lack of services at tier 1 and 2 and the referral doesn't have enough information, and that's qualified later on at paragraph 28, talking about the level of detail provided by the referer. Presumably, just for clarity, there's no data on how many of the rejections fall into each of those categories. As we say at the beginning of the following paragraph, paragraph 26, national data on reasons for referral and rejection isn't collected, which makes it very difficult to be clear about that. As Lee said, the Government commissions and work from the Scottish Association for Mental Health to examine what's happened and they've recently published their report, but that's been a specific clinical audit rather than routine data collection, which we think is important. I would agree with you. I'm also just coming back to the referer point, because yes, if there are a lack of services, that goes towards funding and supply, but the other two points, the other two reasons for rejection seem to me to go towards the competence of the referer. I don't use that in a pejorative sense, just the ability, have they got the guidelines to do it. So somebody has identified a need and said, I have a young person here who needs help, but because of, again, not pejoratively, but a failing on the part of the referer, that young person can't access the help. That, to me, sounds hugely concerning. Is that a fair summary? And what's being done? I think the referral pathway is complicated and we outline that in our report. The criteria are varied across all the different boards and it's often not easy to follow either for the young person themselves, parents carers or other potential referers who don't necessarily come from a clinical background, for example. We do outline some good practice in Highland where there's a primary mental health worker and they undertake a kind of triage service, so the young person goes to their GP and the primary mental health worker is there and can assess and offers almost a step up or down, so if they don't think that it's appropriate, the referral to the specialist services will step them down and vice versa if they think that they require more specialist help, they'll step them up. So I think that there are pockets of good practice to try and address some of the referral issues, but, again, as we also say laterally in our report around training, there is a need for more training for non-mental health specialists, so teachers, school nurses and the like. That may help to address understanding the referral process and what's required in more depth. On that exact point, Lee Johnson, if I may, you talk in your report about a revised role for school nurses, but do you have a view on are there sufficient school nurses such that, if there is a change in role, that will make a significant contribution to early intervention or is it a good thing to do, but ultimately the early intervention won't be significantly impacted by that change? As we say in our report, again, the school nurses are a priority for them now—mental health and wellbeing. As we have said, they have indicated that they require a bit more training and help in that area, but I think that the programme for government has also announced that a significant number of school nurses will be brought on board. We need to see the impact that that has. Finally, I just wanted to press you on the data sharing aspects. At page 23, you talked about how multi-agency working together is ultimately going to be crucial. You also commented that the young people found it very frustrating having to repeat their histories and their challenges to multiple professionals. What is the issue with data sharing here? Is it IT, for example? Is it data sharing regulations? What is going on here and what can be done to fix it? This is something that we have been looking at as part of our work on the integration of health and social care services. The Auditor General will be bringing a report to the committee later in the year on that issue to look at it in the round. Some of the issues that we have seen through this piece of work were around the quality of the record. For example, some areas are using paper-based records. By definition, that makes it very difficult to share information. It was a range of factors. We talked earlier in the report about the need for trust and relationships, and that goes for the professionals working in the system just as much as it does for the children and young people who need the support. We see information in data sharing as one part of that. The use of paper records does not help, but we need to understand how the system is working together and what information everybody needs to know. There have been developments in some part of the health system over the past few years to try to move that issue on around sharing things such as emergency care summaries for people who are going into A&E. It can be done. It can work to improve the way that information is shared across the system to make care better. We saw a long way to go for those services. Do colleagues have any further questions on that? No. In which case, I thank our witnesses for their evidence today. That has been very useful. Thank you. I will suspend for five minutes to allow for a change of witnesses. Good morning and welcome back to the Public Audit and Post Legislative Scrutiny Committee. We would now like to examine the report on the fourth replacement crossing. I would like to welcome our witnesses this morning. I welcome back Caroline Gardner, the Auditor General for Scotland, Graham Greenhill, Senior Manager and Gillian Matthew, Audit Manager, both at Audit Scotland. I would like to invite the Auditor General to make an opening statement. This report looks at Transport Scotland's management of the fourth replacement crossing project, which includes the construction of the Queensferry crossing. I will summarise the findings under three areas—the need for a new crossing, the management of the project and, finally, demonstrating whether the intended benefits of the project have been achieved. First of all, the Government identified a clear need for a replacement crossing. Corrosion of the main cables of the fourth road bridge would have meant restricting traffic from 2017, and ministers made a timely decision to ensure that the road connection between Edinburgh and Fife was maintained. Transport Scotland's decision to build a new cable stage bridge was cheaper than repairing the old one or building an alternative type of crossing or tunnel. Its design is intended to be easier to construct and more reliable and resilient, and Exhibit 2 on page 10 of the report sets out the key features of the new bridge. Secondly, at £1.34 billion, the fourth replacement crossing project is one of the biggest public sector infrastructure projects that Scotland has seen. On a large, complex project like this, there are many opportunities for things to veer off track, and it is to transport Scotland's credit that they did not. In part 2 of the report, we highlight the good practice in the procurement project, which helped to deliver value for money. The team had the right mix of skills and experience, and they invested in the external expertise that they needed early in the project. From start to finish, they demonstrated strong, consistent leadership and communicated well with contractors and stakeholder groups. They were strong on budgeting, governance, quality assurance and risk management. We think that there is a lot that the wider public sector can learn from how the project was managed, and we have recommended that the Scottish Government and Transport Scotland to share the lessons as widely as possible. The Queensfree crossing opened eight months later than first expected and 10 weeks later than the contract end date. We concluded, however, that the reasons for this are reasonable, and Transport Scotland managed the changes effectively to minimise the effect on time, cost and quality. There is still some work to complete on the new bridge, which is to be expected on this kind of project. Our only criticism, I think, is that Transport Scotland could have communicated this better to manage the public's expectations, and it should continue to keep the public updated on progress. Thirdly and finally, it is too early for some of the project's wider benefits to be demonstrated, such as improving public transport across the force, cutting journey times and boosting economic growth. We set out progress against each of the eight objectives in Exhibit 9 on page 36, but more detail is needed on how success will be measured in future. Transport Scotland now needs to produce a clearer plan for how it will measure the success of the project's wider benefits. It plans to carry out a full post-project evaluation later this year and will continue to review progress through our audit work. Colin Beattie Firstly, this seems to be our report that is unrelenting good news, so I guess we need to congratulate the Government on that. There is reference throughout this report to good practice and success story after success story. I am looking at other projects that have perhaps been less successful than the way they have been managed. How do we effectively transfer that good practice, what has been learned in relation to the project, to other projects in the public sector? We think that this is a well-managed project, and it is to the Government and Transport Scotland's credit that they have achieved that. We have recommended that they look at how they can spread those lessons more widely, not just to infrastructure projects, but many of the principles apply to big digital projects, for example. We have produced on our website a hub that pulls together all the materials that we have developed in this area that people can refer to. I will perhaps ask Gillian what else we think can be done to make those lessons a reality and get some of the benefits in future. As you say, it was a very well-managed project, and it is something that we see often as auditors. You will know that in other projects and audits that we have looked at, it is not always the case. One of the things that cannot be underestimated is that it is absolutely fundamental to get the right building blocks in place, right from the beginning, the time that is spent in getting the scope right and understanding the costs, the risks, optimism bias, which we mentioned in the report about underestimating things that can go wrong or things that you do not anticipate. You have to build in capacity for that. A lot of that is well known, and we have looked at that in other reports. We have talked about what good practice is and what it looks like and how projects should be well managed, but we see that that does not always translate. There are various reasons for that, such as time pressures, but a lot of it is not spending that time at the start and getting it right before you proceed. As Auditor General says, there is a lot that can be learned from this, and not just in infrastructure projects, but in a lot of generic project management lessons that can be applied to many other major projects. It is just making sure that recommending that Transport Scotland makes sure that it does that in future work, and we know that they are already, throughout the project, making sure that lessons learned were being shared across other projects. Things like the dualling of the A9, they were starting to apply lessons learned to that project. Again, the Scottish Government has a role to play in making sure that it is shared right across the public sector. Does the Government give any indication as to how they are going to do that? At the moment, the Transport Scotland and the Scottish Government have accepted that and said that they are going to do that, but we do not know exactly in what way. One of the things that we do with every performance audit is to prepare an impact report, usually about 18 months or so after the publication of the original report. I am the auditor of Transport Scotland, so one of the things that I will be doing is looking at how the Government and Transport Scotland are responding to this report as part of my audit of Transport Scotland in the appropriate time. I think that it would certainly be a shame if we did not build on this success, having proven that a large project can be well managed and delivered in the way that the public would expect. There seems to be no reason why it cannot be done across the whole public sector if we can just take what we have learned from this. There is one thing that I was going to raise, which is paragraph 33 in your report. Is what is detailed here normal in the trade? I have not seen this before sitting in any project coming forward. Is it simply the scale of this project that the payments are made to contractors? That is in relation to bidders being paid reasonable costs up to £10 million, which to me is a lot of money. If the contract did not go ahead, is it normal? It is unusual, and as you can imagine, we looked at it very closely. I will ask Graham to talk you through what we concluded about it. There are two separate elements here. There was the fact that the contracting, the tendering, was being undertaken at the same time as the fourth replacement crossing bill was proceeding through Parliament. There was always a risk that the bill might fail, so the bridge would not proceed, and that would have left contractors out of pocket. Transport Scotland took the view that if that kind of thing happened, it would need to compensate the bidders for the costs that they had incurred. Secondly, there was the offer there that Transport Scotland would pay up to £5 million towards the unsuccessful bidder's costs. That was really there to ensure that competitive tension still existed, and that there was more than one bidder in the process. As Caroline McLean said, it is unusual, but it is not unknown for clients to go down this route. It was done in view of the size of the project and the likely costs that bidders would incur in developing their tender proposals. From the point of view of Audit Scotland, does it seem a reasonable expense to have there? We concluded that it was, yes. The effect of the decisions that Transport Scotland took was to make sure that the procurement remained competitive. It kept two bidders in to the point where a decision was made that helped to keep costs down, and it helped to generate a strong form of contract that was cost limited, reducing the scope for cost overruns to come through. As always, there is a balance for where the risks are best managed, and we thought that that was a reasonable decision, which paid off in the end. Just to be absolutely clear, it is the unsuccessful bidder only that got £4.2 million. The successful bidder gets nothing because they are making the profit on the contract. They win the contract. The unsuccessful bidder was reimbursed £4.2 million for the cost of bidding, and that helped to make the procurement itself a competitive process. Just on that point, if I may, the procurement and the bill that set this up were running concurrently. I understand why, and if I might use your own words, Caroline Gardner, you said that it paid off this time. Presumably, it might not have paid off, and we could have been so credit where it is due. It worked this time, but is that a practice that you would advocate going forward, or did they get a bit lucky this time? In this case, we concluded that it was a reasonable decision. We were in the unusual circumstance in which all the engineering evidence suggested that the old 4th road bridge was likely to require restrictions on the traffic that it could carry by 2017. Given the timescale for a project of this scale, waiting longer to start the procurement process would have run the risk of significant road closures, disruption and impact on the economy of Scotland, and particularly Fife and the Lothians. That is not to say that it would therefore be an automatic approach that should be taken for future contracts. As always, what we are looking for here is intelligent application of the principles of good procurement and good project management, rather than a cookie cutter approach where something that worked to one project is automatically assumed to be the right answer for another one. That is certainly not what we are saying. On that point, the Accounts Commission recently produced a joint report that summarises your findings on various major projects and procurement lessons. Do you have any plans to publicise that and disseminate that report more widely? We are doing quite a lot of that ground. Do you want to pick that up? Yes. We have certainly made people aware of that. As you say, it appears on our hub, which brings together all our relevant reports, looking at major capital projects, together with the summary of findings. We have made sure that Transport Scotland is aware of the existence of that. We have been working closely with the Institute of Civil Engineers for Scotland to make them aware of its existence. We are continuing to work and take that forward. Bill Bowman, if I can take a slightly more personal view of this, I have been crossing the forth either on the original bridge or the new one since 1964, when it opened. I continue to do so. I do not want to rein on anybody's parade, but Colin Beattie's unrelenting good news is maybe from afar, from a bit closer up. I see unrelenting delays. The bridge opened, and then it immediately closed. Both I and I have had correspondence from constituents who have crossed in the evening. We have had two lanes closed. We have bumped across on the hard shoulder. You say that there is some work being done. It seems to be more than some work. Was this bridge actually finished when it opened? You say that you are going to look in future at its operation. Are you going to be a little bit more critical of Transport Scotland? We also have an empty bridge at the moment, virtually with no traffic on it, but we still have traffic being queued up on the existing bridge, the new bridge. I will ask the team to come in in a moment. It is important for me to state, first of all, for the record that the new bridge was not intended to increase capacity for traffic crossing the forth. It was a replacement for the fourth road bridge, with any increase in demand being met through increased public transport. That is one of the project objectives that is still not fully delivered. There is an element there that is part of our recommendation for a plan for delivering it and for clear measures of how it is affecting it. Graham, do you want to pick up the question of snaggings and our conclusions in that area? Yes. It is quite important to recognise that snaggings are by no means unknown for any major project. There undoubtedly was a list of snaggings that needed to be completed. Transport Scotland ministers have informed Parliament of the list of works that needed to be done. None of that, the snaggings prevented the bridge from opening at the time, although you are correct in saying that, subsequently, there was a need for a temporary closure to level off some tarmac joints in the bridge. The one main area of snagging that remains outstanding is around painting the undersides of the bridge. That was supposed to be completed before it has been. It will not be completed until September 2019. That is largely because of the specialist equipment and the cradle that is required to get underneath to provide access to the undersides of the bridge. That work will not be completed until the end of 2019, but that is not having any effect on the operation of the bridge. It is if you try to cross the bridge in the evening and you are down to one lane and there is a long queue waiting to cross it. We all know snagging from when you buy a house and there is a crack up in the corner. It is not as if the whole house has not been finished or something like that. I think that snagging is a term that implies minor repairs. If you say that there is something that needs to continue until the end of 2019, and I presume that we will continue with lane closures, I cannot really agree that that suggests that there is not something wrong in the way that the bridge has been managed. Your report is your report, and I hope that you will come back to this as to how it is being operated afterwards. On the cost side, on page 29, you talk about inflation of 5.3 per cent being included in the estimates. Is that correct? On page 29, it is the bottom graphic. If I understand it, inflation probably came in at about zero using the appropriate index. Does that mean that a lot of the cost saving comes from there being no inflation? There are certainly savings from inflation, but there are other areas as well, and we set that out. On page 60, the third bullet point at the top of page 33 talks about price fluctuation costs, essentially due to inflation being £60 million to £205 million lower than first projected. So what, £200 million being saved by inflation? If you are saying that they have managed it very well, but inflation came in at £200 million less, that takes away quite a lot of the trumpeted saving. That was the range that they had allowed for. The £200 million was the upper figure, but it was closer to the lower figure that was the saving. At the time that they were estimating the costs and trying to understand the various aspects of that, obviously it was a very uncertain time. It was around 2009-2010, not long after economic crash, and it was quite difficult to estimate what inflation was going to be over. You were talking about a 10-year project, so they did allow for an upper range, but there was a range all through the project. You said that there was a cost saving of whatever it was. How much of that was due to inflation not being as high as they estimated? I am not sure that we are talking about cost savings, but we are talking about the amount by which the total cost came in under the budget for the project. Not in savings in the way that you are describing it. It came in below budget and international data suggests that nine out of 10 projects like this don't, so that they overrun on cost or time. What we have done in paragraph 60 is to set out the key changes to the cost between 2011, which I think is when the contract was let, and 2017, when it concluded, and to break it down between things like risk allowance, optimism bias, inflation, the cost relating to the principal contract, and other elements of the project as a whole, as opposed to simply the project construction. Time to get to whether they have done something in the management of the project that has been good, or they have just benefited from inflation being less than what they originally estimated? I frame it a bit differently. I think that they let a good contract, which placed the risk for those elements with the contractor rather than with Transport Scotland, and they were able to do that because of the work that had gone in beforehand, as Gillian suggested, in appraising the options and then making sure that the form of the contract was competitive. I don't want to labour the point, but we probably have other questions. However, if you quote an original number and then you say that the costs were less, how much of that less is due to inflation? That's really all I'm trying to establish. The 5.3 versus what it actually turned out to be. You'll have the information. I'll be happy to get it later, or to get an explanation later. I'm not sure that we can start Gillian Gowan. I have a breakdown of some of the costs and what they were at the beginning of the project and at the end. We're talking about ranges here, so there's a lot of changes happening within the different figures for different reasons, but at the start of construction, so we're looking at when they started with the overall budget of £1.4 to £1.6 billion, so overall estimate that they had. Within that, there was a lower range of £91 million allowed for price fluctuation inflation and the final figure was around £31 million, but that was the lower range, and again, there was a higher range. One of the things that, as we set out at the beginning of part 2 around good project management, is that putting the time in to understand potentially around the costs. Inflation, obviously, is very difficult to predict. At the time when they were setting up the budget, the SPICE did a report around examining the costs, and they were satisfied with the way that the costings around that had been made. At that point, including economic reports, it was very difficult to estimate what inflation would be past two years in advance. I think that there's an element of how well this has been managed as good fortune inflation was less than they estimated. However, as we've seen in the report, far too often that's underestimated and the element around optimism bias as well, and those were the things that we felt that were all considered and built in very well to the project. However, that was monitored all the way through and the budget adjusted accordingly as well. I think that there is an element of good fortune. However, as we say in case study 1 on page 21, there were a number of measures built into the contract approach, which helped to deliver value for money. The fixed price contract was one of them. The value engineering approach that let contractors suggest improvements to the project, all of those things fed through. There was clearly an element of good luck in inflation, but they didn't just get lucky because inflation was low. Can I ask the Auditor General about the wider evaluation of the impact of the bridge? How broad and deep is this evaluation going to be? Is it going to be a wide-ranging economic and employment impact assessment, or is it going to be narrower than that? Secondly, does the baseline data exist for making an objective evaluation at this stage? Is this note closing the door after the horse is bolted? Exhibit 9 on page 36 of the report sets out in graphic form what the project's objectives were, and there are eight of them, plus our assessment of whether they are achieved or whether they are still to be assessed. The wider ones that you are touching on are things like supporting sustainable economic development and economic growth are definitely in the still to be assessed category. Gillian, do you want to say a bit more about the plans to transport Scotland to have four evaluations? They have plans to evaluate it one year, three or five years after opening of the bridge, and some of those, particularly around the economic sustainability development, are longer-term objectives. One of the things that we have said in the report was that there was not enough detail in some of their plans around how they were going to measure some of those outcomes. There was quite a lot of detail around things like traffic flow and things that were easier to measure, but we are not sure at what point the other aspect was that it was not clear at what point they were going to be able to answer or say that they had reached outcomes. We would not be seeing all those being able to measure that they have reached these in the first evaluation after one year, but we would not expect some of them to be met by that point anyway. It is not clear at what point they are going to say that there are going to be three years or five years that we can say what progress has been made against the different outcomes. To measure the outcome, you also have to have figures for zero year. What was the starting point before the bridge was built? Do you have that information? Again, that is where it was less clear with some of the longer-term, more difficult-to-measure outcomes. Obviously, there is a baseline around some of the traffic flow and things like that, but it was not clear. It was quite high-level the information or the methods that they were describing about how they were going to measure some of those outcome measures, and that is one of the things that we are recommending. There needs to be a lot more detail around how they are going to do that, what data they are going to use, talk about surveys or looking at businesses and what decisions they have made about setting up in areas around the fourth and how that has been affected by the project. It is not fairly well-defined treasurer-y good practice in all of this. It used to be called the Green Book, I think, about how you measure economic impact. Why are they not just following that? I think that the Green Book focuses more on project appraisal rather than project evaluation, but obviously you need to follow the line through. We say a little bit in paragraph 66 about transport Scotland's plans for evaluation. The second bullet point there describes their plans, which are to compare pre-opening and post-opening employment patterns from a range of secondary sources. They have plans there, but, as Gillian says, what we have not yet seen is the detail of which particular data sources they are expecting to use and the way in which they are going to pull that together at each of the three evaluation points that we have. In broad terms, the plans are there. We are just raising that caveat that, at this point, they are not detailed enough for us to be clear that they are going to be able to demonstrate whether the benefits have been achieved or not. Do you think that they might not be able to do it or that they will not get around to it? If so, when are they going to get around to it? My experience over the past six years of doing this job suggests that people often don't pay as much attention to evaluation after a project has been delivered or a service developed. I think that we feel that the plans that Transport Scotland has described to us are good in that context, but we cannot yet give you the assurance that they will be able to evaluate all eight of the benefits fully. Clearly, the ones around social inclusion and economic growth are the more difficult ones to do. Graham, do you want to add to that? I will say that, but I will be looking at that as part of the audit of Transport Scotland in the fullness of time. When is your next audit of Transport Scotland, Graham? It is an annual audit of the financial statements. The first evaluation report is expected later this year. I can have a look at that as part of the 2018 and 2019 audit of Transport Scotland. I think that we should make sure that that is circulated to the committee when it is published to see if there is anything that we want to pursue in that. Again, the whole justification for those huge expenditures is, to a large extent, the economic impact. Obviously, there was a very urgent situation with the old bridge here, which in itself justified the investment. However, we might as well look at the wider economic benefits for future reference of nothing else. Thanks, convener. Some of those outcomes related to promoting public transport across the forth and paragraph 71 tells us that Transport Scotland plans to publish an update on progress in relation to its public transport strategy late in this year. We are almost in October now, and I wonder if you have any information about likely publication. Far as I am aware, that is still the plan. No, you do not. I have any indication. I think that, to some extent, the thinking around the public transport strategy will be influenced by the initial evaluation, which is due to be finalised roundabout now. I suspect that Transport Scotland will be looking at the initial results of that evaluation work to feed into the wider thinking about the public transport. Thanks, Ian. A few wrap-up points about the cost. There is a key message on page 1 about the final costs, but the overall cost of £1.34 billion includes the total costs from when it was first scoped in 2007 up to the end of a five-year maintenance period in 2022. Given that that is projecting forward, is there any risk to that cost? Do you think that there is a possibility that it could increase? Not that we are aware of. As you say, that is still to happen, but the maintenance is being carried out by another contractor that does the on-going maintenance. The contractor that built the bridge is responsible for some of the initial and snagging and on-going repairs after the first year. That is taken over by the other contractor that is looking after the Queensway crossing and the existing fourth road bridge. Again, there are detailed costings around that and what is expected, but with a new bridge you are not expecting there to be a lot of costs involved in the maintenance. I want to bring it right back to Colin Beattie's points right at the start about effectively what went right and what went well here. First of all, you talk about the co-location of the contractors and the FRC authorities, if you like. That allows quick decision making to happen and that allows a fluid exchange of ideas. Is that something that could be replicated fairly easily in respect of other projects going forward? Is that part of the best practice that should be replicated going forward? Again, it does depend on the type of project. That one lent itself very well to that setup, because the bridge was in one location and had the site office very close to the bridge and where all the construction was happening. It was certainly something that came through really strongly in our field work when we spoke to all the different people that are involved in the project. Even without prompting, that came through and helped to work really well. Everyone was in the same site and were able to talk about issues as they rose. They built really good relationships across the team and with the contractors as well. It was certainly something that is in their lessons learned and looking at future projects. It depends on how the project set up. Looking at the dualling of the A9 project, it is thinking about what lessons can be learned and transferred to that. That one is quite difficult in that respect of having one site, just the nature of the project. The A9 covers a vast area, so it is not so easy to transfer things like that, but it is transferring other aspects from lessons learned around some of the education programmes, the wider stuff around involving school children, universities and getting people involved in subjects engineering. It is certainly something that they would look to replicate if they can in a project that lends itself to that set up. Different projects do not always apply everything to the same projects. I will just come in with a resupplement in that. It ties in with my question about the economic and employment impact. If you let in a big contract, particularly where, for example, steel was procured from China, that the bigger the contract, the less of the potential economic and employment impact to the Scottish economy there is, because those big contractors tend to take their profits out of Scotland and they do not reinvest, because they are not based here. Secondly, they procure steel, for example, which is a substantial part of the contract overseas, whereas the A9, the dualling of the A9, is being done in much smaller chunks for obvious reasons, for pragmatic reasons, but nevertheless one of the potential benefits of that might be that local contractors, local to Scotland, indigenous employees, training opportunities for apprenticeships—all that kind of stuff—is much greater than what it would be if it was a huge contract like the fourth bridge. I realise that you cannot always, with the fourth bridge, it would be very difficult to build that in anything other than one contract, but are we doing enough to maximise economic impact? I worry at times that we are just taking accountants' point of view of those contracts and looking at the immediate savings that are apparent to the public purse, but when you look at the wider benefits foregone potentially in terms of economic and employment impact, the public purse might actually be losing more than it could have gained. I have to start off by defending accountants and saying, I don't think that we do take a narrow view of just the immediate costs, but instead looking much more widely at what's being achieved. In broad terms, you're right. Particularly at times when finance is tight, it's very easy to look at just the short-term cost and benefit of what you're trying to achieve rather than the bigger picture. We know that the best value criteria, for example, is to let people take account of wider benefits than just the cost in pounds of a particular project or initiative. I think that there's room for that to be applied right the way up through the economic strategy in thinking about some of the wider trade-offs that are involved, procurement, but also other investments in services such as lifelong learning, retraining for adults and those sorts of things. I think that it is something that the public sector is increasingly aware of. Transport Scotland is increasingly looking at the extent to which subcontracting to local companies is taking place in some of their big projects, such as the number of apprenticeships that are taking on as part of a major capital project. Particularly with Brexit, because there are restrictions on what you can do built into the Lisbon treaty and the like, some EU rules restrict you on how much preferential treatment you can give, for example, to local contractors. Clearly, if those rules no longer apply and we don't know yet if they will or won't, when we come out of Brexit, there is potential for improving and enhancing the economic impact of procurement policies on the Scottish economy. I'd feel more comfortable responding to that if we had more of an idea about what might happen when we leave the EU, so I'll leave it there. Absolutely. I'm an optimist. Final question from me, then, if I may. Going back to the good practice of successes, are you able to identify from your report? Was this simply a confluence of serendipity and all the right things got together at the right time? What a wonderful coincidence. Or, are you able to say, look, this was a function of some key individuals and who are they? This was a function of some good planning at the start by certain agencies. Were there any key things that happened that both can be replicated, but also, if in the case of individuals, they can be used further? I will ask the team to talk you through the handful of key points that we think they got right that made a real difference. It's worth saying that, although they did get lucky on some aspects, like inflation being low during the life of the contract, they also got unlucky on weather, which was worse than had been the case in previous years, and they planned for that quite carefully. It certainly wasn't just serendipity. Julian Graham wants to pick that up. I don't think that you could put your finger on a single point, which made the difference. I think that there were a number of factors that were at play. Paragraph 21 on page 17 has a list of bullet points indicating critical factors for major projects to succeed. If you look at those bullet points, the first three of them are really about planning. Good planning is essential. We are all aware of the five P's when it comes to planning. The final three bullet points are all about how you discern the likely costs, the extent to which you allow optimism bias, how you get independent advice on your costs, how you compare your expected costs with other similar projects. To that list, I would add leadership culture. Leadership culture sets the tone of the entire project. As we have previously said, it is all about openness and transparency. It is all about willingness to discuss and negotiate over problems and all parties coming together. It is about people having clear responsibilities and governance arrangements in place to make sure that people are held to account for what they are doing. It was really an accumulation of factors. However, the fact that the bridge was an iconic structure played a part too, and there was a genuine source of pride in everyone working together to deliver the bridge. If you would push me, the project director came with a highly thought-of international reputation that he delivered. I think that that is the point that I want to get to. We have an awful lot of bodies in here, agencies rather than individuals, where we look at the leadership, no doubt, has exactly the investment that you describe and the desire for things to succeed, but for some reason they have not been able to deliver that. How much impact do you think it is that the project director that you refer to, for example, was key to that? Is it just about them? Difficult to quantify the overall influence, but all I can say is that he was well thought of and he pulled the full thing together, and everyone was working in conjunction and co-operation. I was just saying that what you are trying to get to was a one-person, one-key thing, and I do not think that it is, because if you did not have all the good planning building blocks in place, you could bring a really oppressive person in and it could still go wrong because they do not have the right team under them or they did not have the right costs, they did not get the scope right. It definitely is a combination of all those things. That page that Graham referred to at the start of part 2, that is why we have just tried to get into some of the key success factors. Paragraph 23 at the bottom summarise and if you have all the good aspects of project management in place, that is a really good start, but then it is these additional things and it is about getting the right people with the right skills, getting people in early, having good leadership, openness, transparency and the team spirit and the good work in relationships. It is all that together that made this work really well. Final comment, as an accountant not giving an accountant's perspective. If the money aspect was good, that is good. Having been through the whole construction phase as a user and a continuing user, if users matter, I do not think that the experience has been good. I do not expect you to perhaps comment on that, but that is just my view. We have to look at the project in the round. There are some real successes on this. We have made a criticism in here that we think Transport Scotland could have kept people better informed about the need for further work after the bridge had opened. I recognise that people's expectations may have been for a bridge that removed congestion across the forth, and that was not one of the project objectives. I absolutely share your view that the user experience is an important part of this. In which case, I thank the Auditor General and her colleagues for their evidence this morning, and I now close the public part of this meeting as we move into private session.


Today's tech talk is on the Moodle grade book waiting. Sorry you had to wait for this one. I was out sick last week Today we're going to talk about waiting and the basics of getting started and getting your Moodle grade book waiting The grades appropriately for you. I'm going to go ahead and get to my screen sharing mode here We're in the webinar workbench Moodle course of course and I'm in the grade book view Let me just get to where the default will go when you first click into the grade book off the Moodle course start page And I'll click into the setup area. You can do this from either setup or in the categories and items view Before I forget in my follow-up message I'm going to send an email out to the LMS group if you could make it this webinar or not a couple of videos that I found helpful in Getting the concepts around waiting in your in your grade book. So I'll send a couple of links out to you also and a follow-up so basically If you saw our previous session on Moodle grade book categories We've got the an assignments category full of 10 assignments Each of these assignments has got 10 points in value each We've got a forums category and we've got a quiz category So you'll see that the max grade back up to the assignments category the max grade potential for each assignment is 10 points The max grade for each forum. I should say advanced forum that these are the seven points and The max grade for each quiz is five points Now by virtue of having the overall grade book aggregation methods set up for natural waiting Moodle itself is automatically calculating the grade percentages for the assignments and the forums and the quiz categories as 45.455% Etc. And Etc. Now let's say a quick way to Let's say imagine you wanted the the quiz category the category as a whole to Equal just a straight across the board 25% of your course total an easy way to do that would click this checkbox So any of these checkboxes beside a category and The weights column I should indicate if there's a checkbox beside the category itself or an individual Gradle item in there. If you put a check in the checkbox You can edit that in this case. I mentioned 25 percent And if I click on save changes Notice how these other category weights will change automatically in just one second And you know these other categories have adjusted a little bit to make room for that new quizzes category of 25% of the grade I Can go back to the default setting that the natural aggregation method gives me By unchecking that box and clicking save again Okay, so now we're back to the default And I should have mentioned earlier a little bit more context about how Moodle is calculating these weights for you It's basically Calculating the number of of course the number of points and the number of gradable items within a category And all that's up to do the math for you. And if you were not a math major like me, this is a helpful aggregation method Okay, let's say that I want to We've already covered how to how to make a category itself be a Have a Potential a fixed number that you want in mind versus the what the default setting is Let's say that you've decided your week 10 quiz is going to be your final quiz And maybe within the category of quizzes You want the week 10 quiz to be worth more than just the default 10 points like all the other quizzes are Let's say we want to make that 25% of the total for just within the quizzes category So I put a check in the box. I indicate that 25 to represent 25 percent When I click save changes All of these tens will be adjusted automatically for me. And there they are so Within the quiz category inside this course shell gradebook area The week 10 quiz which I could maybe I want to also rename that to be like final quiz to some context Now that is now worth 25% of all the potential point values for all the quizzes in the quiz category Independent of these advanced forms and independent of these assignments, et cetera, et cetera Pretty handy pretty sweet Now you'll see it as you as you go through the term you might notice a column in your gradebook that has got like a like a tan Color in and a given column. That's a that's a visual indicator to you That somehow either intentionally or not you may have indicated an override for a Waiting method. So if you don't want that you can go into either Where we are right now Set up or this drop down up here categories and items view And look for that the corresponding item that is has got that tan color in the column and Uncheck that click save changes and that will remove that override for that waiting and there are back to the default So we covered What waiting basically is in your course if you leave everything unweighted manually and let it wait automatically Your natural aggregation method will calculate for you Again based on how many? Gradable items and how many points within each item of potential grade point value they have It calculates by category What the weight for each category is and you can easily make an override of a category weight Or a wait for an individual gradable item with these checkboxes Make sure you always click save changes at the bottom and That covers the basics of the little gradebook waiting tech talk. Thanks for joining us We'll see you next week