itw, ond wrth gwrs yn ddigonwyd gyda'u gweld o'r ddechrau i'r ddweud o'r gyfnodau a'r gyfnodau i'r corff. Rwy'n gweithio i gyd yn y cyfnod, ac mae'n mynd i ddweud i'r ddechrau i Derym Macai, o'r rydwg ffordd. Mae'n ddweud i'r ddweud i ddweud i'r ddweud i'r ddweud i'r ddweud a fydd yn ni'n o'r gweithi o'i ddweud o'r ddweud. chi gydrwyddo i ddim yn dda i ddiddordeb! Gydwyddo i ddim yn ddiddordebad rŵr i gyd nid, ac ydych i'n gwneud i ddim yn dda i ddiddordebad ym mddangosu gyrhau. Yn i'n gwneud i'r cadwiau, minister? Yrgrifebeith yw'r ddigwydd amddangos i ysgolion i ddim yn ddigwydd ar y llwyth mor yigol. Ac mae'n dda i ddim yn gwneud gallu g redo o'r hawdd mae'n meddydd dros ar ddau ddiliaeth theirside pedestrians on the night of Thursday 3 December. I'd like to update Parliament on the reasons for closure, provide information on the mitigation measures that have been implemented and the next steps that we're taking to repair the bridge and return it to normal operation at the earliest opportunity. Firstly, I'd like to thank communities around the fourth road bridge, commuters and road users for their continued patience at this time. I'd also like to reassure them that we're aware of the significant impact of this situation and that we're working with all our partners to minimise the impact where we can. This is an issue of national significance, and with everyone playing their part, we can limit the impact on the local, regional and national economy. Following the discovery of a serious defect near the north-east tower on Tuesday 1 December, during a routine inspection, the decision was taken to restrict traffic to the northbound carriageway away from the defect area. Detailed analysis of traffic and different traffic load scenarios was then undertaken to evaluate the structure and determine if it was safe to keep the bridge in operation. Results on the morning of Thursday 3 December showed that the existing restrictions needed to be augmented with a further restriction on vehicles over 7.5 tonnes except buses, which had been modelled into the load analysis. The defect affected one of the two trust end links that support the main trust at the north-east tower. If a further failure had occurred, then support would have been lost to the end of the main span stiffening trust, which would drop by between 150 and 700 millimetres depending on the loading at the time. That would mean that the load will be redistributed across the link in the north-west tower, increasing loads on the other elements. The carriageways would also drop further damaging the structure. That kind of damage is a likely outcome and would require bridge closure for a repair that could last several months. The operating company, Amy, conducted a series of additional inspections and testing on the welds and joints to the other trust end link immediately after the initial defect was found. The first focus was to inspect the welds of the adjacent member, now carrying additional load from the failed member. The results of this inspection and testing were completed by late afternoon on Thursday 3 December and presented to Transport Scotland. The outcome showed that cracking, having started at the same weld location and spreading along the load-carrying weld at the critical pin joint. At this stage, the extent is small but the implications are large. The main trust of the bridge relies on this joint to be at full strength to cope with the additional loading due to the adjacent defect. With continual loading, our experts concurred that the identified crack is likely to propagate leading to the failure of the remaining trust end link. The timescales for this to occur cannot be estimated as a large number of factors involved, many of which cannot be fully quantified at this current stage. However, by removing the remaining traffic load from the structure, that reduces the load and stresses on the remaining trust end link and ensures that the travelling public is not put at risk. As a result, the decision was taken to close the bridge to all traffic from midnight on Thursday 3 December in order to help safeguard the integrity of the structure. It is anticipated that following the completion of a successful repair, the bridge will reopen in time for people to return to work in the new year. The decision to close the fourth road bridge was not taken lightly. It is firmly based on the expert opinion of the engineers who operate the bridge day to day and that of independent experts in the field. Every effort is being made to open the bridge as quickly as possible but safety is the main priority. Unfortunately, those works are weather dependent given the height and location of defect on the bridge. We are aware of the potential economic impact for strategic traffic in the east of Scotland and on people living in local communities. That is an unprecedented challenge in the operation of the fourth road bridge. On balance, following advice from engineers and independent experts, full closure is the right decision and is essential for the safety of the travelling public and to prevent further damage to the structure of the bridge. The bridge operators Aimee have a robust inspection resume in place, which aligns with industry standards for a structure of this nature. That regime is a continuation of the methodology used by the 4th Estuary Transport Authority, FETA. Due to the thorough nature of that regime, specialist engineers are confident in their view that those defects are problems that have only occurred in the past few weeks. We are taking every step that we can to alleviate the impact of this closure. Action taken last week will mean that any closure will be much shorter than it might have been if we had waited to take action. We continue to work closely with all partners to co-ordinate our efforts to alleviate the impact of this closure. Every effort and resource available is being deployed to repair the damage to the fourth road bridge and minimise the disruption to the public. To be clear, the 4th Estuary Transport Authority, FETA, reports that are being discussed in the media refer to the other end of the trust end link where it connects with the north tower at the top and not to the pin joint at the base of the link, where this defect has materialised. Works to the top of the trust end link were already under way. Specialist engineers believe that this new defect identified on Tuesday as part of a routine inspection that has only occurred in the past few weeks. Based on the advice and evidence that we have received from specialist engineers and for the avoidance of doubt, we believe that the current fault is entirely unrelated to the above project and there is no indication that the ongoing repair project in the towers has caused the defect. The Scottish Government has fully funded all FETA programmes since taking over the funding of the annual grant in 2008 and prior to the dissolution of FETA earlier this year, FETA made decisions on their programme and priorities of repairs completely independent of Transport Scotland. The timing of the closure was communicated to the public within minutes of ministers taking this difficult decision and was covered on evening news programmes, advising of the closure and the measures to take when travelling on Friday morning. Local authorities were involved from early stages and undertook to inform their communities where possible. There is on-going consultation with business organisations such as Scottish Enterprise, Federation of Small Business, Chamber of Commerce and the Road-Hallage Association. We are also in discussions with other partners such as Police Scotland, other emergency services, NHS Scotland and public transport operators to minimise disruption and deliver our contingency plans. A comprehensive travel plan was launched for commuters and affected communities on Sunday afternoon to allow people to plan their trips for the working week ahead. We have also created a dedicated website that had over 80,000 or 85,000 hits on Sunday. That is details of the travel plan along with some questions and answers to help people to tailor their own travel plans. That comprehensive travel plan was put in place in time for the Monday commute to work. With 100,000 people using the bridge every day, delays and longer journeys are inevitable, so it is important that everyone, workers and employers, are flexible in working arrangements during this period. Together with the public transport operators, we too will be flexible. The plan will be monitored and adjusted to give the best possible service to the travelling public. Additional rail capacity has been provided by ScotRail as part of the full travel plan. That was made available to the travelling public on Sunday to allow them to plan for their journeys to work on Monday. As a result of our monitoring, it has been modified to accommodate commuters where possible. In response to passenger demand, an extra early morning train has been laid on, leaving and working before 6 am. Overall, an extra 8,000 seats are now being provided, and that will increase further. Additional subsidised bus services have also been provided, 33 extra buses providing 11,000 additional seats. Along with bus priority measures, those have allowed for reliable journey times to Edinburgh even in peak periods. Both bus and rail services are being served by dedicated park and ride sites at Halbyth and Ferry Tull, and we continue to work closely with Fife Council to monitor the operation of those sites. A dedicated HGV and bus route was implemented from Monday morning, and that involves creating traffic and ensuring that we put measures in place to prioritise bus movements to get the maximum number of people to work and to ensure that the journey times for HGVs were improved, reducing any impact on the economy. The travel plan also included alternative routes for road users using the Concardin and Clickmanager bridge. Updates to the public and are continuing to be broadcast using Traffic Scotland via the website, Twitter feed and Travelline Scotland app, as well as Traffic Scotland radio, and ScotRail and Stagecoach are also providing regular updates. That will be monitored throughout the bridge closure and adapted as necessary, and I would like to thank the local communities of those areas for their patience throughout, as I am sure that that will cause some additional disruption to them. Indeed, following our monitoring of the HGV and bus route, the restrictions on this route have been relaxed between the hours of 8pm and 5am to help ease the impact on local communities. The plan is in place, and today's situation is as follows. Rail services commenced with the additional service at 552, which carried approximately 160 passengers. Services between Edinburgh and Fife have been busy and were strengthened where possible to cope with additional passenger flows. Queuing systems have been in place at stations in Fife, and ScotRail staff are in attendance at all stations. The 613 from Delgety Bay was full with no room for 60 at 70 passengers at Inverkeething, nor a further 25 at Rossife. That service, as I understand it, was the only instance where passengers have been enabled to board this morning, but all those passengers were accommodated on the next train. The roundabouts at the A977, A907, Gartary, A977, A876, Kilbagge and the M876, A876 and A985 Houns Newk were heavily congested during the morning peak, but continued to move. Fife Council also reported problems on the coast road through Col Ross. Congestion at the A9 at Brockston roundabout was heavy at times, although the traffic management at the A9k roundabout worked well and kept the strategic traffic flowing. At 9.30pm, the roads were running free, while still busy at the key roundabouts. The bus HGV prioritisation on the A985 between Kearney hill and Longannet roundabouts operated well and facilitated parking ride buses from Ferrytoll and Halbyth. Stagecoach reported bus journey time of between 1 hour and 30 minutes and 1 hour and 45 minutes. Uptake of the parking ride was low, however, with loadings averaging 12 per cent at Halbyth and 7 per cent at Ferrytoll, and we are doing everything that we can to encourage further use of the bus service. I remind everyone that the successful implementation of the plan depends on the choices that people make, and we would again encourage the public to use the additional public transport services, particularly bus services. Emergency vehicles will still be able to use the bridge in blue light situations, and arrangements have also been made with NHS Scotland in respect of other critical medical appointments. A call chaired by the Deputy First Minister was held this morning with business organisations, and that was an opportunity to share information and identify any practical steps that could be taken. A number of suggestions have been made by business and ministers have committed to look at them all in detail. This is an unprecedented transport challenge. The safety of the travelling public is of paramount importance and the decisions that we have taken will ensure that this is maintained. Specialists are working day and night to return the bridge to normality, and we will fix the problem as soon as we possibly can. We continue to work with all partners in the emergency services to manage the impact of the closure and to help to ensure that diversions operate as efficiently as possible, and we will continue to share all travel information through the dedicated website. Members of this Parliament will be aware of the issues previously raised regarding the suspension cables on the bridge and the subsequent action taken by FETA to mitigate the impact and halt any further deterioration. The residual risk of a potentially lengthy future fuel bridge closure remained, supporting a decision to progress with a fourth replacement crossing. The gravity of the situation and the impact on thousands of people and their ability to get to and from their work cannot be understated. We believe that the Scottish Government took the right decision to shut the bridge as public safety must be the absolute priority. I would also want to acknowledge that the minister has worked hard over the last few days to keep the focus of the Government on people and on businesses affected by putting in place the emergency measures that he has. I would want to say to the minister that it is important that people have a way of feeding back and feeding in what is working and what is not working. My party will continue to work with the Government to ensure that we are able to do everything possible to support people get to work and support businesses impacted. The minister referred to reports in the media about what has gone wrong and why, but to be clear it is not just the media that is asking questions. How has this fault developed? Yesterday, we had a top engineer claim that the key maintenance on the bridge was cancelled in 2010. We know that two senior engineers on the fourth road bridge resigned. It is understood that these specialists with years of experience on the bridge left because the operation and maintenance of the bridge was effectively privatised. What is the impact and what impact has privatisation of the bridge had on the availability of the expertise and, ultimately, on the on-going maintenance of the bridge? We know that in 2012 Audit Scotland confirmed that there had been a capital funding cut to the bridge. We know that in 2007 Transport Scotland identified repairs that were needed on the bridge that were never carried out. People have questions and people need answers, and that is why today I am calling for a parliamentary inquiry into the circumstances leading up to this crisis. I believe that a parliamentary inquiry is in the public interest, and I hope that the minister and the Government will support such an inquiry. I thank Alex Rowley for his participation over the course of the last number of days. The key communication has been important in terms of suggestions as to how to improve the travel plan and share the message around safety. It is a fair comment and question around the use of social media and information sharing. We also listen to the public and communities, and we are doing that in real time as we have adapted the travel plan, for example, on lifting the restrictions on the priority corridor. We are engaging with community, elected members, parliamentarians and local authorities and businesses, and I think that that is the right thing to do, to listen and to respond. In terms of the fault that has been identified, let me be clear that that fault was not predicted or identified. That is not a location of the member who was deemed to be overstressed. Therefore, it was not predictable. Separate works were already underway on the rest of the member, but that specific part of the element was not predicted to fail or to crack in the way that it has. Because of the comprehensive inspections and the daily and weekly inspections, it is that information that leads our expert engineers to conclude that this fault has only occurred in the past few weeks. Hopefully, with the offer of a further technical briefing to parliamentarians, that will assist in terms of the understanding of the technical fault that has occurred. We will be transparent around the nature of that fault, which I think shows that the Government has taken all appropriate action. In terms of funding, there are no critical repairs that were requested to be funded by FETTA that have not been funded by Government or Transport Scotland. The operation and management of the bridge was independent of the Government. It was led by that operating committee. Finance was in place to carry out the identified work programme. Indeed, the work programme that was developed by FETTA was being delivered by our operatives through the new operation. In terms of staffing and the current practices of the operator Amy on the bridge, the impact of staffing is that there are actually more people working at the bridge than was the case before the transfer on 1 June. Their substantial indeed enhanced expertise for the bridge is part of the current operating arrangements. The nature of staff change is not as Mr Rowley has described. Indeed, most of the staff who were working on the bridge before transfer on 1 June are still working on the bridge and all the operating manuals and history of the bridge have been maintained. There was a seamless transition from operators to ensure that continuity of work. Indeed, the Government through Transport Scotland and the operators have prioritised the elements of work that we inherited from FETTA. It hardly needs to be stated how damaging the closure of the bridge is to the economy of Fife and the east of Scotland more generally. I have been contacted by businesses who stand to lose considerable sums in an important trading period and by commuters who face weeks of disruption, additional cost and frustration. The efforts of the Scottish Government, its agencies and transport companies to put alternative travel arrangements in place are appreciated. However, there are still problems and, for now, the priority should be in resolving those. For example, this morning, I used the ferry toll park and ride, which was very quiet, but I still found myself waiting 40 minutes for the promised shuttle bus to Inverkeithing rail station a little more than a mile away. In fact, I could have walked the distance had I realised that the wait was going to be so long. While those travelling from Fife into the centre of Edinburgh have options that they can use, there are many of my constituents who need to get to work in West Edinburgh airport, elsewhere in the Lothians or in the central belt. What more can the Scottish Government do to provide them with public transport alternatives? There are two other points that I want to cover briefly, if I can. Firstly, businesses that are losing large sums of money as a result of the closure are understandably calling for compensation. What proposals does the Scottish Government have to assist them? Finally, there has been a great deal of speculation, as Mr Rowley said, that the bridge closure was the result of inadequate maintenance. We have heard from John Carson who has blamed incompetence on the part of Transport Scotland. Will the minister now agree that we need a fully independent inquiry into what went wrong and reporting as early as possible in the new year so that we can find out the truth of the matter and learn lessons for the future? I thank Murdo Fraser for those questions and his support around the public transport alternatives. They have been strengthened. There is real information around that that is of assistance to the public on the questions of alterations and improvements. We are looking at further strengthening the public transport interventions. I am happy to hear any constructive suggestions on how we may further improve that as we enhance what is currently being provided. In terms of business support, the Deputy First Minister has engaged with businesses this morning. As I have said in my statement, there will be a full consideration of the suggestions made. However, the key thing has to be to get the bridge open as quickly as possible, and we are working around the clock to do that. The Government has taken the right decision in the bridge closure so that the preparation and repair work can be carried out. I believe that we have averted a much more serious structural incident that would have been more damaging to the economy in the area if that had occurred. In terms of the transportation of goods and other supportive businesses, that is why we have prioritised HGVs and we are looking at extending that further to further support businesses. By way of our intervention, our prioritisation and our partnership working with businesses will continue to do everything that we can to support them at this very difficult time. I say that we are very tight for time this afternoon, and I have absolutely no scope to extend this particular session. I am therefore going to give priority to the constituency and regional MSPs, whose constituents are the most affected, and those constituency and regional MSPs who indicated to me by the usual time their wish to ask a question. David Torrance, followed by Willie Rennie. Thank you, Presiding Officer. Like myself and many of my constituents who use rail to Edinburgh daily, the minister clarifies exactly what measures has been put in place on a rail network to minimise the disruption. And what discussions has the minister heard with local authorities on relaction of parking restrictions and on an increase of available parking spaces? Presiding Officer, I will try and be briefer in my answers. In terms of rail, we have through ScotRail identified extra carriages to increase the number of carriages trains provided and an extended timetable that is amounted to an extra 8,000 seats provided, and, in fact, that will be enhanced further. Staffing is in place at all affected stations, and we are sharing that information through the dedicated website. In terms of consultation with local authorities, they are key partners in this. They have worked in partnership with us to help us to have Police Scotland manage local traffic impact, and they will look at activities such as removing unnecessary roadworks to try to encourage a free flow of traffic where possible, and we will continue to engage with them in terms of our travel action plan. Will Rennie fall back, Sarah Hilton? I thank the minister for an advanced site of his statement, but also for his engagement over the weekend. In particular, I was pleased with the announcement about chemo and radiotherapy patients and the special arrangements over the bridge. I want to be clear with the minister, however, that I expect him to return to Parliament for a proper examination of the decisions that were taken by ministers in the past few years that might have contributed to the defect on the bridge today. However, for today, people want to know what changes the minister is going to make to the transport plan. In particular, will he agree to lift the restrictions on the A985 for off-peak during the daytime? That is having a dramatic impact on the local community and on traffic flows. I would appreciate it if he could look at that again. Will I have said to Willie Rennie that I would invite him to take up the offer of a technical briefing? I believe that Willie Rennie has done so, and we will do that today. That will give a fuller understanding of the technical issues that I have referenced in my statement in terms of how that fault has occurred. As a listening Government, we are adapting the travel plan to take account of local circumstances. There is on-going monitoring of the traffic system, as well as the demand for transport provisions, such as the enhanced rail services. There is more capacity on buses and on park and ride, and I encourage people to use that. The priority route is working in terms of the reliable journey time, but if we can relax it further to support business and communities, we will absolutely look at that in the way that we have lifted some of the restrictions to reflect what was working, what was rational and what can make the biggest difference. Of course, remain open-minded to the right interventions. As the constituency member has covered, I am familiar with West 5. There are very few of my constituents who have not been detrimentally affected by the bridge closure. I am very grateful to the minister for his speedy responses to the issues that I have raised already. The minister is aware that, of the lack of parking at train stations, that is an on-going issue in West 5, with spaces full before 8 o'clock in the morning. The bridge closure is causing a real headache for the growing number of commuters who have no option but to travel. On Friday, I wrote to the minister asking that free shuttle buses be provided between Halbyth park and ride and local train stations. I would like to ask the minister if he discussed this option with stagecoach, and if so, to let me know why this option is not being pursued, because it does not surprise me to hear that the uptake of park and ride is low. I mean, no commuter wants to spend two hours sat on a bus when they could be there, and half the time if a free shuttle was provided to the local train station. I would appreciate if the minister would let me know what discussions have happened with stagecoach, because our front-line workers have to be put before stagecoach shareholders, so I appreciate if you would act in that. The reason that stagecoach has been deployed is that, quite frankly, they are the largest operator in the area. With that expertise, it is very useful to use them. The extra buses that have been provided has created the capacity. There is huge demand on rail, and maybe that is because of the certainty around that, but can I correct the journey time issue that Kara Hilton raised? The average journey time is actually an hour and a half, not two hours. I think that that compares quite well with people using the private car. I understand that some people will continue to require to use the car, but for those who can use public transport, I direct them towards the bus provision. Rather than having more buses going to the park and ride the railway stations, it is possibly more about trying to transfer some people from queuing at stations on to the buses, where there is that extra capacity. However, again, I have been in regular communication with a number of members, including Kara Hilton. I have considered every suggestion that he puts me and worked that through the system, and many of them have been implemented, and that will remain under constant scrutiny focus and adaptation, if it makes sense. I need shorter questions. I have within my constituency one business 98 per cent of whose goods are exported out of Scotland. I am grateful for those comments about the dialogue between the Deputy First Minister and business. Will that dialogue continue so that the urgent situation can be resolved? Yes, it will. We will have constant dialogue with the key business representatives and, of course, major employers in the area. It is clear from questions around the chamber that we would benefit from a parliamentary inquiry on the issue. The minister has said that it has anticipated that the bridge will be open for people returning to work in the new year. How confident is he of this timescale and which factors might lead to a delay? It is obviously of great concern to Commuters in 5, particularly shift workers, that we see the bridge open as soon as possible. The last briefing that I had, which was just before I left the National Traffic Control Centre, where the multi-agency response is being co-ordinated to come to Parliament, was that the work is on track, as per that timetable that has been published. Factors that may change that, of course, are the fact that the works are weather-dependent, because we do not have people working in unsafe conditions, but we are working around the clock to get the bridge open as quickly as possible. For those who are being critical, there is a slang phrase that recognises the life-full of unpredictable events, but I better not use that in here today. In that light, can you tell me what discussions you have had with the UK Government on the relaxation of rules for HGV drivers who may face issues around working hours as a result of diversions? I am sure that you recognise that the haulage industry is very important to the Scottish economy. Recognising the pressures on business and the haulage industry, I moved very quickly to have discussions with the Secretary of State, and he has had discussions with the Department for Transport, and there will be a relaxation of drivers hours to support businesses at this time. James Baxter, followed by Jim Eadie. With 100,000 people using the bridge every day, it is clear that this entails a wide range of journey patterns and journey purposes. The minister's statement made reference to a willingness to monitor and adjust the travel plan according to feedback. Can I ask whether any consideration has been given to offering a telephone helpline that would gather first-hand experience and provide reliable advice and information and so enhance the monitoring? There is a telephone helpline available through Transport Scotland and people are able to use that, and it has been scaled up in anticipation of demand. While the main priority in the coming days and weeks must be the need to minimise disruption to the travelling public, once the bridge has reopened, will the minister instruct a full and thorough assessment into the causes of the closure so that we can understand why we have arrived at this position and what lessons, if any, can be learned for the future, and what further assurances can he give that the Government will act to ensure that there is proper transparency and accountability in all of the historic decisions that were taken prior to Transport Scotland assuming responsibility for the bridge? That is a helpful question. We have shared a lot of the technical expertise that has come from expert engineers. We can continue to do that as we update people on the progress of the bridge. Of course, we will review systems, inspections and processes to ensure that, if there are any lessons to be learned, we will learn from that. I think that that is the right thing to do, considering the unprecedented nature of the incident and the closure and the impacts that I will commit to doing that. Presiding Officer, after discussion with the haulage industry in Highlands Islands, I have two very quick practical points. One is whether we can relax the driver's hours issue, which I think Bruce Crawford raised, which I think is very important. Secondly, can we increase speed limits as an emergency measure on single-cards ways to 50mph in light of what is happening currently in the A9? As I have addressed the issue of drivers hours, there will be a relaxation and that has been taken on board between the Scottish Government and the UK Government. However, on the issue of speed limits, David Stewart and I have exchanged views on that in the past. Apart from the A9, there has been a very specific package of measures put in place to allow for an increase in speed for HGVs. I do not think that it would be appropriate wholesale and I do not think that it would be appropriate on this specific measure only because there would be an increased risk of fatalities and casualties if there were more incidents. Indeed, south of the border, when the UK Government is increasing its national HGV limit and its assessment, it says that there would be an increased risk of fatalities and casualties. I am afraid that that is not a gamble that I am willing to take with lives in Scotland. Does the minister recall my announcing in June 2007 that there would be a new bridge delivered in 2016? Was it a proper permanent response then, an effective management now, to be on schedule and £1 billion below the budget that I announced in 2007? I think that it is fair to say that, as well as all the other contingency plans that have been put into place as a consequence of the bridge closure, it is a rather substantial contingency plan, a decision made in 2007 to build at replacement crossing. I think that it shows that the Government has been vindicated in building a replacement for crossing, which is very much on time and under budget. Does the minister have any comment to make on the document that was published on the Public Contract Scotland website on 25 May 2010, which is headed trust-end links, fourth road bridge, and the status of which is now cancelled? I have offered all political parties a full technical briefing and an explanation of mitigation measures. The only party that I know that has not taken this up as a Conservatives, which is maybe why Alex Johnstone is so ill-informed. I would have thought that, in listening to the statement, some of this would have been covered and understood by Mr Johnson. The key point here is that that was feta in operation at the time. The works identified is not where the fault has occurred. This specific fault, this specific crack, was not predicted. It has emerged in the last few weeks and identified on Tuesday recommendation to ministers on Thursday, an action taken within minutes around that. Feta, responsible at the time, had a works programme that they were working through and had not identified. In terms of the technical nature of that contract, Feta re-scoped their own works. They were getting on with the job after 1 June on transition to Scottish Government in our operation with AMAY. We have inherited the work programme. We were delivering that. We were strengthening brackets that had been identified. This fault, quite unrelated, has emerged and Government has taken the swiftest action possible. I recognise the frustration for communities and businesses and indeed the cost of the economy is estimated at some £50 million. The First Minister told the press today that there was not a cut to the budget. Audit Scotland, in its report on Feta, said at paragraph 34 that the budget for capital expenditure was cut significantly. Is not it the case that Audit Scotland are quite right? Plans were made by Feta, done in conjunction with Transport Scotland and in the context of reduced budgets. No, it is not the case that budget decisions have had an impact on this fall. The expert engineering advice is saying that this fall was not predicted and has only appeared in the last few weeks. Feta were amending their work programme, Scottish Government having inherited it, were delivering it and, of course, we will see through this necessary repairs. When it comes to capital grant, there was on-going investment and funding this year would be £10.7 million, matching the programme of works that was developed by Feta. It has not been subject to any reduction, in addition it will have never restricted funding for critical works. I thank the minister for his statement and the measures put in place to deal with the closure of the port bridge. Could I ask what actions the Scottish Government is taking to alleviate the difficulties that are faced by commuters using the already under-pressure Delmene railway station in my constituency, which is the first halt on the southern side of the fourth well bridge, who are unable to board trains due to capacity issues as well as the local road congestion and parking problems around South Queensbury? Well, as the extra carriages that have been identified and deployed are staffing at stations to support commuters and the travelling public, we are looking at further enhancing the number of seats through extra carriages and further amendments to the timetable to support everyone who is affected on the rail line, which has been enhanced to support commuters at this very challenging time. Sarah Boyack, followed by Mike McKenzie. Can I follow on that question to ask if consideration will be given to early services? One of the bits of feedback that I have had is that, for many commuters, it is simply not possible to use the train because there are not early enough services and that is having a big impact on businesses and companies in the public sector in the Edinburgh area. That is a very reasonable question because our advice is that if people can avoid the peak periods, that will help to alleviate the congestion and the busy periods. Therefore, we have extended the timetable for an earlier departure, the 552 service, and we are looking at further enhancing that for earlier trains and overnight trains, if that is possible as well. Hopefully, to take the number of extra seats provided during the duration of closure to over 10,000 extra seats, we are actively looking at that and I will update members through the channels that have been established. Mike McKenzie, followed by Patricia Ferguson. I would like to ask the minister with a greatly increased number of commuters travelling by rail or bus what discussions he has had with local authorities on rail-luxiation of parking restrictions or in increases in available parking. Local authorities and Police Scotland have been asked and were being proactive in taking all reasonable measures. That might be removing restrictions, supporting parking where appropriate, and, as I said earlier, removing unnecessary roadworks to try and make our road system as accessible as possible. However, it is simply not possible to display 70,000 vehicles on to the rest of the network and not expect a degree of congestion. That is exactly why we are encouraging people to car-share to avoid travelling if possible, but we are using public transport. On demand and public transport, rail is in huge demand, so let us add the focus to bus, where there is plenty extra capacity and prioritisation through the bus and HGV corridor in journey times at around an hour and a half, which is much better than was anticipated. I sympathise entirely with the plight of those in Fife and in other areas that are most directly affected by the closure of the bridge, but the 8,000 additional train spaces that the minister spoke to have been brought to Fife and to surrounding areas at the cost of some disruption in other parts of the country. I wonder whether the minister can assure me today that the disruption will be kept to the very minimum and whether he will, in future, speak with Abelio about the possibility of there being additional spare capacity in terms of rolling stock, because the constituents that I represent have suffered already in this year disruption to the service between August and November because of engineering works, now disruption because of the closure of the fourth road bridge, and while during the period of the Egypt project suffer a further 22 weeks disruption, my fear is that they may not return to the train. All of those disruptive works that Patricia Ferguson has outlined is to achieve the outcome of more trains for Scotland, faster trains, greener trains and enhanced capacity with more seats as well. Indeed, the Edinburgh Glasgow Improvement programme will be about the future proofing of the railway, so some of the pain and the disruption that people have endured. I appreciate that. It is worth it for that expanded rail service that is being provided in terms of the other disruptive works that are necessary for electrification and the refurbishment of Queen Street Stage. That is the first point around some of the works earlier this year. On the second point of supporting Fife at this time, I am sure that the whole country understands that that is an issue of national significance and we need to pull together to support the region at this time. Again, I appreciate the impact that it has had on some other people, but it is important to give the area as much support as we can and what is a very challenging period as a main artery is not in operation. Finally, in terms of the rolling stock, we have used the rolling stock capacity to the max so that we are getting the best out of the railways for the rolling stock that we have, but more trains are ordered and will be delivered through the new franchise agreement, and I think that that will be a good deal for Scotland. In terms of the temporary impact, we asked ScotRail to try to identify carriages and rolling stock out with Scotland first before impacting on services within Scotland. That has been achieved in that some of the rolling stock has come from elsewhere, but yes, there was some impact in some ScotRail services, but surely we all understand that the Fife area is under considerable pressure and the intervention required was the right thing to do to pull together as a country and support that region in a way that most other members were encouraging me to do throughout the last few days. Would the minister ask his officials in Transport Scotland if they could ensure that the signposting for diversions is accurate, something that was not the case between the Cairney hill roundabout and the Concordant bridge yesterday? Yes, of course. We will try to make sure that all relevant information is accurate and updated in real time. Some of our equipment was subject to vandalism as well, I have to say, but of course we will try to make sure that all information is up to date as possible, but the nature of this incident and the listening government is that we are changing the travel plan to suit in terms of what is working and what will provide the best intervention, and then all information should flow seamlessly from those decisions. Thank you. That ends the statement from the minister. I will have a short pause while the members get themselves ready for the next item of business.


I am Mr. P. P. Mitragootri, Associate Professor in Department of Mechanical Engineering at Vulturen Institute of Technology, Solaapur. Today we are going to study what is merchant circle, what is merchant circle diagram and how it can be utilized for graphical representation of forces acting along shear plane and acting along tool chip interface during metal cutting process and how we are able to graphically calculate magnitude of shear force, normal to shear force, magnitude of tool force and normal to the tool force with the help of this diagram. And outcome of this particular session will be, learners will be able to draw merchant circle diagram and will be able to calculate various forces acting along shear force, shear plane as well as along tool chip interface by orthogonal cutting and this is graphical method of representing the forces. Now, for drawing this diagram and for further things, first and basic assumption is our metal cutting is orthogonal cutting, orthogonal cutting is two dimensional cutting, it is strictly confined to two dimensions. Now, for drawing merchant circle diagram, required data is we have to measure cutting force denoted by letter fc which acts in the direction of tool travel or tool feed and thrust force denoted by letter ft which acts perpendicular to the cutting force. These two forces are measured by tool dynamometer during metal cutting operation, rake angle of the cutting tool is denoted by letter alpha is recorded from specification of the cutting tool that we are employing for orthogonal turning or orthogonal cutting. And uncut chip thickness denoted by letter t and chip thickness after cut denoted by letter c are to be recorded. Now, we will begin the procedure for construction of merchant circle diagram.  After that, we have to draw a cutting force fc which is measured by letter tool dynamometer acting in the direction of tool travel with proper scale of correct magnitude as shown in the diagram. Then, we have to draw a thrust force perpendicular to the cutting force with the same scale and with proper magnitude. Then, we have to draw the resultant of these two forces. This resultant cutting force and thrust force will form a triangle. Now, next important thing is we have to carry out geometrical construction which we have studied in school days to find out midpoint of the resultant because this midpoint is going to act as a center of the circle which we are going to draw through these three points of the triangle and that is called as merchant circle. This merchant circle is visible in the diagram now along with merchant circle we are observing cutting force and thrust force. Then, next construction will be we have to show uncut chip thickness with proper scale and as uncut chip thickness is measured along the direction of tool travel, line representing uncut chip thickness will be parallel to the cutting force as shown in the diagram. Then, chip will always flow along the tool face. So, cut chip thickness will be parallel to the tool face and cut chip thickness is denoted by later TC. Now, these two lines intersect somewhere and that point of intersection is visible in the diagram. Now, through that point of intersection and through the tool point we have to draw the line cutting the circle at certain point. This line represents shear force denoted by later FS and you have to understand the angle made by this line with cutting force. I think you can understand it this is reflection spot in our video. I will wait for a moment. I understand that you have followed it and this angle can be no other than shear angle denoted by later Phi. Now, we have to draw the line perpendicular to the shear force which will join the resultant force and this shear normal to the shear force is denoted by later Fn and resultant top shear force and normal to the shear force is also the resultant top cutting force and thrust force that we can very well understand from this diagram. We have constructed shear force and normal to the shear force we have to measure their magnitude and this magnitude with proper scale and we have to transform them into Newtons or KGs as per the scale that we have adopted and angle between cutting force and shear force we can measure and we can say that this is the shear angle. Then we have to draw a tool force along the line of the tool face from the tool point and that will cut the circle at point shown over shown in the diagram this tool force is denoted by later F. Then we have to draw a perpendicular line joining the resultant which will represent normal to the tool force which is denoted by later N. We have to measure the magnitude of F and N and these magnitudes if transformed into proper scale they will give us the number that is the value of Newtons and this way we can graphically represent and graphically calculate all the forces. Now I will once again rapidly repeat the procedure that we have followed what we require we require to know cutting force and thrust force we have to draw cutting force thrust force and resultant triangle then we have to find out the midpoint of the resultant by geometrical construction with that midpoint as a center we have to draw and resultant as radius we have to draw the circle which represent merchant which is called as merchant circle. Then we have to draw a line representing uncut chip thickness parallel to the cutting force and we have to draw the line representing chip thickness after cut parallel to the tool face and through the point of intersection of these two lines and through the tool point we have to draw the line cutting the circle which will represent shear force and we have to draw the line perpendicular to that which will be represent now which will represent normal to the shear force and angle between shear force line and cutting force line is none other than shear angle similarly line along the tool face will be tool force and perpendicular to that will be representing normal to the tool force hope you have understood the procedure to draw the merchant circle and procedure for calculating various forces acting along the shear plane and tool chip tool interface by merchant circle diagram for further reading I will recommend a book by machine tool engineering by book machine tool engineering by jr nakpal and textbook of production engineering by pc sharma thank you


OK, testing. We good? OK, awesome. OK, welcome, everyone, and thank you for coming. My name is Alon Philanenko, and I'm an engineering team lead at Bloomberg. And I'm joined here by my colleague, Aki Tsukagawa, who's a senior software engineer with me on the Enterprise Data Science Infrastructure team. So today we'll be talking about managing multi-cloud Apache Spark on Kubernetes. So in general, starting from the top, managing data science infrastructure in a multi-cloud environment is really hard. The variation of behavior between different hardware and cloud providers is vast and rather daunting. Now, Kubernetes in its cloud-native design provides an extraordinarily powerful abstraction across all these hardware stacks and enables the building of highly composable and transportable infrastructure substrates. This talk will be primarily focusing on that feature of Kubernetes as we're looking to address complications that might arise, specifically that we had when we ran a data science platform both on-prem and in various cloud environments like AWS EKS and Azure AKS. Given the Kubernetes as a foundational abstraction that we're using, we found that when designing various compute runtimes like Apache Spark, each of these runtime components require the same standard managed services like we have here identity management, network policies, federated job APIs, and more. But in this talk in particular, we will be focusing on the journey that we took to build out the Spark compute runtime within our data science platform. So what is Apache Spark? I'm sure of Hans who is familiar here with Spark and who's not. Good. OK, so there's a couple. OK, there's a couple. So we'll do a little bit of run-down. So what is Apache Spark? So the quick background for those who are unfamiliar is that it's an ETL application that enables us to provide data science at scale. So traditionally, ETL for files smaller than a gigabyte uses pandas, but when you start scaling up to 100 gigabytes, you either need a chunk pandas data frames or use Dask, but Spark can also be a pretty good candidate. But where it really shines is when you start looking at petabytes or more. It has a variety of features that make it a powerful general execution engine, especially when communicating with cloud object stores. But for the purpose of this talk, we'll be focusing on its distributed architecture and its relation to Kubernetes. Spark runs as independent processes on a cluster, coordinated by the Spark context object in your driver program. So the Spark context can connect to several types of cluster managers, like Mesos, Yarn, or Kubernetes, and for the purpose of allocating resources across applications. So Spark's cluster manager, functionally, is just purpose to provide IP addresses, with which Spark uses them to communicate. So this enables the cluster managers to be rather flexible or composable for a variety of different scheduler backends, like Yarn or Kubernetes. Now, this has benefits for plugability via a scheduler interface, but this does mean that Spark isn't good at looking up or trying to bubble up too much Kubernetes information as it won't necessarily fit into the general interface. We call this kind of a lack of information flow. And this is a lot of drawbacks that we'll talk about throughout this talk. So upon receiving the provision executors, Spark sends the application code to the executors, after which then the Spark context would send tasks to the executors to run. The Spark context could be initialized, for example, as a Jupyter notebook, as you can see kind of in the diagram, which means that the notebook pod would have a service account for directly creating executor pods. So if we were to parallelize the Kubernetes speak, we could kind of think of the driver here as a controller of sorts. It's now tasked with provisioning and managing state. However, this is all done from within Spark. Are folks familiar with controllers or Kubernetes controllers? Sweet. Now, what does it mean for a driver to have controller-like privileges? So this means that the user's pod will have elevated service account privileges that in some environments, like ours at Bloomberg, is rejected due to security concerns. Some examples are, for example, some environments don't allow for users to create pods. Another potential problem is that the full information flow from Kubernetes is expected to reach Spark. But because of the agnostic nature of the scheduler, as I said earlier, this might not be entirely possible, as we discussed a little bit earlier. So to solve this, Bloomberg, as I also talked about in my previous KubeCon talk, looked to enable a pluggable executor creation template that pushed the pod creation to an admin controller. So this allowed now for the communication with the admin controller to kind of push out that service account requirement. Now, to communicate with controllers in general, this is traditionally done via custom resources. So this means that the Spark driver now modifies a custom resource instead of creating pods directly. So this moves the pod creation to an entity that doesn't share service accounts with the users. And that solves the first issue and could potentially fix information flow issues that might come up. But we'll get into that later. So to help understand this, let's work with the visual. Your driver pod, which in this example is running within the user's namespace from a Jupyter Notebook pod, would communicate with the API server to create and modify a Spark app custom resource. With the intention of having the controller in the admin namespace create those executors securely in the user's namespace. The admin controller will watch for Spark app custom resources, where the spec field of these custom resources has some desired state. In our case, we're specifying the expected executor pod templates. And the controller will then act on the contents of that spec field by creating these respective executor pods via its reconciliation loop. The driver will then wait for those executor pods asynchronously, which is functionally then the same as it would otherwise. So therefore, the rest of the Spark paradigm kind of remains the same, even with this pluggable design. Now, we've been running our personal fork of Spark in a managed data science platform across bare metal and various clouds for some time. So let's walk through a couple user studies that address the success stories and complications that arose. So let's start with all of our success stories. We've got a comment that it works. OK, now for what matters, the complications that arose. So without spending too much time on each of these issues, as you can review them and their solutions offline, I think we posted the slides, what I want to highlight here are the two categories of issues. One's relating to the pending status of the executors and the other's relating to the failed states. Now, something to note is that these issues went in a bare metal, non-autoscaling environment with a constrained resource quota can vary from the pending issues that arise when running in the cloud with autoscaling and preemptible spot instances. So to cater to both is quite challenging. Now, regarding failure scenarios, node scale down, preemption, and OMS, these are quite ubiquitous in Spark applications, especially in a large-scale Kubernetes cluster, and especially when you're running in the cloud. And so catering to all these variations of failures is another challenge. Now, Kubernetes with its cloud-native resources provides all the necessary information to cater to these problems across a multi-cloud environment, as you can see by the solutions to these complications. So we will generalize these problems and their respective solutions into three categories and then show that by solving the information flow from Kubernetes to Spark, we can solve for most of the common user complications that have arisen in our platform. So the three generalized limitations that result from missing information flow include autoscaling information, the capture of scale down, preemption, OM, and failure events, and post-job introspection, where a user might want to triage their Spark job after the job completes or fails. So this complements their existing monitoring or logging dashboards that might be available in their platform. So I'll now pass it over to Aki so that he can walk us through our approach and our solutions to addressing these limitations in Apache Spark. So our solution to the lack of information is to collect information ourselves somehow and store high-level statistics into status object, I mean, status field of the custom resources we described earlier. So as you can see in this slide, missing autoscaling information is now stored. I mean, it will be stored in the cluster of scaling field on the right-hand side. And scale down and OM, out of memory failures, information are stored in a terminated status field. And for post-job introspection, we can keep the job object for a while and query this object. And this is how it will look like. So we build contours to collect necessary information somehow and aggregate it as a high-level statistics and make it available in the Spark app custom resources for each Spark app or context. We can, for example, build a UI on top of the information we gather from the custom resource or bubble up the information to the Jupyter notebook that is running the driver program. We could directly collect the information and provide an endpoint to get the statistics in one service. There's nothing wrong with this approach, but we prefer having a separate component for just collecting the information inside the Kubernetes world. So by having a controller to collect information from underlying Kubernetes layer, we have a nice separation of concern. And also, we can follow a common controller pattern for handling Kubernetes objects. So what does it mean? So as we saw earlier, we used the Spark app custom resource for creating executables. So in the controller pattern, the spec field represents the desired state of the world. And the status field is the represented state of the world. So controller tries to represent the desired state and represent back the actual realized state. So we are following the controller pattern here when we store the statistics about the executers to the status field back. So other considerations about the controller pattern itself is that its reconciliation loop needs to be very important, meaning that it needs to be stateless. It cannot rely on the previous results. So it works from scratch every time the reconciliation loop is invoked. So another way to think about this approach is that we are effectively using the custom resource as a data store for the high level statistics we want. We are effectively storing information to SCD via Kubernetes API server. So here's the characteristics of this approach as a data store. So it introduces no extra dependency. And it provides out of the box, pub-sub mechanism via watch. And we can do basic query and aggregation via selectors and controllers. And latency is not ideal, but it's not too high as well. So we can expect an order of a second latency. And not to mention its availability is really excellent. Since we are providing high level statistics in there, we want to push updates via long polling or with socket. So we want pub-sub mechanism in the data store. And the users are not clicking on something and actively waiting for the response. The latency requirement for our UI is not so tight. And we only need the basic query mechanism in this case, in our use case. And since we run it in bare metal and multiple cloud services, we really want to minimize extra dependencies in our case because the maintenance cost and the complexity in each environment add up. So there are two alternatives we consider as a data store. One is that our external log metrics services, such as Hume or Splank, which we already use for our debugging purposes, they are excellent for retrospective troubleshooting via adhoc queries. But building real time UI on top of this is not really trivial. And also, latency is of concern as they are typically backed by S3 storage to handle a large amount of data. So another way we thought about is to use an additional database. This is suddenly doable, but it introduces extra infrastructure. And we also need to choose a solution carefully and design on top of it with care as well. So as we saw, we are actually choosing SCD for our database for the advantages listed before. Now that we have proposed using a custom resource to store executors, high level statistics, let's walk through each of the limitations in information flow we identified earlier. We will start with auto scaling. So what is auto scaling up? So class auto scale up happens when existing node, existing Kubernetes nodes cannot fit in, newly created to pause. So if there's not enough nodes already, the class auto scale up tries to create more nodes. So when it happens, there are actually two problems from the perspective of Spark user experience. Firstly, it takes much longer than usual potentialization. And secondly, it's not always possible. For example, if the user is not allocated enough quota for AWS or Azure resources. So without knowing this underlying cluster, knowing this underlying cluster scale up status, Spark users will be left wondering why the requested executors don't come up. And are they going to ever come up? Or whether they need to inform cluster administrators for help or things like that. So now that we know how cluster scale up works and why it's useful for our users, we are going to actually collect the information to detect the status. So if we are running a standard cluster of scale implementation provided by Kubernetes, that particular implementation will inform us about cluster scale up via event objects. It creates an event object with the reference to the affected pod, like shown in the slide. So the reference is in the involved object field. So one reason we chose to run a standard cluster of scale up, by the way, was because of our multi-cloud environment and the desire to keep a single implementation across all deployments. So this is another example of when a cluster scale up was impossible, so the last one was for scaling up was possible and triggered. So we can detect cluster outscaling state of each executor pod by listing these events from cluster outscaler involving the particular pod and pick the latest one. So we can have one controller watching pod and cluster outscaler events and aggregate all the information into SparkUp custom resource. A minor concern here is that the reconciliation loop is about SparkUp spanning all the executors in the SparkUp context. And the one reconciliation loop will go through the pause of the app and also the events of each pod in the app. And on top of this, the outscaler event can be repeated and it can cause spike of incoming updates and it can potentially make the controller less responsible at the time. So we can separate the concern by introducing another controller nicely. So the additional controller works on cluster outscaler events exclusively and saw the cluster outscaling status information of the affected pod. I mean, the status into the affected pod itself. And the main controller only watches pods. Simple. And this is possible because we can dynamically store additional information to the pod itself. So how we do this? So one way is to put the custom label or annotation to the pod. We define an arbitrary custom label name and put arbitrary information with some constraints. And another less known way is to put the custom condition to the pod status field. We can define a custom condition type with additional field with more flexible constraint. Now that we handle the scale up, the next problem that we will work on is the cluster outscaling down and out of memory. The information source we need here is actually the same as before. The cluster outscaled down information is provided in the exact same way as cluster outscaled up. And out of memory information can be already found in the pod itself in the terminality status in the pod status field. So the same pods and events are the information sources. However, the challenge here is that since these pods have failed already, they can be deleted. As we are following the control patterns, we need all the information source available at the time of reconciliation because control is stateless. And it needs to be even potent as I covered before. So one way to deal with this deletion is not to delete pods in the first place. So there is a relevant option introduced in Spark 3.0 that prevents Spark drivers from deleting failed executable pods. By default, Spark driver deletes failed pods. But by setting this deleted on termination option to false, it leaves the failed pods as they are. So this solves the majority of the problem. And this solved the issue for out of memory error. But this only prevents Spark driver from killing pods.  For example, in the case of a cluster outscaled down or eviction, pod will be deleted by Kubernetes core components, not drivers. So it's not prevented by the above config. It turns out that even for those cases, we can actually prevent the deletion by using Kubernetes finalizers. Finalizers is a field in the metadata. And it's just an array of strings. And as long as it's not empty, the object cannot be deleted. So we can put a custom arbitrary finalizer there and prevent executable pods from being deleted as long as we want. We did not choose this approach because the scope of the implication is really hard to know. Pod is such a central component of Kubernetes. And so many types of resources are attached to it that it is very hard to foresee every single edge cases we can face down the road. So we decided that we cannot keep the pod resource itself. We need to store the information somewhere. So given the previous considerations about the data source, another custom resource in SCD is a natural fit. So now the pod status controller, what is the executable pods and the cluster scale events, and the necessary information into new pod status custom resource. So we just created it. So it's worth noting in this diagram that some of the executable pods may be deleted. And so they won't exist in SCD anymore, which is why this pod status custom resource is necessary. This is indicated in a dash to circle. So the Spark app controller will now watch pod status objects instead of managing aggregated status. So the main controller is still watching the single object. So this is what pod status custom resource actually looks like. We follow the common pattern of the spec and the status field here. And the spec is a desired state, and the status is the realized state. And in this case, it's a variant of that. Spec field specifies the target pod to track in the object. And the controller replicates the necessary information into the status field, meaning that the status field itself is the realized state. So we can selectively activate this functionality for a single pod by creating one pod status object like this. So optionally, we can have a separate web or controller on top of this. And they automatically create such pod status objects for each pod that matches certain criteria. Like in Spark, there is a requirement that all executor pods have a label. Spark role equals executable. So in this example, in the example configuration in this slide, if a pod has a Spark role equals executor, a corresponding pod status object will be automatically created. And then the controller will be watching that and keep updating relevant information from event and the pod into those pod status. So now it looks like we are copying field values around only from one object type to another. So we might as well make it more generic and describe it by a declarative field copying specs. So this is an alternative design of the pod status controller. So for this controller, we have a configuration like this and in the left-hand side, we specify the source and the destination field declaratively and the type, of course. And the controller will watch the source objects and keep replicating specified fields. Resulting destination object in this case is shown in the right-hand side and the owner reference field and the container status field is replicated. And this is another example for the cluster all scalar events. On top of the previous one, we also let the controller watch events with the configuration watch and replicate the latest field values and replicate the latest field values from the cluster all scalar event to the destination status object. So our implementation of this approach is powered by an unstructured type in a client goal. And it's actually quite lightweight, let's say like several hundreds of line of code, goal code, and it might be even more straightforward in Kubernetes clients in dynamic languages. And downside of this approach is that everything becomes a goal map in the implementation and we lose static typing. You might want to do more than just copying, for example, define more complicated trigger rule and have more volatile actions triggered by those rules, not just copying, et cetera, and it's totally possible and not that hard to do. But we would recommend to keep the configuration like this really simple as you don't want to, for example, unit test your more configuration down the road. One realization from this experimental implementation is that this way we can really make sure that the additional control, additional controller, that power status controller only does copying and nothing else. So it can be good design pattern or discipline to follow in certain use cases. So in other words, you can make sure that all the business logic reside in the main controller and not randomly spread across multiple places. But in our use cases, we actually wanted to encapsulate auto-scaler related logic into additional controller, the power status controller, so we didn't choose this route. So as a recap, now that we've walked through providing auto-scaling information and the various failures scenarios by fixing the information flow to Spark, we can look back at the Spark app custom resource and know how it was produced. Regarding the persistence of the Spark app information for the purpose of postage when it's inspection, we've went through various storage options with the conclusion that the custom resource is the effective candidate for our multi-cloud environment. The building upon what we talked about in this presentation, similar features can be implemented in other distributed execution systems such as PyTorch or TensorFlow that rely on Kubernetes as a resource manager. In that case, the power status controller will be reusable as is and the main job controller can handle job type specific statistics apart from that. While the auto-scaler detection behavior via event object is specific to the standard class auto-scaler, we can actually switch to alternative auto-scalers such as Carpenter without affecting the main job controller now by only switching detection logic inside the power status controller. This is especially useful if we have multiple job types and multiple main job controllers. So feel free to check out the rest of our talks from Bloomberg and also check out the links below to learn more about us. And by the way, we are hiring. Thank you for that. So do we have any questions for these? Yeah, there's a question, of course. Hi, thank you for the presentation. How do you manage the data? I mean, in a multi-cloud environment, how do you manage the different aspect of the data? Isolation, security? Do you share the data? How do you run Spark in multi-cloud with different type of data, different type of teams? Got it. So is general data management, do you guys hear me? Sorry, general data management questions? Yeah, the context of the talk is mostly about the compute but in terms of data, it varies between the restrictions and the security concerns of the application. Like if it's something where we're taking data that's only within a spurt and like say network we're constricting it to would have like calico or cilium policies locking everything down in terms of moving data in and out, right? And so it varies based on the implementation. So the platforms ourselves are composable so they vary with security concerns. But if you run the same code in different places, you have to share the data. You have to have only one place or different place. So when you see, when you say the, okay. The pods themselves are on the user's namespace, right? And so the data itself is either co-located to the namespace or you have specific verification on what data source you're communicating to. So the pods themselves are always the user namespace. The admin controllers are just the control plane, right? The pods are not in the control plane. It's only in the user's namespace for the Spark app. Does that answer the question? Hey, so I, yeah, there was one behind. Hey, I have two questions. The first one is you had on-prem data center footprint. You had AWS and GCPs. I was curious if all the cluster compute nodes on all these three footprints across cloud providers are they homogeneous or do you have heterogeneous computer environments? So homogeneous in terms of the hardware types like CPU, GPU, that kind of stuff. Yeah, yeah. Got it, got it. In terms of Spark, there are Nvidia plugins all you to run GPU. And so it's based on once again, the configuration of the cluster. The infrastructure itself is portable. So functionally, yes, it could be, yeah. Thanks. And the second question is I'm curious to learn if you built cluster order scalar before Carpenter was announced or what were the reasons for building it in-house versus using another open source project? Do you mean? Yeah, so we were using the standard cluster auto scalar from Kubernetes. The intention, as Aki mentioned, was that because we're in a multi-cloud environment and we need portable infrastructure, the built-in standard cluster auto scaler, something that would run in EKS, AKS and on bare metal because it's all Kubernetes abstraction. So we just use the base cluster auto scaler so we didn't build our own. But the idea is because we have a separation of concerns between the job controllers that reconcile however that job operates and the auto scaling information, the auto scaler, the auto scaling controller could have either the cluster auto scaler events or it could use Carpenter's way of doing it or not. Got it, yeah. Thank you. And the last question. So I guess, like because you're using STD, like it's kind of a database. I'm curious if you like hit any scaling or performance constraints. And if you did any testing on that sort of thing, seems kind of like pod statuses are just created and imagine it'll pile up over time. So with regards to kind of considerations, so traditionally it's kind of about like what is your Spark app clusters like scale to, right? Like we had some battle tests, I think 400 to 500 kind of executor clusters and that's kind of what I've seen mostly like sub 1000. I don't really know too many have more than a thousand kind of executor pods. So obviously, again, if you're talking about very large, it's also about numbers about how much you can store in STD. We felt that it kind of fit within our models of 400, 500 pod clusters. We can talk offline in terms of like other kind of maybe strategy tests in terms of actual sizing itself, but it seems to fit for most applications that we're running in prod. Thank you.