G usually a question the number one. To ask the First Minister what her responses to the United Kingdom Government scrapping the EU settled status fee. I'm pleased that the Prime Minister has finally seen sense and listened to the deluge of calls to scrap the unfair settled status fee, including the call of this Parliament, with the exception of the Scottish Conservatives. However, dropping the fee does not change the fact that the UK Government is still making EU citizens apply to retain their current rights. Therefore, to assist EU citizens in our communities in order to apply for settled status, the Scottish Government's advice service, delivered in partnership with Citizens Advice Scotland, will help to ensure that EU citizens feel welcome, supported and valued. What's more, the Prime Minister's approach to migration makes it all the more clear why it's time for this Parliament to have powers over immigration to determine a tailor-made policy. Linda Fabiani, I thank the minister for that answer. May I advise the minister of a constituent who has lived in Scotland for decades but who has not renewed his EU passport? Now he has to do so at a cost to prove to the UK Home Office that he is in fact an EU national. In order to do that, he has to prove to his EU nation that he is not a British citizen. That is also at a cost by obtaining a confirmation of non-acquisition of British citizenship from the UK Home Office. Does the minister agree that this anomalous situation is insulting and concerning to someone who has lived and worked here for more than 30 years, raised his family, has a national insurance number and has paid taxes? Surely this man and others like him, no doubt, already has a proven right to continue to live in and contribute to Scotland. It is, after all, his home. Thank you, Linda Fabiani, for raising her specific case. I share the concerns about it being a situation that is insulting and of concern. I am sympathetic to the many families and individuals who have difficulties navigating the complex and increasingly restrictive UK immigration rules, as illustrated in the case raised by Linda Fabiani. It is right that EU citizens who have built their lives here and chosen to make Scotland their home should have all their rights protected. If the UK Government persists in its ambition to remove Scotland from the EU against the will of the Scottish people, it is vital that those EU citizens who have chosen to make their home here in Scotland are provided with the documentation that they need through as simple a process as possible in order to evidence their right to continue to live here as they do now. I would be happy to raise such cases, including the case raised by Linda Fabiani today, with the UK immigration minister, with the consent of the individuals concerned. I want to reassure all MSPs that we in the Scottish Government are pressing the UK Government for a fair and managed immigration system that recognises individual circumstances and provides a welcoming environment for new Scots and their families. I remind everyone, I mean everyone, that short questions and crisp answers would be very helpful. In other words, that is what I want. I welcome the scrapping of the fee for EU residents. Will the minister join me in calling for the UK Government to take the next step and end its hostile environment policy and its anti-immigration policies and rhetoric? Absolutly. I welcome the sentiment in Neil Findlay's question and wish we'd seen such a coherent position from Labour in the House of Commons earlier this week on the UK immigration bill. The hostile environment policy from the UK Government has been discredited in both practice and principle. It should be noted that, since June 2016, the UK Government should have provided assurances to EU citizens separate to any withdrawal agreement, and it could have done that in every month up until this point, and it didn't. It should think about that carefully. To ask the Scottish Government what assessment has made of the impact of Brexit on the life sciences sector. The Scottish Government has engaged widely across the breadth of the life science sector and commissioned Ernst and Young to undertake a sectoral impact analysis and Brexit readiness assessment. That assessment has been shared with the UK Government as clear evidence of the negative impact that Brexit will have across sectors in Scotland. It also accords with evidence from industry and unions that suggest that Brexit will be damaging to the sector in terms of tariff and non-tariff barriers, supply chain resilience, legal and regulatory compliance, free movement of people, loss of EU funding and disinvestment from foreign investors. David Stewart will be well aware that the Highlands and Islands have over 80 life sciences companies employing more than 1,800 people. All those companies rely on academic talent from the other 27 nations in the EU. Does the minister share my view that Brexit is a clear and present danger to the future viability of the life sciences sector in the Highlands and Islands? David Stewart makes an important point about the detriment of the removal of free movement as a whole if the UK leaves the EU and Brexit takes place. In a constructive manner and in good faith, I encourage David Stewart and colleagues to continue to engage with us as the Scottish Government and work together to push for flexibility in the UK immigration system by way of a Scottish visa so that we can support key sectors in our economy, including the life sciences sector. Innovation and research and development is key to the success of the life sciences sector. As the minister knows, a significant number of EU citizens work in R&D, what assistance therefore can the Scottish Government provide to ensure that competitiveness in R&D is not seriously compromised by Brexit? There are two aspects in which we can all work together proactively in order to continue to support R&D in the life sciences sector and elsewhere in terms of the challenges faced by Brexit. One is to continue to oppose the removal of free movement and to oppose the restrictions that are being set out in the UK Government's white paper on immigration, and two, we should all collectively be working together to urge the UK Government to continue to participate as a third partner country in the event of Brexit in EU programmes like Horizon 2020 and, indeed, at the Joint Ministerial Council on Europe in London on Monday. That is exactly what I pressed on behalf of Scotland. To ask the Scottish Government whether the Minister for Parliamentary Business will provide an update regarding the lodging of the legislative consent motion for the healthcare international arrangements bill. The legislative consent motion in question was lodged on Monday, 14 January. It was moved by the Cabinet Secretary for Health and Sport on Wednesday, 16 January, when it was agreed unanimously by the Parliament. Alison Harris I thank the minister for that answer, but surely finally giving consent after months of threats not to just shows up the Scottish Government's empty bluster and suggests that the right thing across all Brexit issues is to work consistently and dump the grand standing. The Scottish Government has shown itself through its approach to Brexit roll-ins legislation to be reasonable and pragmatic, but until and unless we can be assured that the decisions of the Parliament will be respected by the UK Government, we will not lodge legislative consent motions on Brexit-related provisions except in the most exceptional of circumstances. We will continue to contribute full weight to committee and parliamentary consideration and ensure that the Parliament is able to express its views on Brexit-related provisions in UK bills, but overarching all of that, Presiding Officer, is a simple truth. Our role as the Scottish Government is to stand up for the interests of Scotland, something that the Conservatives might want to try sometime. Question 4, Alexander Burnett. Thank you, Deputy Presiding Officer. Can I ask the Scottish Government what discussions ministers have had with civil servants regarding the EU withdrawal legal continuity Scotland bill since the Supreme Court's judgment on it? Presiding Officer, Scottish ministers regularly discuss matters within their responsibilities with officials. In addition, the Cabinet Secretary for Government, Business and Constitution has held a meeting with representatives of the parties in the Scottish Parliament to discuss the bill and the options for proceeding with it following the Supreme Court judgment. I thank the minister for that answer. The original largely unlawful bill was pushed through on emergency procedures, giving MSPs hardly any time to scrutinise important legislation compared to the EU withdrawal act, which had over 11 months. Can I ask the minister whether he will rule out using such emergency procedures in this way again? Presiding Officer, as I have indicated, the decision on how to proceed is the subject of on-going discussions between Mr Russell and the other parties of this Parliament. A meeting took place shortly before Christmas, I think that Mr Tomkins was there, and a further one is due later this week, I believe. An announcement on how we proceed will self-evidently be informed by those discussions, but an announcement will be made in the coming weeks. However, just to be clear, no matter how many times the Conservatives seek to claim otherwise, the Scottish Government's position on the continuity bill was vindicated by the Supreme Court. No ifs, no bats, no maybes. Willie Rennie Does the minister not think that the answer is a proper dispute resolution procedure on areas of common interest? Rather than a Scottish veto or a Westminster veto, surely the acceptance of common endeavour in areas of common interest is the way ahead? Presiding Officer, as I indicated earlier, this is a matter of discussion between the parties of the Parliament where each can genuinely input to the process. I would encourage Mr Rennie to bring forward those points, if he so wishes, at the next meeting, which I believe takes place tomorrow. To ask the Scottish Government how it will ensure that EU citizens living in Scotland can maintain democratic participation. EU citizens will retain their right to vote and stand in Scottish Parliament and local government elections after Brexit. We have previously set out our intent to protect EU citizens voting and candidacy rights in Scottish Parliament and local government elections. The programme for government includes a commitment to bring forward an electoral franchise bill that will extend the franchise to include citizens of all nationalities legally resident in Scotland. Rona Mackay I thank the minister for that answer. Several EU citizens in my constituency have been in touch with my office concerned about the UK Government's EU settlement scheme. Does the minister agree with me that the system is currently not fit for purpose, apart from the gross unfairness of it, as the UK Government's own settled status scheme app is only available to Android phone users and not to those using other mobile devices? I thank Rona Mackay for that supplementary. As I laid out in our programme for government for this year, we have stated already committed to bring forward a franchise bill. We have opposed the settled status fee charge, and we have set up our advice service working with Citizens Advice Scotland in order to advise the EU citizens in our communities who make such a huge contribution in order to help them through the settled status scheme. We are doing that going over and above anything that the UK Government is doing because, as was highlighted in Rona Mackay's question, there are serious misgivings around the technical and practical delivery of the UK Government's mechanisms for bringing EU citizens through the settled status scheme, but also how much their efforts are going in terms of reaching out into communities and assisting those who have accessibility issues or are not comfortable using digital technology. Our advice service will provide face-to-face advice, telephone advice and online advice, and we hope that it will make an important difference. To ask the Scottish Government whether it will provide an update on the preparations being made by it and its agencies for a potential no-deal Brexit. The Scottish Government remains committed to keeping Parliament informed of our contingency planning for the prospect of exiting the European Union without an agreement. We have repeatedly made clear that the UK Government can and should take immediate steps to exclude the possibility of a no-deal outcome. Until that happens, as a responsible Government, we will continue to intensify our preparations. That includes work at the directorate level on identifying the risks and potential impacts, as well as mitigating actions across a wide range of issues. The Scottish Government Resilience Committee leads on our preparations for no-deal, officials' key agency leads, ministers and a representative from COSLA meet each week to assess progress. I thank the minister for that answer. He will be aware that no-deal cannot be removed other than by our deal or by revoking article 50. Given the vote last night in the UK Parliament, does the minister agree that no-deal is now significantly more likely? Importantly, in this context, and following Mike Russell's statement in Parliament on 18 December, will the Scottish Government now place in the public domain technical notices covering devolved areas similar to the 105 notices published by the UK Government? By aligning herself with the hardline brexitier wing of our party last night, the Prime Minister has brought a no-deal scenario even closer, as Andy Wightman has alluded to. It is imperative that, while continuing to press the UK Government to see sense and step back from the brink, we continue to plan for the worst. It is entirely appropriate and necessary that we do so. Mr Russell addressed the issue of a no-deal planning in a statement a few weeks ago. The Scottish Government is happy to consider how we continue to up members, and I will take the point that Mr Wightman has raised a way to discuss it with Mr Russell. There is a meeting chaired by the Prime Minister in London today to discuss preparations for a no-deal Brexit. The First Minister of Wales and Scotland were both invited to attend that meeting. As I understand it, the First Minister of Wales is there, but the First Minister of Scotland is not. Why not? I find it staggering, Presiding Officer, that, with the horrendous situation that we find ourselves in around Brexit, the Conservatives have taken an opportunity to simply make a point like that. Mr Russell, as Mr Tomkins well knows from the exchanges in this chamber, is a perfectly adequate, far more than adequate, representative for the Scottish Government at that meeting. To ask the Scottish Government whether it will provide an update on its discussions with the UK Government regarding Brexit. Last week, the First Minister and the Cabinet Secretary for Government, Business and Constitutional Relations met with the Prime Minister and the Chancellor of the Duchy of Lancaster to discuss Brexit. The Prime Minister has said that she wants to give the devolved administrations an enhanced role in the next phase of the Brexit process. However, the UK Government continues to ignore the views of the Scottish Government. The votes of this Parliament are indeed the 62 per cent vote of the people of Scotland to remain in the EU. With the clock ticking down to exit day, the Prime Minister must start listening to people outside the Conservative Party and the European Union Party, abandon her damaging red lines, seek an extension to article 50 process and immediately rule out a no-deal outcome. I hear Mr Tomkins chunking from a sedentary position. Just a small point—this Government is getting on with the day job alongside Brexit, unlike the UK Government's. I thank the ministers for that response. Last week, the Prime Minister showed she was running scared of the verdict of the Scottish people. Is it not the case that the mandate for an alternative path to Scotland is cast iron? A majority of MSPs and Scottish MPs returned at the last two general elections support holding an independence referendum—an option that is endorsed by this Parliament with the manifesto on which this Government is elected. Does the Government agree that the people of Scotland should be in charge of their own future and not live at the whim and dictate of a hardline inflexible out-of-touch Tory Government? As Fulton MacGregor says, this Scottish Government was elected on a clear mandate that this Parliament should have the right to hold another referendum if there is a significant and material change in the circumstances that prevailed in 2014, such as Scotland being taken out of the EU against her will. This Parliament voted on 28 March 2017 in support of an independence referendum in light of Brexit. Brexit has been the consistent position of the Scottish Government that we will set out our views on the next steps for a future referendum on independence when there is clarity about the outcome of the Brexit negotiations. Sadly, as we all know, there remains no clarity over the outcome of the Brexit negotiations, even as time runs out before March 2019. However, what is clear is that Brexit changes everything utterly. Thank you very much to ask the Scottish Government when it next plans to meet Welsh Government ministers and what matters will be discussed. The Scottish Government routinely engages with counterparts in the Welsh Government on a range of businesses at both official and ministerial levels. Last week, the First Minister and Cabinet Secretary for Government Business and Constitutional Relations met with Mark Drakeford to discuss Brexit matters. On 28 January, the Cabinet Secretary for Government Business and Constitutional Relations spoke to Dewey James at the time concerning electoral law issues. Earlier this week, the Minister for Europe, Migration and International Development met with the Welsh Government Council General. Tomorrow, the Lord Advocate and I will both meet with Mr Miles ahead of the next meeting of the ministerial forum on EU negotiations, which takes place in Edinburgh. I am grateful for that comprehensive reply. The minister will be aware that Welsh Cabinet ministers recently provided the National Assembly of Wales with comprehensive analysis of the devastating consequences of a no-deal Brexit for the economy and the people of Wales. Further to his reply to Mr Whiteman's question, is that an example that the Scottish ministers may be minded to follow? We have much in common with Welsh colleagues. Today, I understand that the Welsh Assembly is uniting behind motion considering, in detail, the impact of a no-deal Brexit—indeed, Brexit—and the catastrophe that it would be for Wales. We continue to share much common ground on that. With regard to the point that the member makes, as I indicated to Mr Whiteman, we are happy to take that request away and consider it. However, I think that it is important that we recognise that, as a Parliament, most of us have the catastrophe that a no-deal Brexit would be for Scotland. I can take a very brief supplementary question from Ms Gilruth, so within time. To ask the Scottish Government what the implications of Brexit are for Scotland, which voted 62 per cent to remain in the EU. The implications for Scotland of Brexit are extremely alarming, as the Government has detailed and the Parliament understands. All Brexit outcomes will be bad for Scotland, but the Prime Minister is now running down the clock to the most damaging of exits. The DUP voted to reopen the withdrawal agreement and amend the Northern Irish backstop, and the EU has categorically stated that the withdrawal agreement is not open for renegotiation. Does the minister agree that pursuing the impossible is simply running down the clock and risks a no-deal at the behest of Tory Brexit? That is fine. That is briefing up. No, no, no. Minister, please. Presiding Officer, I agree with Ms Gilruth. Last night, a majority of MPs for Scotland voted to extend article 50, rule out no-deal and to respect the overwhelming vote in Scotland to remain in the EU. The UK Government decided once again to ignore Scotland's democratic voice. The Scottish Government, however, will continue to do all that we can to protect Scotland's interests, and we urge the Prime Minister to extend the article 50 process to avoid the disaster of no-deal. Thank you very much. We now move on to the next set of questions in culture, tourism and external affairs. To ask the Scottish Government what impact UK Government immigration policy is having on Scotland's working population. Migration is vital to Scotland's population growth. Each year, for the next 25 years, all of Scotland's population growth is projected to come from migration. Therefore, the UK Government's commitment to cut net migration to the tens of thousands could seriously harm our economy. If implemented, Scotland's working age population is projected to decline by 4.5 per cent, a reduction of 150,000 people between 2016 and 2041. A Brexit-driven reduction in migration would see GDP in Scotland drop by an estimated 6.2 per cent by 2040, equivalent to a fall of almost £6.8 billion a year in GDP and £2 billion in government revenue. That is an unacceptable price for Scotland to pay and is why we need a migration policy tailored to Scotland's needs with more powers for the Scottish Parliament. Maureen Watt. I thank the minister for that answer. A NHS Grampian survey recently presented to Aberdeen City Health and Social Care Partnerships states that the risks not only to staffing but also medical supplies, accessing treatment, regulations and cross-border issues are high risk due to Brexit. That is compounded by the Home Office's stubborn refusal to engage with MSPs under our offices. Does the minister agree with me that the Home Office should stop treating MSPs and members of other devolved parliaments, such as second-class representatives, start engaging with us to resolve urgent immigration cases as soon as possible? Is it not time that the Westminster Government stopped using EU nationals and others as pawns in this game? Maureen Watt, I was a good point, but it was not brief enough. I am getting techy. Brexit could indeed have significant impacts on health and social care in Scotland with potentially serious consequences for the recruitment and retention of health and social care workers in Scotland. As Maureen Watt has raised, it also raises concerns in areas such as medicines, medical devices and clinical trials, access to future EU funding and the rights of Scottish citizens to access state-provided healthcare across the EU. On the point about correspondence, MSPs are understandably concerned about those issues and the impact on their constituents. I have met with the UK immigration minister Carling notes several times to highlight the Scottish Government's concerns and reiterated our willingness to work collaboratively to safeguard the interests of EU citizens in Scotland. However, despite committing to meaningful engagement, the Home Office continues to refuse to deal substantially with the concerns of Scottish ministers and MSPs on immigration cases. That is completely unacceptable. The UK Government has repeatedly committed publicly and privately to the full involvement of the devolved Administrations by so far, unfortunately, I have been frustrated by the quality of that engagement. Nevertheless, the Government is clear that it will do all it can to support EU citizens through this difficult time. I am happy to receive correspondence from MSPs to write minister to minister to the UK Government. I have done that for MSPs across the chamber. Willie Rennie briefly. I have concerns about the immigration policy, too, with sectors such as the fruit and veg farms in my constituency. The universities, the tourism industry are already being impacted by a drop in the number of workers in Scotland. I do not support the devolution of immigration policy as those problems are not unique to Scotland. What actual and practical steps has the minister taken to influence UK Government policy? Just under a year ago, we presented a substantial paper to the Scottish Parliament on how we take practical steps and seek to influence UK policy. We have written repeatedly to UK Government ministers about our concerns with the MAC report and the white paper. We have done that in person. We are going to be submitting to the consultation from the Migration Advisory Committee on the Shorties Occupation List. Across every area of Government that we can, we are working to influence UK Government policymaking and to work with stakeholders across business who are deeply concerned about what is in the white paper, particularly the proposed salary threshold. I say to Willie Rennie and I say this in good faith that we are proposing in terms of flexibility in the UK system is taking a solution-focused approach in response to what is being proposed in the UK Government's white paper. He said that, in his constituency—this is the case for many across Scotland—that key sectors are going to be affected by what is being proposed in the UK Government's white paper. I say to Willie Rennie, come and meet with me, engage with us as a Government and, together, let's be solution-focused for the benefit of your constituents and the common good of Scotland. To ask the Scottish Government what it's doing to maximise the benefits of Robert Burns' legacy in the south Scotland region. The Scottish Government wants to maximise the social, cultural and economic benefits of Robert Burns' rich legacy for the whole of Scotland and, through events Scotland, we provided 30,000 to the big burns supper in Dumfries as part of Scotland's winter festivals. As part of this work, we have commissioned the Centre for Robert Burns studies at the University of Glasgow to produce a report that will help us to understand the impact of burns on the Scottish economy and associated prospectus for driving inclusive growth. We expect that the report's findings will be helpful in ensuring that the enduring legacy of burns can help to accrue on-going benefit for Ayrshire, Dumfries, the wider south of Scotland region and indeed the country as a whole. I thank the minister for that answer, and I would highly recommend the big burns supper to all members who run until Sunday. The minister will be familiar with the family home that was built by Burns in 1788. The future of Ellisland is very much at the crossroads, and the trustees have developed proposals not only to save Ellisland but to transform it as a major attraction. Will the minister take the opportunity to find out more about the proposals, which are exciting plans, and hopefully meet with the trustees to discuss how the Scottish Government may be able to assist in delivering a viable long-term future for Ellisland farm? I understand that representatives of Ellisland have recently met with Historic Environment Scotland officials to discuss the trust plans for developing Ellisland, and if they have not done so already, I would encourage them to continue their dialogue with Historic Environment Scotland to ensure that the vital part of Scotland's history and heritage is preserved for future generations. I want to take supplementaries, but they must be brief. I have Finlay Carson and I take Joan McAlpine. As the Presiding Officer will know, the south-west 300 is a stunning 300-mile drive with awesome coastline hills, glens, forests and locks and abundant history, which much rivals have publicised in North Coast 500. The Burns country run is 160 miles and dedicated to showcasing the many locations that are associated with Burns. Those routes have huge potential— No, that is not brief. Get your questioning. What will the Scottish Government do to quantify the potential and what support will it give to the Burns route? The Scottish Government is committed to investment and tourism in the south of Scotland. For example, there has been a half million visit Scotland marketing campaign, half a million of south of Scotland capital funding and 2.5 million for the development of facility in Gaelishields, as well as investment in the David Livingston centre in Blantair. I thank Finlay Carson for raising the point on the proposed route and would be happy to receive more details on that if he wishes to provide that in writing. The cabinet secretary, if rather the minister, apologies. Join me in welcoming the purchase of the historic globe in Dumfries, one of the bar's favourite houses, by Professor David Thompson and his wife Teresa Churchill on Annandale distillery. I am delighted to report that the investment in the historic inn is already apparent during the Big Burns upper. I invite the minister and the cabinet secretary to take the opportunity to visit again in the near future. I thank John McAlpine for that question and indeed welcome the investment that has been made and congratulate all involved. I would be happy to receive invitation as part of our planning for the winter festivals next year and thank her for that offer. To ask the Scottish Government for what reason its external affairs budget increased by 52 per cent from 2017-18 to 2019-20. Between 2017-18 and 2018-19, the external affairs budget increased by £1.5 million to support our expanded external presence in Brussels, Canada and Paris. All the increase of £6.7 million in the 2019-20 draft budget is due entirely to a change in the way that running costs, including staffing costs, are presented across the Scottish Government. Those were previously presented separately but are now included within ministerial portfolios at the request of this Parliament and its finance committee for transparency. There is therefore no net increase in this year's 2019-20 resource available for spending for external affairs. Reserved matters are excluded from the devolved competence of the Scottish ministers. This reservation is particularly important in the case of international relations. So ruled the Supreme Court in its unanimous judgment a month ago in the continuity bill case. What legal advice has the cabinet secretary taken to ensure that this increased budget is lawful? The Supreme Court judgment does not affect the Scottish Government's ability or determination to prepare for EU exit or continue our international work. Indeed, the UK Government and a whole variety of different agreements with us understand and support our international work. It is extremely small-minded of the Conservatives to think about limiting our ability to help our universities, our tourism sector and indeed our trade and economic activity by saying that everything has to be done within the confines of Scotland. I think that it is about time that we raise our horizons and certainly this Government has done and will continue to do and in relation to our activity. He needs to be quite clear that the Supreme Court judgment does not affect our ability to carry out our duties and our functions as good internationalists and good global citizens. To ask the Scottish Government when it last met Airbnb and what issues were discussed. Scottish Government officials last met Airbnb on 29 November 2018. The meeting discussed the regulation of short-term lets, including the stage 2 amendment to the planning bill regarding short-term holiday lets and noted the meeting redacted to remove personal details together with the notes of other meetings with Airbnb were published recently in response to a request under freedom of information and is available on the Scottish Government website. I thank the cabinet secretary for that response. Recent evidence released by the Scottish Government revealed that there were 2,200 Airbnb listings in Glasgow in July 2017, a 45 per cent increase in the last year. Shelter Scotland have expressed a concern that short-term lets may be exacerbating the existing housing crisis. Does the Scottish Government share that concern? What action is it taking to ensure that Glasgow's tourism industry is both sustainable and delivers for local communities? I think that the member makes a very important point, sustainability both for tourism but also for the housing market is really important. That is why in relation to short-term lets, I know that there is a debate currently happening in terms of the planning bill and I understand in relation to housing legislation as well, the short-terms letting working group is engaging with local authorities, I am sure that they are doing so with Glasgow but will make sure that Glasgow's situation is brought to their attention and engaging. I understand that Glasgow City Council regulations introduced in March 2017 have also had an impact, so their current powers that local authorities already have but in terms of looking at the overall picture and indeed the housing market there has to be an integrated look to this and I think that it is something that all of us will take a keen interest in. Thank you, Presiding Officer. To ask the Scottish Government what it is doing to support Scotland's festivals in 2019-20. Scotland is the proud home of a fusion of cultures, arts and creativity in reaching us all with festivals and events. Today, I am sure that everybody would want to join me in congratulating the Imaginate Festival on its 30th anniversary. Visit Scotland will have invested over £3 million across 109 cultural events and festivals this financial year, while Creative Scotland provided £485,000 to 18 festivals around Scotland in 2018. The Scottish Government's Expo fund has supported the Edinburgh festivals with over £21 million since 2008 and the current applications are being considered between 2018 and 2023. £5 million through the platform for excellence programme is supporting strategic projects across the internationally-claimed Edinburgh festivals as part of the Edinburgh and City regional deal. Donald Shaw, a leading figure of Celtic Connections, one of Scotland's foremost music festivals, recently warned that the added pier orchestra required to book QK musicians in EU member states following Brexit will put Scottish musicians at disadvantage. He also predicted new problems for musicians coming to Scotland. Does the Scottish Government share my concern over the detrimental impact of Brexit and the Scottish music sector? Of course, Celtic Connections is a hugely international festival. It, for the first time, has received £100,000 of festivals' expo funding. It truly is international. It is welcoming to musicians. They want to come here, but the UK's immigration white paper would drive a coach and horses through our music industry sector unless there can be changes made to ensure that there is no bureaucracy in terms of the visas. There is an issue about festivals wanting to book our musicians as well. We want to make sure that we remain a welcoming, inclusive country, and that extends to everybody, particularly including the many musicians that come. When we have senior leading figures in our cultural sector warning of the consequences of Brexit, which would be absolutely compounded by a no-deal Brexit that was accelerated by the farce that was Westminster's activities last night, we have to take them very seriously indeed. The cabinet secretary will be aware of recent reports of exploitative working practices within some major festivals. Will the cabinet secretary commit to frank discussions with festival organisers and relevant companies to ensure that that is not tolerated within the sector? The member makes a very important point. I have already done so with some festivals. I think that it is important that anybody in receipt of public money embraces the fair work agenda. I think that that is an improvement that everybody in the sector would want to see happening. We just have to make sure, in terms of their abilities, that they carry out what I think is a commitment to try and address this issue. Question 7, John Scott. Thank you, Presiding Officer. To ask the Scottish Government what it is doing to support tourism, including business tourism, between Ireland and Scotland. Cabinet Secretary. Our national tourism agency visits Scotland actively promotes Scotland to visitors from Ireland. Our sixth largest tourism market, it undertakes a large range of marketing activities. Business tourism is one of the many reasons Irish visitors come to Scotland with a majority coming to visit family members. Last week in Dublin, I met with business and university interests to discuss Scottish and Irish connections and collaborative working and spoke to several tourism businesses that are interested in Scottish investment. If it was left to Mr Scott's colleague, Adam Tomkins, none of that would be happening. Thank you, cabinet secretary, for your answer. She will be aware that the First Minister announced that the Scottish Chamber of Commerce dinner in December will give £2 million of new funding over the next three years to Scottish chambers to promote business tourism and business development through local Scottish chambers of commerce. Can the cabinet secretary tell Parliament and my constituents at the Ayrshire Chamber of Commerce when that money might be available, as they are keen to further strengthen and grow already-established trade and particularly tourism links with Ireland? Cabinet secretary. I am delighted that the Scottish chambers of commerce are working internally, but also with the Scottish Government to encourage business tourism. Their links in terms of key sectors in the collaboration would be a great advantage in attracting business conferences and other activity here. In relation to spending, I would point out that we are facing a budget vote shortly. If the member is wanting us to invest in expanded investment in relation to chambers of commerce investment for exports or, indeed, any of the export area, helping to support the budget would be one way of doing it. Question 8, Sandra White. Thank you, Presiding Officer. Justice Scottish Government, what analysis has the culture secretary carried out regarding the impact of Brexit on the arts and creative sector in Scotland? Cabinet secretary. Leaving the EU would have significant wide-ranging negative impacts on Scotland's culture and creative sectors, including access to funding, trade, working internationally, access to skills and talent. Those impacts have been highlighted consistently in analysis undertaken by sectoral organisations and by the Scottish Government, including a recent Ernst and Young report. That is built on previous analysis, undertaken by Scottish public bodies in the sector. Indeed, in looking at the extension on the extent of EU funding received by cultural sector organisations, the work found that EU funding of at least £59 million was received by Scottish cultural organisations over the period 2007-16, demonstrating the significant importance of EU funding to those sectors. Sandra White, please. Thank you very much. I thank you for that answer. In Saturday, Cabinet secretary, I attended Celtic Connections and organised by Bemis, celebrating diversity and culture throughout Europe and throughout Scotland. Your answer obviously implicates that we could lose those types of cultural and international events. Will the cabinet secretary agree with me that that would be a great loss if it disappeared both culturally and financially to Glasgow and Scotland? The Bemis connections with Celtic Connections and their association has been a great success in recent years. A lot of that funding comes from the Scottish Government funding from our winter festivals, but there are events at Celtic Connections, which Donald Shore, the artistic director, has made clear. We would not have taken place had there not been collaboration in terms of EU funding opportunities. Therefore, it matters, and we have to remain an outward looking country. We have to welcome people, musicians, artists and others, to celebrate—and what better way to celebrate—the wonderful Celtic Connections, which has a period to go. I encourage anybody who has not been to go and visit and take part and buy tickets and support our artists. Thank you. That concludes portfolio questions. I apologise to Willie Coffey and Mark Griffin for failing to reach them. Try, though I did. I now must move on to the next item of business. I give members a moment to take their places.


All right, good morning. My name's David Moore. Very happy to be here. I just want to quickly say thank you for being here. I really appreciate it. And welcome to the Embedded Linux conference. And I want to say thank you very much for the Linux Foundation for inviting me to speak today. Just a few quick details about my background. I've been a professional software engineer since 1994. Had the opportunity to work with some great companies, starting with Next Software, worked for Steve Jobs back in the 90s doing web objects consulting, one of the first web frameworks. Became an Apple employee. I went on to work with some really excellent companies and really great people in engineering, sales, consulting, and business development roles. But I was having a great time at that, you know, great career. But I decided to kind of switch into consulting. And so there I am consulting in Indonesia. I was able to do that for a little while.  And then I actually spent a couple of years as a semi-professional opera tenor. I trained into opera. I took a lot of voice lessons. And I reached essentially a regional or semi-professional level. So it was pretty exciting. Kind of got kicked out of opera, though. It wasn't loud enough. You have to be very loud. And at the same time, I was seeing some pretty amazing things going on in the security world. Even starting with Stuxnet and a lot of the big breaches that happened recently, or 2012, 2013. So I really piqued my interest in. I wanted to get back into tech. So I decided to do that and focus specifically on security, really offensive security. And to do that, I trained into it. I used different vulnerable web apps and things like that, just did tons of reading. And then I started doing bug bounty programs. And these are where you can attack a website. And the company gives you permission. And if you find something and report it responsibly, they either pay you, usually a little bit of money. And they certainly give you public recognition, which I think is more valuable. And it's also a very great way to learn hacking. You're going after live targets that are secured by professional security engineers. So it's much harder than the practice things we do. And so that was great. But I pivoted into fuzzing and memory corruption as well. That's kind of what we're going to talk about mostly today. And I found a few things in Ruby and PHP and other stuff. One quick story, I found a bug in the kernel. And it was in EXT4. And I wrote a script to highlight duplicate lines and source code. Anytime there's the same line twice, flag that. And I ran it over tens, maybe hundreds of lines of open source code. And it was quite time consuming because usually that's a legitimate thing, or often it is. But sometimes it's not. I'm really looking for copy and paste errors. So I found one when they were checking the mutex on iNode1 twice. And there was also an iNode2. And so that was a bug. It was like a copy and paste error that the programmer forgot to change the one to a two in that second line. So I did. I sent in a patch. It got accepted. So I have a one byte fix in the kernel. And then last summer, I really got into fuzz testing. I think it's a really powerful thing to do. So I started a company. And we do fuzz testing at scale in the cloud, as well as training. OK, so today we're going to talk about memory corruption, handling memory corruption bugs that fuzzers find. When I first started fuzzing, I wondered to myself, what am I going to do when I find a crash? Like, I can do this fuzzing thing, and it's good at finding crashes. But I really wasn't even sure how to deal with that. So I worked it out. And this talk is essentially the results of what I found. By the end of this talk, we'd like to be able to triage crashes, meaning determining the likelihood of exploitability. In some cases, it might be a bit of a review of secure coding as well, hopefully. And we are going to be addressing memory corruption issues in C and C++ Linux programs. So this is unmanaged code. Fuzz testing, at the moment, really looks for crashes, seg faults. And so it's really focused on unmanaged code in C and C++ programs. And so this talk is going to kind of be about the middle section of the process. The first section is doing the fuzz run itself, things like choosing a fuzzer, choosing the seed files, how long to run the fuzz run for. That's another talk. And then we're not going to cover exploit development as well. That's certainly another talk. And so this is the middle section, where you do the fuzz run, you get crashes. How do you process those? With the goal of fixing the bug or possibly reporting it to the maintainers. And so we're going to go through a quick review of memory corruption bugs. We're going to go through my workflow to deal with them and then show a couple of real world examples. And so what do we mean by memory corruption? Pretty much it's invalid reads and writes. And so that's if the program can be tricked to allow an attacker to either read or write memory beyond the bounds of wherever a buffer or variable has been assigned to that. And so if you can write out of bounds, read out of bounds, that's fundamentally an invalid read or write, most fundamental form of memory corruption bug. These are also called out of bound reads and writes. And often abbreviated OOB reads and writes. And just a quick review, the causes of this kind of memory corruption are very often off by one errors. I see those so often still. It's so hard to do. So about well over half of the crashes I find are really fundamentally off by one errors. So it's good at finding those. Unvalidated input obviously is an important one. Any untrusted input has to be carefully checked. And then still some use of known unsafe functions such as stir copy and get string. And so there's two areas in memory that you store variables in, in a process, the stack and the heap. Local variables are stored on the stack. You say int x equals 5. That's a stack variable. Memory obtained using malloc, like a string buffer. That's in the heap. So if you're allocating x number of bytes, that's always a heap variable or heap buffer. In the case of heap memory, you have to explicitly call free as well. The programmer has to free that memory. So we're talking about buffer overflows as well. That's another word for invalid reader, right? And you cannot, buffer overflow is pretty commonly still in both the heap and the stack. And one quick note, you'll hear people talk about just strictly a stack overflow. Strictly speaking, that's not a memory corruption error. That's a stack overflow as opposed to a stack buffer overflow is different. A standard stack overflow is when you have out of control recursion. You're using recursion and you're not checking things and it's going essentially infinite until it runs out of memory or until something else determines that there's a problem and stops the program. So we're talking about stack buffer overflows today, among other things. This is a quick example of a really basic stack overflow. So you have a program that reads from standard in, puts it into a character array of 8 bytes, and then uses a known unsafe function to copy it from the arg into the buff. That could very easily be a stack buffer overflow. In this case, we're sending it 12 bytes into the program and that's going to overflow by 4 bytes. So what happens if next isn't called properly? I'm sorry, if free isn't called properly, that can usually be another class of bug called a use after free. And a use after free is really just what it sounds like. A program continues to use a freed pointer. These have a pretty good likelihood of exploitability still. We're going to talk about mitigations in a minute here, but use after freeze are still fairly likely to be exploitable, especially in C++ code. These often show up in error handling or other weird corner cases. And especially when there's some confusion or it's not clearly laid out, what part of the program is responsible for freeing allocated memory? For instance, on a function call, who calls free? The caller or the colleague. So if you're not really careful with that, you can easily get into a use after free situation. Here's a quick pseudo code example of a use after free. So malloc 4 bytes, character buffer, do some amount of work with it, free it. Some other stuff happens. And then inadvertently, it gets referenced again, dereferenced again, print f, or maybe you log it, or whatever else you're doing. That's a classic use after free. And then there's other memory bugs too that are typically not as critical, but are important to talk about a double or invalid free. So that's if you free it twice. You free the pointer once, and then even later you free it again. That is hard to exploit, but it's potentially exploitable under race conditions in multi-threaded programs. Another kind of memory bug is when a conditional, like an if statement or a case statement, depends on uninitialized memory. So if you just say int x and don't assign anything to it, and then you say if x do something, that's at least undefined behavior at best. And possibly a hacker could use a control flow attack and try to control the flow of the program. And then finally one I'm sure all very familiar with is memory leaks. And that's when the programmer forgets to call free. Free never gets called on allocated memory. And in that case, memory keeps getting allocated. That could be a denial of service attack. If attacker keeps allocating memory, finds a mechanism to do that, where it's never freed, you can exhaust some memory and bring the program to a halt. So what is exploitability? I've talked about it a little bit already so far. At most basic level, it's reprogramming the application with input data and not code. So if we can trick the program in executing attacker controlled input data as if it were code, that's a code injection exploit. Another way to think about it is that input streams become instruction streams. Or from the attacker's perspective, can I make your program run my program? In that case, it's definitely exploitable. And this typically involves controlling the instruction pointer, EIP in 32-bit or RIP in 64. But this kind of attack is getting really hard to do because it's some of the mitigations I'm about to talk about. It's rare and rare that attackers can get their own code, their own shell code, often into an existing process. Now more and more attacks are about code reuse. And that means reprogramming with existing code. It's already in the process. So the attacker can't bring his or her own code in, but they can leverage existing code both from the application itself or even more so from all the libraries that are linked with it. And so there's enough code where that person can string together existing functionality and reach an exploitable state. And that's called return-oriented programming, or R-O-P, because it relies on manipulating the return pointer. And it's also called weird machine programming. And so most hacks that we're seeing nowadays are some kind of R-O-P attack. And then does exploitability matter? Like why doesn't matter? Why do we really care? And in a way, it doesn't, because if you can find a memory error, it's good to fix it. But certainly, there are issues of prioritization. We all have lots of bugs to deal with. And it's very important to know if there is one that is critical, or that could be exploitable, you obviously want to fix that one sooner. And if it could be exploited in the wild, if the code with the vulnerability already exists in users' hands, then it's also important to understand that it's exploitable so that you can arrange to push a fix out to the user base. And then for what I do, if you're doing security research, it certainly matters, too, because you want to understand what you have when you report it. You want to report it as a security bug if it is, and not if it's not. And it also is important to motivate the vendor or the maintainer to fix things, if you can demonstrate some level of exploitability. But then it's exploitable by whom? There's a wide range of exploitable, exploit dev skills out there. And one being Google has a group called Project Zero. They're Google's internal white hat group. And their job is to test any code that Google customers might use, any application. Is there a purview to find bugs in that and to demonstrate exploitability? And they have about 15 people, very, very top engineers working there, the top hackers on the world. And they have quite a bit of compute power behind them, of course, too, because they're Google and they have Google compute engines. So they can make fuzz runs as much as they want. They can have unlimited resources. Another group with really amazing resources is the NSA or any other nation-state or intelligence agency. And they obviously have even more people and even more resources than Google. And then, who knows? Who else is out there? Whatever. Who is trying to do the exploit? What is their motivation to exploit? How important is the target? How widely deployed is the application? And this is a good time to just reiterate, security is never 100%. What we're trying to do is raise the cost of the attack to the attacker. We want the attack to be more expensive than the value of whatever data they're going to get out of it or whatever nefarious purpose they're trying to do. So that's fundamentally what we're looking at in terms of securing applications. And then another important point is that nowadays, most exploits are not off just one bug. There's been a lot of mitigation. A lot of things have been fixed. Things are getting much better. So to really get an exploit nowadays, you're looking at a bug chain. You have to chain together multiple bugs to reach a fully exploitable condition. So given that, it's a little bit moot to talk about whether or not any single bug is exploitable. Because even one that's definitely not exploitable could play a critical role in an exploit chain. And then sometimes things are pretty surprisingly exploitable too. There's bugs that you think would not be exploitable, but are very difficult, and they are. There was one disclosed recently. And this is in a DNS library called C Aries in the Chrome OS operating system. And someone found full remote code execution, a pretty serious bug. When I say someone, it was anonymously reported by some researcher or more likely a group. And what's interesting about this is that it was a buffer overflow of a write. It was an invalid write. But it was just a write of one byte. It only write a single byte past the buffer where it was supposed to. Not only that, but the attacker could not control what was written into that byte. A lot of times we can. We can control what gets written in bad writes. And we'll see an example of that later. But in this case, it would only always be the digit one. And that's all the attacker had. But they were able to chain this together, or actually it wasn't even chaining. It was heap grooming, where they make lots of calls to arrange the heap in a specific manner to make it exploitable. Or that's what I'm speculating. That's really what it is, a 37 page report about this bug. I did not read all 37 pages. And it was reported to Red Hat as well. And they rated it as moderate security impact. This was before the exploit was disclosed. And I would have too. It seems pretty hard to exploit something like this. And one final thing, this is what triggered it. If you had an escape dot at the end of the DNS name, that's what would trigger this bug. And this is the kind of thing that fuzz testing often finds. It's kind of a weird corner case. OK, so then we've talked about memory corruption, talked about exploitability. Here's a few of the mitigations that have made it harder. The first one being stack canaries. And this is a random integer that gets pushed onto the stack in between stack frames. So just like a canary in a coal mine, if the canary is not looking good, something's gone wrong. And if these random integers, these canaries are actually random integers. So if something goes wrong, let me put it this way, if the attacker can overflow a stack from one frame to the other and try to work on the return point or whatever they're doing, they have to go through the stack canary. There's a little better way to illustrate it. Sorry about that, I missed that slide. And so there are actually integers in between each stack frame. And only the OS knows what those integers are. They're random. They're not available to the attacker. And so if the attacker wants to buffer overflow from one frame to another, and they typically need to, to run an exploit, they don't know that number. And so they're going to change it. And the operating system is going to realize that something changed. Something's gone wrong. It's time to stop the program. And then there's ASLR, another really important mitigation, address space, layout randomization. I'm sorry, I skipped that. My apologies. Data execution prevention is another mitigation. And this simply marks some region of memory as non-executable, meaning if the instruction pointer is ever pointed into that, it's time to stop the program. This is supported at the hardware level by the NX bit in modern Intel CPUs. Now this is embedded Linux. I'm sure there's people working on other kinds of hardware. And so I'm not even sure if data execution prevention is available on different hardware platforms. But it is in generally anything in a laptop. And the combination of stack canaries and data execution prevention makes exploiting stack bugs especially pretty difficult at this point. Another important mitigation is address space, layout randomization. And this scrambles memory. Just like you buy a deck of cards, it's in order by suit and number. You shuffle it. You no longer have that. With ASLR, it's sort of like the mapping is maintained. And so just like we map virtual memory to physical, ASLR adds an extra layer of scrambling and mapping over that. And again, the operating system knows that mapping, but an attacker cannot. So if you control the execution pointer, you don't know where to jump to. You're jumping to a random place. And that's fundamentally how ASLR works. Now one caveat about it, it's only really effective on 64-bit systems because you have a very large memory space. With 32-bit, you're down to 4 gig heap size. And there's actually a tax on ASLR where they use random heap spraying or brute forcing. And it takes obviously many, many calls, but they can keep trying stuff until they land where they want. Land the instruction point or where in memory they want to run the hack. OK, so we talked about memory corruption bugs and mitigations. So we're going to talk about the workflow that I use to go through these things, deal with crashes. First step is to minimize the crash corpus. And that means reduce it down to the important parts. No extraneous information. And then use memory corruption tools to gain information about those crashes. And then finally, get down to brass tacks, determine exploitability, or find the root cause of the bug. So minimization, when you do a fuzz run, it's trying stuff. It's just making up crazy random files and running them through the program. And so there can be a lot of extraneous information in each of the files, as well as you can wind up with lots of crashes, if something's never been fuzz before, you can easily see a couple of dozen crashes. And the fuzzers do their best to make those crashes unique, to make them represent unique bugs. However, that's pretty hard for the fuzzer to do while it's doing the fuzz run. And so we need to minimize two things. The first is the corpus of crashes. So if you do a fuzz run, and you have 20 crashes, you want to make sure that those actually represent 20 bugs and not say eight bugs. And fuzzers have tools. I didn't mention this yet, but I use American Fuzzy Lope. It's a fuzzer, often called AFL. And that was developed by Google. It's an open source fuzzer. And it has a suite of tools. And one of them does minimizing of crash corpuses. It's called AFL-C-M-I-N. And so what it does is that it reduces the corpus of the crashes down to only those that represent actual bugs. That's the idea. So it can easily reduce 20 crashes down to eight or six or whatever it is. And those are individual crash cases. And to be really specific, I mean a file, a crash case is a file you run against a program that's going to make a crash that got found by the fuzzer. So once the corpus itself has been minimized, then it's important to minimize each case individually. So as I mentioned before, you can have lots of extraneous bites that are not relevant to the crash. And so there's case minimizers too. And AFLs is called AFL-TeamIn. And it runs the file with the target program over and over again. And it just experiments. It tries removing a bite. If the same crash still happens, that bite never comes back. If it removes the bite and the crash goes away, or even a different crash happens, then the minimizer knows that that bite is important to that crash. And so doing that, you can just reduce the case down to only those bites that are relevant to the crash, which makes it much easier to do the debugging. And then quickly, there's one more tool called F-dups. And it's a hash-based system. It simply takes all the files in any given directory. And this will work on any kind of file. And it checks the MD5 hash on them and compares them. If two or more files are identical on a bite-to-bite basis, F-dups will eliminate all but one of those files. And I've definitely seen cases. I mean, these tools do their best, but they're not perfect. So I've run the corpus minimizer, I've run the test case minimizer, and still found bite-identical files. And so F-dups is a nice, quick way to get rid of those. OK, and then it's time to start doing some analysis. And there's some pretty useful tools to do that with. Before I go into those tools, it's important to remind everybody that all bets are off. Once things go south in a C program with memory, you really don't know what's going on. You have to be open to anything. And it can affect the tools. The tools do their best, but it's important to not trust anything that comes out of a computer. So the first tool is called Address Sanitizer. And this is found in compilers, GCC and Clang, both have it. And that's the flag that you use to use it on the compile line, compile with it. And it operates at both compile time and runtime to detect memory errors. At compile time, it adds instrumentation to the code, to the binary itself. And then at runtime, it replaces Malik with its own runtime library for allocating memory. So between the two of those mechanisms, it can be very, very accurate at detecting memory errors. And this is an example of the output. You can see it found a heap after free UAF, found a UAF in the heap. Here's the address, some other information. Here's a stack trace. Now, I didn't compile this with the symbols in it. So we're just getting memory locations here. But it gives you a lot of other pretty nice information. We had a bad read, a read of size four. And so it also finds invalid reads and writes, any kind of buffer overflow, use after freeze, double freeze, and some other kinds of memory corruption bugs. Another really useful tool is called Valgrind or Memcheck. Technically, Valgrind is a family of tools. And Memcheck is the one that checks memory. A lot of people use both terms interchangeably. And it's distinct from address sanitizer because there's no need to recompile. You can simply take any binary and run it under Valgrind. And it will give you the information. So that's pretty nice. It does have a lot more output. And it's not as clean as address sanitizer. It doesn't just tell you, hey, this is a UAF or whatever. You have to interpret it a little bit more. Here's an example of the output from Valgrind. We have a bad write up here and a bad read here. Anytime you have a bad write in the heap, you have to consider it exploitable. And in this case, they're close in memory, too. These memory locations, the bad read is here. And the bad write is here. They're not that far away. So that also suggests that maybe the attacker could get something going in there. If you have a bad write, you can write something. And then if it later gets read, all bets are off. And then there's one more really useful tool. And it's simply called exploitable. And it was developed by cert. And it's now been open source. It's available on GitHub, maintained by a person named Jonathan Foote. And it's an extension to GDB, although you can run it as a standalone script as well. And what's really nice about it is that it actually categorizes the crash into various levels of exploitability. It has four categories, exploitable, probably exploitable, probably not exploitable, and unknown, some cases it just can't tell. And it gives you a really nice layout of exactly what's going on in here. So between the three of these tools, you can get quite a bit of information in terms of understanding what's really going on. So then you have the next step, which is actually figure out what's going on. Determine exploitability or find the root cause. So we're just basically into hardcore debugging. Before you do that, it's important to disable ASLR. After talking about it, it's a good tool to avoid hacks. If you run the program over and over again, or any time you run it, you're going to have all different memory locations for everything. And so that's going to make it essentially impossible to do any kind of triage or debugging on it. So disable ASLR, certainly do it behind a firewall or a NAT router or some other case where you're safe, you're not exposed to the internet. This is the umbutu file under PROC that controls whether ASLR is on or not. And you can't edit stuff under PROC. So this is the command to disable ASLR. If you do it this way and reboot the machine, ASLR will be back in place. And then the next step is to identify critical memory locations. So it gets confusing when you're doing this sometimes.  I like to essentially be very careful about understanding the critical memory locations. And so that can vary in terms of the crash, but certainly where the crash happened in the code, what instruction was being executed when it crashed. That's a very important point. Where the invalid read or write occurred in the data? Where in the memory in question was allocated and or freed, if not, that's a code issue. Like where in the code the allocation happened? Or where it was or was not freed or freed too many times? And then whenever the data is reassigned, perhaps it gets reassigned to a different variable or buffer or gets copied, that can be important too. And that's actually in the code and or the data. So once you've kind of mapped this out, and I like to actually write them down, because it does get confusing after a while, then it's time to just fire up GDB and start digging into it. And so uses of GDB is certainly outside the scope in today's talk, but the basic idea is that you set a break point where the crash happens and work backwards from there. Certainly you want to compile with dash G to get the symbols and dash O0, zero level optimization. That way you get as much information as possible in terms of variable names and things like that. And so pretty much you run the target with canary values like capital A's. And that way you can track those through memory. It's easy when you dump memory to see a bunch of OX4 ones. So that's the hex number for capital A. It's 65 in decimal. This is its ASCII number. And so we're pretty happy when we see a bunch of 40 ones where they're not supposed to be, or four ones. And so we kind of use that to track things too. But GDB is a little bit tricky because you have to work on it a lot. And if you set a break point where the crash happens, kind of the idea is you run it over and over again and set break points kind of early and earlier and earlier and dump memory and see what's going on. And that works. And a lot of times that's what you have to do. But it's kind of time consuming and cumbersome. So I've been using another tool in addition to GDB. It's actually a GDB plugin. And it's called RR. And it does reverse debugging. It allows you to actually step backwards through the control flow of the program. So you have to run the program under RR. And it records the execution. Once you've done that, you can run the program normally with any input you want. And then set a break point. But once you get to that break point, you can step backwards.  And that can be, for a lot of bugs, a very efficient way to figure out what's going on. Now a lot of bugs are pretty easy to figure out. Or certainly, off by one errors are typically fairly easy to both find and identify and obviously fix. But some of the stuff is not. So it's important to kind of tape breaks, be persistent, this kind of low level debugging and memory dumping and all that kind of stuff. It's hard work. And then one more thing. Once bugs are fixed, if a crash is fixed, that crash might have been masking another crash. It might have stopped execution before some other bug might have been discovered by the fuzzer. And so it's important to fix those bugs and then rerun the fuzzer again on the target in the same manner to try to flush out a few more crashes. OK, so for the final section, I'm just going to go over a couple of real world examples that I found in my own research. First one being PHP. I was fuzzing PHP. I found a low bad read. Low meaning low in the heap space. It was actually very low. It was a null pointer dereference, essentially. So not exploitable really at all. Here's my report. And I was running it from the command line. And I was using a dictionary in the fuzzer. So I gave it all the standard commands. This isn't the PHP any file, which is typically not used at the command line, which is part of how I flushed out the crash. And so the fuzzer was always using these. Like I set up with a dictionary so the fuzzer didn't have to generate these parts of the any file. But it just happened to put a 1 there. And that made PHP think it was running in the context of a web server when it was not. And that just triggered this null pointer dereference. And you can see here the valgrind output of this, bad read of size 4. Address 0x10, so that's 16 in decimal, is not stack malact or freed. So this is a standard null pointer dereference, no longer exploitable. In the old days, you used to be exploited these. But those days are over. And then this is the fix that the PHP devs put in there. And they're simply just checking now that it is operating in the context of a web server and doing the right thing as appropriate. So then I turned my attention to Ruby and started fuzzing the regular expression compilation in Ruby. And I found actually a few bugs in that. This is one of the more interesting ones. And it was a pretty nice buffer overflow here. And this is my report. It's pretty weird. I mean, if you open a character class in a regular expression and you don't close it, and you have an octal number in it after that, that's the weird corner case that the fuzzer found. Here's the output of the asan. Have a heap, buffer overflow. But what's kind of interesting here is that Valgrind gives us even more information about this bug. Valgrind's saying there's an invalid write here of size four and then an invalid read later. And they are, again, fairly close in memory. And so as I mentioned before, anytime you have a bad write in the heap, you really have to consider that exploitable. And I do. So this bug was really an exploitable bug in Ruby. However, the attack surface for it was very small. Because we're talking about a regular expression as the untrusted input. And it's already not a good idea to accept untrusted regular expressions in any application. Because you're opening yourself up for minimally a denial of service attack. You can think of there are regular expressions that are regular expression bombs or whatever. Or they can really exhaust memory. And so setting aside, that's not memory corruption. That's just a DOS attack. But that's why you really shouldn't even expose yourself to untrusted regexes to begin with. Here's another reason, because this code could have been attackable in Ruby as of eight months ago. It's been fixed. And so this is definitely exploitable, but thankfully a pretty small attack surface. And here's their fix. This is the Ruby devs fix. And they're talking about the octal and all that kind of stuff. And then I found a bug in a system called dynamite. And this is written by Netflix. I found an invalid write in it. And dynamite is a replicator sharder for Redis and Memcached. Key value storage systems is what Redis does and Memcached does. And so dynamite sits in these storage systems between these storage systems and the internet. And it does either replicating in case these, for reliability here, or sharding if you're dividing it up by last name, first letter the last name, that's sharding. And so that's what Netflix dynamite does. And so it's on the open internet. And they are using it. And lots of other people are using it too. It is open source. And I decided to fuzz it and see what happens. Now what I was fuzzing is kind of a weird thing to fuzz. It was what I call an oblique attack. I wasn't fuzzing it head on. I fuzz the admin file. And it's a YAML based file in Netflix dynamite. And I fuzz this file. So I hit it with lots of weird versions of this file. And I did get a pretty nice crash. And now that's kind of a weird thing. Like, OK, so an admin can make a crash or a bad admin. And that's still an important bug. It has to be fixed, because you could have a malicious or compromised admin. But that's kind of what I thought I had. But I'm like, OK, I'm going to file this. This is the case that I filed. So this is the minimized case that I sent to it as a YAML file. And so here you have six capital A's. And to demonstrate, this is what I filed with Netflix just to demonstrate the issue. And so I ran the program. Down here, we got an error. And this is actually from Glib C. Sometimes Glib C gives you a nice error as well. And it goes to standard out. I mean, sorry, standard error. You won't see it on standard out. And we've got the crash here. Here's the next invalid size. And so in GDB, I just dumped that address. I'm sorry. Yes. And here we have the capital A's here. And then I dumped the next address. And there's more capital A's. And it's Intel. So it's little endian. So these are actually six contiguous writes. And this is attacker control. Like the attacker could put anything here she wants to in lieu of those A's. You get six bytes to do whatever you want. And so OK, this is a cool bug. But again, it's like not that OK, it's an admin file. And so I filed it. And to my surprise, they didn't fix what I thought. I thought, OK, maybe it's somewhere in the YAML code or somewhere else. But it was actually in their string functions. And so this is actually something that gets run a lot, especially in something that handles key value storage. All it does is handle strings and duplicate them. In fact, essentially, dynamite sits in between. And it is man in the middle itself. So if it gets compromised, that's a very serious thing. And so I was pretty shocked when I saw this fix. I didn't think it would be down on the string code. And essentially, they were using string functions, a string library, but they were kind of a little bit writing their own code over it, too. They weren't using it really strictly. And so here's their diff. And you can see this is a standard off by one error. And so that's how they fixed it. So this got fixed. And so I did get in the Netflix Hall of Fame, which is a good place to be. And it's good to have it fixed. And I was just really surprised about this. This could have been, theoretically, a pretty critical bug. If you have a hacker could have gotten man in the middle between the internet and Netflix or a lot of other people. So pretty serious bug, I would say. And so that's going to do it for now. I'd like to talk about some references. Rents of Their Polytechnical Institute has a great course called Modern Binary Exploitation. And this is both on GitHub. And they also have materials as well on their site. So it's good to search for this.  And then on their site, they have really the full course materials. So you can just take this course on your own. And I did. And I learned a great deal from it. And they also cover reverse engineering in addition to binary exploitation. And then the Bible of this kind of work is called Hacking the Art of Exploitation by John Erickson. I got the first edition in 2005. There's a second edition now. And then Project Zero is a great blog. Some other really nice references there. More than I can list as well. So that's going to do it for now. I really appreciate your attention. And I think we have a little bit of time if there's any questions. Yeah. So we're going to use the mic here for the questions. Let's see if that. That was a good talk. I like it. I wanted to ask, have you had any experience with using any kind of fuzzing tool or dynamic tool to try to exploit Java code or something that runs in Java? Yeah, thank you. Yeah, great question. So the question, yeah, so do we use fuzzing to break Java or even other managed languages? And that's the next step. So my company is actually researching that right now. I'm working on that right now. The idea is that fuzzers with unmanaged memory, well, let me put it this way, a fuzzer fundamentally throws random data at an application. And it monitors, in this case, for crashes. But a fuzzer could monitor for anything, for any kind of condition. The problem is, or the hard work, is specifying those conditions. And so I'm trying to think of ways in Java to do that. And obviously you could do that by hand, but that's sort of like writing unit tests. So I'm trying to work on ways to do that. The answer for now, there's really not a whole lot that fuzzers offer out of the box as they stand for managed code. But we're working on it, looking for ways to do that. Anybody else? Yeah, OK, cool. It's like one of those talk shows or something. Thank you. Does your workflow change any for multi-threaded applications? Oh, yeah, good question. Yeah, so does the workflow change for multi-threaded applications? Not mine, or I don't do that. I actually, when I'm fuzzing stuff, I'm usually, or I am fuzzing it in a single-threaded context. And I typically, I really don't fuzz stuff that's multi-threaded at this point, or I haven't. Now you could, but it's certainly, it's obviously harder for a lot of reasons. And actually, the fuzzer that I use itself is single-threaded. And so I mean, you could certainly spin up multiple ones of them to do it at scale. But in terms of specifically fuzzing, maybe to try to find race conditions and things like that, I've actually never done that. That's a pretty interesting idea. Anybody else? All right, thank you guys very much. Really appreciate your attention. Hope you have a great rest of your day. Thank you.


Hey everyone, welcome to today's day trading recap for Monday, August 2nd, Green Day to start the week, plus $555 on the day, $555, lucky number five, Facebook was a one mighty 90, we took plus $145, no pairs trades today, and then on the runners had four trades, Roku for plus $400, Square plus $580, Boeing minus $330, and Roblox minus $240. So let's take a look at these, we'll start with the mighty 90 that we took in Facebook. Facebook started off just fleshing lower, just kept fleshing, fleshing, got this nice spike in volume, put my order to get filled right down here, and it came down and hit it, actually went a little bit below that, but pretty much hit it on the nail, got long here, so caught that bounce, got out a half in this area, and then scaled out of the rest and was out right up there. So nice trade in Facebook for $145, did really small size there, obviously if I would have done my normal size, I could have booked more, but just kind of trading small, getting back in the swing of things this week, Roku was my, was a runner, oops, pull up Roku here, so Roku was fleshing, had this nice spike in volume on that second bar, so when it bounced up here, got short, and flushed down, and I got out of my last P, I just did a couple contracts, got out of my last piece around right there, so plus 400 in Roku, square was a nice square, just super strong today, oops, that's not square, that's BA, this is the one I want. So square had this big push up, first part of the day, big volume, and so I ended up getting long right here, or no, no, I'm sorry, I didn't get long until right here on this little pull back, got long right here, pushed up to the pivot, and got out, kind of actually got out right here on this first initial push, kind of chopped around, chopped around, when it pushed back up, I ended up just closing out of another piece here, and then when it pushed up here, got out of another piece, so nice $580 winner in square on just a few contracts. Boeing was a loser, Boeing, you loser, Boeing looked like he was just going to keep taking off, and so when it pulled back here, I got long, looking for another extension higher, started to push up here, didn't quite get to my order to get filled on half, and then just rolled over, so I cut out pretty quick, took a loss on that one, and then pretty similar on Roblox as well, Roblox made this big push, big volume spike, looked like it might take off, and then it, so when it pulled back, got long, just kind of chopped around, and then when it broke down here, just cut out, now you can see after the fact this thing did end up rallying, but I was already out of the trade by then, so took a loss there, but still $555 on the day, I was done by about $915, $930, so less than an hour of trading, so good day, good way to start the week, look forward to seeing everybody in the live stream room tomorrow, talk to you then.