Thank you, Kjara, taking over swiftly here. Martin Benish, Kjara already mentioned that I'm acting head of the legislation division in the legal services of the ECB. And it's a great pleasure to chair this panel on the second day of the conference. This panel will send around procurement issues. The starting point for this panel will be the ECB internal review procedure for contract award decisions, i.e. the procurement review body or PRB. This is a mechanism that does not exist in any other EU institution and therefore is interesting and unique. We will compare it with the legal framework for the review of such decisions in other EU international and national organizations. It will include stop taking after 15 years of the decisions and experience with the PRB and assessment of the impact of recent case law on review procedures and the requirements for obtaining interim relief from the Court of Justice of the European Union, as well as practical problems that arise, for example, in terms of the national regime. I'm super proud and happy to be in a panel of super experts. I can already mention that. I just mentioned the names at the moment, Isabel Köpfer, who is sitting next to me. The camera will pick her up when, as soon as she says it, peep. Hello. Say more. Okay. Yeah, now I'm there. Hello. Welcome to the conference. This is the feature of video conference that sometimes the camera is not following what you want to do. Isabel Köpfer is advisor in the legal services of the ECB in the institutional law distribution. Secondly, we have Jeff Derricks, senior advisor, head of group legal department in the National Bank of Belgium and we have last but not least Laura Andre, member of the legal service of the European Commission. I will introduce them in more detail, but always at the beginning of their respective presentations. Please allow me some introductory thoughts on the issue. A common theme of all of these presentations will be the importance of communication in the relationship between contracting authority on one hand and tendress on the other side during but even after the selection process has ended. This raises the question, why is it beneficial for contracting authorities and tenders to communicate in a transparent manner and keep doing so throughout the process? What role does communication have in this relationship? Or to put it a bit more bluntly, assuming that all legal obligations have been complied with. Why would contracting authorities want to communicate and rethink rather than simply going through with their decision unilaterally not care about the unsuccessful tendress? On the other hand, why would bidders be at all interested in communicating with the contracting authority and staying in dialogue even after a non-successful procurement procedure? There must be a benefit for both parties to stay in contact. Cheap communication channels open and take time to explain the own perspective. How does a perfect communication look at the first place? Academics define seven C's that effective information has to comply with. Information must be complete, concise, clear. Cautious, that's an interesting one, so polite, but also avoiding expressions that hurt validity. Concrete, considerate, and correct. So no false information, that is an interesting requirement. This means a lot can go wrong when bidders and contracting authorities exchange information in the process. For instance, information can be forgotten or go missing, can be too bulky with the risk of relevant aspects being overlooked, be ambiguous, delayed or offensive, be too vague, not clear enough, not meeting the expectations of the receiver or simply false. This confirms the clear need of all participants in procurement processes to keep all involved parties, contracting authorities on the one side and bidders on the other side informed in a consistent manner. Subject, of course, to what is permissible in the procurement process at hand. Now imagine that the tender process is decided and bids have been rejected. What is the role of information and communication then or one step ahead in the case of imminent judicial conflict? The UK Ministry of Defence considers that in terms of conflict management much depends on the quality of the debrief, hence how well the reasons are explained why bidders were not successful. Academics confirm that good debriefing diffuses protests also by giving contractors a better understanding and control over the risk of improperly losing the award. Information helps suppliers to reassess and understand changes necessary which they have to do to improve their own business and be better in the next bit. Information also makes the bidders more tender ready or contract ready and makes future bids more successful. An evaluation of small and medium-sized enterprises as an east access to public procurement in the EU reported that no debriefing was one of the top four biggest problems companies face when participating in public procurement. Meaningful feedback and clear communication allow small and medium-sized enterprises to understand why they lost and why another company won. This has also a strong potential to reduce litigation and that is why it is indeed beneficial to communicate in transparent manner and to keep communicating throughout the process. So with this small introduction now I would like to go over to the presentations of the panelists and we start with Isabel Köpfer to my side. Isabel will talk about the ECB's internal appeal procedure for contract award decisions. She will be outlining the ECB's status as Supernational Organization of the European Union and will give an overview of the ECB procurement rules we have our own set there for. The main part is then about the communication of the outcome of selection, award and rejection decisions and remedies of tenders and in public tender procedures, in particular the ECB procurement rules allow unsuccessful candidates and tenders to challenge the ECB's award and rejection decisions with an appeal to the already mentioned PRB procurement review body which is quite unique within the EU institutions to say it again. Furthermore the presentation includes an assessment of the impact of recent case law on review procedures and the requirements for obtaining interim relief from the Court Justice of the EU and then of course a stock taking after 15 years of decisions of the PRB in which I think Isabel was involved for some part of it, some substantial part of it. Isabel just who the person is currently I mentioned advice on the banknotes procurement in the counting session within the ECB institutional law decision. Isabel joined the ECB in 2007 and has 20 years work experience in contract and procurement law. She has previously worked with a GIZ, Gesellschaft für Internationalen Zusammenarbeit International Corporation, a German government owned organization in the field of international cooperation for sustainable development and before that at an international law firm Allen Overie LLP. Isabel has studied at the University University Nanter Paris, the University of London SOS and Wolfgang Goethe University Frankfurt Amain. She is a post graduate in new law from the University of London and obtained a PhD in environmental law from the Wolfgang Goethe University Frankfurt Amain. So that was the introduction and now we go over to the presentation. We are looking forward to listen to Isabel's presentation. Isabel please. Thank you very much Martin for the kind introduction. Can I have the slides please? So as already announced, the first topic is the ECB's internal appeal procedure for contract award decisions. Thank you very much. Yeah it's as Martin already mentioned it's a formalized procedure which is quite unique because other EU institutions don't have such a mechanism in place and it was already praised by the by the European Court of Justice as an effective remedy and more detailed to that later. So let me start by giving you an overview on the presentation. I'll first start with the status of the ECB as Supernational Organization of the EU and then I'll give you a quick overview on the ECB procurement law framework and to give you a little bit more background how how how procurements work at the ECB. Then I'll move on to the communication of the outcome and the remedies to the tendras and then I'll give you some insights on how the ECB internal appeal procedure the with the so-called procurement review body works. Also explain about the rules of procedure and requirements and the scope of review of the PRB and last but not least how effective is this remedy as a matter of fact. Okay so the ECB as the Supernational Organization of the EU has the autonomous power to lay down its own rules for internal organization and administration. You can already click the next column exactly that's what I need first. So due to this principle national procurement and budgetary laws do not apply to the ECB. The EU procurement directives do also not apply as they are addressed directly to the member states and the EU financial regulation does also not apply it only applies to to the other EU institutions as the ECB has its own budget. Of course this does not mean that the the ECB is entirely free to establish its own procurement law regime. The the principles of the EU treaty supply in particular the free movement of goods for the freedoms of establishment and the freedom to provide services and also the the principles of procurement which are derived there from and which were established by the Court of Justice of the EU such as equal treatment and non-discrimination, mutual recognition, proportionality and transparency and since the award decisions of the ECB and tender procedures are subject to judicial review by the European courts it's also the the jurisprudence of such courts that are relevant for the ECB. Next slide please. Against this background the ECB has established its own legal framework for procurement procedures first by way of internal guidelines and as of 2007 by way of finding decisions adopted by the executive board and published in the official journal of the EU. As you can see and one more click please as you can see from from the slide the ECB has changed its its decision since 2007 six times it has completely recast it the ECB decision in 2016 and the the woods mainly draw on the EU procurement directive. Okay then let's move on to the next slide. In terms of types of procedures you can also see that it doesn't differ from the the roots of all the tender procedures stipulated in the EU procurement directive that's why we will not go into into details you will you will know how the the procurement procedures under the EU procurement directives work so we have the thresholds that are also the same as in the directives because we directly refer to those above the thresholds we have the typical public tender procedures the restricted negotiated procedure competitive dialogue innovation partnership and the open procedure and below the thresholds we have installed a simplified procedure and we call it three five quote procedure which is less strict than the public tender procedures so less deadlines we can invite tenderers depending on on how high the value is and usually we do not publish an official an official notice in the official journal. Okay next slide please and we also have the the typical exceptions like in the EU procurement directive which establishes for certain type of contracts that fall outside the scope of the also outside the ECB procurement rules such as cooperation agreements between other ECB and other institutions public institutions the acquisition and and rental of buildings for example also employment contracts and legal representation of the ECB in court proceedings then we have what we call the single supplier exceptions where we can award direct contracts to certain companies due to for example technical requirements or also in the case of extreme urgency and secrecy and we have special contracts where the cross-border dimension is not so important or where an element of trust is required like for security or legal services or restaurant services which are more regional so it's largely the same as in the EU procurement directive. Next slide please so then let's move on to the communication of the outcome of the award decisions to unsuccessful tenderers we send an outcome letter within 10 days stands the period and we indicate the reasons for rejecting application and tenders so what are the reasons and those are the main and the sub-scoring of the own tender and the written explanations and that's usually an excerpt of the evaluation matrix and upon request of the tenderer they can ask for additional information regardless of the standstill period so that request can also come after the standstill period has elapsed and if the tender was admissible they can ask for the name and the key characteristics the relative advantages of the winning tenderer and we have the possibility for public tender procedures for an appeal procedure within 10 days upon receipt of the outcome letter. Next slide please yeah we had so this regime is in place for two years now since the last update of the ECB procurement rules previously we had a different notification regime which we call two-step procedure it was a little bit different in so far as in the first step we have only disclosed in the outcome letter a high level score which is a high level comparison of the scoring of for example quality and price no sub-scoring was was provided and only in the second step upon request additional information could be requested within 15 days and the information that can could be requested was the reasons for ejecting the tender and on top of that copies of all evaluation documents relating to the own tender and if the tender was admissible again like in the previous approach the name of the winning tenderer key characteristics and so on and copies of all evaluation documents relating to the winning tender so this is very comprehensive and it of course there's a lot of sometimes when we had additional requests from tenders a lot of administrative burden and each time they ask they asked for additional information the stance the period was extended and and this of course the contract conclusion was also suspended in some cases we had several additional information requests and in the worst case an average duration of three months for for the whole procedure so next slide please we actually we wanted to always change this approach and as we we also gave out more information that we actually weren't legally required in particular the copies of all evaluation documents the contracting authority is not obliged to provide that and then another thing happened so we had a game changer fund the fund breeder judgment in 2015 which sort of obliged us to to change the old approach as some of you might be aware fund breeder eased the conditions for the award of interim measures so until then it was practically impossible to to to have to gain interim relief for unsuccessful tenderers because damage could always be compensated in in money so now in in the fund breeder case the court said that it's sufficient to prove serious instead of irreparable harm in order to preserve the principle of effective judicial protection and and this made us think that we had already concerns about our previous notification regime because what we provided in the first step was not a minimum amount of information that was necessary to trigger the stance the period the court now emphasized and referred to previous case law in fast web that award decisions have to be challenged before the courts within the stance the period before the contract is concluded after the expiry of the stance the period contracting authorities can can award the contract and tenders can merely claim for damages so the consequence for us was that we have to we had to provide more information in in the first step because we were concerned that especially for those tenders who did not ask for additional information that the stance the period would not start running and this is what was the main reason for for changing our two-step procedure okay next slide please so let's move on to the to the internal appeal procedure as such we established the internal procedure in 2007 and the prb so the procurement review body is an internal body composed of three ecb senior managers and supported by the ecb's directorate general legal services so i'm the secretary of the of the of the procurement review body and i have special tasks which i'll explain to you in a minute the prb is independent from the office carrying out the procurement procedure and we have established rules of procedure sr set forth in article 39 of the ecb procurement rules and we further specified them in the internal mandate of the prb which is not public next slide please um so the the secretariat of the prb assists in the performance of its tasks it provides for example all members of the prb with a relevant tender documentation it liaises with the procurement committee which is responsible for awarding the contract to clarify the facts and it summarizes the facts underlying the appeal and we also provide a preliminary legal assessment prepares the first draft of the appeal decision takes minutes of the prb meetings and ensures notification of the appeal decision to the appellant and it and we also manage the prb archives so now this is of course a lot of um tasks which we have to do on top of of our uh normal um job and uh so i'm not the only one who's dealing with with these um internal appeal procedures i also have the help of my colleagues uh in in the in the legal services department yeah so the the main aspects of the internal appeal procedure the remedy is available to unsuccessful candidates or tenderers only in public tender procedures so for all other procedures like the the simplified procedures or call-offs the the the remedy is not possible and the the prb may reject the appeal and it can sustain the appeal and and order that a tender procedure or parts of it are repeated or take a final decision then it's also important to note that the appeal has a suspensive effect in relation to the award of the contract which means that as soon as an as as a tenderer submits an appeal the the contract signature is suspended the appellant must await the ecb's appeal decision before bringing the matter to the court of justice so the appeal procedure is mandatory and a tenderer cannot go right away to the court of justice then the time limits for judicial remedies are are two months from the receipt of the appeal decision and we also have established an additional standard period for contract awards of at least 10 days upon prb the the the appeal decision has been issued so that's what we have to do also as an aftermath of of fanbreda because a tenderer if we make it mandatory on the one hand to go to the appeal body it of course has to have another 10 days after the the decision of the prb has been issued to bring the matter to the court if it so wishes next slide please so what are the requirements and the scope of review by the prb the requirements for the tenderer is that the appeal has to include all supporting information and reasoned objections this means that a tenderer cannot simply send us a request please review the decision i'm unhappy with it and the the the tenderer has to really substantiate and its its appeal and the reasons why he's unhappy and wants to to submit objections it cannot submit precluded objections we have a in the in our procurement rules a a clause which requires a tenderer that if he realizes that the tender documents are inconsistent or flawed that that it has to be notified to to us immediately within 15 days and if if the tenderer doesn't do though those objections would be precluded in in in the later appeal procedure and also in the court proceedings perhaps another interesting fact to know is also that appeals can be submitted in all languages so we do not only accept english we also sometimes have received appeals in german or in french even so we would also accept that and have it translated then if need be what does the prb examine the prb examines first whether it's admissible so whether it has been submitted in in in due time so within 10 days of the stand-in period and whether the decision of the ecb to reject the application or tender was taken in line with the ecb procurement rules and with general principles of procurement law so next slide please the scope of the prb is is is limited as the ecb has like any other EU institution broad discretion in assessing factor the factors for awarding the contract so the prb checks compliance with the procedural rules and the duty to give reasons it also checks the correctness of the fact found and it ensures that there is no serious and manifest error of assessment or misuse of powers so that's in principle what a court would would review as well then that's what what the prb assesses as well in an appeal procedure next slide please so how effective is this this remedy the general court and the european court of auditors have confirmed that it's an effective remedy and a robust internal review mechanism the general court has said that it's an effective remedy whose purpose is to ensure that the lawfulness of the award decision is verified by a body separate from the one that initially took the decision and giving rise to a decision open to judicial review so that's very important that the office the prb is independent from from the office that initially took the the the decision and so for example we may how do we how do we ensure this we make sure that that if i were involved in the initial procurement procedure i couldn't act as a prb secretary or also the one of the managers who's in the prb if that manager was involved in the initial procurement review procedure or only gave approval then this would be a conflict of interest so in that matter guard it's really completely independent and the prb is supposed to given a really objective assessment of the award decision taken by the ecb the ecb receives about between five and ten ten appeals per year but this really differs sometimes you only receive receive one or two and sometimes receive ten since 2007 we received around 70 appeals and i would say approximately 10 percent were upheld and the majority was either rejected or settled by presenting the appellant with additional information the appeals mainly relate to the evaluation of of the tenders the scoring and also to formal requirements and selection criteria all in all we are convinced that with with this appeal procedure we are and ensuring that that tenders do not have to go further step and bring the matters to the court so i think that by verifying the initial decision this is a very good mechanism in terms of transparency so next slide please the court of justice of the u has exclusive competence in disputes between the ecb and suppliers concerning procurement procedures so tenderers cannot go to the national courts they have to go to the u courts the general court is is a competent and we have the time limits of two months which which starts two months from the seat of the appeal decision and complaints can also be lodged with the european ombudsman separately from from judicial review okay last slide thank you very much for your attention i hope that this insight in our internal appeal procedure was interesting for you and yeah if you have any questions already at this stage please go ahead thank you very much isabel i propose um note your questions i already had two uh note it down on isabel's presentation and we will then collect questions after we have heard all three speakers and then we can enter into discussion possible also the speakers can can do so uh we will go next to laura andree um she will speak on the communication between the tenders and the contracting authorities after the award position and on that we will base her presentation on the regime of the financial regulation she will describe the information that may be provided to the tenders depending on their ranking and the admissibility of their tender but also the actions that can be taken by the contracting authority and by the tenders following such exchange of information presentation will be supported by a summary of the relevant and most recent case law in which respect we benefit from the fact that laura is quite involved in the number of these cases without of course going giving us any juicy details unfortunately we will see maybe we can get something out of her here um just a few words on laura such laura andree is a member of the european commission legal services and an agent of the european commission before the court of justice of the u she focuses on legal issues relating to the public procurement of the european institutions and to the interpretations of contracts before joining the legal services she worked as a legal and procurement officer for the director general for informatics of the commission from 2016 to 2019 prior to joining the european commission laura andree worked as a lawyer in the litigation and dispute resolution department of the law firm stupid in brussels that was from 2011 and 2016 where she specialized in cross-border litigation with a particular focus on contractual and commercial law laura andree and this is something that we had in the prep meeting found out uh normally we don't mention internships but here's sometimes nice you were the blue book stagiaire in in the fall of 2010 in the cabinet of miss vegan reading white's president then of the european commission and commissioner for justice fundamental rights and citizenship laura andree studied law at the university of yesh ludig belgium laura without further ado uh you have the floor thank you for being with us thank you very much martin and thank you isabel for your presentation i hope you can hear me maybe you can oh perfect so um as martin just said the topic of my presentation today is the communication between the tenders and the contracting authority after the award decision as said by isabel the european institution are not subject to the procurement directive but we are actually subject to our own set of rule namely the financial regulation which is regulation 2018-1046 the idea of my presentation today is not to describe the entire applicable legal framework with all the exception and the specific rules but more to highlight the key topics that are currently being discussed and challenged before the court of the eu and providing you with concrete examples that i have placed in my daily practice next slide please oops we are frozen perfect so um i will discuss three points today um the first one being the communication of the outcomes or the information that is provided to tenders and candidates after the award decision the influence of the stencil peers and then the influence of this communication and the stencil on the pre-constructural remedies and the most important being the interim procedure before the general court next slide please so uh as isabel mentioned the prb mechanism is a very unique mechanism and the eu institution are not subject to this mechanism and do not have a similar mechanism in place so how do we manage to ensure that we give the tenderer still a possibility to have a kind of review well in the eu institution we ensure that the principle of transparency and the right of effective remedy are respected this is uh of um the utmost importance when communicating the outcome of a procurement procedure but also when dealing with this stencil peers as you will see later on next please first of all a scene setter what is the legal framework what is the exact legal framework applicable so the financial regulation for c specific rules relating to the information provided to candidates and tenderer in article 170 of the financial regulation and in the point 31 of its annex one which is a an annex dedicated to procurement rules the rules on stencil appeared on the other hand are laid down in article 175 but also in the point 35 of the annex it is important to note that the directive 89 665 which is the remedy directive does not apply as such to the eu institution however um the van breda caseload that isabel already mentioned before and that you will see will come back quite often in this presentation has clearly stated that the principle that are laid down in the remedy directive also apply to the eu institution next slide please let's first discuss the communication of the outcomes or the information that is provided to tender and candidates after the award decision it is important to note that the rules forcing in the financial regulation should be distinguished from the rules that are foreseen in the access to document regulation regulation 1040 92001 the regulations to 1040 92001 foresee a general right to access to documents and this is a keyword and it applies whether or not the applicant is a candidate or a tenderer it is a general rule applicable to all citizens here we are discussing the privileged access that can be granted to the candidates and tenderer in the context of procurement procedure and this access is to information which is of course broader than documents so what is the general rule and the general rule is that there should be a privileged access for all candidates or tenders simultaneously and individually at each step of the procurement procedure each steps means what it means that every time that a decision is taken by a contracting authority the concerned tenderer should be informed and understand the reasoning behind this decision next step next slide thank you and the financial regulation still foresees a two steps approach what is this two steps approach the first step is we will inform the tenderer of the reason for the rejection of its tender and then only if this tenderer makes the request in writing will it be informed about the name of the tenderer that is the winning tenderer the characteristic and relative advantages of its of the successful tender and the price or the contract value please note however that there are certain limits that are foreseen for this privileged access of course the contracting authority must withhold information that that contrary to the public interest would prejudice this the legitimate commercial interest of the other tenders and would of course distort fair competition next since van Breda Isabel mentioned that the ECB has adapted its way of working and has moved away from this two step approach however it is important to note that the general court of justice has confirmed the validity of the two step approach this has been clearly confirmed for example in the trans tech judgment that you will see quoted here however for the practical reason that Isabel mentioned during a presentation the commission always strongly recommend to its services to proactively provide more information than the first step information the commission would always suggest to its service that they already proactively directly without even a written request from the tenderer provide information on the identity of the winnings tenderer the relative advantage of its center and the price or contract value next piece I will make a small appartee to insist again on the importance of the principle of transparency and the right to effective remedy when approaching the communication of the outcomes of a procurement procedure why is it so important that the tenders are made aware of the reason behind the rejection well it's a novel it handenwells the tenderer to verify if the duty to state reason has been respected by the contracting authority the court has confirmed that the objective of article 170 the communication between the to the tenders is to ensure the right to effective remedy this is once again a very important principle that has been again highlighted by the court in the van Breda case law that we will mention later on on this slide I quote to extract from the case law that exemplify this importance and the link between duties to state reason and effective remedy next please why is the duty to state reason so important in procurement procedure maybe it is important also to recall the basic principle a contracting authority has a very very broad margin of interpretation when deciding on procurement matters and this is why the court is always putting an emphasis on the duty to state reason and a proper communication with the losing tenderer so that they are able to understand the reason for their rejection once again the link between transparency and the right of effective remedy is crucial for the court next the court has confirmed that this communication is not a single action it is an exchange between and the tenderer and the contracting authority the contracting authority has a certain obligation but the tenderer can also participate in the exchange of information and request additional information it is very important to know that the court confirm that the duty to state reason is to be assessed on the basis of the information that is available at the moment of the lodging of the application by the applicant this means that the communication has an influence on the extent of the duty to state reason by the contracting authority the contracting authority is in a position to provide an additional information additional motivation and build a stronger case even after the first letter have been exchanged you will find here two cases that highlight this principle next piece coming back to the legal framework it is also important to really grasp what is the notion of characteristic and relative advantages which information in particular should be provided to the tenderer what does it actually cover the court has specified that the characteristics and relative advantages are the information that allow the tenderer to understand the reasoning for the rejection of its tender this does not mean that the contracting authority has a duty to provide very detailed information on each and every single point and this is important the court has confirmed that a detailed summary of each detail is not to be provided and actually could not be in practice provided a detailed comparative analysis is also not necessary the purpose is not to compare the different offer but to provide the different explanation that we're underlying the different decision next oh seems like a slide is missing maybe on my side um no problem I will just um um confirm that the court also established that no copy no full copy of the evaluation report should be provided I will here refer to a judgment T 536 slash 11 european dynamics against a european commission the the full copy of the evaluation report is not to be provided just a relevant extract I'm just saying this because I know that cheffrey will shortly explain that in belgium law this is a bit different thank you um in the q&a I will be happy to discuss also some recent case law should you have any question on this I'm putting two examples here of a growing trend of the court to pay more and more intense attention to the beauty to state reason and to the communication and and the importance of the exchange of communication between the defenders and the contracting authority after the award next here is next one as already said the presentation today aims at demonstrating the importance of core principle of procurement and how they interact with the rules and the practices of procurement officer this is very important when discussing stencil the stencil is the periods after the award decision and before the signature of the contract it is a period in which the contract may not be signed this is a key period for the communication between the contracting authority and the tenderer because it is actually when the tenderer will have the opportunity to raise their objection what is the objective of the stencil period the stencil period gives the opportunity to tender to raise requests commands provide information that the contracting authority had not known about the winning tender or raise an objection about the procedure in general in order to be granted a possibility of a remedy or review this is also a period in which the contracting authority has the opportunity to be made aware of such problem sometimes the information provided were not complete and the contracting authority just didn't know some information in this period the tenderer has a proactive role and should notify the contracting authority about any issue next slide please there are two consequences for such exchange of information during the stencil period as you will see 0.35 of the annex foresees that if information provided by the tenderer and lead to the needs of additional examination the contracting authority can suspend the signature of the contract in order to do this examination sometimes during the stencil period following the information provided and the suspension the contracting authority can come to the realization that indeed there are some flows in the procedure and decide not finally not to award to the first rank tender but to the second ranked tenderer this is the second paragraph on the slide it is also to be noted that when the flows in the procedure does not concern the tenderer that was winning the tender but the procedure itself the sometimes the contracting authority will come to the realization following the the information provided in the stencil period that the procedure might have to be canceled altogether next slide please the stencil period is very important the court has confirmed that the suspension of the procedure following additional information provided by tender should normally only occur if the information is provided during the stencil period this means that this is really the period in which the tenderer should communicate proactively and quickly with the contracting authority if they want to be in a position to request a suspension of the signature of the contract next please the interaction between the communication of the outcomes and the stencil period on one hand and of the principles of transparency and the right to effective remedy on the other hand is essential to really grasp the evolution of case law relating to interim procedure as we have said already several times the vasembreda case law has really been a groundbreaking evolution for a procurement practitioner as you know to be granted interim measure before the general court and before the court of justice an applicant should demonstrate that this application is urgent and this means that normally the application should cause a serious and irreparable harm and to the applicant however this is very difficult and isabel was just saying this a couple of minutes ago this this mandatory requirement was in this creating a barrier to effective remedy because the tenderer were never in a position to really prove that they would suffer an irreparable harm if the procurement contract was to be signed in the end this is why in the van breda case and the court of justice established that in the matter of procurement procedure in order to warrant the proofs the principle of effective remedy it was necessary to ease the condition of urgency the condition of urgency in the matter of procurement is fulfilled as long as the applicant can demonstrate serious harm and not a serious and irreparable harm it is important also to specify that this can only occur if two conditions are fulfilled the first one is it can only be the case in particularly serious prima facie case so the court is really putting an emphasis on the fact that this easing of the condition of urgency is only possible if we have a very very flagrant violation that really occur at first sight we are talking about a particularly serious prima facie case and not a prima facie serious prima facie case as is usually required in interim procedure also the the application should be lodged before the signature of the next case however as said many times transparent communication is key in procurement and as isabel mentioned the court has always as specified in in development of case law that the stencil period could only start running if those interested party the one that are lodging the application for interim measure have sufficient information to ascertain whether the award decision was unlawful in any way this means that if the communication was not sufficiently complete on the side of the contracting authority the stencil period could not run this is one why once again the commission always proactively insist that the services provide as much information as possible from the start the stencil period is also key because the general court has specified over the year that the condition of introducing the action before the signature of the of the contract should actually be understood as introducing the action during the stencil period this means that the stencil is really a key period for the tenor because not only do they have to communicate proactively with the contracting authority but also if they are faced with a very particularly serious prima facie case they should also introduce an action before the general court in the stencil period if they want to benefit from the easing of the condition of urgency and be in a position actually to be granted a suspension of the contract it is worth noting because isabel mentioned that it was a case in dcb and i think jeff will also mention this later on that the interim procedure before the general court is in principle not suspensive however in our practice we have noted that in the last years it is nearly systematical that the the general court will issue a preliminary order when faced with a procurement case to suspend the signature until the issuance of the order it is also worth noting that the procedure before the general courts in interim usually lasts between two and four months next piece as a conclusion i just want to highlight that in procurement it is always important to come back to the basic principle such as the principle of transparency and effective remedy that have been highlighted here i think it will be very interesting in the following year to note to see how the case law of the general court will still evolve on this matter as we have seen in the recent year very interesting developments and more and more the contracting authority being requested to provide very accurate information in a very precise way thank you thank you or somehow i don't know one can sense that you are involved in some of these cases there's always a glimpse of inside the knowledge which shines through the explanation of the case is very likely many things for that introduction and sides into the financial regulation approach to this information i go to the final last but not least speaker who will round off the two first introductions jeffrey diricks will speak about the legal remedies against the vote decisions from a national law perspective as a staff member of the national bank of belgium the speaker will first highlight the provisions of the belgium procurement law on the obligations of contracting authorities to inform tenders after an award decision the provisions are of course based on the eu procurement directive and therefore there are similarities to the procurement law of other member states as well obviously but they're not always directly furthermore the presentation will focus on legal remedies against the vote decisions on the belgium law with special attention for pre contractual remedies and here we may see some national features which deserve highlighting jeff will do that finally in the absence of a national law framework on internet internal review procedures the presentation will examine alternative methods for contracting authorities to increase transparency and monitor the legal soundness of their award decisions to the aim of avoiding unnecessary litigation with tenders a little bit on jeff jeff diricks is a senior advisor and head of group in the legal department of national about belgium on the belgium joint national bank of belgium nbv in 2011 immediately after finishing his law studies at the university of leuven carwell k u leuven currently he works as a senior league council and head of the corporate law division in the legal department of the nbv in 2017 jeff was seconded to the ecb on a short-term basis where he worked for the institutional law division so we both had the pleasure to work in closely with him of the director general legal services jeff is experienced in various areas of law in particular institutional law corporate governance accounting law fiscal law public access and public procurement within the euro system jeff is a member of the legal committee task force on value added tax issues vat issues and of the ethics and compliance conference a little bit on the background because it's important i think jeff has has also a musical career which he actually started with but he became a brilliant lawyer as well he's active as a musician a pianist and conductor and i experienced him live at the last meeting in this context of choirs among other musical activities he's the conductor and co-founder of the bank notes the choir of national bank of belgic staff members so noted on this musical note hand over to jeff thank you jeff for being with us thank you so much martin for this kind introduction i want to thank you and of course also chiara for the opportunity to uh to participate to this panel and also of course my co-panelists isabel and lora for their very interesting presentations it already has been a very interesting morning for me and i will try to complement it with some of my views on the topic as martin already mentioned i'm a staff member of the national bank of belgium member of its legal department and as such i will look into the topic from a national law perspective complimenting the previous presentations that tackled the the issue from from an e-u law perspective next slide please so my presentation is built up around four four four topics i will briefly highlight the legal framework and then of course first we have to discuss the transparency obligations because as was already highlighted very very clearly in the in the previous presentations when we speak about legal protection in a procurement law framework then of course we have to speak about giving information to tenders and in particular unsuccessful tenders because providing sufficient information really is a prerequisite for any form of effective legal protection third i will dive into the external review procedures that are available to tenders under belgium law and i will focus indeed on contractual remedies because those are the most effective and we will discuss a bit how the procedure the application for suspension is can be submitted before the council of state in belgium which is the highest administrative court last i will look into the internal review mechanisms of it this is of course because i understood that the the topic of today was inspired by the 15th birthday of the procurement review body of the ecb the prb which is of course a highly sophisticated internal review mechanism and when i worked for the ecb i benefited from seeing how it works from from the inside a bit and it was very very interesting we do not have such mechanisms in belgium we do not have it in the national bank of belgium and we do not have it in belgium law so there i mean it my presentation would be quite similar to to lauras presentation but i will highlight some of the approaches that we try and that we developed over the years as an alternative because we have some informal ways of providing some internal review which is not as formalized or as sophisticated as the prb but but which might serve as an alternative next slide please okay briefly on the on the legal framework i think a lot of you in the audience also work for national institutions so this will be quite similar to you of course procurement law is to some extent harmonized in the eu we have in particular two directives that are of particular interest for us the procurement directive and the remedies directive when we speak about legal protection and of course these directives are transposed in belgium law in various legal instruments which are shown on this slide next slide please so i think it was already highlighted and clarified very extensively in the previous presentations but i just want to come back briefly on this of course the starting point for effective legal protection is always the information that is given to to attend or once an award decision has been made there are of course transparency obligations in in the legal frameworks whereas in the ecb legal framework in the financial regulation and also in the procurement directives and in the national legal frameworks and the aim the objective is always the same in order to be able to assess the award decision and to be able to assess and evaluate its correctness and decide whether or not it makes sense to challenge the award decision in a review procedure of course a tenderer has to receive all relevant information and the necessary information to be able to do this next slide please so in the in the following slides i've summarized the obligations under belgium law i think for other legal experts working for national institutions this will be quite familiar because of course belgium law is based on the EU directives i would say and we can go immediately i think to the next slide the the most important message is that that of course the idea is always that a tenderer has to has to has to understand why a certain decision is made for example why his bit was considered non-admissible or why his bit being admissible was not chosen but another tender was considered more favorable so this this information has to be shared with a with a tenderer in particular a successful tenderer and in belgium we have we don't have a two-step approach as was explained before but we have a one-step approach at least for contracts above a certain value this doesn't apply for for very low value contracts but the general the general approach is a one-step approach where we immediately and proactively share all relevant information and even in in cases of tenders which which submitted success and an admissible offer even the entire award report this also this also applies to situations where contracting authority decides not to award the tender because of course the contract authority always has the option not to award but this also has to be a reasons decision subject to transparency obligations i'll come back to this later next slide please of course as already as was already mentioned by Laura there are some exceptions to the transparency principle of course some information might be of a confidential nature and then the contracting authority should not share such information okay so now we have shared all necessary information with a tenderer and then of course there is something very important that happens this is the standstill period we have this standstill period of 15 days under belgium law during this standstill period the contracting authority cannot conclude the contract and this period allows the tenderer to assess the correctness of the award decision if an application for suspension which is the pre-contractual remedy under belgium law is submitted then this standstill period is extended until the review body being the council of states in belgium has delivered its judgment there are cases when where no standstill applies i won't go into those cases in detail i think the most important to mention is that for contracts below the european threshold amounts there is no standstill periods which means that under belgium law for contracts below the thresholds the the legal protection is is is clearly less effective than for contracts above these thresholds next slide please okay so now let's stop so now let's talk about these this external review possibility before the the belgium council of states and in in particular the pre-contractual remedy because this is the one that is most effective that is most often used and which we also see in practice we of course have also other remedies other review procedures i will briefly mention them but i will not go into detail on them what does this mean pre-contractual remedies well it means pre-contractual remedy well it means that during the standstill so before a contract is concluded in the in the period between the award decision has been made and the conclusion of the contract there a tenor really can challenge an award decision and try to seek a remedy that will correct the infringement where the tenderer will still have a chance afterwards to to to be the chosen tenderer and to and to and to be to be the contractor with which with who the the contracting authority finally concludes the contract so this i would say is a is a very important feature of such pre-contractual remedy because it really allows a possibility of correcting an infringement so what do we have in belgium we have the possibility to apply for suspension or other interim measures before the council of states the time limits to introduce such such such application of course is the same as the standstill period it's a quite effective remedy in the sense that the council of states works quite fast and delivers its judgment in principle within the month after this submission of the application and it's also i would say the bar for unsuccessful tenderers to cross in this in this sort of procedure is relatively low in the sense that they don't have to demonstrate urgency or serious or irreparable harm it is sufficient that they that they demonstrate a prima facie case based on serious grounds or an apparent illegality i'll come back to this later so this i would say is our pre-contractual remedy under belgium law as i said we have other procedures we have the application for annulment before the council of states and then on the next slide you will see that we also have of course a possibility to apply for damages declaration of ineffectiveness and a review body can also impose alternative penalties but i will not go in detail on this next slide please so my conclusion would be that we do have under belgium law based on the eu directive effective pre-contractual remedies for tenders at least above those above the european threshold amounts stand still in combination with this possibility to apply for suspension a quick judgment it is sufficient to demonstrate a prima facie case of course and this is different from what is a bel clarified on the prb the council of states cannot substitute for for the contracting authority and it can only sanction manifest errors so it of course has to respect the broad margin of discretion contracting authority enjoys what would happen what happens if the council of states does suspend the the award procedure then the normal course of business would be that the contracting authority would revoke its decision and then of course the contracting authority can reassess the matter taking new award decision taking into account the judgment or decide not to award the the tender and possibly read tender next slide okay so this is we just discussed the external review mechanism under belgium law in particular the pre-contractual one as i mentioned before i will now highlight um what we can do internally as a contracting authority under belgium law um of course the remedies directive does provide for the possibility to uh to uh to cater for internal review mechanisms um but the belgium legislature did not transpose this so in in belgium law we do not have a legal framework comparable to the ecb with its prb procedure we do not have this sort of institutionalized formalized internal review bodies and mechanisms next slide please um you might consider this as a as a missed opportunity by the belgium legislator we also see that in other areas of law we do have these kinds of internal review mechanisms for example uh in education law we see that there are internal review possibilities against uh decisions of educational institutions we also have similar similar review mechanisms internally uh when it comes to public access decisions but we do not do not have it in procurement law um in the preparatory texts of the belgium legislator no reasons were given uh legal scholars did not comment it so we have to guess about reasons i put some in the slides but it's of course pure speculation uh perhaps reasons were of a budgetary nature because of course these sort of uh uh highly sophisticated internal review mechanisms also demand resources um as you know the the the the government structure in belgium is also quite complex so this might also have been an obstacle and perhaps the legislator thought okay um this external review mechanism before the council of state will already be highly effective we do we do not need an additional layer but this is of course all speculation um let's go to the next slides so but even in the absence of of a legal framework on the internal review mechanisms we as a contracting authority do try to seek um um alternative approaches and we try to seek ways to to reach the same i would say to in name of the same objectives as the prb of the ecb uh with a name of um of um of of providing additional guarantees for unsuccessful tenders enhancing good governance and avoiding unnecessary litigation so what do we do and i think i can refer to to the introducing uh introductory remarks of martin where he discussed the importance of efficient communication between a contracting authority and and and and and and and tenders and i can confirm that we see that this works in practice so uh if and i mean a crucial period as was said by lora is of course this stands still period um and if uh even unsuccessful tenderer approaches us during stands still period with additional questions after we already proactively provided the entire award report then of course we we answer such questions in detail of course taking into account possible confidentiality restraints but we really try also if necessary to enter in a dialogue with tenders and possibly also their lawyers to really try to make them understand why the bank made a certain decision and we already saw in practice that this might help to clear up misunderstandings and really avoid litigation afterwards of course if there is really a substantial substantive complaint by a tenderer against an award decision and we also try to look into this in a thorough way and and and and try to do um and as independent as possible second assessment of the decision and there we as lawyers from the legal department also are involved in this in this uh in this uh in this uh in this procedure and if we find that there is a manifest error um be um um with that resulted in the in the in the award decision in question then we will also advise our decision makers to revoke the decision and this is something that we already that we've already done in the past once our decision is revoked we can reassess the situation uh we can uh we can come to a new award decision or decide not to award the tender and possibly uh retender next slide please um this this was during standstill but also after the standstill and once an application for for for suspension has been submitted to the council of states also in this case we still do this sort of own reassessment of a decision so if a if an application for suspension is is introduced then normally the council of state would give us two weeks to prepare our defense and during these two weeks we do a reassessment of the decision we do with uh together with our external law firm and again if we find that there is a manifest error then we will proactively revoke the decision which means that the procedure before the council of state becomes devoid of purpose and will not be continued and we avoid further litigation and the council of states doesn't have to look into the matter uh on substance then of course again we can reassess the situation uh taking new award decision or decide not to award the tender in question next slide uh no uh let's go back perhaps to a previous slide because I just wanted to tell you uh that that this is something that we also did in practice quite recently so there was a recent tender where uh two tenders were very close to one another uh they were separated by less than a point in uh in on a scale of a hundred points um and so in the first award decision we awarded the contract the the award decision was in favor of the the tender that was ranked first uh in a on the basis of a first assessment this was attacked before the council of state by the tenderer that was ranked second and when we looked into this um uh into this um application for suspension we found that there was indeed a procedural flaw on the basis of the first award decision and we revoked that award decision then afterwards we reassessed the matter and we made a new award decision this time uh awarding the the tender to the tenderer that was ranked second in the initial award decision of course this provoked a new a new litigation procedure before the council of state by the tenderer that was initially ranked first but this time we were convinced that the second award decision was the right one was correct we defended it before the council of state and the council of state upheld this decision and implicitly um showing appreciation for um for our uh for our approach where the bank the national bank um admitted the the problem that existed with the first award decision revoked it and then reassessed the matter um and came to another uh conclusion okay I think we can go to the final slides these are just some I would say thought experiments that I added um I will not go into these in detail um I think I can I can end my presentation here I would like to thank you all for your attention and I can give the floor back to to Martin thank you Jeff that was really nicely rounding off for this panel we have now looked at the PRB um mechanism which one institution has and which we have at the ECB we have the context uh on the uh from the European Commission as the experienced let's say um litigator on on many cases also explaining how the framework is for all other institutions and Jeff you gave them the framework and rounding off out on the national side and how you also try to of course have a very effective remedy process in version we will now have a few minutes for questions we are we probably have five to ten minutes and we try to finish up by five past and I will take my uh chair privilege and ask uh to all three one question actually to Laura and Jeff one and to Isabel another one uh Laura and Jeff and maybe Laura can start would you wish to have a PRB procedure and to Isabel I would like to ask would you wish to not have a PRB procedure Laura can you what what's your view well I'm excited I've been very impressed by the presentation that Isabel gave on on the PRB and it seems to be a very effective mechanism in the sense that it is quite time efficient but also it really gives the contracting authority the opportunity to review again with a different information the one provided by the tender and the award decision however from a practical point of view and I think that Jeff hinted the matter earlier in his presentation I say that I I don't understand how you manage to to review all this procedure internally because in terms of resources and and in terms of of workload this must be enormous when I see the implication that the litigation required before the court reviewing all this procedure must require a lot of expertise and a lot of time and and and effort thank you Laura Jeff what do you think I fully agree with Laura I I'm perfectly willing to to to take on this challenge of a PRB in Belgium but then I will have to convince my board to strongly reinforce my team this is this is I would say a first point of thought I really see the merits in in what the ECB does and I really think it provides for an additional guarantee of an additional layer of of legal protection however I must say that I'm also convinced that what we do in Belgium on a more informal basis also works and that we also see see the results the results of this thank you Jeff the informal communication after the award is really key and and as Jeff mentioned we have seen many cases where just an improved communication as as lead to avoiding litigation very effectively which is what we all strive for Isabel would you not wish to have a PRB being the secretary and in charge and you heard Laura and Jeff we need more resources yeah if you ask for my personal view of course you know wanting to go on vacation and receiving several appeals at the same time I would say I don't want to have this procedure in place anymore at least I do not want to be the secretary of the PRB but for for legal reasons it's true what you say that also in an informal manner lots of these cases can be solved so if sometimes we receive requests where it says this isn't a PRB appeal but in reality it's a request for an additional information and so also on that we interpret it then as an additional information request and if we get in contact with a tenderer and speak and explain more the background sometimes these cases are solved and on the other hand it also helps to have a formalized procedure and with rules of procedure and which and and to be sure that it's really an objective procedure so I think and seeing also having a look at the figures we can see that 10% of the appeals that we received were really appalled and and a lot of a lot of appeals didn't go through the to the court of justice so I've been at the ECB for 15 years and I've always most of the time I've been in this procurement law team and I've only had two court cases on on general procurement related matters we had a couple of court cases earlier which was related to the building of the new ECB premises which was perhaps a little bit different but yeah I'm convinced so for transparency reasons and to to provide an effective remedy before bringing the matter to the court this is a very good mechanism thank you I would then give the floor to the audience and I understand we have questions from two participants we have at the moment 90 participants in this in this panel there's been a loopy first and then Anjeliki Mavromati please there's been a there's been a gun not then I would give the floor to the next one on the list Anjeliki Mavromati there she is good morning hello Anjeliki do do you want to yes yes can you turn your video on thank you okay you might be able to see me now hopefully okay so first of all thank you to all the panelists for the very interesting and informative presentations it was an insightful morning indeed as a follow-up to Martin's question I was wondering given the differences between the PRB and the ECB's approach to the process and then the two-step approach in the Commission and the Belgian one-step approach we just say that the system you have established works best given the specific characteristics of your respective organization and based on your experience so far do you recognize specific advantages and disadvantages on your established internal review procedure so the question is related to either of the panelists I would say okay good then maybe let's start Laura and Jeff well thank you I must say that on personal note I'm of course not representing the Commission when saying this I would consider that if one-step approach is more efficient and maybe more clear and this is also why we strongly recommend to the service to always go for this approach providing proactively all information from the start not only on the loose on the tender that has lost the tender but also on the winning tender and its relative characteristic and advantage it's maybe more clear and also dismiss the whole discussion on when the standstill should start so this is definitely something that that could be maybe improved and then the second part of the question was okay can you maybe if you recognize specific disadvantages or advantages and if it works best given the respective characteristics of your organization thank you I think that having and not having a PRB works quite well at the moment for for the Commission and I said and the communication and the informal communication mechanism is actually quite helpful we have a lot of interaction with standard I think what what is important is that the legal service is involved as soon as there is some question that are raised in the post award phase usually if the legal service is involved and we managed to to have a proper communication and a quite comprehensive one with with a tenderer most litigation can be avoided thank you Laura Jeff any comments from your side yes I would I would agree with Laura I think that that's giving proactively as much information as possible is the is the best way to tackle the issue to avoid litigation and to really to really make tenders understand why a certain decision was made so I think our approach we are I mean we don't we do not have a choice because under Belgian law we are obliged to to apply the one-step approach and to even share the entire awards decision the entire award report I mean in this way we already give information that would be asked for anyway in a second step so I think we also win time by doing this and yeah I agree with everything that was said previously by Laura and Jeff and I would just like to add that one of the reasons why we also changed our approach not just because of fanbreda forced us to is that we felt we should give out more information in the in the first step right in the in the outcome letter for transparency reasons and and I think that's the way it should work and of course another step more which is the the fact and which are you are doing in the commission and in in the national bank Jeff is that you give out even more information than you would be obliged to and that would be something that we would consider as well of course always weighing against the administrative burden and yeah and also wondering whether it would prompt more requests than less but that's of course another matter thank you thank you Isabel and thanks for the question Aniliki I see that Despina is now online Despina you had another question hi good morning I hope you can hear me now and first of all thanks to everyone for the very interesting presentations they were very concise and informative and I would have two short questions mainly for Isabel and you mentioned that the PRB the internal procedure is only available for public procedures after all of these years of experience and since you I think we can conclude that it is an effective way to reduce litigation in front of the court would this be an argument to extend also the scope to non-public procedures this would be my first question and a first a second small question would be if you can give us a practical example where a tender brought a procurement case before the european albutzman since you mentioned also the albutzman in your presentation thanks a lot am I unmuted now that's great thank you Despina yeah to your first question whether it would also be feasible to to apply the PRB procedure to non-public procedures yes definitely if we get more headcount I would say this would be a possibility we only have one month to to respond and to issue a PRB decision and so it's a quite tight time frame and if we applied this this procedure also to the non-public procedures all the three five quotes that we have even call off procedures it would be very difficult to manage so I suppose that this was at the time when the PRB was established one of the reasons why we only allowed for it in public tender procedures so it's basically it's practically or mainly a question of of effort as well and yeah your second question european albutzman so the request to the or complaints to a european albutzman can always be submitted in in addition to an appeal procedure and I don't I don't have I didn't have one myself and it was we had we had already complaints to the european albutzman but it was more related to employment law matters so on procurement law matters not that I remember so it's rather if if tenders want to complain it's more a few procedure or court and yeah so hope that answers your question thank you very much it's a very thank you it's been a for the for the question I'm looking to the list there are no further questions from the outside well I would have loved to have a still a conversation amongst the panel members but looking to the to the watch I think we a bit 10 minutes over so probably rather close also for the benefit of the following panel many thanks for the excellent contributions I start external Laura Andre bravo thank you for also defending the interests of the european commission and in these cases and making sure that the court makes more good case law on this excellent presentation many thanks isabel also for the is it 15 years of pab secretariat maybe not fully but yeah almost involvement in this for very long and jeff for really blending into this and and finding a place that really rounded off it was really as we hoped this this presentation and giving the kind of national aspect of this thing of the of the of the procedure that we have in relation to the tenders and the access for information I would also like to thank Monica Mayutas who was here in charge of the of the clicking the the the slides and all the setup and touch it's a lot of church is also in the room we also have a job a job at a gap from the vex team and many thanks to the whole legal services team who made a lot of work to make that happen thank you very much and we close the panel for here and have a little break until 11 o'clock when panel five will start thank you for attending goodbye


It is my pleasure to introduce our next lecturer, Donata Giglio as an assistant professor at the University of Colorado. She worked as a postdoctoral scholar at the Scripps Institution of Oceanography, and her interests are mainly focused on large scale ocean dynamics, ASC interactions and geophysical fluid dynamics. And one of her favorite foods is sushi. Donata, I'm looking very forward to your talk. Thank you, Judith, for the introduction, and thank you to you, Anisha and the program for inviting me to give this lecture. Hi everyone. Today I will be talking about ocean observations for S2S. Before I start, I would like to thank all that contributed to the studies that are cited in these presentations, and if you are interested, there will be references in the slides. So we will start with some basics on why we care about ocean observations for S2S. We will then discuss what observations are available versus desirable. At the end, I would like to briefly discuss uncertainty quantification in 3D oceanic fields that are based on observation. So the goal is here to improve understanding modeling and forecasting of S2S variability as you have heard last week and also today. So observations, as we will see in this talk, are important for this, as they provide the ocean is a source of predictability. The community has seen that coupled atmosphere ocean modeling and data simulation may be actually key to forecast S2S variability. It's always important, as you have heard at the beginning of last week, as many critical decisions need to be made several weeks to months in advance. For instance, for naval and commercial shipping, we want to be able to design the shipping routes weeks in advance to take advantage of favorable condition to avoid hazards for drought forecast. It's important for farmers to know in advance about the forecast so to make the best decision in terms of which seed variety to use. Finally, one last example, water resource management. The decisions about reservoir levels have to be made weeks, months and seasons in advance of the actual usage of the water. We all know that we are facing a climate crisis. And as a result of this, there are more severe and frequent extreme events that are developing. NOAA has recently published a report that underscores the importance of ocean observations for forecasting, including for forecasting at S2S time scales. And quoting directly from this report that was published in May, NOAA states that NOAA must embark on a new way of doing business by fully integrating the ocean observing communities with its weather, water and climate programs in order to provide society with state of the science actionable data information and services. NOAA goes on to elaborate on this and to point out how the insufficient integration of the ocean observations in numerical models is a clear limiting factor for improving the skill of earth system prediction. And this is across time scales, including S2S. In this report, they show one summary graphic for the importance of integrating, better integrating ocean observations in model and data simulation efforts to predict, to improve the forecast on different time scales. In particular, they show this graphic where on the horizontal axis, you can see time here, specifically on the vertical axis, the number of events that are billion dollar disaster events in the United States during the period 1980 to 2021. And on the right, you see the cost in billions as a consequence of these events. One thing that the NOAA report points out and I would like you to pay attention to looking at this graphic is that 2020 was the sixth consecutive year in which 10 or more billion dollar disaster events have impacted the United States. This is clearly a very important topic for society and NOAA in this report describes the strategic goal to improve NOAA earth system prediction to provide more accurate and I quote reliable and timely forecast across time scales. And to do this, they highlighted the focus of closing gaps in ocean observations and also better integrating the available ocean observations into the medical models. Here on the right you see a schematic of the strategic objectives that are described in the NOAA report. And you can see how aligning and enhancing and maintaining ocean observation is one of the key points on the right. But let's think about why this is, why this focus on ocean observation. So the subsurface ocean stores and releases heat and this is the key reason to understand the important role that the ocean plays in regulating climate. The ocean is a major reservoir of energy for both weather and the climate system. If we want to think about some numbers and think of an example that in one sentence describes this if we think about the top 100 meter of ocean, top 100 meter of ocean at 40 times the heat capacity of the entire atmosphere. And you may have heard of the heat capacity of the ocean in the context of climate change. The climate is changing as we're all aware due to the fact that the input, the energy input to the climate system is greater than the energy that exits the climate system. But where does this heat go? The oceans actually take up more than 90% of the energy that is added to the climate system by humans. For comparison, the atmosphere takes up 2%. You may have seen this schematic on the right from the IPCC report where on the horizontal axis you have the ear, on the vertical axis you have the energy. And this graph shows how the different components of the climate system, the energy stored in different components of the climate system has changed over time. You can see prominently how most of the energy has been stored in the upper and the portion. So these would be the light and dark blue shading. But let's think of an example that has been discussed also during this workshop for S2S forecast. Let's think about where tropical cyclones get their energy from. Tropical cyclones get their energy from the ocean. We have that water evaporates and latent heat goes into the atmosphere. Once this water condenses, this heat is released and fuels the tropical cyclone. And if there is a nice video that explains this mechanism and you can easily find it either looking at the link down here or literally googling where the hurricanes get energy, get their energy from. This is a NASA video and I encourage you all to look at this animation. We can see the effect of the ocean changes as the tropical cyclone comes through. Here we have one example for tropical cyclone Harvey in 2017. We have a map on the left with the track of Harvey and then the color is the maximum sustained winds in knots. And you can see that there are also these magenta stars on the map. And these are ocean observations from the Argo array that are collocated with the track. So these are Argo profiles basically. If we look, if we focus on the Argo profiles that are available next to the track, close to the track in the Gulf of Mexico. We can look at how the ocean changes between before and after the cyclone. And this is actually included here on the right. So on the right we have a graph with temperature on the horizontal axis pressure on the vertical axis and you can think of pressure in decibel as depth in meters more or less. And what you can see is that if you compare the profile before the tropical cyclone with profiles after the tropical cyclone so you clearly see this cooling. And if you're interested in learning more about Hurricane Harvey, I encourage you to look at the publication by Trenberg et al. in 2018, which describes all the observations that are available during Hurricane Harvey. And if you're interested in more in general to look at how the ocean changes as the tropical cyclone comes through, you can look at this kind of plots for other hurricanes using the Argo with web app and database and the Python notebook that is in this that you can find in this in this GitHub repository down here. The ocean is very important to understand how tropical cyclones evolve. And to improve forecast of for instance tropical cyclone energy. Here, I wanted to show I want to show one study that compares the anomaly correlation skill of forecast of tropical cyclone energy at day 16 to 45. So what you see in this map are different regions and then bars that represent the anomaly correlation skill at this. And the different color of the bars indicates a different kind of experiments. In one case, the orange bars represent what happens, what the correlation skill is, if we initialize the ocean using all the observations available in the system. And then we have the green bars that instead show what happens to the skill if we do not include in the initial condition, the information from Altimetry. And you can see that in almost all cases, the orange bars are taller than the green bars indicated the importance of ocean observations to better initialize the ocean and achieve a better forecast. Another example is for seasonal forecast in the tropical Pacific, specifically a seasonal forecast of El Nino. We are all familiar with the normal conditions in the tropical Pacific and specifically the warm water volume in the western Pacific and different studies have showed that the recharge and discharge of this warm water volume in the tropical Pacific is closely related to ANSO variability. So accurate observations of the evolution of this warm water volume can help to improve ANSO process understanding, first of all, and predictions on seasonal time scales. This is one example that we have heard last week about to ocean observations are key to better understand and represents in models RC interactions. We have learned about the MGO and the propagation of the MGO, particularly through the maritime continent seas and understanding those interactions in these regions are key to understand the propagation of the MGO through the region. And this is a part of the world where we haven't had historically a lot of observations and the international program here so the maritime continent has provided for the period 2017-2020 long needed observations in these regions that are helpful to better understand RC interaction. Finally, I want to mention sea ice related predictions. These need to have good initial conditions in terms of operation currents, temperature surface radiation and it fluxes surface wind way by. We need observation at the edge of the ice. We need observations in the marginal ice zone and increasing the amount of observations we have in time will allow us to do a better job at predicting extreme events like the unprecedented springtime retreat in Antarctic sea ice in 2016. We have a schematic a graph showing that here on the bottom on the horizontal axis we have the month in 2016 on the vertical axis we have sea ice extent. And if we compare the the mean for the period 1979 to 2015 which is the black line with the red line that is for 2016 we see this unprecedented springtime retreat which we would want to be able events like these are events that we would want to be able to do a better job at forecasting. So to summarize, we really need ocean observations to understand the interactions and the rule of interaction in S2S variability and prediction. We need ocean observations to improve S2S forecast in high and mid latitudes, including sea ice predictions as we just discussed. And also the NOAA report that I mentioned earlier, highlights how the ocean is among the most poorly known and understood components of the earth system. And this is largely due to the sparseness of the data to the difficulty to do measurements in certain regions or in certain oceanic layers. So what observations are needed in light of what we discussed so far, we want observations of the ocean atmosphere interface and we want these observations to be simultaneous measurement of the ocean and the atmosphere. We want observations in the subsurface. Just a few examples, upper ocean current and temperature, mixed layer depth and properties, ocean boundary currents and we want a resolution observations in ocean boundary currents which provide a source of predictability for S2S variability. We want measurements of way height in and around the marginalized zone. We want observations below the sea ice. We want observations that capture the processes of sea ice growth and retreat. And we want observations of sea ice thickness and concentration. So what observations are available at this point in time. Here are some examples. We have, for instance, moorings that provide a lot of information with a high temporal resolution, yet they have the limitation of being observations at a single site and maintaining the mooring is expensive. This is particularly challenging if we are in a region where there is a strong current, but moorings are a key source of information for Fc fluxes, for instance. They are a key source of information for the urinal variability because they provide the resolution that is needed to describe the urinal variability. We have gliders that are a key source of information for regions where we need high resolution measurements. I mentioned earlier, boundary currents and gliders are very helpful at doing that. We have, more recently, we have had sail drones. And these instruments, as we will discuss in a second, are very, very useful. For instance, to supplement the information in regions where we have moorings to provide information about the gradients between moorings. Surface drifters, they provide a view, a global view of mixed layer currents near surface temperature and sea level atmospheric pressure. We have ship data that although they are sparse, they are concentrated along specific lines and the sampling along these lines is sparse in time. The sections are repeated only every so often if we think about, for instance, the Go ship program that is a key resource of information for the global ocean. This kind of network is key, not only for the measurements that it provides directly and can be analyzed to study climate changes, for instance. But also these are key to provide a gold standard to assure that arrays like the Argo array of profiling floats remain at best quality, provide best quality data. The Argo floats have provided an unprecedented coverage of the global ocean in space and time. Here, if we look at the map on the top right, these are dots that show the location of Argo profiles collected in a three-day window. And just looking at this map, you can realize the magnitude of the revolution that Argo has initiated. Obviously, satellites are also key to gather information on a global scale. They have provided unprecedented coverage, especially and temporarily. So we have these observations and we shouldn't give them for granted if we think about what we had in the past. For instance, if we think about what we had for temperature casts per one degree box where if you think about casts that profile at least to 900 meters and we put together all the casts available from the world in the world ocean database. For the years before year 2000, we end up with a map like this where the black and we see a lot of black and blue dots in this map indicate that there are only in the case of the black color one cast per one degree box in the case of the blue color two to five casts in a one degree box. So all the observations available over time up to year 2000, especially in the southern ocean, there is a scarcity of data as you can see from this map. Even if you include the initial years of Argo you still see that the southern ocean is not well covered. And now if you compare instead with how much observations we are available, we are able to collect now in particular these are the number of observations for years 2015 and 16 so in only two years. We can now do a much better job at observing the ocean on a global scale, and especially we can do a better job in the southern hemisphere. So finally I want to show the casts from glider data. I mentioned earlier how gliders are very important to observe gradients in boundary current systems. And these are the gliders in the world ocean database, and we can see where these data are available. I mentioned about sale drones and how promising and useful these instruments are these instruments are autonomous surface vehicles that basically sale in the ocean and some power that the instruments on board are some power, and this can be used to target specific events if we think about a linear development. These sale drones could capture could could make measurements during the linear development and and as I mentioned earlier they can supplement other sources of observations, they could cover gradients in between fix the moorings, for example. So we want to make to increase the use of sale drones that are of course the sale drones have been used in specific projects and studies we want to increase the use of these observations, which can the sale drones can provide both observations in the surface of the ocean and in the ocean and in the atmosphere. So this collocated information simultaneous observations of the ocean and the atmosphere are very important. Also, we want to think about new mission and concepts to enhance the range of information that are available for from space. The current satellites do not provide measurements for all the surface variables that are needed to estimate RC fluxes. And I mentioned many times today, how important it is to have simultaneous measurements of the ocean and the atmosphere to have a better descriptions of different types of fluxes and concepts like the butterfly concepts are proposed to move to advance in this direction. Also, satellites, most of the satellites measure at a couple of times per day at any fixed location. And we want to have instead multiple satellites that have similar objectives that measure at one location for a different day so that we can describe the urinal variability and the scales of fast moving synoptic storms. Finally, I mentioned we need more observations, many more observations of the marginal ice zone, more observations at the edge of the sea ice to better understand sea ice growth and retreat. Okay, let's see. I have a few more minutes and I would like in this remaining time to discuss a bit about one more use of optional observations beyond what we have heard a lot so far in terms of assimilating these observations in data assimilating models. So obviously that's a very important part, very important use of ocean observations. Another way ocean observations are useful is to produce gridded products that can be helpful to assess oceanary analysis and improve the initial state of the ocean for S2S forecast. As Magdalena and other speakers have described, this is of key importance to improve S2S forecast. So when we have, when we think about Argo profiles for instance, I mentioned how Argo has provided unprecedented coverage in space and time. We have more than two million profiles. How do we go from these profiles to not only gridded products, which we have quite a few out there, but gridded products that come with meaningful uncertainties. How do we do that and what are the challenges? So if we think of the example of field of ocean it content estimates, that is of key importance, obviously, because we know that as I described in the beginning, we know that the ocean has taken up more than 90% of the excess energy in the climate system. So we want to be able to do a good job at estimating how the ocean it content changes over time and estimating the uncertainty of the ocean content. The challenges are that there are different sources of uncertainty and not all of this and not for and we haven't yet produced the products that take into account all these different sources of uncertainty. So one, a certain sort of uncertainty is the quality control of the data. And as Magdalena pointed out, this could be a problem also for assimilation of the data into a model if bad data are assimilated that will degrade the what we can tell using the model. There are errors in bias correction for some data. And one example is for the XBT data in the past. We have, as Magdalena mentioned, special temporal data sampling density that changes over time. This is just one schematic that shows on the horizontal axis, the year on the vertical axis, the number of casts per year. And as you can see, as the Argo array and the glider kick in, we have a large increase in the number of available observations. And this is challenging, not only for data simulating system as Magdalena described is also challenging when producing maps. Another key component to this that we need to consider is when we map these observations, one of the first steps is to generally to remove a baseline climatology the choices that we make in for that baseline climatology have impact on the end product and have impacts on whether some of the assumptions of the method that then we used to map the residuals are met or not. But let's have a look for this. For the trend, for instance, of the data for oceanic content. This is a very relevant. Very important question. What's the trend in oceanic content and this trend. If you use cross validation, you can show that not including the trend in this baseline climatology leads to an underestimate of the actual warming trend. So the choice we make have are very important for the end result. Also, the choice of mapping method, you could think of going with a very simple box average versus objective mapping this in as impacts on both the estimates and even more on the usability of the data that you can produce along with that estimate. You can make choices about how to estimate the correlation scales and this again as implications on on what happens next. Finally, one source of uncertainty is the special temporal correlation of the measurements. And in a recent work that I'm doing with collaborators. We have seen that using local conditional simulation can help to provide uncertainty with the estimate that are taking into account the special temporal correlation of the measurements. So to summarize, we need to design and implement low cost I value surface and subsurface ocean observations. And not only that. So we need to, of course, maintain the observations we have, we need to enhance these observations adding more measurements, especially to capture. the observations in regions that are sources of predictability, but also we need to develop forecasting systems that are capable of extracting the observations potential in forecast applications. We want to take advantage use what we know from the models to also guide the design of observing system, we want to combine the information from the observation of oceanography with the modeling community to design the best observing system for the future. The, as I try to highlight today. SOS forecast will benefit from high special temporal resolution in regions that are sources of predictability and these include coastal regions, tropical regions and polar regions. I made the example of gliders and how useful they are in boundary current systems to capture strong gradients for instance in sea surface temperature in your surface temperature. We want to think beyond the current technology and advance current technology to observe the ocean atmosphere interface simultaneously using autonomous vehicles but also potentially doing this from from space. And finally, we discussed briefly about observations based gridded products and the importance of producing observation grade based gridded products that have meaningful uncertainty so that we can assess ocean analysis and improve the initial state of the ocean which is important for SOS forecast. So we'll stop here and take questions. Thank you. Thanks very much. Very interesting aspects about ocean observations.