Hello, everybody. I'm Dan Jurgen. I want to welcome you to the session on the Geo-Economics of Energy. Both welcome all of those of you here in the room with us and all of you who are watching this on the web. Obviously, this is a very timely subject that's demonstrated by the fact that this room is packed and there are a couple hundred people outside who are still waiting in line to see if they can get in but will join us on the web. So when we're going to talk about the Geo-Economics of Energy, we're really talking about the new Geo-Economics of Energy because so much has changed and even the discussion from a year ago, you know, it had been commonly understood that the last 10 years that demand was all going to be, the growth was going to be in the emerging markets shifting to Asia. But there are big questions now about the nature of demand. And of course, the supply picture has changed. What happened with the development, the disruptive technology in North America of shale gas and then shale oil has had a profound impact, changed the balance in the market. And we haven't seen a increase of this nature in this fast since the early 1980s. And so this has posed major questions that we'll be talking about. Even as these supplies came in, they seem to be balanced out by disruptions in the market. And it was assumed for about three years that $100 a barrel was the price of oil forever. We've seen that that's not the case right now. There's no shortage of geopolitical risk. We had seen a great deal of geopolitical risk in the price. Now there doesn't seem to be any, even though there's plenty of geopolitical risk. So we might say, while there's a surplus of geopolitical risk, there's a greater surplus of oil. So we'll talk about the oil price collapse, why it happened, what the impact is, what it means for countries, what it means for companies, what it means for the world economy, where things will be going. We'll also be talking about geo-economics in terms of natural gas and in particular European gas. And the way we'll organize the discussion is that we have a very distinguished panel and I'll ask each of them a question on kind of their topic areas to get the discussion going. And then we'll open it up and looking forward very much to a vigorous discussion among our panelists. So I'd like to start with President Gruber-Skyta of Lithuania. She had been a European Union Commissioner. Lithuania two years ago held the presidency of the European Union and in December they took the first shipment of gas to their new natural gas floating regassification facility called Independence. So I'd like to ask you, Madam President, why you called it Independence and how you see the issues of gas security in Europe. I think the panel is unique from the point of view that we here are consumers, the producers, and some who were recently 100% dependent on monopolistic supplier. And for all Europe, really the new challenges, new geopolitical situation in energy sector and especially in gas crisis gives new challenges for further and faster creation of energy union which European Union is trying to achieve now because for Europe it is partly slow and sluggish growth and mainly because of low competitiveness and especially because previously of very high dependency on one monopolistic supplier and very high energy prices. So now tendency is a little bit better and my country was one of recent countries in Europe which introduced energy package very fast, means unbunding reform of energy sector and we dependent on gas problem supply 100% and now with the new facility, practically all free Baltic states, not only one, but all free Baltic states can receive and be independent from this monopolistic supply and now our prices are really not take or pay but related to their different formulas, for example in our case it's a British index and this allows us with the changing of prices in the world to reduce and to renegotiate and to change the prices for us also each free months even. It depends on situation in the world. So from this point of view, Europe today is eager to proceed with the internal market finalization with a better gas, especially joint gas market which is not yet finalized. Reverse questions of gas supply in Europe is still under question and why this new challenges, new geopolitical situation with the prices gives us some better chances to invest in other facilities especially for Europe it is important that we are targeting also non-conventional energy resources by 2013 our target is close to 30% of alternatives in our mix of energy and the new phenomena or let's say shell gas revolution which we are seeing in the world will also very much affect our capacity and the LNG terminal buildings in Europe. So from this point of view I think that the game changed forever especially game change for the countries who used not to invest into technologies but only to export, passively export their resources and for the countries who imported and investing in new technologies, new energy sources are gaining and addiction to the gas pipelines already becoming a huge burden on progress and these countries who are still addicted to the pipeline only will be not developing its resources and potential fast. So for Lithuania as example, small country no any natural resources really, chance which we took and took it on time and very fast we build this terminal in three and a half years and as I said we fully will support our needs and we capable to help our neighbors in Baltic states and in Poland if there will be necessity so this new changes is only I will say a small example how today's energy global markets are changing and the countries, especially suppliers if they will be not able to run after their changes at least to be with them together will fail a lot and I don't think that in two or three years we will be at all able to think or talk or see their prices of gas or all as we used to see more than $100 so I think that these changes what we see now are practically irreversible. So well, sitting right next to you is a country that exports gas and oil. So first deputy prime minister for Kovic if you'd like to A, comment on that and B there's been much discussion about what the impact of lower oil prices is on Russia in terms of economic growth, in terms of budget and many different opinions are expressed but you would have a very authoritative position so I'll let you take both of those. Thank you. It's a good day in Davos very sunny day and I hope that our discussion will not destroy it. Yeah. And well first on gas, the gas market is one of the most active markets in the global economy and it is clear from the global energy balance perspective that gas will remain one of the most important energy source for the world economy throughout 21st century and the demand for gas will only grow in the upcoming decades. The structure of this demand will change over time and some regions of the world will be in higher demand for gas, in some regions the demand will either stable or grow very slowly like in Europe since European economy is not flexible enough and not vibrant enough to have increasing demand for energy at the rate like Asian Pacific region but again in the overall energy balance it's clear that gas will not keep its leading role and for all producers whether those producers are in the United States, in Europe, in Africa, Australia or in Russia will be a place in the market and probably all major producers will continue increasing production. Point number two, those investments are huge of long term nature. So to provide on a sustainable basis gas at reasonable prices to the world market you need transparent and clear market conditions, predictable market conditions and predictable rules of the game and we are not just a supplier to the market, we are partners. And on the basis of partnership we can achieve sustainable long term stability at the gas market. We are dependent on our clients as much as our clients are dependent on us. This single important thing that is very important critical to understand if we discuss the gas market. We are okay with any country going away from monopolistic nature of gas from us and again the market is quite big and we will find a place and we are shifting some of our, some of the focus to Asian Pacific region, China in particular but not only China, other countries as well. But in every aspect of this activity the long term relationship is a very important factor and we hope that all countries, all our partners will understand it. And the point number three about alternative energy and alternative sources including LNG and in spot market it comes with the risk and it comes with higher prices. So we need to find the right balance between the price and stability price and safety for our consumers. So for Europe alternative gas will be more expensive. The situation will be less stable. Solar, wind, hydropower, all those things are going to be more expensive than Russian gas. It's okay for us if European taxpayers are ready to pay higher price but we need to talk about this openly and politicians should explain to people why they give their taxpayers higher prices not lower prices. On implications of lower oil prices, implications are big and our balance of payment is adjusting to the new equilibrium. With oil prices of about $50 per barrel we have exchange rate adjustment, huge exchange rate adjustment that will lead to lower imports and those trends come with higher inflation and difficult situation for many people but also it comes with the potential since potential for import substitution with lower ruble exchange rate, potential to promote exports from Russia not only of commodities but other products is bigger now and exchange rate competitiveness is much higher and we will use it. We will use all possible instruments to develop our technological base and capability to export more high quality products to the rest of the world. Does your sovereign wealth funds give you a cushion here? Yes, it gives but the important thing is to use this time to diversify even more sufficiently to be prepared for the next crisis. I know President Krupp-Skyder wants to comment. Let me just offer a very short comment about the partnership and reliability of supplier. We will be happy to have a reliable supplier for Europe in general but our experiences, historic experiences says very clearly that always energy prices, at least from monopolists, Gazprom was used at all unreliably. It was at least for Lithuania always 30% more than Germany was paying recent years and it was used as a political pressure and political tool. So from this point of view, such a liability absolutely counts into price and we can explain to our people and why our energy terminal is named independence. Right, thank you, okay. Exactly. We'll return to this subject but I think now we're gonna turn to oil. But we'll be back to gas. Last November, the OPEC countries came to what is seen as a historic decision and want to turn to the Secretary General Abdullah Al-Bhadri of OPEC to tell us where the decision came from and why it was made and how you see the implications today. Thank you again. I was here about, I think four years ago and everybody was asked me and it was grilled because there is some shortage of supply. Everybody's asking why we don't increase production. Now it is the other way around. Now you're gonna grill me for cutting production but I will take it. Which do you prefer? You know, our ministers met in Vienna and they debate the market situation, the economic situation of the world and for three, four hours and at the end we decided that we're gonna keep our production. The decision was a collective one. Everybody agreed with it with some reservation from some ministers but at the end of the day everybody was agreeing to this. Now the problem because yes, there is an oversupply, there is a lower demand but OPEC did not increase its production for the last 10 years. We've been producing 30 million per day for the last 10 years. None OPEC they've been producing but they increase the production by about seven million per day. Now this seven million per day it is a very high cost. They need a price of $100 and more. So how, you know, we thought maybe none OPEC and OPEC they get together and solve this problem together. Now it didn't work out. So we decided to keep production as it is. Now if we, the problem at that time was the price was declining very fast. The fundamental exchange does not deserve this 40, 50% decline in price. And we said, you know, if we don't know how much we're gonna cut, is it one million, is it two million, is three million, nobody really knows at that time. And if we cut in November, we have to cut again. Maybe in March or in June.  But they don't replace us with a normal price. They replace us with a very high price. So we decided to keep it, you know, to keep our production and see how the market will behave. By the way, this decision was not directed to any country. It's not directed to United States. Tite oil is not directed to Russia. It's not directed to any country. It is a pure economic decision by our ministers and we supported it. Do you think at the time people sort of thought the price would start, stop at 70 or do they have a view and a picture in their mind? No, the price was declining very fast. The price declining was not looking at the fundamentals. No, the price was coming down. We know the price will come down maybe to 40 or lower. That's why we really, it's a problem for us. How much we're gonna cut? Right. Well, let me ask Fatih Barol, who's the chief economist of the International Energy Agency, following from what the secretary general said. What do you, one of the aspects, I think, in your decision was the weakness of global demand for oil? There is another supply and there is, yeah, the economic growth was very cloudy. We really cannot see it is growing. And so that's affected the demand. The demand was less as we forecasted it and maybe also the IAA forecasted it. And also there is a more supply coming from non-OBIC, from all of it coming from non-OBIC, nothing from OBIC. So Fatih, let me go from that then to say, what do you think as an analyst has been going on with world demand? Why has it been weaker? And as you look out in the perspective of the IAA, what impacts do you see of the turbulence in the oil market today having on future supply and demand? Thanks, Dan. First of all, IA is known to be the group of countries of the consumers, but the supply glut, as Mr. Secretary General mentioned, was mainly as a result of the achievements in two of our member countries, US and Canada. So more than the OPEC, so this is just the first point. Second, I believe the 2014 was a unique year, both for supply and demand reasons. In terms of supply, when we look at the non-OPEC countries, we have seen more than 2 million buyers per day almost an increase, and this is the highest in the last 30 years in the record. We have never seen almost 2 million buyers per day increase in production from non-OPEC countries. This is number one. Number two, on the demand side, there are two reasons which ended up very weak, again exceptionally weak demand. One of them is the economic growth. China, as we all know, China is the lowest growth of economy in the last 25 years. Europe is still showing as major problems with the growth, very sluggish economic growth. Japan being in recession, so the economic growth was very, very low. And on top of that, there is one new fuel which is computing with oil in the transportation sector, which is efficiency. We have seen efficiency improvements in cars, trucks and jets in a very big way, and this is not a one-off issue. Today, three out of four new cars sold in the world are subject to fuel efficient standards now, and this is new. Started with Europe, followed by Japan. First Obama administration pushed the CAFE standards, and now China, India following this trend. So efficiency is becoming a structural issue which slows down the oil demand growth, and we have seen the first results now in the 2014. So as a result of that, I would see that coming this major growth from non-OPEC countries and, I should underline, this growth come from non-OPEC countries which are the higher-cost areas, which were driven this growth mainly by also the $100 oil prices. So when the prices come here, perhaps we'll discuss later on, then you make a lot of analytical work in IHS, you would know that we will see this price levels will have an effect on the high-cost areas, and this will have implications for the next years to come. So therefore my view is the current $45 is a temporary phenomenon, and we may well see it towards the end of this year an upward pressure on the prices again. Right, and in terms of the effects on supply. I can say a couple of things, perhaps. A very fresh analysis from us. In 2015 this year, we expect oil and gas upstream investments to decline about 100 billion US dollars, about 15%. 15%. And the big chunk of it will come from the high-cost areas. And this will have implications for the, perhaps not immediately, but 2016, 2017, and this may have a significant implication. And if this comes together with a stronger demand, if they merge together at the same time, this may well have strong implications for the markets and for the prices. Let us now turn to people who are actually in the business of producing oil and gas and discuss the impacts. First, Khaledov Lee, who's the CEO of Saudi Aramco, a large oil producer. Tell us, from your perspective, the fundamentals, the circumstances that led us to where we are today in the marketplace, and then how it impacts your company in terms of your own investments. Well, I think the market circumstances have been covered well, and I would just add that what is perhaps most surprising is that people are surprised that we're going through a downturn in the oil markets. After all, it is the ultimate commodity, and commodities do go through cycles like this, and it's inevitable to happen. The high-price environment, stable as it was, but it was elevated and high for a number of years, have fueled the growth in supply that the other speakers have addressed. At the same time, efficiency and high oil prices have crimped demand growth to levels that were not matching of the supply, and this imbalance, of course, was present actually even before 2014. We saw the excess supply earlier. I think the third factor that had not been addressed today is that the market has been propped up to a significant degree by geopolitical fear that was fueling financial investors to drive prices up and keep them up for a period of time. Ultimately, the reality of the fundamentals in 2014 burst, you know, the bubble of geopolitical fear propping the price up, and then you have a fourth factor, I don't know how big it is, but the financial industry, the strong dollar, winding down on QE, pulling and movements and investment flows have also contributed to acceleration of the collapse of oil prices. But I go back to the two big factors, supply and the impact of elevated prices, of pouring a lot of money, especially in the area of unconventionals in North America, return of Libya in the middle of the year, although it was only for a while, I think just made it apparent to the market that there is no reason to fear shortages in the oil market and that oil was overpriced. Since then, no amount of bad news would move the prices. And I think it's going to take some time. I think the market will have to balance itself. Higher oil prices, as the president of Lithuania was chatting with me earlier, is good for consumers, so it's going to incentivize demand, and we're going to see higher demand later on this year and hopefully for sure in the years to come. And at the same time, investment flows will be a lot more careful going forward. But we're all talked about what is happening this year, but I think expectations of elevated price in the future that many investors, including us in the oil industry, are shaken to a large degree, and people will now be careful, will be careful before committing large sums of money to the mega projects that the oil and gas industry. In the short term, that will help balance market in the long term, it's a worry to me and to my colleagues in the oil and gas industry because we know that in our industry, we have to invest and we have to invest big to replace, to meet rising demand from developing populations, but also to replace the natural decline which is happening every year. We're losing 5% to 6% of production unless we invest large amounts of capital. So those investment flows have to somehow be maintained and we shouldn't be scared away by this current downturn we're experiencing. Now in Saudi Aramco, we're balancing the short term with the long term. Like everybody else, we're using the downturn as an opportunity to sharpen our fiscal discipline. We're cutting on a few things that we could cut, but we're as committed as other to our long term strategy. So we're investing big in gas and gas will not only provide prosperity for the kingdom and meet the energy demand of prosperous population that is growing, but it will displace liquids that will become available for export. So in a way, it's investment of providing more energy to the whole world. We're investing in downstream that includes refining and petrochemical which is not only creating value for us but it's part of what I call building resilience in the company, making sure that our portfolio is not overly dependent on the upstream only. And in fact, Dan, if you remember in your own conference in 2013 and Sarah Week, which is by the way a great conference that I'd like you to attend. I did speak in 2013 that there will be an inevitable downturn in the market and that the way to strengthen ourself is to build resiliency in our industry but also in the individual company and part of that resilience for us in Saudi Aramco is investing big in the downstream. So we may slow down a few programs here and there just to make sure our financials remain robust but we're committed to that strategy. We're investing in gas, we're investing in downstream and we're also adjusting but by and large maintaining our maximum sustained capacity to keep the ability to bring up supplies and production in case of unexpected disruptions which have happened many times before. So Collard, you mentioned gas. What's happening with your drilling for gas and including shale gas?  Yeah, I mean, I think the short answer is we have had great success in three different regions within the kingdom. The priority is in regions away from where the conventional resources are in the Eastern province. So we've put a lot of money in the north of Saudi Arabia just south of the Jordanian border where phosphate mining and industrial development is going to be fueled by this unconventional gas and we will start commercial production in 2016 but we've drilled tens of wells in that basin and the results are extremely encouraging. We've also drilled in two different areas in what we call south of Gawar and Rob Al-Khali in the traditional areas of operations. We have found not only good productivity of gas but we've found huge reserves, very thick pay zones within the shales and they're also liquid rich. So we're encouraged that their costs will be reasonable because we will be able to monetize the condensates that will come with them. So the story we're seeing in North America for unconventionals is, in my opinion, transferable elsewhere and I think Saudi Arabia will be the next big area where unconventional gas initially. We don't need unconventional oil but in decades or generations to come it will follow. Well that's very noteworthy because of course one of the big questions has been when does the unconventional revolution travel from North America? So thank you for that. Claudio Decauzzi, E&I operates all over the world. You've had to adjust, look at your strategies, your investment plans. How has E&I respond, how is it responding to these new circumstances? Thank you. Before I make some comments about what our colleagues talk about, said about the price and the situation and the fundamentals. So we joined the group saying that we think that this cycle is not a structural situation. Cycle, I think that is the disappointment in 2014 has been more from the consumers. So demand point of view at the end of the day than about the growth. Because the growth we had mainly in the shale oil so we did an additional 700,000 bar per day but we have also to consider that the depletion in the shale oil is very, very high. There is a depletion of 50% in the first year. In the US we have a 25, 30% if we put together the lower 48, the shale oil and the Gulf of Mexico. That means that you need to invest much more than what you have to invest or follow up with your operating cost than what you have to do in Saudi Arabia and North Africa and Middle East. So I think that the growth is there but the demand failed. So that is the first point. The second that we discussed about physical issue or just demand and also offer or supply but there is also a financial issue. A financial issue or the exchange, also the dollar, a euro is change rate but I think from a financial point of view we see that from a structural point of view so demand offer we had until November a decline so a slope, a clear slope. Then and we reach about 75, $80 per bar. So then when OPEC said, I don't want to do anything. I want to keep my productions steady. There's a lot of change immediately. There's a lot of change immediately and rapidly with a strong acceleration we reach $44, $43 per barrel. And you saw in the market that all the long position in the stock exchange has been dropped with a short position. And that creates a swing and instability. You see that price in the last week, 10 days straight to overcome $50 per barrel without be able to overcome $50 because there is some kind of recovery in the market, over in the physical market so a compensation between supply and offer but now the market lost the parts of the long position and in the short position more speculative so it's playing a strong role. So from a 75, $70 to $43 is more the short position that are playing because the financial paper is 15, 20 times more than the physical one. So I think that that is a component and I think that is a potential component so it's not a structural one. And coming to the investment, the big issue because we need stability, if now we think that we have to cut 15, 20% first of all, we are going to do that in a certain way affecting the future but we have to think that oil and gas investment to build up and go to a final investment decision is a matter of years. So it's a big preparation, inspiration at risk then the development then the production and this is something that you design in four or five years time so a long cycle and reacting over our reacting to a cycle situation that we already experienced of two, three months and running, I think that is very detrimental and negative for the business because we will be in a situation in four or five years that the price can jump to $150, $200 per barrel because we are not enough supply because in the meanwhile the demand and the industrialization can react because they are using very low price and then we have a problem. And this swing is terrible, it's something that we have to avoid for that reason I think that a stabilizer that at the moment I understood very clearly the reason of OPEC but we need a central bank of the oil like happening in the financial system where we have a federal central bank that gives stimulus and try to stabilize the economy we need this big economy worldwide to be stabilized the way we destroyed ourselves to a swing that is not sustainable. Right well before we ask the central banker here what he thinks about that idea to just finish Claudio on Africa E&I is very strong in Africa. What impact do you see these prices having in terms of development of projects specifically in Africa and the African middle class African growth has been so tied to commodities. What do you see there? So talking about Africa we have to distinguish two or to spread two points. The first of all the assets, African assets generally speaking are conventional assets or open shore, shallow water or deep offshore, not ultra deep with a very good breakeven. So I think that in terms of assets at least for talking about our exploration result and our existing assets is not a big issue because the breakeven price is very low. Absolutely the producing asset is less than 15 or $20 of a breakeven price. Looking forward for the new project we are in the range of $40, $35, $30 per barrel in Africa. So I think that Africa is well positioned to phase discontinuity in the market. The other point is the budget breakeven of the country that is a different issue where you have to accommodate different kinds of things. So I think that Africa as a producer and a country, single country can suffer, suffer I think not for a long time because we see that recovery. They cannot get any benefits from a low energy price because they produce and they use their own energy when they don't sell. So there is no this kind of relief that we have in Europe or in Asia. So I think that I'm not worried because we are not from our point of view and we are not going to stop any project that we started and any project that we think to start in Africa. We are reducing our costs. I think that we'll be in the average of the industry which you talk about 10% and 15% and I think that we are not far from being there. We already started in May before because we work on a budget of $90, $80. So we thought that we can have some decline in the price not like that because as I said is more a financial also component. But I don't think that for Africa will be so difficult because of the good products, good asset that they have. Right. So I'd like to give the panel opportunity to comment on each other's comments but I think one question already for the agenda. Mr. Secretary General, how do you like this notion of being the central bank of world oil? You've just been a political. Thank you very much. Let me just say that I think we tackle this problem with a lot of excitement. We've been in this before. We have seen this two times or three times. Prices went down and we've been able to solve it and overcome it. Now, we have to be very careful with this crisis. We should not really, as Mr. Rafaleh said, we should really invest. Obic and non-obic. We should not lay off our people like what happened the last time. We laid off a lot of very experienced people we did not introduce a new generation. Nobody goes to the universities so we have problem with our projects. There is a delay, there is a problem here and there. So we should really benefit from last experience and price will rebound. You don't need a central bank, you don't need anything. Price will rebound, but we have to get benefit of this. Now we have to get things in order. Do we reduce the lower cost or we reduce the higher cost? Now what they are telling us, Obic, you should reduce your production because we want the higher cost of oil to come to the market and you keep your oil. This is not logic, this is not logic. We really, it is the other way around, like all Obic countries, one of them is my country, Libya, they have a huge shale oil. So in Saudi Arabia and all our member countries, they have. So why don't you just go in order? Let us be logic, I don't see the understand that now everybody is crying. This is a decision against United States, this is a war between Saudi Arabia and shale oil in the States. None of this, this is all nonsense, this is not correct at all. It's just the logic, let us reduce the lower cost first and then go and reduce the higher cost. So I think this psychological, it will come again and a price will rebound and we will have a higher price and then we'll have a lower price. This is normal, this is normal business, I have seen it. It's my age now, I've been in our business all my life. I sold three, three, four times, but we have this time to be very careful to handle it right, thank you. Do any people want to comment on any of the comments? I would say this is music to my ears to hear this thing from Obic. So that the market forces are the ones who are going to take care of the oil market is excellent. I completely agree with the Mrs. Secretary General. Having said that, I believe we need some time, the market forces go through the system and we will find a new balance. But I should also highlight that we may have some emergency cases in the oil markets, which we don't have now, but in those cases we need some, we may call central bankers or the spare capacity or whatever. We had always Saudi Arabia who played that role in a very, very distinguished way and they were always there to come to markets and to take care of the emergency cases. But we are not at all in this stage. One more thing, I think the oil markets are, as we all know, but it is in fact the title of the session today is changing. There are new suppliers coming in the market, such as U.S. and Canada, as I said in the beginning. We are known, my member countries are known are the consuming countries, but the production growth from my countries were higher than the OPEC countries last year. Are you suggesting a rearrangement of membership between OPEC and OPEC? No, no, no, we are very good with our membership for the time being, at least, but what I should say is that it is, we should look at it from a different lens now, the markets, U.S. is becoming a major, I think, the oil and gas country. And today, when we talk about the winners and losers, we talk about Russia being one of the losers, some African countries, one of the different winners is the United States. If today the U.S. President says that we have the crisis behind us, I think we turned the page. I think the shared revolution gave the energy to turn that page, among other reasons that the recession is behind the U.S. economy now. So therefore, I believe U.S. is one of the different winners and one of the new major oil and gas suppliers. So Fatih, what country do you think is the biggest beneficiary of the drop in oil prices? I would say there are definitely winners and the winners if. The winner is, for me, the United States here because of the low oil prices. Not Japan. But I will come. Japan is if category because United States is because of the cheap prices and also the trade deficit went on substantially and many, many new jobs are added. Your analysis then also highlighted about 600,000 new jobs come in the picture as a result of the shared revolution. There are some winners if because of this low oil prices, many consumer countries have a comfort zone, a breathing zone. If, this is the if, if they can make use of this zone this time period, because it will not last forever, I believe, and Mr. Secretary General Egret and many colleagues as well, they can definitely put right policies in place. It is Japan, it is India we were discussing. The Modi government for me made a very good move by regulating the diesel prices. Fossil fuel subsidies can be now phased out in the time when we have the low prices. So some of the consumer countries can make use of that they can use this breathing zone in a wise manner and can be one of the winners. And losers are, I will say Russia, starting with Russia because Russia, 70% of Russian export revenues come from oil and gas exports. And according to our numbers this year it will go down by 150 million dollars. 150 billion dollars is a huge money and therefore Russian economy will suffer plus the impact of the sanctions that we may see in a few years of time. I think our Russian policy, as Mr. Deputy Prime Minister highlighted, will have serious challenges in the energy sector and in the economy as Nigeria and Venezuela, I think these are the countries which we have tough times. So Arkady, do you want to comment both on that and also President Lithuania talked about reorienting their markets, Russia's going through a reorientation towards Asia, towards China and then you talked about the efforts to reorient the economy in this new circumstance which picks up on a long-term theme about reducing dependence on commodities. So maybe talk about the pivot to the East if there is a pivot to the East and how significant that is. All right, first, we are not losers.  Point clarifying. At the end you will win. Temporarily losers, let's begin. Actually, with any price level you can win if you are getting more efficient and this is what will be happening exactly in Russia, we will be much more focused on efficiency, much more focused on doing things that are commercially viable and we will not enter into the projects that have huge risks. And when oil prices are low, there is more inclination to do other things that are probably more important in the long run and develop industries that when oil prices are high are not on the top of the agenda. And for Russia it's good. Russia is good to have moderate, I would say, oil prices, but stable oil prices. On prices, and I was speaking about this in the other session, we are better off when prices are between some levels were mentioned, 70 or 80, then 110, one day and 40 another day. I understand that cycles are inevitable but we believe that for the oil market to be stable on a long-term basis, we need predictable policies that in turn lead to predictable market situation. And policies are critically important. We do need to understand, and markets need to understand what the US policy will be towards energy, its own energy balance and global energy balance. What European stance will be? Middle East, China, of course, and other countries. If policies will be transparent enough, markets will have more information to balance themselves in normal way. Our shift to Asian Pacific region is quite normal, of course, and with growing demand in China, Japan and other countries it was quite natural for us to turn attention to those markets. And those markets present good long-term opportunities, that was important. It's not about short-term supply shift, it's about long-term relationships, especially with China, and we signed contracts not only in gas for supplying gas, we also signed quite a few contracts for oil supply to China. And both Russia and China are comfortable with that. And for China, it's a way to diversify its energy balance and to have suppliers from all parts of the world for Russia. It's also the diversification motive at the first hand. And we are not turning any supply from Europe. We are increasing our investments and we are increasing supply from the distant parts of Russia, from Siberia and the forest, and we are developing those regions by doing that. And it's our first chance in our history to provide gas for the whole Far Eastern region. And quality of life in the forest will improve with the project we have with China. But probably even more importantly is that we we have a very flexible policy towards our oil and gas industry in terms of taxation, regulations, and the framework that allows to live both with high and low oil prices. So oil companies are certainly not losers in this situation and they will keep production probably either constant or even if you will have some drops, those drops will be minimum. They will continue modernization of their refining facilities and will have a higher quality fuels domestic market and will supply some of those fuels to other markets as well. And oil and gas sector even with lower revenue can play a role of innovation driver and technological progress driver across the economy. And this role of the oil and gas sector is probably even more important than the role of improving our balance of payments. Right, so the exchange rate has given you a cushion right now. Exchange rate is a short term inflation problem for our consumers that's clear inflation hike due to the exchange rate adjustment. But in the long run, lower exchange rate will lead to better price competitiveness and better growth perspectives for Russia. Khaled, the energy governors under your chairmanship has done a series of studies. One of them was launched a year ago was on this very difficult problem of costs in the oil and gas industry going up. It's obviously a more urgent issue. Maybe you and Claudio would want to, maybe you want to comment on the study and Claudio just talk about this whole problem of managing costs. Well Dan, let me just reflect as I'm listening to the panel talk about this period where the dominant theme is abundance of supply and the fact that we have the right kind of mismatch where supply is higher than demand which is really good for the global economy, good for consumers, good for important countries. But what I hope to a result in is decoupling to a certain degree the politics from the flow of investments and the flow of oil as critical as it is to consumer countries. Today we see heavy handed involvement of governments and their energy policies of restricting the industry from doing what the industry does best which is invest, innovate, reach consumers and move products and commodities per market forces. So as we talk today about the markets needing to do their things, one thing that we would hope would happen, you see it in the US for example with the restrictions on exports that has created this not insignificant dislocations and product flows and refining which have caused some of the problems that the industry is suffering from outside the borders of the US. It has impacted Europe because of all of the product flows that are coming into Europe which has in turn lower demand in Europe for crude oil and contributed to the issue we're talking about. We see it in consuming countries. The big consuming countries don't have open access for oil producers to go and invest and reach their customers and provide reliable energy supply. So as they, and perhaps it was justified at some point of time because of the fear factor that I referred to in my earlier remarks but today as we see the distribution of energy being more even than ever and the abundance that we're talking about and the ability of investment and technology to provide energy. I'm hoping that strong political intervention and energy investments and keeping them from happening will be lightened. Now in terms of what we're doing as the oil and gas energy governors, last year when oil was 100 to 110, many of my colleagues were talking about canceling, deferring and reexamining projects because they couldn't justify them economically even at that price. We were not able to deliver on our capital expenditures with certainty. Our supply chain was tight and unable to deliver. That is the oil field services. That is the APC contractors, equipment suppliers and our skill levels within the industry of delivering projects on schedule, on budget to the quality standards that we traditionally have done was getting our confidence in it was getting shaky. So we commissioned a collaborative effort amongst the oil and gas community. We worked on it all year. Many companies contributed. We extended our hands to our partners in the oil field services and engineering community and a number of proposals came about. Most of them are in the area of standardization, not just of equipment and specification but also of contracts and working openly and making sure that the whole supply chain and the skill sets are built into the pipeline and that there is visibility of the work and the investment pipeline so that we will invest together with our partners and suppliers and be able to deliver. I go back to what I said earlier. The industry will have to develop 50 to 60 million barrels of oil capacity in the next couple of decades and that is going to require trillions of dollars of investment. That is going to require every supplier on the planet to extend their capacity so that they're making the oil and gas equipment that we need in our field. And more crucially, we need to hire the talent. We need to attract the people. We need to innovate and do what I just described more cost-effectively, more environmentally responsible way. We need to respond to the call on the industry and all of the stakeholders to the environmental challenge, to the climate change challenges that we face and to the regime that will emerge out of the co-op negotiations in the not too distant future. So the industry has to think beyond the current downturn and the good news is that we are. We spent three hours last night in an oil and gas community working dinner and that was a dominant theme. How do we attract the best and motivate them and excite them about working in a technology intensive industry like oil and gas and an industry that is absolutely required to meet economic development? How do we deliver on our capital projects together with our partners and suppliers? And how do we rise to the climate initiative? Claudio, do you want to have a quick comment before we turn to the audience for questions on this cost issue? Just two quick comments. The first one is on energy or European energy security. And the other one is on our managing cost in the future. The first point is about Europe. I think that Europe is a very important opportunity now because we have a very low energy cost. Europe suffered for at least the last eight years to have a double or three times more the cost of energy respect to the US. The US reacted. So they found a way to reduce their cost. And we didn't. First, because we don't have all these reserves, but also because we are doing nothing to develop our domestic reserves. So that is a very important point now. The second, we didn't act as a European community, but each state have a very single different agenda because we are still suffering about a infrastructure layout grid that is completely disconnected. So we cannot absolutely exploit the two big South hubs, I mean Italy and Spain, because there is no connection with the north of Europe. Italy is not connected. And Spain is just connected to a very sharp pipeline. There is no interconnection between storage. So I think that these moments, to be able to think about diversification, to find new and complement our resource base, we have to create a real common network like a state and not like a group of states with different agenda. So that is an opportunity. We have low energy. We have to run faster, not just to cry that the energy is too expensive. It will be, again, very expensive. We must be ready for the moment when the energy will be more expensive. So that is the first point. Second point on cost, management cost, I think that is not just a matter of cut costs. We have a good opportunity to renegotiate some contract. We have a good opportunity to go back to the basis when in the 80s or in the 90s, with very low prices, we were focused on having a very strong grip on our project without rely only on contractors that are very useful. But we have, as a company, inside the project, we have to be very close to the project to control our cost. So I think that go back to the basis in the 80s, in the 90s, we had all our projects on schedule, so in timing, and respecting our budgets. Then in 2000, when we became too rich, we became less efficient. And I think that the other point is really technology. With the low price, we develop a lot of technology. With the high price, we were not able to really use very good technology to develop new reserves. The only country that did that has been successful, the US, with Shayla and Shayga. So that, I think, is the future now and now is the moment to think about that. Thank you. So we have time for two or three very short questions. I'll take them all and then direct them. And then I want to end with President Group of Skyta with a last question for her. But so just identify yourself in short. Udhay Kotak, at what level, at what price of oil does Shayla get uncompetitive? OK. That's question one. Is there a second question? Farhad Rambarak, I have a question. Shayla, tight oil will be there and it's going to be an important part of the supply in the future. Going forward, which one will be the price setter? Is it OPEC or the tight oil marginal cost producers? OK. Is there a third question? Ken Hirsch. I, Ken Hirsch, with NGP Energy Capital Management. The oil price has dropped from 100 to 50, yet everybody's a winner. I'm confused.  OK. So we know there are no losers. Who wants to, on the shale question. I guess the shale, I think are, oh, there you are. I think it depends. I mean, there's different levels of shale. There's some in the US that's economic at 40 and some that's economic at 120. So there's a range of it. But about half of the shale is economic under $60. It seems in the US. Yeah, about half. But we're not at 60. So OPEC or tight oil? What's going to, who's the swing producer? Gosh, everybody really wants to answer that question. What he's saying is who's going to set the price. OPEC does not set the price. We never set the price. We are looking for a price not very high, not very low. A price where it's comfortable to producers, consumers, where we can invest, where we can have decent income for our countries. But we don't set the price. So but the tight oil is, we are reducing $30 million. We are not talking here about tight oil only. We are talking about the non-OPEC supply. The non-OPEC supply, all of them, they are a very high cost. So what we are looking for is they are increasing. Because of the price of $100, $110, everybody is producing. Even in the States, even a three barrel a day, they produce it. It's fine. Everybody is producing. So what we are saying, we OK. We cannot, since we did not increase our production for the last 10 years, if we reduce now, we reduce in June, we reduce in December, we reduce in 2016, we reduce every time we have to reduce so to give a floor to non-OPEC producers. I'm not talking about any one of them. Non-OPEC producers because they see the price is very high. So this is not really logic.  And everybody is trying to jump on the car and jump on the car and the car was not able to move. Say, you know, you have to slow down so we can manage. We cannot do it. It's just let us taste and see how things, but I tell you that the price will rebound and we will go back to normal very soon. So Arkady should can be confused? Yes, very well. Thank you, Major. I was going to do the next question. First, actually about price setting, I think that governments also have a role in price setting since policies affect prices, including tax policies, depending on how your taxation is structured. And the point was made, like there is a story about production cost, but the story about budgets being balanced and not, and this affects price equilibrium. But about winners and losers, well, there are win-win situations in the world. If you invest in the right way, then both parties can be winners. And two different situations. Let's imagine two scenarios. One is where the world develops long run with the price of 110. Another one, when the whole world develops with the price, let's say 60 or 70. And I'm not fully convinced that 110 is the best scenario for the world. And if the world is doing not good enough, then Russia is also not going good enough, since we are all connected, interconnected. And maybe we'll all better off with prices at 60 or 70, not with 110. Thank you. You wanted to add something? No, I just say this price, of course, will affect the oil industry because of the investment, because of producing cost. Everybody will look at it. But it's not really a winner or a loser. Now, the one that they are getting less income, they will look at their budget. They will look at their budget, try to cut it here, try to cut it there, try to reduce some of the cost that they should not incurred in the first place. So it is really a bill that, OK, be careful because the price will not stay as high as for so many years. It's good for us. And for the consumers, I hope that the government will not increase taxes, OK? You keep it in mind now. Low prices will increase demand for sure. But if the government will say, yeah, we have this very low price, let us increase taxes. So the individuals will not get anything. And Dr. Fatih Ibaral, I think I hope that you will not encourage them to increase taxes. No, no, not at all. But I will tell you two very brief things. First, US oil versus Middle East oil. We have to put things in a context. We need to increase the global oil production in the next 20 years, more than 10 million dollars per day. And US alone cannot fill that gap. US is still an oil importer, and will be oil importer for many years to come. Therefore, we should all understand that we will need Middle East oil, and Middle East oil is and will be critical for the global oil markets for many years to come. It's number one. Number two, I think the question from Mr. Hish was not a question, it was a statement itself. So there are definitely losers. And this is important to understand. The losing countries said they are losers so that they are prepared for the next oil price crash. So it is important for those countries to diversify their economy, not to have a single-party economy so that they are more diverse and they are more less vulnerable to the price shocks. Therefore, to say that we are not touched by the collapse of the oil prices may be rather on the optimistic side, I would say. So my last question is for President Gruber-Skyta. We've talked about one reason for the whole circumstance we're in is weak demand. One reason for weak demand is that Europe is in poor economic condition. And that is the excuse to ask this question to you as a former commissioner of budget in the EU and having held the presidency of the EU two years ago. What should Europe be doing in terms of economic recovery in your view that it's not doing now? On European level. Just the European economy. Not about energy, but the European economy. So with new commissions program, and I want to join a little bit on the summer comments about interconnections, infrastructure investments into Europe. Exactly what new commission is planning for seven years and with the seven years of European budget, which you know is about 1% of all European economy. So it's not very large. So why? It is directed towards multiplication effect. Then, for example, 300 billion euros will be invested into Europe's economy in years, three years, but it mainly will be done via multiplication. Multiplication effect, because there is no cash as much, but with multiplication, it will increase. And investments will go mainly into infrastructure, into interconnections, and into creation of energy union in European Union. We've first step is gas market finalization. This is important because, as I said, for Europe, energy is the largest impact to all economic development. But of course, the sort of debt crisis still, which is pending, the very slow recovery still pending really makes some difficulties. But Europe is very different. We have parts in northern, central, more fast developing. And let's say, south part is less fast developing. But in general, the prospects we see already this year increase about 1, 1.5% and to 2016 even faster. While our region, as my country, we're growing more than 3% now, and we will keep this phase. So from this point of view, the investments into energy and the plans which European Commission now has in European Union for seven years exactly goes in the same direction we need to tackle not only European situation, but probably also because European situation is influencing also what's happening in US and other trade partners of European Union. And here I do see good prospects, while of course will be difficulties. But with these prices for us in the world, we are consumers will give a better chance, better chance to invest in new technologies, infrastructures, and especially to interconnections. To meet and I know exactly the figures that by 2020, there are plans to increase interconnectivity, especially in gas, interconnectivity by 67%. So that's not bad. And that's exactly timely. And why with this note, I will say that Europe already was thinking and is preparing to use this opportunity, which is coming with this world energy prices for us. Well, thank you. I think we've had a very panoramic view of the new geo-economics of energy. We realize a lot's changing and a lot's happening very fast. I think we learned a lot from our panelists. I hope that the only person who'll leave the room confused is Ken Hirsch. And everybody else will feel confident that they've really had a deeper understanding. And please join me in thanking this exceptionally good panel. Thank you. Thank you.


So, I hope you got some time to think about that question over T on the create level question in your topic. So, let us actually begin with that and what I would like you to do at this point is if you have not written it, take another 2 minutes or so to write it and then share it with your colleagues in your remote center and RC coordinators if you could please share one such question so that we get an idea of what are such questions we will then go to the next part. So, at this point you can use chat to write to share your question will take about 5 minutes for this activity write your question share it with your colleagues ok great looks like many of you are sending it looks like many of you are getting the idea and I am going to read out some which are let us say really open ended because what we want to get into now is assessment we are going to now look at a couple of examples and then we will go to the next part. So, let me read out a few one says create a workflow for hospital management system. So, even depending I mean it does not matter which field you are from, but if you see a question like this you know that there is something fairly open ended about this and students also have to use a lot of analysis and evaluation in order to answer this question because they have to make several decisions in the workflow. So, it can be workflow for any process which involves a number of different decision making points. Another example ok here is one design a website for online shopping again this is a question which requires a lot of different inputs in terms of decision making in terms of an analyzing which might be more efficient than something else and so on. What is not a good example of a create level is writing a C program for Fibonacci series and the reason I say that is really not at the create level is it is a routine algorithm and students mostly have to code it. So try to make a difference between questions where there are multiple valid correct answers multiple paths to solution and so on and others where there is a fairly straightforward algorithm that needs to be implemented. Let us look at one or two more questions. So, here there is something that says design a super elevation for a two lane national highway. So, if you are doing civil engineering transport and it also says assume all data as per IRC. So, what is IRC, but it looks like some standards so where you have to take real life constraints and standards and then design a solution for an actual real life problem that is open ended design problem. Maybe one more create level and do not write questions like write a C program to sort elements those are more routine apply level questions, calculation questions ok here is one. Develop an experimental setup for the following physical principles. So, there is some physical principle given and the students are asked not just to do the experiment, but they are told to come up with their own experimental setup and actually create the setup. So, I think you all got an idea of what is meant by what kind of problems we are talking about. So, now let us see a challenge that we have. So, the main challenge we have is how do we meaningfully assess such create level question. So, what we will do is a quick think pressure activity here. So, now at this point do a think phase where you write two challenges of assessing the above question. So, one minute for the think phase and then we will move on to the pair phase. If you have done the think phase you can move on to the pair phase where you discuss with your partner and come up with one technique that to meaningfully assess the question that you wrote you can pick any of your questions. And make sure that the technique actually addresses the challenges that you wrote in the think phase. So, the question is if you have to assess an open-ended create level question like the examples we discussed here, what are some challenges and what is one way of assessing it which can address these challenges. And if with your partner you have done that share your technique and let us hear some of your techniques. Challenges are something I think all of us are aware of. So, that was just to get you warmed up to answer the actual question as to what kind of a technique will you use to address some of these challenges. Most of you are saying that one of the main challenges is that there is no unique answer and you are all absolutely right that is one of the biggest challenges that there is no unique answer. But yet as teachers we have an idea of poor answers and better answers. So, how will you meaningfully assess various solutions that you get. So, some of you are saying you will do a mini project evaluation that is ok, but what is your basis for evaluation. So, some of you one person has said that he uses a grading system based on milestones that he can define and these milestones can be at different cognitive levels. So, it is like a roadmap that if you have got until one particular milestone you may get some credit. If you get to the higher level milestone you may get some more. So, this is one way it can definitely be implemented. Some of you are saying developing rubrics. So, that is what we are going to talk about, but tell me what you think what you mean by rubrics. So, those of you who are saying rubrics let us hear what you mean. Another challenge that some of you are sending is that they often this is group work. How do you assign points and so on ok. So, let us actually talk a little bit about rubrics since many of you have mentioned rubrics. This word is used for a number of different things, but to effectively assess open-ended solutions and to meaningfully assess them the rubric needs to have certain characteristics and that is what we will explore in this session. So, if you want to use rubrics as an as an assessment tool let us actually start with some of the challenges which I am putting under goals and constraints. Our solution must make sure that it addresses all these goals and constraints. So, multiple valid approaches, multiple valid answers and solution approaches. This one many of you mentioned. We as teachers want to evaluate not just the final product, but also the process. That is important to us because that also tells us what the student is learning. No single measure exists. In the sense the assessment has multiple diverse criteria of to say whether something is let me put in quotes good or poor. Often the products or the artifacts that you get from students they are not text, they are not equations or numbers. So, you cannot slot it into right or wrong. Often they are very descriptive or they may be diagrams or they may be actual products. How do you look at these? So, for all of these traditional assessments are really not very useful because they will fail on one or multiple of these goals and constraints. So, let us look at what rubrics do. Rubrics are one solution. Now, this is sort of the open blank template of the rubric and I will go to I will go step by step.  This is only one row of the table. So, first we will understand the columns then we look at different rows. So, the first column this is the most important column it defines the criteria a very specific performance criteria like a learning objective. It is also called a competency sometimes. So, this says the student is able to structure open ended using specification. So, let me explain what this is. Here one of my colleagues Madhuri that you saw she is a PhD student in our group. She is coming up with assessment rubrics for design problems in electronic circuits. So, what she did is first listed various competencies or criteria that we need to see from students when they are designing electronic circuits. And one of the top ones there was to take an open problem and structure it into a more well defined problem. So, if I take your example of designing a two-lane super elevated highway that is a very broad problem. So, in order to do that what are the more structured domain specific problems that one has to answer in order to make this large highway. So, the structured problem in the highway example could be that should be able to take a load of so much should be at this certain height. It should have materials which have these properties we still do not know what those materials are. But the first step is to convert this broad problem which says design a two-lane super elevated highway into saying something design a highway with which can take a certain load and can be used for certain time and so on. So, the first one of the early competencies that she was trying to measure was that student is able to structure the open problem using specifications that are given. So, load in the highway problem can be one of the specifications. So, in a rubric for an open ended problem you will have multiple such criteria and each criterion or each competency is fairly specific. You can have as many criteria as you want, but try to make each of them concentrate only on one specific aspect of the problem and the reason for that is if you looked at some of the constraints there are multiple criteria of assessment for open ended problems which makes it challenging. So, we are trying to address the challenge by writing each of these criteria in different rows. So, we have to address each of the multiple criteria, but not together. What happens after this is we write down a description of the desired or target performance of a student that if we want a student to do this what should his or her performance actually look like. So, let us look at an example. All specifications are used to take decisions to structure the problem all interconnections of the system are specified. So, this is for lack of space on the slide I did not put in more text, but in the target performance you can write down all the things that you wish to see that you desire to see from a student's performance to be able to judge if they have achieved this competency or criteria. Let us look at why this is actually defining a performance. You still have to translate it or apply it to your own problem, but several specifications are given like the load and may be the number of vehicles which travel, may be the amount of rainfall, may be the temperature variations during the day. So, there are several specifications for the problem of designing the highway and the student must show demonstration that the decisions that they take must be connected to all the specifications. These specifications may be connected to each other they are not independent often. So, has the student made sure that the connections between the various specifications actually have been identified and used that is the target performance. As I said like this you will have multiple rows. What happens next? Once we write the target performance you as a teacher write down some other levels of performance. The lower levels of performance we will see how to do this and why this is important. In this particular example three the target performance plus three levels are written. So, we will just walk through them one by one. A student who has made an attempt to use specifications but has left out some minor specifications in the decision making process. That student will fall under the category of needs improvement. So, let us say there were five specifications and the student has used four of them and not the fifth which may have been somewhat important for decision making or they have used all specifications but have incorrectly used them that can also be come under needs improvement. On the other hand if a student has made an attempt to use the specifications but they are entirely wrongly applied or some specifications which are really needed to make the decisions have not been used at all have not been even thought of that student can fall under the category of inadequate. And finally if a student has made no attempt to use the specification we will say that this person has not done anything missing. So, because it is an open ended problem there is some subjectivity again at the boundaries but at the same time it is very useful to know that there is a boundary between most of it has been used reasonably with some minor mistakes. It does not matter if there are one or three minor mistakes so long as we can say that overall holistically it is a minor mistake versus inadequate where most where some important specifications have not been taken into account in the decision making process as a result of which the final design does not work very well. So, your rubric for assessing an open design problem has columns usually other than the criteria and target performance. So, this has what is called a 0 to 3 scale if you want to give numbers and the numbers are not really grades if you notice there is not a single number in this rubric they are descriptive. We give these to students we tell them give it to them at two places one is right at the beginning when they are solving the problem or during their learning process we give it to them saying that these are the expectations. And then after they have done their product or after they have finished their problem we give it to them and we circle that look you are here right now you need some improvement. If you have multiple criteria I will try to show you a slide for that one you can have maybe one student falls under needs improvement in one criterion and in some other criterion they are inadequate. So, we are not saying that a student is good or poor we are not simply saying 5 out of 10, but we are specifically saying that in the ability of structuring problems they need improvement, but in the ability of implementing their solution they have reached the target. So, they can have they can fall under different categories in different criteria. This gives students lots of specific feedback on where they are and how they can improve. So, formally rubrics are descriptive rating schemes that define performance levels. Each word is important here they have to be descriptive, but yet there is a rating system and what they define is performance levels just like we explained. What is in a rubric? It has a set of scales which are the ratings we had a scale of 4 missing inadequate needs improvement and target performance. The top level is always the target and the bottom level is always missing. Some people sometimes there is a 3-point rating, sometimes there is a 7-point rating it is really up to you how much you pick there are pros and cons. If you pick a rating scale that has 7 different columns you can get more specific, but then the boundaries become even blurrier. If you have fewer ratings it is easy to slot people, but then the precision gets lowered. If it is really your call there, please note that this is not a yes or no it is not whether they got it or they did not get it, but it is to what extent they got it. So, it is a continue. It has target levels of performance and intermediate levels as we saw each level is highly descriptive and specific you can put examples and each examine this each element of the complex performance has a different scale. What this means is that there are multiple rows to the rubric. Right now we have only seen one row. So, what you can do at this point is take a few minutes to write your own rubric. Take the question that you wrote and we will look at multiple rows also. The question that you wrote in the beginning of the session and if you want to do the think and pair together that is okay, but first write one rubric item actually do the things separately it makes more sense. Write a single rubric item to assess the question that you wrote. What does it mean to write a rubric item? You have to write the competency, the target performance and all these levels. So, this entire table for one row please write it. Once you have written one item then share your criterion and performance level with your partner. So, that you will be able to get more than one criteria for the same problem and then you can do the share. I am going to make a comment here. Some of you are sending criteria which are still too broad for a rubrics. So, if you say complete procedure for designing something you cannot measure all that in a single criteria. So, try to break up your criteria into very precise specific points. It is okay if you have 20 such criteria, but write one or two very precise criteria and this is not about students should be able to understand something, but what students should be able to do in order to solve the open-ended problem. Okay, since many of you are asking about examples, let me try to let me do one thing. I will show you one example and then you can go back to this problem, but I think now we are getting some good examples that students should be able to use appropriate data to solve the problem that is a reasonably good rubric item. But you are only looking at whether they have chosen the appropriate data or not. You can have another rubric item after that which says students have done appropriate calculations using this data. Implementing the data analysis and choosing the data, either keep it separate or together it does not matter, but it is all about choosing the data and using it in the problem. Let us look at one more which is at a right level, I will give you some examples which are at a right level and some which are not. See, you are not writing learning objectives even though it looks very similar to them. What you are trying to do is to try to understand how to assess these projects of students. Okay, here is one, students should be able to justify why they use the principles to design a particular system. That is a good rubric item because there you are trying to look at the process. So let me do one thing, I will show you one full example of a rubric and then you can try to work on this particular question even later. There are two separate issues here. One is to come up with an open-ended question itself. So when you say students should be able to design something or they should be able to create a solution for this problem, those are all your actual assessment questions. Now we are talking about how to evaluate those questions or how to assess those questions. These are rubrics to assess engineering design competencies. Please look at the examples on the slides. The various criteria have been called as sub-competencies here. So I will just read out some of these criteria. Is able to construct representations of the problem, is able to maintain consistencies between different representations, is able to use representations to solve the problem. So these criteria came under the ability of using representations. So this entire thing called engineering design has been broken up into several sub-pieces. This is about representations. In the next block below, it is about structuring problem that we just saw, is able to extract required relevant specifications in detail from the given open problem, is able to actually structure the problem, is able to sequence the problem based on specifications and so on. So this particular rubric for assessing an engineering design circuit had close to 20 or so of such rubric items. Let us go back to seeing where and how we use these rubrics. So when is a rubric effective? If all important elements or criteria of the performance are included. That is why in the rubric that I showed you, there were some 20 odd rubric elements. So each row is either called an item or an element or a criterion or a competency. You can use any of these terms. Important that each element or each row must have a single dimension, otherwise the student will get confused on what exactly they got wrong. The purpose of a rubric is to tell the student what parts they got right and what parts they got wrong, but still maintain a precision. The ratings must be descriptive and comprehensive. So after you have written your rubrics, what you can see is that you can go and check if your rubrics has these elements inside it. And finally where and how to use these rubrics. So as we mentioned earlier, you use them for assessment of open ended problems, projects, presentations, portfolios and so on. To give you one example, if you want to assess students presentations, you can have some elements or some criteria related to their oral presentation, some related to their slides, some related to flow and logic, but each of them must be accompanied by what is the target level that we expect and what are the lower levels. That will help the student understand where exactly they need to improve and where they were fine. So when you come back January 19th, 20th and 21st, we will do a whole session on group projects. At that point we will come back to these rubrics. So today's was more of an introduction to what are rubrics and how you can use them. And it was relevant to talk about these today because we started with the open ended create level problems. We recommend that rubrics should be shared with students up front so that the evaluation criteria are made transparent to everybody. These are, we are setting the expectations and you can give it to them not just at the end but also during the learning process. It's also a good idea to ask them to self assess their work at the end of their work. That's also a very powerful way of helping students learn. Teachers can also use it to give feedback and it's very useful to measure student progress over time. Because you see that a particular student today is falling under the category of inadequate in some criteria but after 6 weeks maybe they have moved from inadequate to target performance. So you can track students progress over specific criteria. So let me also share some personal experiences of what were the challenges of using rubrics and how we overcame them. So I am assuming that you have written rubrics or sometimes you can in fact find rubrics for certain goals. Initially it may seem that there is a lot of reading you need to do in order to give some assessment feedback. It seems so much easier to give them marks like 7 out of 10 or 3 out of 10 but the 3 out of 10 or 7 out of 10 is not at all transparent because it doesn't tell students what exactly they got wrong and why. Usually when they have complex performances like open design problems or presentations. After you mark a few papers let's say 5 to at most 10, the time it takes to use rubrics we found was very similar to giving marks. Giving marks, numerical marks are easy when you have very defined steps but when you have these multiple solutions and open ended problems giving marks also takes time. So there was a short learning curve for us let's say the first 10 papers take time and then it becomes quicker. The second question which people usually ask and it is a very valid question is how consistent can we be with rubrics in the sense if there is so much subjectivity. See what one teacher gives is different from what another teacher gives. So what we found again is that when teachers are trained in using these rubrics and train means they use the rubrics to give scores to some student answers and the experts do the same and then we compare. So you can do it with a colleague. This iterative training takes I would say 2 to 3 sessions but after that there was a very high correlation between a traditional teacher giving marks and a teacher using rubrics to give these open ended feedback and the reliability of these rubrics was also high once a teacher got trained. Reliability means will this rubric give similar answers when different people use it. So it is called Interator Reliability. So after training Interator Reliability of rubrics was high and it correlated well with marks plus as we said it has the advantages of giving some specific feedback for students. There is one last point where here there is a debate should I use rubrics and convert it to marks later or should I just leave it as it is. So what the question is is that rubrics are very open ended and I cannot maintain a spreadsheet for all these open ended things. So how do I convert rubrics to marks? Educational researchers say it is not a good idea because the purpose of the rubric is to give very specific feedback to students over multiple criteria. But teachers still use them and if you must do it what you can do is the following and this is not to give students but just to maintain a log for yourselves. You can call this as 0, 1, 2, 3 if you would like let me get the pointer. If you want you can call this as a 0, this is a 1, this is a 2 and this is a 3 and maintain logs of how different students perform on different criteria and how it changes as time. Going to marks again defeats the purpose of seeing which criteria students are good at and which criteria they need improvement in but sometimes we do need to maintain a record and in that case you can give scores.  We give scores to each of these and then that is how we track the students. So let us end this session here.


Hello and welcome to the NPTEL course on an introduction to programming through C++. I am Abhiram Ranade and today's lecture is on the standard library of C++. This is discussed in chapter 22 of the text. So the standard library comes with every C++ distribution and it contains many functions and classes that you are likely to need in day to day programming. These classes have been optimized and debugged thoroughly. So if you use them, they will be reliable and you may be able to write programs with very little work. So definitely use them if the need arises. So definitely do not invent your own classes but learn these classes and use them. So here is what we are going to do in this lecture. So I am going to talk about the string class. So in the last lecture we talked about a class that we ourselves designed and which we called string but with a capital S. So this is going to be string but with everything lower case. In some sense this class is implemented in a manner similar to our capital string class. And in fact all the classes that I am going to talk about will have implementations which will need the heap memory. So they will deallocate and allocate heap memory but yet all of those things will happen behind the scenes and to that extent the implementations will be similar to capital string class implementation which we had. But of course these classes are going to do something different. So there will be differences as well and in this course as well we are not going to talk about exactly how these classes are implemented. But just to get a sense that how they might be implemented we have discussed the capital string class. So you will have some understanding. All of this will not seem like too much magic to you. So we will talk about the string class, we will talk about a class called a vector class and we will talk about classes a map and unordered map and then we will conclude. So the string class from the library. So this is a much more powerful version of the string class we developed in chapter 21. There are more constructors, concatenation is still there using plus and you can directly print out or read into strings. So you do not have to go through null terminated arrays or anything like that. And there are operations for extracting substrings or finding one string inside another. So we are going to see some examples. This is not going to be a comprehensive discussion of the string class. For that I will recommend that you look at the online documentation. So to use this class you need to include the header file string. So here is how you might use it. So I might create a string and while creating I might initialize it to this string constant abc dab. Now I can write string w and in parenthesis I can write a string from which you are going to copy that string. So this is another constructor but this copies directly. So that v is directly copied. So I could have written this as string w equal to v as well but this is another style of writing initial values. I can write string x equal to v plus w. So that is just concatenation and the concatenation is put into string x. I can index and the index operations can be on both sides. So in this particular case v2 is changing to v3. So we had abc dab before. So that will now become abd dab. Here is an interesting operation. So we are doing cout v dot substring I guess there should be a semicolon over here. But anyway so that the substring 2 means the substring of v starting at 2. So this is v so this is 0, 1, 2, 0, 1, 2 but remember that we just change this c to d. So the substring of this v starting at 2 will be ddab. So v dot substring 1, 3 says that I want a substring which starts at 1 but whose length is 3. So substring of v which starts at 1. So v is now this, so 0, 1 so it starts at b but its length is 3 so this will be bdd. So for this part we will end up printing bdd. Then we can find, we can look for strings inside a given string. So for example I can write v dot find so this is our v and we are finding ab inside it. So where does ab appear? So it appears at 0 so this will return 0. But suppose I execute v from position 1. So this says find ab but execute it from position 1. So this is our current v so I am not going to start over here, I am going to start over here. So in this case my occurrence is going to be at this is 0, 1, 2, 3, 4 so this should be at occurrence 4. So this will end up printing 0 and 4 because I will be set to 0 and j will be set to 4. So the find function needs some explanation because if the find member function does not find the argument in the receiver then it returns a constant string colon colon n pos. So which is not a valid index so what you can do is you can check whether the returned index equals string colon colon n pos. So that way you know that the string is not present. So going back over here if we were looking for say x, y then I would have been set to string colon colon n pos. So n pos probably is some negative value for example so which cannot be the position of any occurrence. So that is how you can check whether the string actually appears or does not appear. So a string object can be passed by a value in which case it is copied or by reference. So if you are passing strings to a function this is how you can do both but of course I mean the usual rules apply. String objects can also be compared and for this the lexicographical order which we talked about some time ago is used the dictionary order. So just to clarify A, B, C, D, E, F appears earlier in the dictionary so this will be deemed smaller than this. So here is an exercise for you write a function which counts the number of occurrences of one string inside another. So in our old string there were two occurrences so your function should compute you should print out both the occurrences. So you will have to use the find command in slightly imaginative manner. So what have we discussed? So we have discussed some features of the string class which is a powerful safe way of handling text data. So you really should try to use this string class as much as possible and avoid null terminated care arrays because those are quite error-prone and also cumbersome. In the next segment we are going to talk about a class called the vector class but before we do that let us take a short break.