Enrico Giovannini is the moderator for the next panel. Enrico, hello. Come here for one second. Morning. Good morning. What is your moderating? Good morning. Okay. All right. Much better. Because we're going to talk about data. So if you sleep just before talking about data at the end, it will be better. I know. It's like tired and data, no hope. What is your moderating style? I think you gave us a little bit of a clue there, but what is your moderating style? First of all, trying to have people wake up first and then try to engage them, although the time is very short. So prepare your questions that you had in mind and never asked. All right. Very good. I will leave you in the very capable hand of Enrico. Thank you very much. Good morning, everybody. I'm Enrico Giovannini from Italy and co-chair of the Trends Group, which is one of the task teams of the Sustainable Development Solution Network dealing with data. As I said, data is something that people say is dry and so on. But do you know the origin of the word statistics? No. It's science of the state. And it can be about the state of things, but also state with capital S. This is why we are talking about this today, because without data, we will never know whether we are moving in the right direction, but also data are an ingredient of sustainable development, not only a tool to monitor. Today, we will present a great report produced by our group. And so I'm glad to ask the main author of this report. Please come in, Jessica Espi, who is not only a great expert in this field, but work very hard for delivering the report on time for today. And we have also two distinguished panelists, Jenna Slotin, who is a senior director for policy and strategy for the global partnership for sustainable development data. And Antonin Cia Azare, who is a director general of the Ghana Health Service. And he will help us to understand how data can really change policies and change people's lives. So, Jess, you have the floor for a presentation. Thanks very much. Thank you, everyone. It's an absolute pleasure and honor to be with you all and to follow such distinguished panelists in the previous sessions. As Enrico said, my name is Jessica Espi. I'm a senior advisor to the Sustainable Development Solutions Network. And I've been working very closely with an incredible assortment of leading minds on data and statistics from around the world. Enrico is one of our co-chairs. And we're joined by some of the other members here today, such as Shada Badi, who's one of our other co-chairs. It's been a real honor and a pleasure trying to work on this topic. A few years ago, if someone had told me that I would have become one of SDSN's data gurus, I would have laughed. But suffice to say, as Enrico said, it has become sort of one of the hot topics of politics and of the Sustainable Development Agenda. And I've found it's become a really exciting area of work for us across the network. So today, we're going to just briefly summarize our new report. I say briefly, because I want you all to read it, so I don't want to give the game away. But it is available on SDSN's website. So, just a quick overview from me. So, everyone's talking about data and statistical systems within, certainly within the UN corridors, and why is that? As everyone's heard today already, in any 15, 193 heads of state and government put us in the path to a completely transformative new agenda for the world towards sustainable development. And when you get down underneath the politics, when you start to look at how you do this in practice, very quickly the data rubber hits the road, if you will. And it all becomes about predominantly quantitative measures of progress. So, we've been starting to think through and trying to help member states, local governments, local actors, think through the kinds of information they need to achieve these ambitious goals. But it doesn't take very long to kind of dig underneath the covers and realize that the data that we have currently is grossly sort of inadequate. We have huge problems of timeliness, coverage, quality, and so on. Just a fascinating new report came out a few days ago called Ready to Measure, which shows us the state of gender data relating to the sustainable development goals. And they showed us that of the 53 gender related indicators we need to fulfill the objectives we have set ourselves, only 20 are ready to measure. So, if we don't know where we're coming from, how are we going to get where we're going? I flagged a couple of the issues to do with quality, timeliness, statistical capacity, and so on. But what are the solutions? And that was really the mission of this report and this piece of work that our data expert group has been focusing on. The first is always investments. We need more money. I think everyone knows that. It's the constant plea. We need to build robust statistical systems that help governments really understand the information they need to affect change. But perhaps more excitingly, we need to harness the so-called data revolution. So what is the data revolution? What's this data thing everyone talks about? The short answer is it's everything. Every time you go on to book a flight, every time you order a takeaway, every time you send a text message, you are passing information from one entity to another. And that information has inherent value. This is a really interesting new study that's just been done by the Australian government who looked at what are the economic implications of the data revolution. And they concluded that it is the biggest shift that's going on, structural shift in the economy within a generation. It's fundamentally changing the way we run government. And it's got a really long way to go. We're at the tip of the iceberg of understanding how new information systems are going to transform politics and economy. So with all that in mind, we've been trying to think about what a modern fit for purpose 21st century data system looks like and how governments can achieve this in order to support their efforts to meet sustainable development. And very quickly, it's obvious that there's no one right way, there's no one perfect statistical system. And I think we'd be naive to suggest we had the answer to that. But there are some big things we do know. Firstly, we know that the scale of the challenge is immense and consequently it needs to be a multi-stakeholder partnership. And with new actors coming on board, it can't just be the government anymore who's leading the charge and trying to collect official statistics. Second, we know we need to take the best innovations from the private sector and from aspects of the public sector and apply those to how we generate information. And third, we know we need the highest level of political engagement. We really need heads of state, government, the UN system really, really saying, okay, data is a big priority. It's a hot topic. It's not relegated to a small department somewhere buried across government. So what is this modern statistical system? Firstly, it needs to do three things. For the first time almost ever, people are starting to realize that statistical systems don't just look backwards. They have to look forwards. I think we all know from what's happening with Hurricane Jose, with Hurricane Irma, with all the climactic shocks and events we're seeing at the moment, that we need to try and plan for and predict this kind of change and try and manage some of this uncertainty. And that means we need to model future scenarios. We need to forecast what's going to happen. And that requires quite different skills than we've ever seen before within data and information systems. We need people who can do modeling and so on. So it's not just about collecting static points of data. It's about looking forwards. Second, we need data that's going to enable us to manage more effectively. We need to know whether a health service in real time has got the vaccines it needs, has got the staff it needs, whether school teachers are arriving and attending their classes. And we need that to be current information, not two years, not three years, not five years out of date, if we're actually going to have responsive government. And third, we need to monitor historical progress. Now this is what we have traditionally been doing relatively well in spite of all the gaps. But what we haven't done is make the most of this historical data. There are so many reports and I think everyone in this room would know how many reports and data sets and so on that are sitting on shelves gathering dust. And so this is about pulling all those pieces of data and information together and trying to make new value out of those inherent legacy systems. Now I don't expect you to even remotely read this slide. It just goes to show the range of different actors we have involved in this new statistical system. And just a quick example from the health sector. If you think about collecting something as simple as mortality data, an essential fact and figure that we need to know to understand the well-being of our populations. Normally you would collect that using administrative data, data that's coming out of a hospital or a health service. You might have a civil registration and vital statistics system which tells you when births and deaths and marriages and so on have happened. But a huge number of countries don't have that. They don't have the capacity. They don't have the financial resources. So they might ask the UN to help or perhaps the US government. They might come in and help them do a multi-indicator cluster survey or a demographic health survey. Once they've done that they might decide they need academics to help them clean and tidy the data. They might turn to the University of Washington that has one of the best international health metrics and evaluations units. And then they might decide oh well we could also use data from the insurance sector or we could turn to Google analytics to look at where future trends of health and you know, unwellness are likely to occur. So the point is there are loads of actors that are contributing data these days but very seldom is all of this information taken back to the national statistical system and is it actually used and compiled and owned by government. So that's what we're trying to move towards. But of course there's huge institutional barriers to these kinds of partnerships. People concerned about ethics, about privacy, about security, about the ownership of data. So those are some of the things we need to tackle. Our report has four big clusters of recommendations and I can't do them justice in two minutes I probably have left. But I'm just going to give you a quick highlight of one of each of the recommendations. The first topic relates to governance and leadership. I've already said we need the highest level of political commitment around data and information systems if we're to make sure we don't leave anyone behind. And one of our recommendations is for the creation of chief data officers. Now much like CDOs as they're commonly known in companies will oversee all the bits and bytes of data that flow through a private company. We need that kind of commitment in government. At the moment national statistical officers often sit separate or independently of government and rightly so with some distance. But we need someone sitting in the center of government preferably in the executive office who's able to oversee all the bits of information the government has, can review new sources of data, can try and establish these new partnerships and then turn to the NSO to assess their quality and rigour. So we think that kind of high-level political commitment is essential if we're going to take the SDGs commitments to data seriously. Second we need more principles and standards. This isn't just about principles for principles sake. This is about trying to make sure that if you've got a private company or a university or a government all working together they are singing from the same song sheet. So for example we need very clear standards about how you do data sharing across these different groups. And we in particular are calling for a new data compact to be agreed at a multi-stakeholder meeting of the UN Statistical Commission which means every March but is currently very narrowly defined to just national statistical office members. With more participation with private companies and various others involved in that we could come up with a robust compact around the SDGs and data that everyone would subscribe to. Third we need technology innovation and citizen led analysis. Now this is a huge bucket of recommendations but just one quick example. We've seen some brilliant innovations coming out of different private companies around data sharing. If you look at what happened with the Ebola crisis in West Africa particularly we saw private telecommunications providers working in the government working with the government working with the UN and creating sort of platforms and innovative ways of sharing this proprietary data across these different actors. That's the kind of innovation we need now on a much bigger scale. And so we're calling for an innovation challenge at the World Economic Forum to kick off at the next forum amongst various other sort of high-profile moments where we can encourage this. What I won't skip on also I just want to highlight recommendation three here which is a city group on CGD that's citizen generated data. Citizen generated data is becoming more and more and more important not only to try and fill gaps in the official arsenal but also to understand people's experiences. A good example is you can use a hurricane information system to know where the house has been knocked down or you can use GIS software and a map to look at what's been destroyed but you have no idea how many people were in that house what kinds of services they've been deprived of what their personal experiences were whether there's emotional trauma and so on. So we need citizen generated data in order to help us really understand what we're trying to fix and we're calling in particular for what we call a city group which is a interagency and expert group on citizen generated data to make it a new a much more widely accepted science that we can integrate into achieving the sustainable development goals. And then we need capacity and resources and this isn't just about money that is a big part of the recommendation there's a lot that could be done through a new global financing facility for this these kinds of investments particularly in low income countries and lower middle income countries but it's also about making sure everyone understands what we're talking about when we talk about the data revolution when we talk about data systems when we're talking about our data. So we're thinking about things like partnerships between the statistical commission and UNESCO to actually encourage widespread data literacy training in schools from a very young age so we all know how our information is being used and how we can control it and contribute it to the support of achieving sustainable development. So that's a quick overview of our report I'm sorry it was so brief but I hope you will read it. It was only we've only highlighted a few recommendations I think this is a huge huge huge topic and as I said it requires so many actors to help achieve a new fit for purpose statistical systems for this but we hope that they're the beginning of a process and that if these things can be achieved in the next six months to a year we can really set the world on a effective path to achieve the SDGs. Thank you very much. Thank you very much great presentation let me add a couple of points before opening the discussion with our panelists. Two years ago the Secretary General Ban Ki-moon asked me to chair a panel on the data revolution for sustainable development in preparation of the final agreement on SDGs and we made several recommendations and we have to recognize that the international statistical community took seriously our recommendations and made an effort but let me also say that after two years we are not yet there and we are far from being able to deliver what people, governments, companies want to know about sustainable development. The key issue here is to make a big shift in the way in which statistical offices and others work and work together and this is what our report is about it's about policy it's not about technical issues. The the risk that we are facing is that in two three years time when the General Assembly will meet again to see what we have done over the last five years they will say we are not on the right track and we'll be too late to have what we expect to have from data providers and then there is a big issue of breakdown because the no one left behind principle is a great principle but is a hell of work for statistical offices and very costly way to produce data if you want to have them desegregated by gender region and so on and so forth. This is why you need to exploit the big data. This is not an easy issue and as Jess said we need more cooperation between the private and the public sector and we have to mobilize the public and also the philanthropic sector to invest on these and this is what the global partnership for data for sustainable development is about so I'm very glad to have Jenna here and we will hear from her how they look at this issue and what kind of proposal you can make just to take this project forward. Thank you Enrico Jess everyone for having me here today. It's really a pleasure to be here representing the global partnership for sustainable development data which was actually a recommendation of the report that the the secretary general's independent advisory group on the data revolution made in 2014 and then it was established in 2015 to try to help address this data challenge for the SCGs. So first off I want to commend the trends group on this excellent report that Jess has just described. I think it does an outstanding job of laying out what we really need in terms of the modern data ecosystems to support the achievement of the SCGs and sustainable development more generally and I think what's particularly exciting about it is it talks about both the what and the how. What's needed across the data ecosystem and makes very concrete clear recommendations about how how to get there. I also really greatly appreciate the recognition that it needs to be an iterative approach that it's articulating a big vision while recognizing that each country is going to be in a different situation each group each actor and so there's a lot of there's going to be also a lot of variation. One of the things I want to really really emphasize from this report and emphasize about I think the data ecosystems that we really need as we work towards the SCGs is this point about a multi-stakeholder approach and really appreciate that the report takes a very clear-eyed view. We hear a lot about multi-stakeholder partnerships in the context of the SCGs across sectors across goal areas in the area of financing and to be sure in the area of data but it's tricky it's very very tricky particularly when it comes to issues of privacy issues of ownership issues of interoperability which is just a technical way of saying how to get data to talk to each other and it's difficult but it's absolutely essential and so it's one of the things that the global partnership exists to support. The global partnership is a network of now over 275 actors and entities. Members of the global partnership include governments such as the government of Ghana but we're very proud to have Ghana as a member of the global partnership. Many many companies large and small many civil society organizations and international organizations and many academic and philanthropic groups and it's the contribution of all these actors coming together bringing their different perspectives of how they support and engage and can provide data to foster unique collaborations that can drive innovation ways in which governments can work with companies to unlock data to get access to new software and technology ways in which public and private sector can work with citizen groups as Jessica was describing this new and burgeoning area of citizen generated data to really generate more meeting around the data and information we need to inform decision-making not just on the back end for monitoring asking how are we doing and having data that might be out of date by one two three five and ten years but the data we need now today to know if the programs and policies we have in place are actually working are the health clinics sufficiently stopped with medicines are they reaching the people in the community are the schools actually are the teachers showing up at schools there is data and information that can be fed into the system to know if the policies and programs in place and if the resources are being directed in the right places to achieve our goals so this point about multi-stakeholder engagement and this point about the data that's needed now looking to the future and for monitoring is incredibly important another point that I want to make though about multi-stakeholder engagement and this is very much at the heart of what we do in the global partnership is when we're working with governments to look at what a multi-stakeholder approach means at the country level to put the data systems in place that are needed is it's not just across sectors looking at private sector civil society government and others it's also the cross government dimensions one of the things that we see is that the national statistics offices which we tend to focus on when we talk about data are not necessarily able for a range of reasons to work with the different line ministries to share data and to collaborate across government I think there's a lot more work to be done in that area which is why I'm really looking forward to hearing our colleague from the Ghana Health Service and really salute you for joining us today because that the relationship across the health sector and the national statistics office and all the other sectors is incredibly important. There is one point I would like to raise because I think that questions already came is the idea that generating data from citizens this can put at risk people's privacy which is indeed a big risk this is why we need the principles applied also by non-statisticians while the statistical offices are in general safe space but of course more you bring together data from different sources more let's say the the privacy is at risk especially in countries and I will would like to ask you also a question so I heard that in the global partnership some solutions have been developed right yeah so absolutely so this new area of citizen generated data but really all these areas raises big privacy concerns we actually have a group of partners many of whom are working at the grassroots level and been using different methods of generating data from citizens they're coming together to document their methodologies to collect use cases and to work towards some principles and standards to ensure that the level of robustness and certainty we have around official statistics can also be applied to citizen generated data to make it useful and defensible thank you Anthony what is the situation in your country yeah the situation might thank you very much the situation in my country is that we work together this interceptorial collaboration between the minister of communication minister of health the vice president office is also steering all distance and we meet together we are using the laws in the country we have data protection laws minister of health also have sat down and we've come out of a data protection for our patients because we deal with patients so it's very important that we protect the patients and we protect ourselves as well so that was that's what we are doing and how do you see this data revolution and the recommendations of the report applicable to your country and other developing country in africa yeah we look in health we look at how what can you do to take advantage of the data revolution so for the past few years we said we should use appropriate technology in Ghana what we realized is that in 1992 about 190 000 people were subscribers of a mobile phone as i speak now about 35 million subscribers if at every child born in Ghana we have a mobile phone by now so we took advantage of that let's go into mobile health and use the low lowest level that is a community health workers train them how to collect data onto mobile phones and then trans they are also supervised by the community health officers which are the trained nurses who are also supervised by the health information officers and technical officers who who know data and then it fits into the system which you call the district health management information management systems so it's a seamless system and we've taken advantage of that train took one region in Ashanti region trained about 20 of the district officers about now we have our 1800 people who have been trained to collect data at the community level so they are collecting data for disease control disease surveillance and also clinical data and are supervised by the community health workers and they send it to the sub district level goes to the district level and comes to a national level so we are having a timely data that we can use for decision making and that's the advantage that we have taken and so doing we have also moved forward so why don't you also do tele-consortation and luckily the same Ashanti region that was worked down by Millennium Promise and people are consulting and the community health worker can now send the data that he has collected if the patient is sick to the next level and then there's consultation and back consultation to the level and we've taken advantage say look let's go into telemedicine so as we speak now we have five centers or six centers where we have tele-consortation and we have also gone forward again to take one region 26 facilities from the teaching hospital up to the lowest level which is the chief center which is the community based health planning service that's the lowest level and now we are doing the pilot to find out how we can move seamlessly from the lowest level to the highest level there are two things that we are doing we are trying to achieve the universal health coverage but not only achieving the health coverage but achieving quality health coverage for the whole people of the country and that's exactly what we are using data for we've taken advantage of data revolution in the world to make sure that we give quality services to the people of the country so I think that this is a great example of using data to make sustainable development happening and not only to produce statistics to monitor where we are going well there are several questions that we received but unfortunately we have to close this session I think that we hope that now you will take a look at the report that which is available of course on the sdsn website and spread it in your networks and contribute to the work of the global partnership sdsn and the other bodies who are committed to make the data revolution one of the fundamental engines to reach out the sdgs thank you very much


So I'll shortly speak, I will talk about 3GPP specification version control. It's probably most important for people working and actually implementing the 3GPP specifications. And I'll illustrate it with a use case. For example, when we did the GSM network at the Congress, at the beginning we had an issue. And the issue is that the phones would not register with our network. So every time you tried on the phone to register with the network, it says, I'm not allowed to register to our own network with our SIM cards. And the mitigation was just restart the phone and then it would work. The problem is that when the phone lost connection with our network, for example when the people went back to the hotel or wherever they sleep or where they were just out of reach, the SIM card would try to connect to other networks because it wants to roam. But for the phone, it said, I don't know the SIM card. I don't have roaming agreements with this network. I don't know this IMSI. So it would say that I reject calls saying IMSI are known in our IHLR. And this triggers on the phone state where the phone is not allowed to register to any network again. Until it reboots. And the permanent solution for that was simply take the SIM card and write in the SIM cards in a list called Forbidden PLMN, the networks where you don't want the phone to register to. So now all phones without SIM card will never register to vote a phone. So the SIM card will never be blocked. But it took us a bit of a while to find out that. Now, almost everything is defined in 3GPP specifications. And this procedure is also defined there. So the section is the specifications is 24 or 8. This is the specification describing how the procedures for GSM work. And I will not explain the numbering system of 3GPP. There is a link at the bottom. And it's already complicated, but I will even add a layer of complexity because I will talk about the versions within the specifications. So the procedure here is really clear. Whenever you do a location update and it is rejected, if it's rejected with cause number 2, IMSI Note in HLR, it's written consider the SIM USIM as invalid for non-GPRS services until switch off of the SIM, until switch off of the phone. So this is what we found out. Or the SIM USIM is removed. Generally, you have to switch off the phone when you want to remove the SIM or put the SIM back again. So it's the same thing. But looking at things, there is an OR. OR the timer T3245 expires. And it's quite an interesting timer because we didn't see this happening. So I looked at, again, the same specifications which define the 3245 timer. And this says that it's a timer where you pick between 12 and 24 hours. And after this 12 to 24 hours, you erase this forbidden list which is saved in the phone and you perform the cell reselection. So it's a really, really large timer. Probably this is used for vending machines or fixed appliances where rebooting the machine or rebooting the phone is just really hard. You cannot do it remotely because you don't have connections anymore to the machine and you cannot send anyone on site very often. But the problem with that is that it says also that this timer is specified in another specification. In this case, it's 31010. And this is a specification for USIM, so whatever is written on the SIM card. And if you look at it, this parameter is in an optional file which is on the SIM card. It's an optional field. The behavior is an optional field and only the operator can write it. So the user cannot say, I want this timer to be enabled. The operator has to enable it. I checked a few SIM cards, it's deactivated by default. So that's why we've never seen it in real. But maybe for vending machines, they have this option to do that. Then I looked at the specification again and you see there is a small A. The subclose, so the clause A. And if you look at it, it says that if you don't use timer 3245, you use another timer which is called 3247. And this time you draw, the timer is between 30 minutes and 60 minutes. So it's a lot shorter than the other timer.  So that's really awesome because this saves you between 30 minutes and 60 minutes. You will re-scan all the network and connect back to it. But this is a specification for 2018. When we had this problem, it was 2012. And you can see this clause A didn't exist back then. So the problem with the specifications is that they keep adding features with the specification, which is normal because you want to have new radio, you want to have advanced RTE, and so on. But it's really hard to know what is new and what is not new. And this is where specification with versionally control is really important. And some specifications, and most specifications, you have release nodes. And one release is just one line. And if you can try to look for T3247, there will be no comments in the release nodes. So it's pretty hard to find when, in which version was this feature introduced. And this is where I will talk a bit about specification. So all the specifications are written in Microsoft Word documents. You can download them, all the documents really, in zip archives. The release nodes only contain one line change telling which feature has been added or which feature has been changed. And sometimes you even don't have the release nodes for the previous versions. So you really have to download the old version and see the release nodes for the previous version. So if you have the last version, it doesn't mean that you have all the new features. And it's really hard to track what the changes are from version to versions. And there are new tools which really allow you to do that. So the specifications are really written by humans, for humans. And only the people who write the specifications actually kind of know what are the new features which are added or not. To change that a bit, I wrote a small script which downloads all the specifications. It extracts them, converts them into text files. And text files are quite useful because you can grab into them quite fast instead of PDF files. And because they are text files, you can version them. So I put everything in a git repository. The script is really just one make file which takes care of everything. And you can find it at 3gr. And the result is that just the archive, so everything compressed is around 100 gigabytes. And everyone tells, yeah, there are thousands of specifications. Well, now we know there are 3,200 specifications. And when you count the versions of the specification, you are 52,000 specifications. So downloading everything, extracting the documents out of the archives, converting them to text, takes around three days. And in the end, you have 230 gigabytes. I have hard drives here with the dumps. So you don't have to do everything here. You just copy my hard drive. It's the easiest. Or if you have a lot of time, do it yourself. The problem is we can also not make all everything which is processed online because the specification version are copyrighted. You are not allowed to redistribute them particularly when they are modified. Transforming them into text removes the picture and so on. So it is modified. So I am not allowed to distribute the end documents which you can grab. You have to do it yourself and wait a long time. Once you process, every time you want to update, it takes around half an hour to one hour. But then you have also all the updates which are versions. So let's try to use it. Now I put versions on all the text. I am trying to search for a section <IP_ADDRESS>a which is the new section in the 24.000 text. And here you see the commit. This feature has been added to release 13.5.0. So the first number is always the release which tells the major steps. So for example, LTE has been introduced with release 8.9. New radio 5G will be released with 15 and so on. Afterwards, you have the small versions. And the last number is almost always 0. Except for the first documents I haven't seen it change. But now we know in which revision it has been changed. So it's quite good. So here I just looked for the big change but you can use bisect to find smaller changes. And you can also look at the diffs and you here we can really see that they added the paragraph. So this allows you to see when it has been introduced but also have the complete diff of the feature. So you know what has been changed. And in this case, they've added also this timer in the authentication reject and a couple of other places. So when you're implementing this feature, you can see the differences. And now you know, okay, I have to implement this new timer here, here, here and here. They've added this feature. Problem with this also is that it shows the diff of the complete version. It's not only this feature which has been added but a lot of other features. And you don't really know which feature belongs to what. This is where we'll talk about how they do revision control. If you see in the change lock, you have an entry line. And now we know it's in the version 13.5. So I took the version 13.5. I looked at the release notes and the closest I could find is correction of the integrity of the reject causes without integrity. This was the closest description for this feature. So on the left, you can see other numbers which are related to these specifications. And I'll decode it rapidly. So the first number tells you at which meeting this feature has been introduced. The second number tells you, and the first number tells you who introduced it. The second number tells you which pack of feature it is. The third number tells you the change request. So it is really this single feature. And then you have other numbers. But based on that, you can go on the FTP, you just add these numbers in the path. And you find exactly the document which introduces this feature. They are called change requests. This is how a change request looks like. And this is what they work on. So it really looks like a commit. But for humans, you have the source file, the commit ID, which is on the top right. The branch name is the CR request. You have author, the commit title, the commit message, and it's a Word document. You have the diff. So this is how they explain what they change, what text they introduce, everything like that. And it's done in Word documents. What's interesting also is that you find more information about why they added this feature. In this case, we now know that they added this feature only for rejects which were not integrity protected. So there were attacks which have been demonstrated where you really reject the phone. And you could do a denial of service because you just roam around with an empty catcher. You deny the phone and you request it. You will never connect again. So this is why they added this feature because generally when you are a rogue base station, you cannot send authenticated messages. So in this case, a real network can really reject yourself. So this would not help in a case with Word Phone. But any malicious base stations, you would be able to recover from the situation. So that helps quite a lot to understand why they added this feature and gives you a kind of background. Then we saw on the top right, you have a commit ID and we have revision six. So it took them a bit of time to write this patch. You can see, you can actually search on portal.3gpp.com or don't remember. You can see a list of all these change requests. So where is my mouse? Here you have the change request. Here you have the affected specification. And here you have the revisions. So the revision and the commit ID. So it's this particular document which they talked about. And using that, you can actually build a graph on how they are doing the commit. But everything is done manually. So I just took it, you have the source file in the branch. They use this branch. Then the branch is the new feature which has a change request. And you can see the different commits and the technical specification group. There are around three of them. It really depends when you look at it. But there are three technical specification groups which cover different aspects. Within these groups we have around four work groups which are more specialized into something. So a change is done by one work group of one of the bigger groups. And they talk about this change, about the document and they really meet every month or so or more. Also they are really their calendars. And they group all there and they discuss and they change it. It has been revised. And it took them six version before everybody agreed on this patch. And what they do afterwards is they push it to another branch which is a CP. CP tells you which group from the subgroup is deciding on it. So the small subgroup said it's a cool feature. We'll add it to our group. And the whole group is meeting regularly but less regularly than the work group. And deciding if this feature is good or not. And so this was the group, the meeting 71 which you saw in the change log. And they thought, yeah, it's a good feature. We merge it. But this is what they discuss about is not this single request, this single change request. This CP includes around 20 to 30 change requests and approve all the 20 or 30 change requests. They can also reject individual ones but they generally talk about them in packages. And this is... And it's done by humans. The whole tracking and all the numbers done by humans. C195. Oh, you mean here and here. So C195 is one meeting. And three months afterwards they had another meeting. And so here they said we agree on this feature but it's not complete. And on the next meeting we will... Some guy said okay, I want to add this more and so on. So it's hard to see here but actually I'll come to that afterwards when I talk about the CR pack. Oh yeah, the CR pack. So it's important to see also about not just single CR but it's... This CR pack is for one particular domain or one particular aspect. And it's important to look at it because so the CR 2927 which I talked about is right here. But if you look at the other CR you will find the exact same thing correction of handling NASA jet messages but for another specification. 2431, 301 is for LTE. So they added the exact same feature in LTE. So it gives you a clue where this timer is also important if you want to implement something in a new feature. You can also see that they talk about almost the same thing. Authentication reject and authentication and ciphering reject in another CR which is 2928. But this was at some point rejected. Not pursued. Meaning that the guy who wrote the specification they said it's almost the same thing but it wasn't as complete. So the one which has been accepted actually integrated this feature. This is what you can see to the agree. And then they discuss later on or the next meeting they agreed again because they merged this request. They merged this change CR into the more complete one which has been agreed later on. You cannot see it directly so if you do the branding by yourself you can see it. And actually... Did I write? Oh yeah, there is a database. There is a machine-readable database about how the CRs which CR and which internal document reference to. It's also maintained by humans but at least it's machine-readable. And there isn't even an online tool which allows you to search for CRs and you will see in which documents they are and you could actually plot such a graph using their database. But again, done by humans it's not automatic. So the conclusion is that the 3GPP has quite good version in control but really done by humans and for humans. It's not machine-readable but it's kind of possible to make it machine-readable. Thanks to Git it's one of the things. But as you could see Git only works on the end specification. I don't track all the changes and so on. I think this is done for human and they had so many exceptions. Even just for the specifications I've seen half of the script is just handling exceptions and I don't handle all exceptions. And a last note. I'm also disclaimer, I'm not part of 3GPP. I never attended a meeting. This is just what I found out while trying to work on this script. I'm sure there are a lot of errors and I didn't talk about the meetings and all the other documents. There are plenty of other documents which are interesting but even harder to read. And also what I talked about are the specifications. They are not standards. So the 3GPP only writes specifications and then each country or nation state or whatever takes the specification and makes standards after them. This is why generally when you go on the Etsy page to download the PDF you don't find the latest version. It just takes some time for the Etsy. So the Etsy is the most known one which is responsible for the European standardization. They take a bit of time to take the 3GPP specifications. They adapt them. So I don't know what change they make to it but then they make them official. But these specifications are also available. So maybe if I have some time I take the same script, change it a bit, download these specifications. And then actually I could diff what the Etsy done compared to what 3GPP done and actually see what the differences are and which country does which change to the specifications. It would be interesting. They probably change from American English to British English. So Etsy is really the best of them. The other ones they are not really complete. For example in the USA they don't care about the core network. They only care about the access networks. Yeah, that's it about how versioning control it works and how the script can help a bit if you want to implement new features or at least keep trap of the features. Any questions? How long did it take? So writing the script itself took around one week. That wasn't too long. The longest is just waiting that things happen because it's a lot of files. Adding exception took another week for the most important documents but I didn't add exception for everything. So I will with time whenever I open a specification switch I need and I see something is missing. I look where in the conversion this thing has been missing and where it's been removed. Also I want to add a couple of features because here in the change lock you actually don't see who is the author. You don't see the date. So which working group and which is the date. I can extract this. So in the change lock commit I could actually add the real date and in the commit message I could add the list of CRs which helps you to find out which is the important CR. So these are features which I could add in the end. And finding out how it works. This was when I was working at Qualcomm I had to track. We had incidents about ATE and I had to track when this attack would be mitigated thanks to one of the specifications. So this is where I learned a bit how the meetings were working and where I really at one case really tracked every document and found out when it was added and so on. You mentioned file size of hundreds of gigabytes. Was that the extracted text or the PDF? So here I could make, there is a thing which shows me the statistics and the PDF. I don't do PDFs. So I take the archives I extract the doc files and from the doc files I convert them into HXJMN and then text. So no PDFs. I don't know the size of the PDFs. But from, I don't have the hard drive here, but if you make statistics it will show you how many doc files and how large they are. So it's 200 gigabytes of text. In the end, yes. So it includes the, yeah, it's 200 gigabytes of documents and text. But it's not plain text only. Also the doc files. But I recommend really to do, to use a file system which draws transparent compression because out of 200 gigabytes you are at 115 gigabytes. So half of the size and works quite well. It's quite fast. Saves a lot of space. And this is only for the TPP. If I do the same thing for the Etsy, we'll have as many files. So my question is about licensing. So you mentioned license terms for the specifications themselves. My question is about if there are, if there's some, some restriction or the same restrictions for the change requests. Or if it would be possible to, to include most or relevant information from the change requests in a public repository of metadata. Without any, any issues. I don't really know. I think they were under the same copyright. They're all publicly available. And I didn't talk to the TPP if they would be interested in this tool. They just do it by hand and meet every time with printed documents. So, so I don't know what the lessons terms for the CSR. I would think they are the same. You're not allowed to transform them because again of the diagrams and so on, but I didn't talk to them. Yeah, I would be surprised. There's the same copyright statement in there. So I think it's the same. I think the only way how you can make this like the result of your process public legally is to find some people at 3GPP who find this interesting and make them hosted themselves because all the information is provided by them. It's just nobody is permitted to redistribute it. So if they would run the resulting Git repo on one of their servers, then it would be completely legal. So at the very least, it would be possible to have a public repository with links to the, to the actual CR. Oh, if you look at the portal. Yeah, it's portal. You have links to all the documented and so on. So it's not bad. It's a bit slow, but they added they improved a lot. This is new. It's not, it's not a tool website. Right. But I mean in the generated Git repo, you could, you could link possibly even deep link to change request level information on this public portal. Yes, you could not change request itself because it changed request and zip files. So I could. Okay. I could link to the zip files. So the group of change requests and then it's up to you to do it. But yes, it would be possible if I extended to do that. Would be super cool for 3GPP to, to take some interest in this and export. I'm not sure they're aware of this, of this project or listening. Well, they should be. So exactly like, what are the different steps that take three, three days actually? Like what's most of the time invested on most of the time is converting the document files to XHTML files. So this took for me took one and a half days. I don't have a super fast computer, but downloading the files the first time took takes around one day. Updating them takes around 15 minutes. So you can, the script allows you also to update. So it just goes through all directories. And so is it a new specification? I will download it, extract it, put it in Git version control. But yeah, one day for downloading one and a half days from converting from doc to XHTML. And then a couple of hours for converting it to text and putting it in Git. I guess you're caching the files when you download it somewhere. So you kind of, you don't need to re, re download them in the script. Yeah, they have two ways or how you can access this specification. One is over HTTP.  It takes, so it's faster to download the first time because I don't know the HTTP server is a bit faster. But afterwards it takes two hours to update. The second method is over FTP. FTP takes a bit longer the first time, but because they have listings, it's, it's really fast to update. So it only takes, it goes through all the directories, but it's just those listing on the specification. No date change. I don't have to request every document to know when it has been changed. So updating over HTTP is, is a bit faster. Over HTTP is possible, but it's slower. And for Etsy documents, you have to sign in to get the archives with the docs. So I'll see how I'll handle that. Since this is a tiny little bit off topic, but also on topic from what you chose to, to the project brought us to this. Because as anyone knows who uses a dead SIM card from an operator, it's really annoying when if it goes away from your test network and then gets blocked. Because you have to power cycle to get back on your own network. But just I had a question about what, what is, what is integrity? What is a, what is an, a reject then with integrity? Okay. Because if, if, if you are, if we're using our own SIM cards, then their network doesn't know the keys. So obviously can't. Oh, um, yes. Or how is that working? Um, this, the, so in DSM, it doesn't matter because there is no integrity protection. It matters in 3G and LTE or MTS and LTE because there you can have integrity. But what is important actually when a rogue base stations gets a Vodafone client on its base station and then rejects it. So the guy from Vodafone knows this is a reject, which is not integrity protected. So the Vodafone client is protected. It is not about protecting the non Vodafone against the Vodafone, uh, against the Vodafone network. So there's a kind of a balance which they decided on. They don't want millions of subscribers to use this timer every 30 minutes and log on again. Because if you have a rogue SIM card and expired SIM card or so on, you would have one update request every 30 minutes. Thanks to this timer. This is why it's only using the timer for non integrity protected networks with their own SIM cards. And because they know their SIM cards, they could protect it. So but we at the Congress network, we have our own SIM cards, right? So could we use this feature to, I mean, yes, we've put Vodafone in the Forbidden PLA man list. But could we also benefit from this feature? I can, so this rejection was just a use case where it's important to track features. But we can talk afterwards about how to use it. How about the timer?


Hi friends, welcome back to my channel and a very special welcome if you're new here. My name is Jen, I'm a certified weight loss and nutrition coach and I'm on WWW and I count macros and calories. It's Saturday, it is grocery haul day. I shopped at two stores and I have a lot of new food find.  Make sure you're subscribed and your bells turn on so you never miss a grocery haul which is every Saturday and I upload five videos per week. Check out the description box down below for nutrition coaching, where I offer personalized to you macros and calories. Highly, highly recommend as well as one-on-one coaching if you would like to chat with me directly. Links, discounts to my favorite things and don't forget, come on over, join our Facebook group. We'd love to have you. So let's jump in to the grocery haul. So here is my grocery haul for the week. I made a trip to Sprouts and then I went to fries which is my local Kroger store. I have some new food finds for you guys that I'm super excited about. So let's jump in to the grocery haul. So starting the sprouts. The first thing I found were these apple pie snaps. So these are like ginger snaps but apple pie flavored. They also had pumpkin spice. I thought these would be really, really good on protein pudding. You guys know how I like to put something a little crunchy on my protein pudding. I thought those would be delicious not to mention they would make just a really good sweet treat. So I decided to grab the apple pie if I like them. Next time I'll pick up the pumpkin. And speaking of cookies, look at what I spotted. They actually had two different partake cookies at Sprouts. They have this one, which is holiday sprinkle and then they had chocolate peppermint. I've tried the chocolate peppermint. They're amazing. They're a really soft, yummy cookie. So I decided to pick up the holiday sprinkle to try those. I really like partake cookies. They have good ingredients. They're low in calories, low in points and they're just really, really delicious. And then they have my favorite cottage cheese on sale for 3.99.  I actually bought this, not last week but the last time I went to Sprouts. This is the brand chocolate love. This is the dark chocolate peppermint crunch. This little candy bar right here is the best thing I've ever had. It has a little bits of candy cane in it. So it gives you that sweet minty and then it's perfectly paired with the dark chocolate. I don't even love dark chocolate but this bar is so good. I ate the entire thing that I picked up last time so I did a repeat purchase. Two big things of shredded chicken. I need shredded chicken for a couple of recipes and for me, this is convenience. It's already cooked. It's all white meat, ready to go. So it's zero points on WW. And like I said, it's just cooked, shredded and ready which is really, really convenient. And then I needed some eggs and I really like the Sprouts brand of pasture raised eggs. I don't know why, but they are so delicious and they're really affordable. And then just a ton of produce. So we love the Moondrop grapes. In fact, we have these as part of our Halloween get together with my in-laws and my mother-in-law loved them to actually send some home with her. So I'm wondering if she's been picking them up too but we really, really love the Moondrop grapes. They are so good and they were still on sale at Sprout.  And then they had their raspberries three for five which is a great deal. But then when I was at fries, they had them four for five. So a better deal at fries or Kroger if you have a Kroger store near you but I picked up three packs at Sprouts. Some fresh organic cilantro. I need that for a couple of dinner recipes. And then I needed one yellow bell pepper, a couple of sweet potatoes, a couple of zucchini, a fresh lime, bananas. I'm gonna freeze a couple of these and I need a banana. I believe for my meal prep, I'm really excited about my meal prep this week. So stay tuned. After the grocery haul, I'll be sharing with you what's on my dinner menu and what you'll see Monday in meal prep. And then no surprise to anybody. Another bag of my mini cucumbers. We eat at least one bag a week. I could probably buy two bags when we go through them. And I found a really good dip, a new dip at fries for the cucumber. So I'll share that with you. And then last but not least at Sprouts, I grabbed a huge bag of sweet onions. Jumping in to fries, which is a Kroger store, a gallon of 1% milk for Troy. And then I needed black beans for a recipe and I got some coupons in the mail from Kroger and one of the coupons was for buy one, get one canned beans from Simple Truth Organic. So I went ahead and just grabbed two cans of black beans. I'll definitely use those. I needed some slow cooker liners because I've been out for a while and keep forgetting to pick them up. I needed one can of corn for a recipe. I had a coupon for a free Philadelphia cream cheese and with the holidays coming up, I will definitely be using a lot of cream cheese. This is the dip that I found. This is the Stone Mill Kitchen's Garlic, Herb and Feta. These are on sale right now for $3.99 and then it had a dollar off coupon. So I paid $2.99 for this and I thought it would be really good with the cucumbers and I found a new cracker that I think it'll be really good with as well. So I picked up that. I saw these in the clearance section and decided that I would try these. These are cashews, these are sugar cookie, confetti flavored cashews from the Simple Truth brand. So a quarter cup, 170 calories. I thought these would be a good sweet treat and a few of these would be really good sprinkled on yogurt or on my protein pudding. So I thought $3.50 couldn't pass it up. Light sour cream just because we've been out of sour cream and it's something that we always like to have in our refrigerator.  I stocked up on my Jell-O puddings. However, they were out of vanilla which was a little bit of a bummer but I did grab a sugar-free cheesecake, sugar-free lemon. I had not seen that before, really excited for that. The banana cream, butterscotch and chocolate fudge. This will tide me over for a little while and next time I shop I'll grab some sugar-free vanilla but this is a staple in my life because protein pudding is a staple in my life. A pound of ground chicken. I have a recipe in my meal prep that calls for ground turkey but honestly I like ground chicken better. So I'm gonna swap it out. This ground chicken is zero points on WW just like the 99% ground turkey. So I thought I might as well get something I like a little bit better so I picked up the ground chicken. I did grab a loaf of Wonder Bread for Troy. He had asked for a fresh loaf of bread. Here's a new food find and this was sitting next to the ground chicken and I thought these looked really good. These are from the brand Mighty Spark and these are the spinach and feta all natural chicken patties. 23 grams of protein, 170 calories per patty and I thought that these would make a really good lunch so we will try these out in Wednesdays what I eat in a day so definitely make sure you're subscribed and your bells on because that is going to be part of a meal in that video. I needed some light shredded mozzarella cheese for a recipe and then some paper plates and I had a coupon for those as well so I picked up a 48 pack. You already know my addiction to the no sugar added bread and butter chips. I was completely out so I picked up another jar. Honestly I wish they sold a huge jar because I would buy it and then we were out of light mayo and I had a coupon for the craft light mayo. I also needed a restock of my Fairlife Low Fat Milk and then here are the crackers that I found. These are new. They're from the brand Crunchmaster. These crackers are a really crispy kind of rice cracker but look at the flavor. Avocado toast, salt and pepper. That sounds so good. You could put some avocado, some bruschetta on there and then I thought they would be good with that dip so I picked those up as well. So that is everything that I grabbed for groceries this week from fries and sprouts. So let's jump into what's on my menu for dinner and what you'll be seeing in Monday's meal prep. So here is what is on my menu for the week. So today's Saturday out to dinner. Tomorrow Sunday we're doing sloppy joes. We've been on quite the sloppy joe kick. Monday I'm doing a Mexican chicken and rice. You guys will actually see this recipe in next Wednesday's what I eat in a day. Tuesday's a leftover night. Wednesday I'm making tacos. Thursday we are going to dinner at a local theater. It's my husband's birthday, our wedding anniversary and then my birthday. If you didn't know, our birthdays are one day apart and we got married on his birthday. So we're going to a Christmas vacation inspired theater event with my in-law. So we'll be eating out on Thursday as well and then Friday we'll finish up any leftovers from earlier in the week. And then here's what you'll be seeing for meal prep on Monday. I am super excited for these recipes. I'm going to be making a zucchini blueberry baked oatmeal and I'm going to amp up the protein in this. So I'm excited for that. Lunch I'm doing a ground turkey sweet potato skillet sub ground chicken. That's what I was talking about. I decided to buy ground chicken instead of ground turkey and then for dessert, for dessert. I'm making maple glazed apple bologna. So definitely subscribe, turn your bell on and stay tuned for Monday's meal prep. Thank you for joining me for this week's grocery haul. I hope you enjoyed seeing all the good stuff that I picked up for the week, what's on my menu for dinner, what we're doing in meal prep. If you enjoyed today's grocery haul, don't forget to give it a big thumbs up. Subscribe if you're not because we do a grocery haul every Saturday and don't forget to check out the description box for nutrition coaching, links and discounts to my favorite things and come join our Facebook group. We'd love to have you. Happy Saturday and I'll see you in the next one. Bye.


So there's been a poll by Harvard, Caps, Harris poll, right? And it found that two-thirds of voters between the ages of 18 and 24, two-thirds, over 66% of voters believe Jews as a class. I didn't know Jews were a class. But anyway, Jews as a class are oppressors and should be treated that way. I've been telling you this. I've been telling you you're rich and successful.  The fact that not that long ago, you were brutally, you know, massacred just in October. But, you know, genocide was committed against you. The fact that not that long ago, Jews were poor and had nothing.