Civil society organization could figure out who is responsible for human rights violations, whether an individual or a state, and how to suggest this and address for victims internationally and nationally. Then civil society could create strong advocacy and push for changes on the ground through legal analysis. No, legal analysis is not something only lawyers or people with some legal backgrounds can do. The essential work involves answering primary questions such as who did that and what did the person do and how did the person do it based on the data and documentation collected by the civil society organizations and defining whether there is a right protected under international law. Anyone can do this. It may not be easy and take time at first, but once you understand the structure and sources of international law, you will become familiar with how to analyze the facts using international law. There might be complex situations, but there are many secondary sources and a case law that can be used to help refine your analysis. Legal analysis requires a thorough understanding of the facts, so all documentation and data your organization collects can be used for legal analysis. To figure out which law to use, you need to know the difference between human rights and humanitarian laws. International human rights law refers to the body of international law designed to promote and protect human rights at the international, regional, and domestic level. International human rights law primarily consists of treaties and customary international law. It is important to note that international human rights law applies at all times, both in times of peace and in times of armed conflict. This means that both international human rights and humanitarian law are applicable during armed conflicts. Unlike international human rights law, international humanitarian law applies only in armed conflicts, both in international and non-international armed conflict. International humanitarian law is binding to all parties to a conflict, state and non-state actors. A major part of international humanitarian law is contained in the four Zeneva conventions of 1949 and the two 1977 additional protocols. In addition, the 1907 Hague Convention and the Annex Regulations set out important rules on the conduct of facilities, notably on military occupation. I think the question could be rephrased what kind of rights are protected under international law and how to assess the state party's compliance with international law. First, we will want to have a clear understanding of the effects and define an issue. Next, we will have to identify a source of law that applies. It would be helpful to try a hypothetical example. Let's say that we identify the situation where a person was subject to ill treatment during investigation in state A and want to know whether it constitutes torture under international law. To figure out which law applies, the first thing to do is to ascertain whether the state has ratified the treaty. It is also necessary to check whether the state has made a reservation or an interpretative declaration at the time of ratification. On the UN Treaty by the Database, you can't search the ratification status by country or treaty. Here, treaties that seem relevant are international covenants of civil and political rights and convention against torture. Also, there might be other treaties on the reasonable level. Now we have specific provisions related to torture under international covenants on civil and political rights and convention against torture. The text of the provision applies to start. In some instances, because of the intensity or nature of the certain acts, it is clear that the act constitutes torture. However, it is hard to decide whether the act amounts to torture in some instances. Then the next question we will have is what factors should be considered to determine whether the case amounts to torture or whether it does not reach this threshold. And then we will need to look into other sources, such as individual complaints, advisory opinions, general comments, reports, resolutions, concluding observations, and control reports, and judgments on the treaty bodies' websites. There are currently 10 human rights treaty bodies. The treaty body issues general comments and recommendations, which interpret human rights treaties, and they can be found on their website. You can further analyze the scope of torture by considering the general comments and jurisprudence of the treaty bodies, in particular the Human Rights Committee, which monitors compliance with the ICPR, and the Committee Against Torture, which monitors compliance with the Convention Against Torture, and other cruel, inhumane, and degrading treatment of punishment. The next question is how do you assess whether the state complies with this obligation and their international law? Once a treaty has entered into force and is binding upon the state parties, they must perform the treaty obligations in good faith. State's legal obligations to protect human rights implies an obligation to prevent, investigate, and punish human rights violators, as well as to restore rights whenever possible, or provide compensation. Tube-specific states that are party to Convention Against Torture has an obligation to take effective legislative, administrative, judicial, or other measures to prevent acts of torture in any territory under its jurisdiction. Therefore, we'll need to look into state's legislation, policy, and practice regarding torture, and then assess whether the state complies with this obligation under the treaty. As mentioned, the force never convention, and their additional protocols form the framework for the treatment of surgeries, prison of war, and non-competence during armed conflicts. It is important note that the proper category of armed conflict is necessary to determine which set of rule applies to the conflict, because international humanitarian law classifies armed conflict as either international armed conflict or non-international armed conflict. The International Committee of the Red Cross hosts an international humanitarian law treaty database on their website. And the website contains many introductory international humanitarian law resources. Their website is a good place to begin your research for international humanitarian law. It depends. It is hard to say exactly how long it takes. If the facts are clear, it can be easy. If the facts are complicated, it takes time. But looking up secondary sources such as journal or article helps your research because it provides overview of the issues you are interested in.


So I will hand over to Amy Duchel, our C4 colleagues, to lead the next session for the following, the next 35 minutes or so. You have two speakers and now yours, Amy, to lead this session. Great, thanks a lot, Pat Daniel, and hi everybody. I'm Amy Duchel. I'm leading C4's Climate Change Energy in the Carbon Development Team. And thank you for the invitation today. It's really wonderful to learn about work that I'm not intimately familiar with. So I really appreciate the chance to be here. We will bring questions for Ibu Mirna and Pat Marcel into the end of this discussion as well. So I see a lot of great questions in the chat and keep adding them as you hear our next speakers talking because we will have time at the end to try to bring everything together and incorporate your ideas as well. So our next speaker is Dr. Harry Pernomal. He's a scientist at C4 and the Value Chain for Finance and Investments Team. He's also a professor at C4 Bay. He's one of our illustrious scientists known, especially for his work on the political economy of fire, but also people in restoration issues. He also works on sustainable value chains, criteria and clinic meters for sustainable forest management and human management. So he will bring his broad governance expertise into the session today. So I turn it over to you, Harry. Thanks, Amy. Good morning. Good afternoon, everyone. I would like to share my presentation. So the title is Criteria Indicator for Tropical Pitland Restaurants. The governance aspect was developed by myself and also my colleague at the VFI team. I will start from the definition of governance. It is about the process of decision-making and how the decision is implemented or not implemented. It's important not only the decision-making process, but also the implementation of the decision-making. It is also the way power of actors is exercised in the management of economic and social resources. It is also talking about the power of protesters. Just the general things about governance element is coming from the wet bank, Fritz et al. The element is about the structures of trade, immigration, and also climate. And then the institution variables looking at the law rules, formal, informal, as well as the actor they influence the way results is managed. And then I would like to share our work, actually, in the previous project called the Sladi Sustainable for Lowland Agriculture. So you see how the structure, institutional actors influence the sustainability and livelihood of the land in the whole Indonesia. So we did a survey to the anti-Indonesian. And this is the score, actually, how strong the influence of the actor and how the connection between structure, incision actor, and sustainability. And also, you look at this more powerful actor based on their attributes, actor-centered power. The central government is the most powerful and powerful at the last, the least powerful. Also, according to the network, the social network analysis, the private company is the most powerful. And the financial agency is the least powerful according to the network, the social network. So it depends on the attribute as well as the network. And this is also very famous, the work of Kaupman, Kree, and Mastruzi, coming from the woodwind, the sixth dimension of governance. First is voice and accountability, how people accountable to their work, the political stability, option of violence. Also, the government effectiveness, how they deliver services to the people, the regulatory quality, as well as rule of law, how the rules enforced. And the last, I think, very pro-Fanto-Indonesian, the control of corruption, this is the sixth dimension of governance data worldwide known. And there are some principles of good governance that's developed by other authors, UK, ASS, and UNDP Indonesia. And the last is participation, rule of law, transparency, responsiveness, consensus-oriented, equity, and inclusiveness. Ibu Merna talked about inclusiveness. There is the principle of good governance, effectiveness and efficiency, as well as accountability. And the UNDP also developed kind of principles of good governance in Indonesia, for Indonesia. It's accountability, how people responsible to their work, equity, and effectiveness of services, deliver by government, efficiency, equity, and progress. Again, participation, how different actors participate in the decision-making process, as well as transparency, how the outside can get the information as soon as possible coming from the author. And this is, I will not explain the definition of them from the participation to the control of corruption. What does it mean? It took from various sources. And this is my proposal of a government principle for pit-land restoration at the national, sub-national level, or you can say a jurisdiction on that level. It's about participation, how people participate at the national level, or provincial level, how people are accountable to their work, and also the stability of politics. This is from my study, how actually the local election influence the fire and also the pit-land burning. There is a connection, it's already published. Also the effectiveness of government, also the quality of regulatory and rule of law, how the law enforcement, I think law enforcement is key. There is no good fire prevention and pit-land restoration without working the rule, without enforcing the rule, as well as the control of corruption. This is the, you know, the PCA diagram. Mostly rules are targeted to individual and also the big enterprises, less to the medium-scale investor. This is at the national level. It's at national level, it's a bit different. It's not talking about the quality of rule, but talk about the equity inclusiveness, participation as well as effectiveness. And also very key to talk about the control of corruption. When I talk about landscape, mostly related to the PHU, pit-land hydrological area, even less on that, a sub-landscape. So there is a principle at the national, sub-national level, at the jurisdiction level, as well as at the landscape level. And this is the CNI structure. This is what we did from the previous project, the hierarchy structure for the principles, then elaborate into the criteria, then each criteria elaborated into several indicators. Principles fundamental truth, everybody will agree, but then the criteria developed and used to judge, to assess the compliance with a principle. And then we need to develop indicator when it is measured to indicate a criteria. And this must be minimum, it's not all of them. And localize to meet the context, jurisdiction, scale, and also value change. And as Ibu Miramai said, there are already indicators, make use of the existing, the collected indicators. Just use it if there is some already indicator, no need to start from beginning. And then in assessing, you will need to wait and then assessing process using multi-criteria analysis. This is what I saw from the BRG as already indicators there. You can, for instance, criterion two, active involvement of parties. This is going to be a criterion, a criterion for participation. So if BRG already collected, just use it. If the Ministry of Affairs already collect some indicator, just use it. Just like the word being used, the existing indicator that you can accumulate and synthesize. So that's it, my presentation. Thank you, Ibu, back to you, thank you. Thank you, that was great. And thank you for being on time. I really like this idea that, you know, implementation of existing policies, programs, criterion indicator is actually the new innovation. We're always trying to do new things, but in fact, there's so much that's in place that we can be leveraging and actually trying to implement and implement better. So that was nice connection between the presentations. I also think, you know, tying the theory of governance to very practical, you know, issues of peatland restoration is very important. And, you know, Marcel was talking about the importance of social cohesion before we actually get to rehabilitation and reforestation activities. And you can see the complexities of such social cohesion from some of those diagrams that you were showing. So this isn't easy, but highly important. So let's move on to our next speaker in the session. This is Ibu Josi, a Katharina, who is from the Terpachaya Initiative Secretariat and represents Nobu. Terpachaya is a really interesting initiative. I'm sure most are familiar with it, but this is, with Inobu and the European Forest Institute, really showing how agricultural commodities can be produced sustainably and legally. So really pushing forward the jurisdictional approach in Indonesia in a very practical way. So I look forward to hearing about that from Ibu Josi. So first of all, I would like to thank the organizers Daniel for personally inviting me today to share the work of Terpachaya. And we hope that it can be beneficial for BRG and C4 in further developing the indicators for fitland restoration. And there are four aspects that I would like to share today of the Terpachaya Initiative. The first one is the indicators methodology that is applied, the platform that we use to share, to let on share the measurement. And lastly, the policy approaches taken by Terpachaya. So this initiative has been developed since 2018 by Inobu and FV under BAPENAS, the National Development Agency Direction. And the work of Terpachaya is developed based on consultation with stakeholders through an advisory committee mechanism headed by BAPENAS. The advisory committee of Terpachaya consists of government, including Ministry of Home Affairs, Ministry of Agriculture and the Ministry of Environment and Forestry. And recently, Ministry of Trade also joined in the committee. We also have non-governmental organizations such as S.P.K. Aids and Epistema, representative from various embassies. And lastly, business communities, including buyers and producers of agriculture, commodities, be it based Indonesia or abroad. And in developing the indicators, Terpachaya adhere to several principles and more importantly, as I have previously mentioned, is it is developed through a multipartist processes. And it is also aligned with national laws and regulations as I mentioned earlier. And currently, based on this work, Terpachaya has 22 indicators which are divided into four different pillars, environment, social, economic, and governance. These 22 indicators are developed to show that agriculture, commodities, and the district are produced sustainably and in compliance with law. And Terpachaya want to do it by defining what is jurisdictional sustainability and by providing reliable and regular information about performance of different jurisdiction in Indonesia of it. So we hope that it can be built up from other speakers. Myrna in her talk already mentioned about importance of combining issues, social, economic, and governance. And Marcia also mentioned that the importance of having jurisdictional approach. And Harry just explained to us about how to measure governance and how to use existing work. That is also the approach that we use in Terpachaya. So I will share with you some of the methodology that we use to assess some of the indicators. The first one is indicator four. Sorry, there is a typo there. It should be indicator four on pitland protection. And the current proxy that we use is the size of pitland that is protected in a district level. So we use the hydrological map of pitland as the baseline to let them measure it based on the policy documents. So to what extent policy documents have protected the pitland. And the policy documents that we use is the moratorium map and the district spatial plans. Both are formal documents produced by government. And based on this measurement, we can then have a national average and the district performance is measured based on the average. And group into three for all indicators. We look at those below, those on average and above average. The other relevant indicators for our discussion would be the fire prevention. And in this indicator, we use the data from the Ministry of Environment and Forestry on the burn scar. And we compare that between years. So those with better performance in the following year get better score. So these are the analysis based on the actual data from the Ministry of Environment and Forestry in 2018 and in comparison with 2019. So because we have limited time, maybe I just move forward to the last pillar that we have, the governance. Currently we have five indicators under the governance pillar. They are first the proportion of district budget allocated for sustainability, access to public information, multi stakeholders participation in planning, compliant mechanisms, and lastly the sustainable and use planning. In this last indicator, we want to measure the utilization of environmental instruments in planning processes such as environmental carrying capacity and strategic environmental study that are obliged by the environmental law. And indicators under these pillars, according to our work so far, is proven to be one of the hardest to measures. Currently, we're only able to measure three out of the five and for some other, we are still testing several different datasets. I'll share just two with you today and maybe we can elaborate some more during the discussion session here at the time allows. The first one is on the proportion of budget for sustainability. The choice of the budget is based on the perception that it can show the ultimate interest of local government as well as the capacity to deal with environmental issues in their area. As you can see here, for central government province, Barito Utara and Selatan are above average, while Amanda and Seruyan are below average. We can see diversity of performance here compared to the other indicator that I will share that is the public access to information. This is the indicator 19 of the Tepachaya indicators. And among the many ways to measure this indicator, based on our discussion with the Central Information Commission, at this stage, we agreed to measure the obligation for all public bodies to have officials that is responsible for documenting and dealing with information requests, not known as PTED under the 2008 Access to Information Law. Because this official is crucial since the availability of data and information requests will go through this official. And more importantly, the data is accessed at the national level because the Ministry of Home Affairs collect the data from district level government. Based on our findings so far, it seems like we need to adjust the proxy because most of the districts already have appointed officials, these acquired officials. Only there are several districts that have not been appointed, that official have not been appointed. So in the next cycle, we will have to elaborate other processes for these indicators. And most likely we will use those that are already being measured by information commission. And so the next one is the Tepachaya platform. The Tepachaya platform is going to use to share indicators and results of the measurement that have been done by the word. And it will be maintained by Bapernas and it will be used by different users with various interests. Therefore we are currently consulting with different stakeholders on how their interests can be best facilitated by this platform. At this stage, the platform allows analysis and scatter plot to show performance of different districts. In analysis of indicator, later on we are thinking to allow users to adjust the value of the indicators. You can see here, analysis of the indicator will allow user to give weight to give different weight to the different pillars. Also a different weight on the different indicators in each pillar. So if you use this as an example where a user only put 30% weight for the environment aspects and 70% for the economy with this type of different weight under the economic pillar, we can see this as the performance of district government for the whole Indonesia. So this is not final because it is still under discussion with different stakeholder group to develop features that at the end of the day it can actually reflect jurisdiction in sustainability. And lastly, I would like to share about the policy approaches that are developed to institutionalize their per-chaya. So based on the direction given by the AC, the advisory committee of the per-chaya, hopefully we have three approaches that we at the secretariat are exercising. The first one is the traditional command and control approach where per-chaya is situated in the planning and evaluation cycle by the government agency that is the National Development Agency and the Ministry of Home Affairs. The two agencies have the power to allocate budget and gives fiscal incentive or disincentive on local government performance. So under this approach, we expect that information given in the per-chaya platform can be used as the basis to give fiscal incentive or disincentive. But at the same time, we also based on the discussion in the committee, the information can also be used to identify districts that need assistance. For example, those who are below average in some combination of key indicators such as poverty and governance will receive budget support so that they can enhance their offer performance. The second approach is the market instruments. And based on our discussion with buyers of agricultural commodities, their purchasing and investment decisions can greatly be helped by the information provided in the platform. But they also request to have more information such as the traceability of products. So this is one aspect that we haven't really touched in the existing work so we need to work more on the traceability. The last one is the multilateral cooperation. So in the last meeting, Representative for Ministry of Trade mentioned they will use the platform to support their negotiation with other countries in comprehensive economic partnership agreement, for example. And they also expect in the future that if we have more information about local government, particularly who can be contacted in the local government, the platform can facilitate communication between buyers from a country to producers directly from a local government. Thus to a certain extent, it is expected can be functioned as a marketplace. But we understand that these different policy approaches will depend on reliable information and measurement. And therefore the main homework for us at the moment is to make sure that this can happen. So that is the first and the most important homework for us. And with this, I would like to end my presentation and I hope this can be available for my colleagues from C4 and BRG to develop relevant indicators. And of course, we expect that in the future, if it is fine to be useful that we can collaborate so we can better protect the land environment while also allowing business and community to thrive. Thank you, Ami. That's great. Thank you so much. I will just ask that the other speakers turn on their cameras and appear in the gallery view because then we can have a bit of a discussion here. Something that we have about 10 minutes for that before we split up into the breakout groups. Something that really struck me about the four presentations was the scalar aspect of what's going on here. I think Ibu Mirna really focused, in fact, at the village level, connecting to national scale, of course, and multiple scales, but really the importance of village level engagement. And I think then Marcel really was focusing at kind of biome in a way, emphasizing the importance of a jurisdictional approach that, in fact, encompasses the ecosystem of interest. So if it's a peat dome, you want to make sure that that full peat dome is encompassed in the political territory that's being managed. And then Josie talked about the district level and how piling at the district level has been really important and made connections to Bapanas, in fact, at the national level, but through a really strong pilot at the district level. So I guess my question to the speakers is how do we link effectively across these scales? All scales are important for action, but how can we really make these connections? Maybe we could start with Josie, maybe, from your experience. How do we make the connections across scales? Thank you, Amy. So yeah, I think it is important to use the existing governance structure of decentralization, which center with the Ministry of Home Affairs and in terms of institutional setup and in terms of the different level, it's actually centered in the district level because the other level would be the reporting process will work based on the authority of a certain governmental, what do you call it, the government authority itself. So for example, in agriculture, the choice of districts is because for food and agriculture, the authority is within the district level government. So I think for peatland restoration in this sense, but Mirna, of course, no more about this compared to me, but I think it would be centered with the provincial level government by Mirna. So even though we know that village level government have its own authority, but it is based on the structure, which makes the actual authority can produce, for examples, what we call an especka at the national level, as well as the exercise of budget at the authority that has the main authority over a specific issue. So I don't know whether it helps in a way because I'm thinking while discussing here. So for example, I will use the initiative by the Asia Foundation and other colleagues who are working on the TAPE, TAPE, and TAKE. And I think that helps to show us how, for example, the district level government can encourage village level government to perform as well through fiscal incentive. So it might be showing that in the case of a culture, for example, the focal point would be the district level government. And the district level government can then exercise the power that they have to make sure that the village level government contribute to the performance at the district level jurisdiction. I don't know. Hopefully it helps. What do other speakers think about this scale or multi-scaler aspect? May I say something, Amy? Yes, they are connected, but also they are different because scale matters. What is relevant at the local scale is not necessary. Relevant at the national scale, for instance. So for instance, we work for FIRE. It's very relevant at the provincial and national, but sometimes it's not really matter at the village level because they have a long time in particular village working with FIRE. So there's a connection, but I don't want to push. It must be connected. So it can be a connection. It can be not because scale really matters. Politics matters a lot at the provincial, at the village scale. That's more lively, more economic matters. So this is very different, very different matters at the local as well as at the national. So there is a different indicator. That's what I meant, actually, a localized indicator. So what on the FSC, for instance, the forest stewardship concern, they only develop principles at the global level, but criteria is developed at the national level. Even indicators sometimes going down. So we do not enforce all principles for the worldwide. So it should be localized and also should be a minimum. People try to develop indicators as much as complete as you can. But finally, it's difficult to measure on the ground. So difficult to understand the performance. It's good to have a minimum set, not many, minimum, but able to explain. Also, cheap is not very, very expensive. So nobody can measure. So it's also a simple thing. So everybody can see it. That's what you can have a look at. If it is complicated only you, scientists can understand, doesn't really matter a lot for the people that they do not understand it. Thank you. Amy, if I also may, it's a key question, I think. We've seen so far in Indonesia that a lot of the peatland restoration has happened at the village level. But these are always often confined to just relatively small projects of a couple of hectares, maybe 100, maybe 1,000 hectares, never the entire landscapes. A lot of these projects have been very successful, I think, socially, also in terms of initial restoration. But we know that, ideologically, all these areas in the peat dome are interconnected and you can't restore just the 10% of the peat dome needs to be done in its entirety. So I think the next step for the Indonesian government will be to bring these stakeholders together, the experiences from these village projects and then see what can we do by combining these? Can we upscale, for instance, the finance mechanisms behind it? So we can also create a sustainable and a sustained finance support to these communities. Peatland restoration is very difficult. Even if you make a profit, it remains very difficult. And so there's a need for continuous and long-term government support in this. How can that, by combining these efforts, you can then also bring in other finance, such as carbon finance, GCF, you can look at what the role of commercial banks can be in relation to the private sector that's active in the peatland. And as long as everybody can work together on a common vision for the restoration of the peatland, I think that can work. So you need to have that platform of our coordination at the landscape level, but then to upscale that further to provincial level and also multi-provincial and national level, you need to have this kind of stakeholder platforms that encompass multiple landscapes. So you can combine all the landscapes in the province and bring the stakeholders from these landscapes together. I think it requires actually additional government structures than the ones we have at the moment. In the Netherlands, where I'm from, we have water boards to deal with the whole issue of water management, which is very complex and is multi-sector and multi-stakeholder. I think in peatlands, you would need to have something similar, maybe peatland boards, where the stakeholders can come together and also the heads of the peatland boards from multiple landscapes can combine their efforts in upscaling finance support mechanism, fiscal incentive mechanisms, and technological support to the communities. Thank you. You're wonderful. I don't know if Mirna is still with us, but if she is, it would be nice to have a final word from the government. She sent me a message that she has to leave for Kanbaru to catch her flight. Okay, that's okay. But I think this is really important, and many of the speakers, you've already addressed some of the questions coming into the chat about kind of these local level experiences, often even based on local knowledge, being scaled up to multiple levels. And that maybe new governmental structures in fact are needed to facilitate this cross-scale learning and implementation. There's so many specific questions for speakers in the chat. I would ask that all of you after the breakouts, if you could go back into the chat and the questions that are directed to you, please answer them for the participants because I think there's some really nice information and content in there that I don't want to miss out on. And this chat is part of the dynamic of the workshop itself. But I do think we now need to move to the breakouts. Is that correct, Pak Daniel? Yes, sis, thank you very much, Amy. Thank you. Very animating session you have. And certainly governance issue is cut across various levels. That's what I learned today. And in the next session, where we are talking about social and economic aspect, I believe the speakers will offer to you very local issues that are going to be very much related to what's going on on the ground. And let's give us the chance for the host to split us into these two sessions. I believe they have the list of you, which session you prefer to join. And we will be immediately move towards different rooms as you're expecting to be.


Good morning everyone. My name is Vipin. I work as a staff engineer in VMware and OpenStack I contribute to Cinder and OS profiler. So this talk is about OS profiler. It's a library in Oslo It's meant for profiling OpenStack API request So here's the agenda. So we first discussed some use cases That's applicable for OS profiler and followed by an overview on OS profiler Then we discussed some internals of OS profiler. So if you're planning to contribute OS profiler or you're planning to implement integrate OS profiler in your project then that will be useful and Finally we show a small demo which describes some of the things we discussed so Let's start with the use cases So I hope you can see the image clearly so Actually, this is part of a call graph generated for a command The command is NOVA boot from boot an instance from block device with an image source So this request it Actually contains it propagates a NOVA, Cinder, Glance and Neutron So this call graph contains method from NOVA, Cinder, Glance and Neutron So let's take a close look at that call graph Okay, it didn't work. Okay, so you can see the call graph so So this is a call graph generated for that command NOVA boot from boot an instance from block device. So So it's it's pretty huge So at the point not he's this doesn't contain all the methods This only contains the API calls, RPC calls, driver and the manager public methods So a lot of methods are not in this call graph. So we can Imagine how big is a call graph the actual call graph will be So as in that we have a performance issue in one of these commands. So how do we troubleshoot it? the performance issue maybe in We don't know which service it will be because a lot of services are involved and The service is running in multiple hosts. So we we don't know where where to start So the problem may be in a single method So it's often the case that some methods doesn't scale well. So and that may be the reason why We have performance issue. So identifying the performance bottleneck is a first step in diagnosing these kind of performance issues and then some cases we need to benchmark two drivers for example LVM and CIF for Cinder and Using a tool such as rally and and often times we wonder, okay Why delete is not it it's taking a lot of time in LVM compared to CIF. So we want to understand how Why it's taking more time so we can use So that is another use case where we would like to understand the performance issues with certain APIs So what what's the solution for this the solution is distributed tracing. So what is distributed tracing? So distributed tracing basically helps you understand how a request flows through the distributed system So it also gathers the timing information how much time is spent in each method so So in OS profiler is actually the distributed tracing tool for open stack Okay, so how do we use OS profiler? To for for troubleshooting these kind of issues. So how to generate a distributed trace using OS profiler? So it's it's really simple you we just need to use a profile option with the command for example for creating Cinder volume we can specify Profile and provide a secret key. So the secret key is something configured by the admin just to make sure that only Certain users can enable the distributed tracing. So when you run this Cinder command It actually prints An ID for your distributed trace which you can later use to generate a report So similarly for Nova we can specify the same option profile with a secret key So once we generate the trace so to view the trace we can use command from OS profiler It's a OS profiler trace show command And we have to give the trace ID printed when we use a profile option So let's take a look at the sample report generated from OS profiler Okay Okay, I don't know Okay, I don't know the reason why it's not working So anyway, we will cover this again in the demo So the sample report, okay, let's move on to the next one. So a brief overview on OS profiler So as I mentioned, it's a distributor tracing tool for OpenStack and it generates a single trace for an API request So the trace basically consists of multiple trace element trace element is is that corresponds to a method invoked during the API request and Using OS profiler, we can generate the trace report in different formats such as HTML JSON and dot so dot is a language used to draw graphs So OS profiler prints can print the trace report in dot format so you can use any dot rendering To actually display the graph. So you have we've seen the call graph which I show in the first slide so that is generated using this dot option and The trace report includes the call hierarchy the time spent in each method the project or service which the method is part of and Some context to do some debugging and also categorize the trace elements into different types such as a WSGI RPC DB or driver calls depending on what type of call it is so Natural question could be why OS profiler? Why can't we use sipkin or Say c profile so the short answer is a scope is totally different so we don't want any tight coupling with a non-OpenStack solution and Also the same time we should be able to enable or disable a tracing without restarting the service and In some cases we should be able to enable tracing for specific API request so we don't want to enable tracing for all all type of API's and As I mentioned earlier, we don't want all users to run tracing and put a lot of stress on the system and also it should be easy to integrate with OpenStack projects and There's no need to There's no need to include all methods in the trace because we start including all methods in the trace The trace report will be too noisy and it'll be difficult to analyze that so these were some of the goals and So we had to come up with a new project of new library for men for OpenStack that is OS profiler So what are the projects supported by OS profiler so? so you can see the complete list here, so some of them are under review and most of the The core projects it's all demudged the integration. It's already integrated so we can straight away start using OS profiler with those commands Okay, so let's move on to some of the internals of OS profiler So these are the different modules in OS profiler So I'll start with the profiler module So that that that module is responsible for generating the trace trace elements for so when you when a method gets invoked it generates a trace element before it enters a method and Once it exits some method it generates another trace element so that we can calculate the time difference The time for execution of that method so the profiler once it generates a trace element It needs to publish and store it somewhere. We distorted something called trace data store So there are different types of trace data so we support So it uses a notify module to publish the trace to the trace data store the notified module it supports a plugin mechanism So which means that we can actually have different kinds of plug-in to support different types of trace data stores So the web module is a one which is responsible for validating the user request whether it's a valid request to enable Profiling so it basically checks whether the secret key mentioned by the user is correct But not in that way. I mean it uses something all a hash signature So So that's a purpose of the web modules initializer as the name indicates it initializes always profiler It initializes the web module with the secret key Specified in the config file the admin specifies the secret key in the config file and the initializer initializes the web module with that and it also initializes the notify So basically tells the notify. Okay. This is my trace data store connection parameters So initialize the trace data store so the notify uses appropriate driver to initialize the to initialize an instance of a driver so So this is these are the main components in OS profiler So the profiler generates a trace element when it enters and exits a method So what's the structure of the trace element? So the most important element the part of the trace element is a base ID So the base ID is a ID of the trace So this is something this is what is printed when you run the the command with the profile option So you this is a ID representing the entire trace the ID is the ID for that for that particular trace element and parent ID is the ID of Of the function of the parent function in the call hierarchy then the project and the service are pretty obvious The project can be sinter and over extra and so is is sinter volume now a compute and The name is a type of that trace element. It's a WS GI or RPC or DB driver So when it before entering the method we generate we suffix start to the the type and when it exits We we put a suffix chop just to understand To calculate the time difference then the timestamp to generate to to find out the time spent in the method and Info is a debugging info. So it varies with the type of the method for WSGA. We will have a path we have a query method and scheme and For RPC and driver we have a function name arguments and keyword arguments for DB We will have the statements and the parameters So I mentioned we support different types of trace data store to store the trace element So that we can generate the report at a later point. So since boys prefer 1.3 We had a dependency on silo meter. So we publish trace data as events to the message bus and Which is consumed by silo meter and persist in its store? so That is a that is not a good dependency because every time the someone wants to run they need they need to enable silo meter and So in 1.4. We introduce a driver framework for trace data persistence. So we could We could we could we could have say log inside a MongoDB as a store for persisting trace data So right now that the current version OS profile is 1.9. It supports silo meter elastic search log inside MongoDB and log inside and grids so so if you're not happy with Silo meter and the Oslo messaging Which which is a default tray store so we can replace it with say reduce or MongoDB so So how is it trace generated? So the first step is the initialization of OS profiler So we initialize OS profiler with a secret key and a connection string. So this is specified by the admin in the config file So the connection string is like it first the scheme indicates what type of back-ended is it's MongoDB or it is then the The connection parameters appropriate for that back-end. So this is actually interpreted by the appropriate driver class so first we initialize the initialize OS profiler with these config options and Then if OS profiler is initialized we patch the manager and the driver classes So and once we patch it then Then it's capable of generating the trace elements. So how does a client initiate a trace request? How does it tell the server that okay? I need to enable trace request. So the user specify the secret key and Once the secret key is specified at the client side We initialize a profiler using the secret key once a profiler is initialized. We send to additional headers so the header contains a data, which is a base ID generated for that trace and hash signature so the ashes signature is computed using this data and and and and the secret key so And and we we pass we we send this data and the hash signature as to additional headers if the profiler is initialized So at the server side, how do we verify whether a trace request is valid or not? So we verify the hash signature So we regenerate the trace signature using the data sent from the client and the secret key configured in the server and We compare the hash signatures and if it matches that means the user entered the correct key So we initialize the profiler with the base ID So so once you patch the class it's capable of generating the Trace elements. So basically it it applied the decorator which is shown here the profiler dot trace to each method and So whenever the method execution comes it Generates before it enters it it generates a trace element and exits it generate another trace element for RPC calls we in the request context serialization before making the RPC call we construct some trace info dictionary and with the with the current ID of the function and the base ID which is the ID of the trace and During DC realization, we check if this trace info is set in the context if it is set We initialize a profiler with the base ID and the parent ID Okay, so let's go through a short demo. This is a record the one or live demo so we will walk through these steps configuring OS profiler and Running the various commands with the profiling enable and and and the trace report generation okay, the First step is to configure OS profiler. So So we call the config file of the project which we want to enable OS profiler. So we need Basically for option we have four options enable must be set to true Then trace equal alchemy By default it's false. So if it's turned on then tracing will be enabled for DB queries as well But it will generate a lot of noise And the HMAC keys is a secret key We can specify more than one key and the connection string is a connection parameter for your trace data store Trace there store. So in this case, I'm using login site. We could use MongoDB already is here or we can just Keep it empty and it will defaults to Cilometer and the oslo messaging So we need similar settings So we need similar settings in other Projects as well. We have to use the same Secret key and the connection string Yeah, so So I'm going to run a command To attach a Cintervolume to a nova instance with the profiling enabled so you can see in the help The profile option is clearly documented. It's my secret key is full So I run the nova command with the Profile option and give the key as full. Okay, so it prints a command To generate the the trace report So you can you can see that the the last part the UID is a base ID of the trace So the command is completed. So we can we can generate the trace trace report so By default it assumes that Cilometer is your trace source. So it tried to connect to Cilometer and Cilometer client was not installed in my system. So that's why It gives an error but actually my trace The trace chore is log inside. So how do I specify to the How do you specify OS profile a tracer because this is running in the client side not on the server side? That we normally run it in two machines. So the config file which we saw earlier that is server side configuration So we need to tell OS profiler. Okay, where to connect to get the Trace information. So we have to specify the connection string Either to the tracer command or we can set an environment variable So so we can see the connection string help here. So We can either Give it along with the command or we can set the environment variable OS profiler connection string. So I'm going to set the Environment variable here. So now it generates a trace report, but it's printed the console. So let's redirect that to a file So now we can go and open that HTML file So this is a default trace report generated by OS profiler So we can we can see we can Collapse it expanded the various various nodes These are different methods. So we can we can see the total time We can see the total time spent on the top at the level zero and we can see the type of the Trace element the project service and The host generated that trace trace element If it's running in different holes, so that will be helpful where it was generated and we can click the detail So that is that's a debugging context. So So in this case, it's a driver method. So it has a function name Arguments and the keyword arguments. So that will be helpful for debugging. Okay, so Let's see one other capability in OS profiler. So I'm going to create a Cinder volume using a volume type So the volume type has this it has a storage policy associated with that So I'm going to use a store the volume type gold So it has a storage policy named bar and this is an invalid storage policy. So obviously when you create the volume It will error out So let's see whether we can get that error information when we run the trace So I'm going to run this Cinder create command with profiling enable so the the volume we just created that is an error state because The extra spec has an invalid value. So this is a trace report generated and And if you look in that method the trace element corresponding that method, we can we can see the exception information as well So so we support Some other formats for trace report. So let's go to the let's go and see the dot format so So we support json. So if you want to inspect the trace report in json, we can use that option So I'm going to show this dot option So when you use the dot option, it generates the trace report in dot format And if needed we can actually so we need a rendering a renter to actually create a call graph using the dot Output so there's an option to render it In OS profiler itself. So it generates a PDF file showing the graph But we need to install graph with package for that. So the volume is now created. So we can run the trace show command Giving the dot option to generate the output in graph format. So we have to use in service channel We have to use dot as output format And we have to also mention render dot option and give a file name to render the output to So we can we can see that the dot format here and We can use any third-party dot dot format rendering to display this graph. So this Since we use a render dot option OS profiler rendered that into a file called trace. So we can open that and And see the output. So this is a simple trace So we cannot put a lot of information here because it'd be difficult to display that unlike the waterfall style graph model in the HTML format But it's it's useful for smaller smaller graphs. Okay, so by that Demo is over so So these are the folks that they are core reviews and OS profiler. So they help me prepare this presentation so thanks to them and Thank you everyone for attending the talk Thank you Christians I'm just wondering if you know if there's how much overhead when you have a trace on does that Production cloud is that gonna be dangerous? Yeah, so I was expecting that question. So so while Implementing the driver for log inside I did some study, but we haven't published it and it's not a formal study So I did it use me a laptop so we cannot say that okay whether the results are valid for a production system So when I did that So I did treat us one was no profiling no patching and one with the default option That is the messaging and the silo meter and then one with the log inside which I'm using So definitely there's an overhead, but it's in the order of milliseconds. I find I was running Few commands like cinder list and Cinder create I was running in a batch and then trying to take the average not just running one test so and compared to comparing log inside and Cilometer I find that the Pushing the trace elements publishing trace alarm was much faster With the log inside, but the report generation was comparatively slower compared to Kilometer, so yeah, so this is something we have discussed so we are planning to do that. Yeah Two questions is there Command that shows you a list of the profiles that have been captured in the commands that generated them And is there a way to aggregate the results from running the same command a few times? Okay, so you just want to know what all traces have been generated in the past Yeah, if I don't copy. Yeah. Yeah, I got it. Yeah, so Unfortunately, it is not there, but our driver framework So we have added the API to extend it So that is something which we can do it. So I'll take a note of it. Yeah, that would be useful Yeah, I have two questions first is how does this support me means the hooks already added in the NOAA or Cinder or this component and How many hooks are added? Do you know? Okay? So you basically want to know how many hooks were added in our center, etc Yeah, and the other already exists in the latest release. Oh, okay. So I don't have the complete Idea of how many places at least I can tell from center because I contribute to Cinder so in Cinder we we actually added hooks to all the public methods in the manager classes that's a scheduler managers in the volume manager and all the WSTI APIs and Whenever depending on your driver all the public methods in the driver will also be patched So NOAA similar things, but I don't have the exact number Instead of yes, yes, yes, that's right The second question is if you use rally or some your horizon to do a batch of operations Will the truth be just one or each operation will have one trace means how to differentiate so you just want trigger from rally, right? Yeah. Yeah, so this so The rally integration is not a completed. So it's actually the spec is much So to do the integration with rally so every time before you invoke an API You should initialize a profiler in the current thread So whenever you initialize a new New trace ID will be created. So it's up to the client if rally is initializing it once all the APS It invokes in that thread will be a single trace So it initialize a profiler run API one API to all these will be part of the same trace But after completing API one it reinitialize the profiler. We will have a different trace So basically each trace is corresponding to one operation instead of one open stack operation So you saw operation we as open stack operation So each trace corresponds to an open stack operation because it So I don't understand what do you mean by open set if you create what use ready to create 100 VMs How many traces will generate? So you may be using the novel boot command right repeatedly In a for loop or some things Okay, so You initialize a profiler then in a for loop you try to create a hundred VMs So all these operations will be part of the same trace But at the end of creating a particular VM we reinitialize a profiler. We will have different different races And you can query the profiler module to get the trace the current race ID If you're doing this programmatically so So, I mean you you mentioned there that you've got a shared secret effectively to to authorize whether the profile isn't put in You know turned on yeah, is there anything you're doing to expand upon that because obviously you can see how in a Production cloud or a production intent cloud then being able to hand out one-time keys to users with problems might be a useful thing Yeah, so this is something the core team has discussed in the past So definitely that is there in the The goal list. Yeah, so that is something we plan to address. Yeah, sorry two questions It's recording profile in synchronous or asynchronous Recording sorry notify profile. Okay, not if oh it synchronize actually yeah, yeah Yeah, so we can do some optimizations there like caching the the Profile elements and sending it in a batch. Yeah, so those optimizations are not in done It's synchronous Okay, if cell meter is crushed The API process is Stop So what's a question Okay as a question Should I set Key at same value between components. Yeah, no machine that you should set the same value. Otherwise because One service will call the other service, right? So it the decline will send the secret key So and it should use the same key. Yeah Thank you. One question over here. This went on I guess two-part question first one is Would you recommend or recommend against using this as a distributed tracing tool outside of OpenStack and kind of related to that You mentioned you didn't want to tightly couple distributed tracing to some other tool. Yeah Were there other distributed tracing tools that you looked at? Yeah So so currently there is a proposal a blueprint proposed to make the trace element information We generate compatible with open tracing API. So this open tracing API once we generate our trace elements in that format We could use we could use a lot of other visualization tools. So that is a plan rather than putting always profiler in other projects because Our use cases may not match. I mean it may not be generic one, right? So so so the plan is to make the trace element compatible with open tracing and then Integrate with a lot of other tooling which supports open tracing. Yeah, thank you So would you recommend this to I mean have do you know of anyone that have used this, you know in the production? Yeah, so I know some of our customers used it. Yeah, so diagnose performance issues and we as developers we use it whenever we get some performance issues Yeah, I see but not on the regular basis. We're on the regular base when you have performance issues. Yeah, I see got it. Thank you Yeah, I think I think the question that got missed over here before was they were asking since the Sending the trace data is synchronous. If you're sending it to Selometer and Selometer is crashed. Does that stop your? Like your sender proxy process? Yeah, so let me see I Think it depends on the driver implementation how the driver catches the exception So it's in and if it ignores that then it will continue otherwise And if the driver doesn't it propagates that exception back then your OpenStack request itself will fail Yeah, so we need to we need to fix that. Yeah, that's right. Okay. I think that's it. Thank you for attending


Today, we are taking all of our folks in First Brigade who are type O blood and we are titering them for antibodies. One of the issues that we've had throughout the last 20 years of conflict was how to bring fresh whole blood as far forward as possible. What we've learned is that when you're resuscitating a trauma patient, the best fluid to replace blood with is blood. Cold stored blood products have to be refrigerated. They only last for so long. So what we're doing here today is we're essentially turning all of our O type blood groups here in First Brigade into donors at the point of injury. So for example, if there's a platoon outmaneuvering and they take contact, someone needs whole blood. What we're doing here is allowing the medic at the point of injury to pull a unit of blood off somebody and give it directly to the patient at the point of injury. Why this is so interesting is because this is a capability that soft elements have had for a long time. And this is the first time that we're going to see the Army start implementing this at scale.


On the next episode of Backroads. You are my special girl I'm changing on each and every night of day You won't hear no goodbye


All right, let's talk about privacy because this was a big one for me. So there's two things that Apple announced with privacy as I see it, you guys might have more. The first is very much focused on mail, which is that mail now has constructs in it that will keep all of those tracking beacons that are buried in email addresses, the little pixel trackers that are buried in email, not email addresses that are buried in email messages that track your IP address and then your location. And then because you've loaded the pixel that tells someone whoever owns the pixel that you have read the message, they are now obscuring everything from all of that. So that theoretically people don't even know that you've opened their email. They don't know who you are. They don't know where you were when you did that. And this, I can think of a million cases where maybe not a million, but many cases where this will cause some issues for the way certain people do business, but it totally fits with what Apple does and I applaud them for it. I think it's great. I think it causes more good than harm, though it will cause a little bit of harm and that's okay. And then there's iCloud Plus. So I wanna put that sort of on a shelf and then there's iCloud Plus, which has three things in it and I'm going out of order intentionally. It has HomeKit secure video, which will open to more than five cameras, which is great. There is hide my email, which is let you create obfuscated email addresses so that email gets to you but no one knows what your email address is if you wanna use it. They say that the pricing on iCloud Plus remains the same, which is- Tell me you didn't think as they were going through this, when you saw that plus sign, you were thinking- Yeah, plus to the price. Only $5 more a night. Of course. Not a month. Well, especially with the third feature, which is what they're calling private relay, where it sounds like they've essentially recreated the Tor network, although hopefully quite a bit faster, where it's bouncing your traffic through two different servers so nobody knows who you are or where you're coming from, not even Apple's servers in the middle. But also they're encrypting everything. Correct, right. So let's think about it and they say no performance lag. Right, which is unlike the Tor network, right? Which is right, yeah. But that's really interesting to me that they've essentially taken this and done it. It's effectively like a multi-layered VPN, although I'm sure someone out there will have a better terminology than that feedback at mackeykev.com is the email address to yell at us and tell us what we got wrong. I would say a VPN tends to fool somebody into thinking your traffic originates from another country, say. Sure. Whereas this wouldn't do that. Maybe what? There's no country of origin or whatever. It's all Cupertino. My concern with that and my friend Dave did say feedback at mackeykev.com just to make sure you guys heard that. I said feedback at mackeykev.com. Okay, good. Feedback? At mackeykev.com. Well, we usually do it in threes, Mark, but we'll let you add one. What concerns me is similar to what they did with the mac address randomization in iOS. This is gonna break things that I may not want broken. Fair. I may want people that email me to know that I opened it or know my location or make my, and to tell you the truth. So some of the, in iOS now, they have a thing now where it's like, oh, do you want to share blah, blah with blah, blah, this info with this person? And I'm like, honestly, call me crazy, but for most of them, I'm like, I like this vendor. So yes, I will allow them to continue to track me because, and I'll talk about it in a future episode, but the other day in Instagram, I got an ad for a product that I've been looking for for ages and it popped right up. Because they know what I like. It was a charging cable, multi-tip, and it's animated and stuff. And I've been looking for this, and they know this because, I look for it every now and then on Amazon and stuff. Like I got, and all of a sudden it came up. So not all tracking is bad, in my opinion. And Apple lets you make that choice, right? They wouldn't have popped up. They'd let you make the choice, and I think that's good here. The mention of masking all this in mail, it sounds like they're gonna by default do this and I'm not happy about this. Just like when they by default would randomize your MAC address, that caused a lot of grief for a lot of people because if they tied their MAC address to their DHCP, all of a sudden it broke. And they're like, what did I do wrong? And it's like, you did nothing wrong. Apple just kind of made a choice for you, so. One thing about this private relay that I want to find out more, and I'm almost certain that I will be able to watch a dub dub session this week to answer this question, is what apps does it affect? Because it sounds like it might only affect Safari. And I would like it to be able to affect other things. Very specifically, I would like to see it affect the podcast app. Because just like there are pixels that are being put into our mail, some podcasts, not this one, but some podcasts are allowing pixels to be put into the download of the podcast. And you have no, like the worst part about that is there is no way to opt in.  And then they track you by going to, when you go to different sponsors' websites and they correlate it all together. Again, it's fine if you wanna opt into being tracked, but podcasting doesn't really have a way of doing that. So I would love to see Apple fix that problem through Private Relay with more than just Safari and we'll find out. You know, I have an app that I use a fair amount that is now broke. I literally just today got an announcement from them that basically said, we've done everything we can to work with Apple. We cannot find a solution that Apple will accept. And it's called Mail Plane. And it's basically a Google Mail app. That's on my end. The squeaky door. I heard the squeaky door. I'm like, did someone just walk into my room? I'm in a haunted house. That's cool. But basically the app does that thing where they put up a, in effect, a web page with Google Mail on the web page, but they do it, they hide everything they can hide. But it's now not gonna work anymore. And they sent out a letter saying basically, we'll be happy to give you your money back, but we can't fix this. And now it's end of life, which it just makes me sad because I'm so used to it. I now need to find a mail client. And I don't know what the good mail client is, but maybe I'll use Monterey's mail. I'll give it another shot. Sure, sure. Yeah. My next time trying Apple Mail. I used to use third-party mail clients. I chose them very specifically for their features. And I ran into that scenario, maybe not quite that draconian and end, but I ran into that sort of functionality limitation scenario too many times that I decided I'm just all in on Apple Mail. Hopefully it remains extensible, at least as it is now, so I can use things like Mail Acton and Sig Pro and the little plugins for mail. Because of that exact thing where Apple can just like decide, hey, today's the day. That doesn't work anymore. I can't have that done to my email. One of the things that it just came up with you and me today was you said, I send mail and then I get a delay. I use that in mail plane. Is that a thing in Apple Mail that I can send a mail and have it be delayed? It's not, but it is a thing in Small Cubed which is the company that makes a, what they call the Small Cubed Suite, which is really a collection of four plugins. One of those plugins is called Mail Acton and Mail Acton lets me do that among many other things, but that is mail plugin or it's a mail plugin. Yeah, I'll put it in the show notes and I'll because we're buddies here now. I'll send you a link to the thing. But yeah, yeah. Yep, no, Mail Acton is great for that. And as a side delaying my email and I choose to have it delayed by two minutes by default that you can choose whatever you want, including none, no delay, but that two minute delay has a saved my bacon more times than I care to admit. And both in terms of like, I really don't wanna send that email, but also you hit send, you go focus on something else and then it hits you like, wait, I should also say this or I should, while I'm in this, I should ask this question and I can go and edit the message and rework it and send it all off and then it's like all good. I also have it warn me if the word attaches in the email and yet there is no attachment. Yep, yep. Same thing. That's funny. So yeah, SmallCube's Mail Suite, which has Mail Acton in it is your answer. Sorry to derail the conversation. It's okay.


Everybody forced here ripping one 12 box case of 2022 tops chrome f1 hobby for Jeff. Hey, here we go Jeff Here we go Yeah, for sure. We'll worry, but even like An early like that's an early Jude card in the span of his career as far as like cards go It'll still it'll still hold pretty good value. I would think So Do a recap of anything that we mag and some of the nicer color and on autos Let's go magic And everybody personals are full for tonight. I'll get to Barry Raymond Simon Michael are in a Keem D tonight. I think anything else Probably goes tomorrow chance it goes tonight, but I'll assume goes tomorrow refractor Felipe Drogovic Bankero and Franz Wagner are the magic's best players Lewis Hamilton speed demons it's getting it's getting popularity for sure Taylor Lane and rich watch it a Kevin Magnuson Checkered refractor Daniel Ricardo. I Watched a couple races last year Fractor Max Verstappen Grand Prix winner in Mexico Sonata speed demons Lewis Hamilton purple checkered flag numbered 145 of 199 It's Grand Prix winner Bach rain Future stars refractor Richard Vershore Vershore refractor again. Yeah, that definitely helped a lot Taylor They did a really good job producing that show put it together purple green future stars Jack Duhan Leclerc speed demons the drive to survive Formula 1 Netflix like doku series many diamonds Daniel Ricardo 180 of 299 Ghastly and Sonoda, I just see I just see red checkered flag Whoa, that's sick four of five Red checkered flag extractor Carlo signs Yup Big card first box what a way to start the case Congrats Jeff That's what's up, buddy That's how you start on the right foot right there card number 32 on the signs Holy cow, man This just became I mean this is gonna miss it anyway. Yep. That will definitely do That uh, that'll go a long way here Let's just see more of that then right refractor you can see no to auto Lance stroll Things are things are shaping up real nicely here refractor auto Lance stroll that is some auto right there Just a giant three giant three Speed demons George Russell and the autos are one every two boxes on average, right? Is that the auto odds guys? One every 63 packs. So yeah, one of every one every three boxes theoretically But they seem to come in like droves 4047 Purple future stars ends up at the Paldi 128 or 399 they're doing Vegas. They got Miami. You got Checkered flag Grand Prix winter pop rain Lewis Hamilton Well toss and Joe rookie refractor Joe Guan Yu Award winners refractor Lewis Hamilton a lot T. Fee speed demons True green Charles eclair There we go 35 and 99 JQ's refractor refractor Marcus Armstrong orange red Billy by drogovich future stars nice orange red one of the hundred packs so about a couple of case on that Speed demon signs purple refractor Sebastian Vettel 215 to 399 Legend right there actor Franz toast and a gold checkered flag of Max Verstappen Grand Prix winner Austria 28 out of 50 Last card of both both first two boxes and pretty darn nice Nice we're stopping on one of those Variations there. That's good stuff, man room instead of boom. I'm down Sergio Perez refractor Grand Prix driver of the day Mexico You're like Max Verstappen Taylor You don't like the goat Vettel checkered flag Mando and Ricardo refractor Carlos signs album Minidiamond Frederick Vesti to 299 for 299 That's the bono con refractor What's wrong with being a whiny baby Taylor? So the sisters who run with that Fractura Max Verstappen Daniel Ricardo refractor Purple Green Lewis Hamilton very nice one every couple boxes on those Speed demons Lando Norris purple checkered flag Kimmy Reichen and Grand Prix driver of the day Abu Dhabi Wingstop chicken sandwich So when you stop on 50 Future charger factor Marcus Armstrong and a gold Sebastian Vettel I guess at the last part of every box just gonna be sick, right? Three straight. I mean, what are you gonna do man 38 to 50 on the gold Vettel? I don't know if it's the way. I'm like stacking the packs. I have no idea Ridiculous, okay. I'm gonna do the same time the same thing I do every time And that that that pack right there That's that's the magic pack Four times in a row four times in a row Lewis Hamilton refractor nice And the auto Marcus Armstrong refractor auto Marcus Armstrong must be related to a Lance Neil yeah Lance and Neil. They're all related. I think Daryl Magic legend Daryl Armstrong Heck yeah, Jeff has been pretty pretty pretty pretty pretty pretty good a purple Future Stars Marcus Armstrong 28 of 399. Yeah, that's how it happened Rory Checkered flag Liam Lawson. So back-to-back nights. I think it was Saturday and Sunday I pulled one of one Zach Wilson's at a high-end products Yeah, refractor George Russell. Oh, yes, I must think about this very very Thoughtfully Yes, yes To be or not to be rookie refractor Joe Guan you like hmm poppy seed or everything Which type of cream cheese? The locks purple rookie Joe Guan you that's a nice one 29 of 399 Latifi refractor Mac should be the star next year. He's better than Zappy Future Stars refractor. I'll be called. Well Telling me I get that dude some actual weapons Gold Purple Clamant Novak Novilock, excuse me You go gold purple one in every 80 it looks like I think on these Yeah, first rounder for Zappy to the Jets. Oh, let's do that. Let's do that trade Mini diamond Fernando Alonso Numbered 109 of 299. Yeah, plus at 15th overall Mac is a cheap option compared to like all other starting quarterbacks in the league So like we want to look at it that way even What are we doing refractor Oscar Piazzari? And then that's the bottom O'Connor pink refractor 73 of 75 No, he's card there. I think factor still nice. So low numbered Solid solid card there not as big as the others but still pretty solid refractor Fernando Alonso I'm not saying I love Mac Jones. There's definitely better options But if we're talking like on the roster currently he's better than Zappy Brando Alonso speed demons there'll be another time Rory or Paul There'll be another time Paul Checkered flag Ali Caldwell So Ghastly and Sonoda Constructor Constructor coalition Sounds pretty cool, huh? Come how do you say the same bull bull Bashi bull Bashi? How do you say the name Sean? Come on the train or the Transition guide over here Alexander albon refractor Speed demon Sergio prez. So what do we think Derek card of the Colts? Maybe they'll go get it a great a great veteran quarterback like Who else could they go get Jared goff to the Colts? Maybe which would have been a big operator for that this year? You're Good when he has weapons he puts up numbers. Yeah, he bawled out purple refractor toto wolf This looks kind of strange. What is up here? It's a printing thing. See how it's like that's a like the bottom of the card here I'll send a box and wrapper this might just be like a printing error Whole cards kind of like that. Don't know if that's Supposed to be like that or just an accident. Let's grab a box and wrapper I don't know if they'll do anything about like a printing error if they're all the same I'll send a box and wrapper Sometimes it'd be like that Logo Fractor. All right guys. All right easy guys easy guys Card of the Dolphins. Let's go. I'll take Sebastian Vettel refractor. I'll take our all day and twice on Sunday Kevin Magnuson refractor card of the Jets Card of the Saints so car is gonna play for your six teams next year is what I hear purple and green p.r. Gossley Car is like the gary Sheffield apparently of the nfl it's playing for all of them Logan sergeant purple checkered flag numbered 61 of 199 Jimmy G would be like perfect for the Jets Just distribute the ball to the playmakers Steady Eddie steady Eddie Steady Eddie refractor f1 award winners Constructors world champion Hey Kevin Magnuson gold checkered flag number 39 out of 50 pretty good Giovanni. How are you? Slap bet car makes Jets contenders Contenders for the playoffs. Oh, yeah, we haven't had that game on for like 45 minutes Joshua We've been watching Orlando magic pregame for since like since like 9 30 eastern Oh, we got an orange. Come on now Pierre Armstrong. Sorry Pierre. I'm from Marcus Armstrong refractor Orange auto Sergio Perez 14 to 25 Sick Sergio Looking good too. That is noise Sweet as the bono cone speed demons home home stays in Kansas City I don't know James. Uh Grand Prix Uh winner 2021 round 15 Lewis Hamilton mini diamond 138 of 299 Checkered flag future stars off Bochum Landon Norris refractor I'm surprised all these people saying Derek Carter certain places wouldn't just want Lamar Jackson instead Max versus stopping refractor Grand Prix winner France Speed demons Lewis Hamilton doesn't matter who starts in San Francisco quarterbacks like the least position the least important position on their team They're all lines so good. The weapons are so good. It doesn't matter who plays quarterback there Dude, Sean could get them to a first a first round victory playoffs Go darts. Where's McAfrey? Where's Kittle? Where's Debo? That's all you have to do in that office, you don't throw a deep purple mix Schumacher number 138 of 399 Put me in the slot to show you got McAfrey Kittle Debo. I have easy one of the best O lines in the league easy Yeah, but y'all don't have the depth that the Niners have Y'all run the same offense, but the Niners have Trent fricking Williams on the O line The are you know top two three tight end in the game Best all around running back in the game in McAfrey Charles Leclerc refractor They bowed that life in San Fran future stars. Daravalla refractor Uh, there's a way to do it rossick. I think the integrate chats on like some other stuff. We tried it before it didn't work out for us Purple gold yuki synoda yuki speed demons I should do a full case of score football. I mean this is personal. So if you want to buy a full case of score football and Go ahead Green yuki synoda number 68 of 99 Nope, there's a way to do it where each chat sees each other's comments We've had it before you just don't use it anymore. I think it's through restream And orange in the last pack. This is a pretty pretty common theme here refractor max for stopping And as they front end of Alonso Grand Prix driver of the day hungry orange refractor number 23 of 25 That's funny james You see his comment James goes Raiders are a smart draft away from deep run elite wide receiver elite back possible elite If waller gets hurt and stuff together a top 10 o line and top five pass archer league, they get brady and draft That's funny The pats would be good if they had a great roster, you know what I mean If the patriots were like the 2016 patriots. Oh, yeah, dude. We're looking great Give me like the uh The o1 pats defense And like the o7 pats offense championship Yeah factor max for stopping Uh, factor auto Felipe drogovic Are you telling life bot to stop life bot does my bidding Mattress mac Broncos they're going after what harbour like Peyton or something we're going after one of the bigger fish purple a checkered flag album to 199 refractor Charles Leclerc I would defund the atoms. He was going to have Aaron roger's quarterback next year. Let me find score I don't even think we have it in stock for personals What space Yeah, I actually don't have a score in stock. Yes. We only have for group breaks Found it. Uh, oh, we only have a ship sealed but a full case would be uh, what 2400 A factor off our softener Speed demon Vettel, but here's a link for score Uh, there's 12 in stock, which is a full sealed case and that'd be ship sealed You don't do rippets of score We're 12 of them. That's the full case And a yumu Iwasa purple refractor 244 of 399 So I'm ready for the pageants are going to contend next year because they're going to get the vante atoms They're going to get tj watt. They're going to get shohai otani. They're going to get lebron james Are we so good? They're so good, dude Sergio Perez or factor they're going to get mahomes They're going to get mccaffrey. They're going to get mccaffrey and barkley George kiddle and kelsie refractor daniel rickardo Freed shipping. Yeah, orange and red future stars climat nova lock Orange red one on the hundred packs Yeah, I filled jacks under the patriots to coach as well They're going to dig up lombardy also, uh, charles the claire checkered flag nice Jim thork plays all positions Uh, many diamonds show guanyu rookie very nice card there as well 13 of 299 Otani give him like Like a full football off season to prepare he could do something in the NFL I don't know what it is exactly, but he could do it And man who knows refractor valetary boltas Gold refractor future stars climat nova lock 9 out of 50 nice gold Red hour back dc Not too much too much cigar smoke is going to be bad for the athletes today's day and age It's different now like you have to be super athletic now. You can't just be Old scooter mc dickerson In the 20s There was a painter every off season and then you know how like a two e.r.a Yeah, exactly Whoa, he just threw a 66 mile an hour fastball. Are you kidding me? Refractor yuki sonoda Back when cars went slower than horses Yeah And they just named people after ordinary things Life expectancy of 41. Yeah Yeah You didn't get to retire from sports.  That's like an actual one though, right? Hey, exactly oil can boy. Like what what are we talking about here? George Russell refractor flat tire Flat tire frank or something No gator in the bench. Yeah, no gator. Ain't no water. It's like out. It's a hard liquor Or a warm beer Everybody everyone's got a pack of lucky strikes purple daniel rickardo Number 298 of 399 couldn't play games at night. There were no lights. No Yeah, the baseball manager was the same guy that had to go light the lighthouse Yeah, so that whaling ships couldn't crash into the rocks Louis Hamilton refractor grand prix winner great britain They're drinking coca-cola in the dugout I mean field castle drinks coca-cola a lot and he's the new he's the nhl iron man So come on now val terry bow tats refractor Yeah, dude leon dawson with the with the sig Purple green joe guanyou Mini diamonds max we're stopping nice card 116 to 299 I mean if you get a if you get all 12 score football hobbies, which would be a sealed case It's gonna be $2,400 If you click the 12 it'll be it'll be a sealed case ship to you Royna sani future stars refractor and a gold wave f1 racer A rookie joe guanyou. That's really nice These inserts are really cool. Love the color on that. That's sick. You can magnus That's awesome, man. It's been a pretty solid case It's funny the jokes are making because in like a few baseball players including ted williams fought in world war two Ted williams also fought in korea What the hell dude? He would have gone down is like the best ever those four years of his career to war refractor total wolf Dude some of his stories that that were like told are kind of kind of wild like he almost died like eight times Uh speak demons lando norris I'm not sure what you mean jovani Oh like how old do you have to be to work for us? I guess 18 I think 16 but You wouldn't be a breaker at 16 years old A ruinous sani checkered flag Refractor gunther steiner Refractor might crack. Yeah, I don't I don't I don't know labor laws honestly You can work at 16 but yeah, you have to have like yeah parents permission and you can only work for a certain amount of time You have to have a certain amount of breaks all that stuff Only so many hours a week so many hours a day so on and so forth was like four hours a day something like that Or like every if you work like six you have to have like a Two thirty minute breaks. It's I don't know. It's just more break time basically less hours Purple feature stars danis holger numbered one of three ninety nine pop galas. You can probably work when you're 12 Yeah, they don't care you're six and you have two hands Yeah, you got a pulse Our refractor lando norris it's kind of like how they give out licenses here if you have a pulse of two hands Boom you can drive in florida refractor christian horner It's ailer Yeah, sam's actually uh 14, so They still have diapers Purple green louis hamilton nice card speed demons we're stopping. No, you can't work when you're 12 in the states No, no, we're we're joking around 16 minimum with like parents permission and It's it's pretty strict like they get really strict when you're under 18 So it's hard to find a job when you're under 18 public everybody goes the yeah public You just go bad groceries for like four hours a day Uh, that's about oak on purple uh checker flag the 199 focused here These extractors are tough to focus though. He is Yeah, he is ray Big baby, we had to do uh, I mean you have to get social security and identification and stuff when you get hired So then it's hard to refractor get a boy gold checkered flag max for sop in Grand Prix winner Amelia romagna 43 of 50 And it literally is the last pack of every single box. We have been opening it. This is wild That's what's up. That's what's up. It's a fun time. Jeff. Hope you're enjoying it, man There's still three boxes to go here. What do you mean what happens if you don't live in florida? You just don't live here and you don't get to have Uh trumpets You don't get to drive on i4 Gold wave after the claire Is it ollie caldwell auto? 33 of 50 No corn fritters for you No butterfly shrimp No sand fleas biting your feet at the beach Yeah, we don't do a remote jobs And andrea sidle mini diamonds numbered 134 of 299. There's wall on the east coast in different places What is like the only place on the southeast of it though? Checkered flag. I'll you mo iwasa future stars Refractor max for stop and grand prix driver of the day saudi arabia Uh, yeah, we'll we'll do a sunday fun a football breaks throughout the post season Refractor white nisani The only way we would hire a remote breaker is if rich like Knew you could break and knew you very well at this point Because we have so much space here for breaking Then we're just paying to ship you products. Then you ship it back to us. We sort it then ship it. That'll make no sense Make no sense purple refractor sergio pares number 29 of 399 Got the hq down here. It's feasible. We've done it before matt used to do when he first started lsc That's how it went Yeah, you're right. Let's do it. Yeah, we have a gaming shop opening up a little bit here Two locations you guys refractor future stars. Frederick vesti The conquest begins Kevin Magnuson refractor international level Red green joe guanyu rookie That's very nice That's awesome. That's about What case hit little less than a case hit, but that's pretty sick Down to recap speed demon sergio pares Yeah, legal.  So there's that 7 of 99 Yeah, custom sounds great mix shumak a refractor and a pink refractor of oscar piaz tri F2 award winners 66 of 75 nice oscar oscar.  Everybody wants to break all of a sudden But then when we're like relocate to florida nobody wants to do it Yeah, we have the best football teams ever You could move from like I'm not gonna I'm not gonna start trash over the places not gonna do it Not gonna do it refractor future future stars. Liam lawson. The legal becomes the boss.  Well, I got news for you. That is not true That's gonna backfire in your face big Oh, man, let me tell you something It does not work like that If only right if only Lewis hamilton checkered flag grand prix winter guitar Sounds like a bunch of great reputable breakers we have in chat here. Somebody great candidate's saying that they would steal cards and Oh, this is a great christian hornet refractor. Oh, yeah sorting and packing We have a whole operation for that though purple future stars. I'll be called well 304 through 99, but I've done the sorting and shipping. It's uh It's a lot different Refractor pier ghastly not many people to talk to when you're doing that although Some days that was the best dude boom podcast on Swirt breaks package breaks. Don't say a word for eight hours. Sometimes I use I love that Sometimes I like that It's weird like some days off when I have will we go like do things like friends or whatever I don't talk a lot like at all. I can't get myself to do it I talk so much here, you know fernando alonso refractor Like what else can I talk about go to go do real life? I'm like, hello I just want to talk trash like a shy kid for say hi. I'm like my mom's my mom's like I'm like three of my mom's dress Orange wave Jack douhan future stars 17 of 25 A lot of great color In this case, you know what we need though? We need a super fractor Two years of hard time at kmart and before we were talking about kmart like 45 minutes ago Like an hour ago In the college bookstore refractor max for stopping grand prix winner netherlands orange orange red max for stopping Grand prix winner belgium orange red one of the hundred packs Louis hamilton speed demons Giovanni's all over the place Refractor fernando alonso grand prix a winner or driver of the day hungry. I know alonso mini diamonds nine of two ninety nine Yeah, this has been a really good case. I think just overall really solid 59 to seven Yeah So is this is this getting to being the most lopsided victory in a championship game? At least in my lifetime it is terrible Just five years to stay penned for stealing cards You do that in apopka.  You got you got a speed trap right now If he doesn't if he doesn't write 20 tickets, he's fired Daniel ricardo refractor speed demons nicholas latifi Yeah, uzi is chief of police actually max for stopping Grand prix winner netherlands refractor. Yeah, I didn't even watch I'm just prison gangrene in tops of panini Refractor carlo signs Just lifts up a shirt just got a tattoo Yeah, I have I have to wear a mock turtle with coverable my neck tattoos Yeah, I got a bunch of tops chrome tats Uh denis holger future stars refractor All right last stack of packs right here. Good luck. Jeff Mini diamond sergio pares. That's such a cool photo Number 277 of 299. Yeah, I go through two hours of makeup a day to cover up the face tats Rookie with a Y That's funny refractor overall bow shung That's funny. Oh, that's good clomont nova lock X for a checkered flag X factor purple green clomont nova lock has to be Yeah, I mean, we'll do a pretty big one for the super bowl. We'll do a few couple hundred. I would think at least Refractor reyna sani. Oh, we're nice lewis hamilton purple 299 of 399 Yeah, I think rich and lane. We're actually working on finalizing it today. So See what they came up with we got There's some older product last year went back to like 2012 On the mixer. I think we go A few years past that this year like Yeah, it's gonna be pretty sick and pares That's a pares short print Yo Yeah, that's a pares short print, right It doesn't say refractor on the back. They always say refractor So look at this nissani. It says refractor That's a short print right there, right? Last card magic every single box You can madness That's what's up. That's a cool photo too with a flag. Is that in me? Is that in mexico? I'm assuming Mexican goat right there That's awesome. I think that's the first variation that I know that I fold Let's do a quick recap man. That was a great case. Jeff. That was a lot of fun, man Thank you for the order Let's recap some of the nicer color and then everything that we magged which is honestly quite a quite a bit It is 1038 eastern some time it is here Okay, so We did that purple lewis hamilton orange red versus stopping They're stopping on the mini diamonds red green rookie joe guanyu purple rookie joe guanyu Green lance troll green yuki sinota green charles eclair pink asabano cone pink oscar piastri gold vettel There's a lot of nice color here a gold clemont nova lock gold checker flag kevin magnuson orange wave jack duhan orange fernando valonzo autos felipe jogovich marcus armstrong lance troll and a gold wave holy caldwell I'm not sure uh, Giovanni, but ebay probably has the answer Let's see. So we had that short print of sergio pares So cool gold checkered flag max for stopping grand prix winner. Amelia romagna Gold wave rookie joe guanyu on the uh throwback insert here the f1 racer throwback insert orange sergio pares auto to 25 gold uh checkered flag grand prix winner austria max for stopping And a red checkered flag carlo signs big time stuff We'll drop the insurance link for you jeff appreciate it man Very much congrats on the hits


My name is Sibirah Akundi. Today I will be signing the Book of Ephesians in order to spread the gospel and reach those in the deaf community. Ephesians chapter 1. Paul, an apostle of Christ Jesus by the will of God to God's holy people in Ephesus. The faithful in Christ Jesus. Grace and peace to you from God our Father and the Lord Jesus Christ. Praise for spiritual blessings in Christ. Praise be to the God and Father of our Lord Jesus Christ, who has blessed us in the heavenly realms with every spiritual blessing in Christ. For he chose us in him before the creation of the world to be holy and blameless in his sight. In love he predestined us for adoption to sonship through Jesus Christ in accordance with his pleasure and will. To the praise of his glorious grace which he has freely given us in the one he loves. In him we have redemption through his blood, the forgiveness of sins in accordance with the riches of God's grace that he lavished on us. With all wisdom and understanding he made known to us the mystery of his will according to his good pleasure, which he purposed in Christ to be put into effect when the times reached their fulfillment to bring unity to all things in heaven and on earth under Christ. In him we were also chosen, having been predestined according to the plan of him who works out everything in conformity with the purpose of his will. In order that we who were the first to put our hope in Christ might be for the praise of his glory. And you also were included in Christ when you heard the message of truth, the gospel of your salvation. When you believed you were marked in him with a seal the promised Holy Spirit who as a deposit guaranteeing our inheritance until the redemption of those who were God's possession to the praise of his glory. Thanksgiving and Prayer For this reason ever since I heard about your faith in the Lord Jesus and your love for all God's people, I have not stopped giving thanks for you, remembering you in my prayers. I keep asking that the God of our Lord Jesus Christ, the glorious Father, may give you the spirit of wisdom and revelation so that you may know him better. I pray that the eyes of your heart may be enlightened in order that you may know the hope to which he has called you, the riches of his glorious inheritance in his holy people and his incomparably great power for us who believe. That power is the same as the mighty strength he exerted when he raised Christ from the dead and seated him at his right hand in the heavenly realms, far above all rule and authority, power and dominion and every name that is invoked, not only in the present age but also in the one to come. And God placed all things under his feet and appointed him to be head over everything for the church, which is his body, the fullness of him who fills everything in every way.