Well, as you can see, my slides are currently not presenting. So I guess I'm going to start my presentation because we only have 30 minutes, and then hopefully by the time we get to the metrics part, it's going to magically appear. I don't like the odds of that, but I think I'd rather have a productive conversation than nothing at all and just staring at a blank screen. So thank you for joining me today, which might be a very different presentation that I was planning on the benefits and pitfalls of open source software metrics with a particular focus on measuring health and risk. For those of you I haven't met, my name is Sophia Vargas. I am a research analyst and program manager within Google's open source programs office, where I support a number of open source measurement related tasks from looking at overall engagement across Alphabet to understanding what's happening in our projects and our communities. I'm also an active member with the chaos community where we design metrics, publish metrics and work with external projects on measuring health and sustainability. Still no slides. And so I'm here to talk about project health, which you might have seen as a topic that's been coming up a lot in various events and such. I'm most certainly been part of that, but I've seen many others give talks on this. And I've been really excited to see the adoption of using metrics to make more data driven decisions versus just gut decisions in a lot of these spaces. And so we've seen project adopt them to track participation, retention, productivity, and particularly companies using metrics to better assess what they should be using, how they should be using it, what they should be contributing to, investing in. And for many different stakeholders, metrics can be really valuable tools to understand what's going on in a project, when again your gut or you're not there in person to better understand and assess the scenario. There are many aspects of a project and project development life cycle that we could look into. For the perspective of this conversation, we're going to be looking at kind of a range between understanding health versus risk. So if a project is at risk from people or declining participation or maybe as a company using a project could increase your risk portfolio, this is really a gamble here. So the first thing I like to talk about is all the caveats health is not well defined or is risk. This is because it can mean very different things to very different people, whether or not using it a personal or a professional context. And I guarantee your company is probably going to be a lot more conservative than your personal, I'm growing in my AV support, we're scaling here, or the use case in which you apply this project, it could be part of a critical service, it could be part of a back office utility, that if it shuts off, maybe it doesn't provide any disruption to your business. I'm just going to keep going, okay, I'm not here. And whether or not you're looking at the perspective of the community or the asset the community is producing, you could be going for a stable code base to a diverse and active contributing population. I do want to have another big caveat for this conversation. Here I have a big slide if you imagine it that says risk is greater than security. There are a lot of security talks here today, I counted at least five and they've been really popular across a lot of open source events lately. This is not a security talk. So if that's disappointing to you, I'm sorry, I mean I'm disappointed for other reasons right now. Mostly I want to talk about the other elements of risk. There are other ways that you can introduce risk into your project, into your community, into your company by using open source or participating in open source software and other types of asset-based communities. If you are interested in security, I highly recommend looking at the open source security foundation, open SSF. They're building a number of great tools, frameworks, best practices, metrics like popular scorecard on how to assess security and security posture within an open source project and package ecosystem. Nope, still no slides. Well, I'm going to keep going. I mean it's like it's not great, but this is a hard problem. It's a hard problem for many reasons right now. Say something fails is not predictable. The data itself is messy, which is kind of a problem if you're building a metrics program where in theory the metrics are giving you feedback signals, but if you can't trust the data that are behind the feedback signals, then can you contrast anything at all? And then when you're building any kind of risk model, everything is subjective to your own case, to your own context and all the elements that we discussed before. So what is wrong with the data? Can we just assume that the data is real, accurate, actually showing things that we believe in? I've been an analyst for a long time in my career and in the last couple of years has been an introduction to working with data around open source communities and projects and version and control systems. And I learned about a whole new slew of data problems that I was not aware of, say inconsistencies between naming and schema coming out of APIs from the same platform or say how you design your pipeline to pull from these platforms if it's too large or how you've designed it is too large. You could be subject to rate limiting, which means that your data set is not comprehensive and is actively missing things. When you count things matters, when you're looking at version and control systems, you're looking at a current state, not a historical state. So if you're looking at trends over time, which are very popular for my role, that we are actively pulling things in time so we can see how things progress because if you pull it again five years later, any prominent forks that have emerged into the main branch are now the history of that project, now what was there five years ago. And I also looking at metrics around open source communities were often focused on people and people based activities. OK, wow, OK, now I'm going to start presenting again. Yeah, sorry, not you miss my cat slides. This is my little chaos agent Moby. So he's a great descriptor for a lot of my slides. So yeah, so the last started here. Thank you so much, AB team. OK, so the last one I wanted to make here on data problems was how you're counting people and I personally have multiple accounts on GitLab and GitHub. Why? Because of my personal professional context. I don't want some things going to my personal email or to my work email and vice versa and also there's a lot of automation, which is great and can be really powerful tool to scale processes and functions, but also does that really attribute to time based time on hands on the keyboard time, depending on what you're trying to measure? If you think if you're wondering how big of a problem this is, this is one of my favorite things to quote here. But the growth in bots on GitHub has been incredible over the last couple of years. So thinking about any kind of metric that we might talk about in the next two slides, say it's last time to update or last activity. If it's dependent bot, which is one of the most active bots, does that really count as a repository being active? Probably not, but that can really muck up your metrics. One of the other favorite ones is Kubernetes has an auto close policy on things that have been open too long in their pull requests, which means that if you're tracking mean time to close and a bot is closing things, does that metric actually mean anything? So be aware of how metrics or automation and scripts are being used in your community as well as a general best practices actively remove them because they could really muck up depending on what you're measuring. So let's come back to risk. How are we looking at defining risks in your own context? Again, non-security related risk. The question here, what could go wrong if this isn't your project so you don't have complete control over the tech, the process or the people? So I'm going to start rattling off some metrics now. One of the first and foremost is licensing. Some would say that open source is defined by the license that governs it. There are many different kinds of licenses that are created for specific intentions around use, consumption and contribution. So the first question for anyone who's about to use or release a project is, is this license compatible with your engagement model, with your intended usage model? If you're looking at it from a more of a portfolio and a consumption perspective, are all packages are using licensed? One of my favorite things to do is to troll through someone's GitHub and look at all the repositories. Are the licenses attached to all files? Sometimes they're aren't. So knowing that can potentially introduce risk if you're using any code base in your company that does not have a license to attach to it, should be a red flag or any legal team. How many distinct licenses that can create more complexity? And if you're wondering, OK, how do I even know what licenses I should be considering? The open source initiative has a set of approved licenses. They review new licenses all the time to help assess their relationship and applicability to the open source ecosystem, as well as helping you avoid things that are not well-defined because if they're not well-defined, you don't know how they're going to be defined and applied in a court of law. Another element of technology risk, you'll see a lot of suggested metrics around activity and reactivity as indicators of whether or not a project is being maintained. Last activity is a popular one. I have mixed feelings about this just because you can have a very functional project that has a really stable code base that hasn't changed in a long time and is still providing a valuable service. So maybe in that case, a better sign is reactivity versus constant activity in that if there is a new issue, a new bug that's been filed, is anybody looking at it responding to it or addressing it? Also, maybe if you start using a project and you submit a pull request because you want to actively change something, is anybody looking at it? If nobody's maintaining it, these are indicators that there might be lacking of that. Within your own context, I'm going to clearly I have multiple ways of looking at this. There are many kinds of things you can measure. So the goal of this exercise is to help you pick and choose the things that are most applicable for your own context. Say in the case of activity and reactivity, maybe you really only care about a subversion. You're only using version two point something, or maybe using it in a specific language or operating system. Quality metrics mixed bag as well. So helping to narrow down what could imply quality, even necessarily how you might measure quality, might be something like the number of bugs that pop up after a release. Another fun fact, typically the latest release of a project might not be the most stable. This is probably because you think of all the flood of bugs that happen once a release goes in. So maybe you want to wait a little bit before you consume the next release. But also, does this project have a history of preserving an integration and compatibility release over release? Or are you potentially introducing more breaking changes into your software? And then also just in terms of your own internal metrics, maybe how much time are you spending on fixes and patches as a result of taking advantage of this net new software. For additional context here, if you're curious of the kinds of bugs and issues that are coming up, you can look at where they're reported, what tags are associated with them. Fun fact here, I will have a little bit of security pointers. A lot of projects don't actually disclose security in the forms of bugs and issues because they're purposely trying to obfuscate their vulnerability, not saying, hey, we've got a target open on our backs. But they'll try to do this in a bit of an opaque methodology, either in closed doors. So ideally, this metric would be how quickly are you responding to security issues and vulnerability issues. But often that information is not available. Dependencies, clearly a popular topic in the broader slew of software supply chain. Being able to identify all of your direct and indirect dependencies, there was a talk by Sonotype, Brian, Brian from Sonotype about an hour ago, who mentioned that I think six out of seven vulnerabilities were coming from transitive dependencies, not direct dependencies. So this is a great way to keep your bookings up direct, sorry, keep your record up to date, maybe looking at S-bombs and you get the number of upstream dependencies. There are tools now like Devs.dev, which was released by the open source insights team at Google last year. So basically you plug in a package, see all the dependencies that are currently known, and they've actually been importing some of the CVE and vulnerability history there as well to help you understand what you potentially are introducing into your system through this dependency. Also ways to minimize this risk are how many versions do you have active in your code base, understanding when the dependency itself is relevant to you, is it during the test phase, build phase, or run time. And many other layers of dependencies, it's not just code, it also could be a service dependency and infrastructure dependency tooling. We could keep going here and you can kind of get the picture. You can't talk about risk without talking about cost. In my former life, I used to build risk models on infrastructure sourcing methods and often it was about, okay, cost as much as you do this, it costs as much as you do this, and what are the potential things that could go wrong that would increase cost. Typically risk can always be associated with the cost from the perspective of a company. So in this case, I'm focusing a little bit more on understanding what your costs are in a baseline and then the risk being how much variation is happening on top of that. So understanding what it costs for you to support this project, either as the presenter or the user from the tools, infrastructure, events, people, labor, training services, again, depending on your context, you choose the cost that makes more sense for you. Clearly complexity here can really change this equation and that's a very difficult thing to measure. I loved if you were watching the Angel Diaz interview earlier, the mention of forking, I was thinking about that, this slide as he mentioned that, in terms of how you're using a package in your system is really gonna change this equation from, again, is it part of a critical system or not? If you fork the code, then it makes it harder for you to take advantage of upstream versions that might be maintained, that might be being patched. And this isn't a bad thing. Forking and customization is one of the great appeals of using open source software. You consume it and make it work for you. But the more you fork it, the harder it is for you to stay up to date with the current version. So that essentially increases your maintenance costs to maintain it, which is exactly what Angel mentioned a couple, like 30 minutes ago, an hour ago. I'm losing track of time. And then again, from a technology perspective, what would it cost to replace this thing, to rip it out, to build it yourself to if your cost portfolio keeps going up, then maybe you do pay for a service to maintain it depending on what you want to keep investing in. Open source software is dependent on people. And these people could be coming from a variety of backgrounds. A lot of them could be and probably are volunteers with varying incentives. So this means that you need to interact with care and consideration to the community's needs and engagement preferences. So there are suggestions around tracking and understanding the role of the community, the health of the community, because without them the project would not exist or not continue to be maintained. Clearly, you need a sense of scale for the number of contributors. Here I've put a little bit of an asterisk because when we're typically measuring contributors, we're typically measuring what we can measure, which is data coming out of versioning control systems, which typically is code related activity, whether or not it's a pull request to review or documentation updates. But there's a lot of work that happens outside of that that isn't necessarily easily traceable in these systems. So I'm putting that as a caveat because the number of contributors is really more than the code base, more than the engineers, everyone that it takes to run this project. But for the sake of execution against these metrics, you typically end up focusing in on the things you can measure. I like to look at distribution of work. So I recently did a project where I was looking at some of our projects within Google and I found that the top 10 contributors by pull requests were doing anywhere between 30 to 70% of the work with hundreds in the long tail submitting one to do pull request year over year. So losing a handful of contributors, it really depends on what bucket you're pulling from. And so understanding the work done by what percentage of people can give you a better sense of the what people like to call the bus factor or I like to call the circus factor because it's slightly less violent, but how many people would have to leave the project during the circus and never come back for the project to not be able to be perpetuated. So this is an idea to kind of track how many people are really part of that core group and looking at the retention and attrition over time, giving you a better sense of is this project able to unwrap new contributors? Is it able to replace maintainers? Allow maintainers to take breaks and step away because we don't want them to burn out. We wanna give them an off-run plan as much as we need to unwrap new people. And then in terms of civility and community, does the project have a code of conduct? I also considered adding, does this project have a code of conduct enforcement committee? But that's a little bit harder to come by depending again on other aspects that we want to go into this talk, but whether or not a project has one can be an indicator of whether or not this is important to the community because you cannot predict how a toxic personality is going to impact the health and productivity of the community. We know that it's a risk. We don't really know how we can measure that coming into it, but whether or not a community is proactively setting grounds for acceptable behavior can be an indication of this. Clearly looking at, we talked about most of this context already, but diversity within company, background, location, I'm thinking about this again from the perspective of risk. So if say a project is primarily contributed to by a single company, that could be both a positive and a negative risk indicator depending on how you're looking at it. Negative from the perspective of okay, is this project gonna be too aligned to the goals of that individual company or maybe one of the company decides to pull back and then there's no contingency plan for how to perpetuate it. So typically you wanna just look for more diversity representation for both signals that the code base could be surviving beyond the whims of any one individual organization. Within a company, I'm always thinking about it, within your own context, how many people are you requiring to support this thing as a service or in house? There are a lot of metrics out there. Some are better than others clearly, but I think it's all really depends on what you need and what your own context is. This is a plug for the chaos community where we define metrics and methodologies as well as have software tooling to express these things. This is a great way to just kind of get a sense of what's out there and available. It could be overwhelming, but if you have a general sense of what you're looking to do, this could just give you more inspiration of how these things can be defined within your own context. So I wanna take a little bit of a left turn. Just wanna check my time because we had a bit of a strange start. When you start to collect metrics, you have to collect data. And if you're collecting data from systems around open source projects, you're probably collecting a lot of PII, personal information, emails, names, locations, companies, and other information around the history of that person's engagement with a platform. This information is not particularly controlled or regulated. If you're within a project, there might be project-specific policies, but if you're now a company aggregating a lot of information on behalf of your organization, now as a company, you are subject to regulatory and compliance and the policies of your own company. So the more that you amass, and depending on where you sit, you might end up being accountable for how this data is processed or reviewed or audited. So basically a general check that the data itself could have licensing policies and practices around it. It's generally good practice to review any of the privacy and policies attached to the platforms that you're aggregating from, say GitHub's privacy policy or your own company's privacy policy or a project's privacy policy. An example here is in the chaos community, we actually did develop our own data policy as a reaction to our DEI badging program that we've been running with events where event participants and organizers had to submit a lot of pretty detailed, sensitive information about the demographic profiles of their participants and speakers. And we realized with the collection of this information, it would be behoove us to create a standard policy of how we're gonna treat it, how we're gonna protect it, how we're gonna preserve transparency and trust within those that we're collecting data from. We did not have to do this, we felt that it was generally good practice to be open and transparent about our own data practices. So basically what I'm implying here is that there is an inherent risk with collecting data that has a sensitive nature when we don't exactly know how these things are going to be interpreted in a legal scenario. My general suggestion is that we're probably gonna expect more regulation. So if you know how to comply with standards currently, that's probably a good thing. So I wanna end with a couple of recommendations. I threw a lot of metrics at you. Clearly you could have an entire practice dedicated to collecting metrics alone. And this was just a subset focused on risk. Again, there's the whole security component that we just left out. So there's a lot more things that you could collect. So I think depending on the scale and practice that you're pursuing, it's always good to try to focus your initiatives and have goals or else you might just be left in a C and without enough people to interpret what you're looking at. Ways to narrow that down are determining scopes that are more relevant for your context. Instead of looking at a specific project, maybe you're looking at the specific version of that project or specific components of the version of that project. And then again, because we're thinking about systems based on people that don't necessarily have explicit expectations associated with them as the maintainer and you as the consumer. So understanding who's responsible for what isn't always necessarily a clean arrangement, especially when every project has their own practices, processes, governance, model, engagement style. Something that I've been encouraging of with maintainers that I have regular conversations with is wouldn't it be great if you just put it out there how you want people to engage with you? You probably don't want a random business trying to send you an email when something breaks in their context. But if this does happen, what's the best way and most constructive way for them to share this feedback with you? Is it using a mailing list? Is it using Stack Overflow? Is it creating an issue in their GitHub repository? And so if project maintainers can be more open and transparent about how they want to receive feedback or maybe they don't. Again, there was a wonderful talk in the keynote today from Sarah around how maintainers can feel about their own projects. And there are some cases where they don't want to do this anymore. And so maybe they need to be very open and transparent about I'm not maintaining this if someone is interested in maintaining it. Here's the steps you should follow. So in general, we need a little bit better communication on both sides as well as for net new consumers of open source to recognize that these are non-standard or contractual based systems. You're interacting with people typically in their personal time who have their own preferences for how they want to engage with you. And if you're looking at a bunch of metrics as someone who's personally been in this scenario, I often just have more questions which I think is actually a good thing. If you look at your metrics and you don't really learn anything then maybe you need to go out and measure more or just assume that you're only in steady state which may or may not be true. So often when you see a signal you have to dig further in and really understand what's happening. So never just trust them at face value is sort of my general encouragement here is to look at more things, ask questions, dig into it, see what's happening and if you can, go talk to someone who's on the ground living and breathing this experience. As a consumer, it's important that you invest in what's important to you. Again, I think that was mentioned again in the keynotes this morning but if this is important to your product or to your company, consider getting involved. There are many ways to be involved from financial contribution to developer contribution to event sponsorship, to mentorship, to many other kinds of programs and engagement style. So see what can be again most appropriate for that community and find ways to get involved that this is something that you want to preserve. And at the heart of it, people are your best resource. Do not let them burn out. We are looking into more methods to proactively detect signals of burnout but it's still difficult to do especially as people work in these spaces in both personal and professional time. That's some of the trouble that I have looking at the data across Alphabet is we can't easily distinguish these spaces. We see activity on GitHub but depending on the time of day we can't necessarily assume whether or not this is a side project or part of your core work. So asking people to do more or less in these different spaces, it's difficult to read those signals. So by proactively ensuring that there is a steady stream of people coming into this community that are supporting it and allowing people to step away when they need to to ensure that they stay engaged as long as they want to and have the ability to hand the reins over. And anyone who shares data, you always have the possibility to influence what you're measuring. I've seen this happen with leaderboards when someone will put up, hey, we've got the top 10 contributors by pull requests. And then suddenly they find that people are breaking up their larger pull requests into smaller components so that they can get higher stats. Is that what you wanted to do? I don't know, probably not. But in some cases you are trying to incentivize a change of behavior or growth and awareness. So just mostly acknowledging the fact that especially when you share data about human behavioral based systems that you potentially influence what you're measuring. So I kind of ran through that quickly because I had to say that was a little bit awkward on time. I think I have two minutes. Is that true? So I think I can take questions because no one is kicking me out yet. I'll answer that for two contexts. Within Google we have many. It depends on the project and what they're interested in tracking some of both roles, some we have supported as Ospo. But in theory there are almost too many things you can track. So we try to do it program specific in terms of say like the security team is actively looking at security metrics across the projects and repositories that are in our own organizations. But other projects have more robust engagement goals or say increasing participation in certain areas and in case they'll build their own dashboards then to sort of track those things or track against their own goals. We had one team that was looking at trying to ensure a similar contributor experience from those inside and outside of Google. So in order to do that we were looking at pull across response rates from those in known entities versus unknown people in the community to see is their turnaround time their response time similar to our own? Does that give us an indication of feedback that we need to update our documentation? So it can be very specific to a project as well as we have some larger scale programs that are looking more aggregately across the company. Within Chaos we've seen all of the above as well. We have a couple of software tools for more labs and auger that we're working on to try to create more tooling available. But you can just pull data directly from the get of API and the rest API is pretty easy to work with. Yeah, so I've seen most people build their own dashboards. So I've probably seen too many dashboards I think is the answer to that question. Anyone else? Yes? Yes, I am a symptom of the size of my company that we have someone who's just looking at analytics and measurement. I think right now I've seen better tooling happening externally in terms of public data sets. So we published a GitHub archive which is then used in tools like DevStats and OSS Insights that are basically just populating a comprehensive GitHub event stream so that you can sort of generate some of these metrics on your own without setting up pipelines. I will say that the data is getting worse. It is not scaling with GitHub and so when it was created in 2011 it was doing a pretty good job of collecting everything and now in 2022, we're down on the 80% range. So it's a pretty good estimate if you have no support at all to build anything. OSS Insights is a tool and if you are using a project in the CNCF they have their own dashboarding tool running on Grafata called DevStats. So I would start there as a place because if not then you are investing in time to build a dashboard. I mean this is, it can be a whole team of people which clearly you may or may not have support resources too. Other companies I've seen have leveraged more of their other business analysts functions. So if you can make the case internally about the criticality of open source as part of your infrastructure then ideally you can start to lever some of these other support services because you probably do have other business analysts which is kind of my role essentially. It's just I'm hyper focused on open source. So ideally you could get some other support because I would say as a team of one this would be hard to do on top of all your other work. So then being very focused and trying to leverage things you already have. We try to do as much automated as possible. A lot of the manual collection tends to be on some of those other styles of contribution. Say participation that's not collected in automated like system based. So like say board members or event sponsorship on participation that sort of one off other styles of contribution that aren't necessarily recorded on GitHub. Within the chaos community a lot of our assets are published metrics not software. So in that case we have explored using just markdown files to literally just write up what people's roles were. Just as a way to just provide more transparency of who worked on this and also to provide credit around people that spent time on it. But that tends to leave a lot of gaps still. So we focus a lot of our automated data collection on these larger systems that are giving us trace and activity data. But the softer ones are usually other ways that people are engaging in projects that we wanna give credit for and we wanna count. But we currently have no system to do that. Sorry, repeat that question? Yes, in some contexts. So it's a most certainly insecurity investment but it's more around teams that are already working in these communities are pretty well versed in it. And so we've used it to indicate where things are changing where we might not necessarily have full visibility. Say in very large teams where we are contributing across a large number of repositories we might not necessarily know how many people are engaged and what aspect at any given time given that there's a lot of personal engagement as well. So we can look at sort of our own footprint as a risk profile of just whether or not we still have the number of people we think engaging in these spaces because they're doing so again in personal and non and professional context. So we look at it mostly from a population perspective. There are other elements that we're looking at from a security perspective. Cost, not as much, mostly because we're very much invested already. Isn't a question of like what we should use but mostly how we minimize what we're already using. Controlling versioning that's something that we do as well. That's more based on policy versus risk analysis and that we have a policy that restricts using a larger number of versions on things. So we try to do things more proactively in policy than retroactively in risk assessment but that informs what we put in our policy as a way to reduce our risk. Any other questions? I think we might be out of time and getting if you're okay with it, I'll take one more. No one's kicking me out. Yes. This is Moby and this is Newton. Moby's named after Augustus Mobius, also a mathematician. He made fun shapes in Euclidean space is how we named him. He's not very smart though, which is why he's our chaos agent. So I usually end up using him in most of my slides. If you caught the risk one, that laptop was later damaged. He removed the B key. Just found it later under the carpet. So thankfully he did not disrupt my partner's work in the moment. It was more after the fact when he was missing part of his keyboard. Thank you so much for your time. I'll be hanging out around.


Everybody forced here rip in 2022 Donors baseball hobby 5 box number 17 Random team style you can always find the results of lane sports cards calm. Thank you very much for joining everybody and good luck See what we got just want to make sure I believe it's vet base or all base split even Yeah all base cards split even to all spots you get all your hits inserts parallels all that goodness good luck Let's see a what are the short prints all whammy? Crosstown rivals. I know there's whammy. Yeah, let's see some big stuff here. Hey, Travis. How you doing, man? All right, got an independence day O'Neill cruise rated rookie Solid one there for the pirates blue foil Mookie Toronto T. Oscar Hernandez to 303 blue foil Lou Brock Javi Baez dominators Tigers to 999 blue foil Salvador Perez. I pulled a Mets Javi Baez card yesterday I was so confused God for Blyla into 2022 twins Newt bar blue foil Backyard baseball. Yeah, I'd be cool Blue foil Matt Olson. He has the only baseball break left tonight. Is that dual case sterling and tributes? Trevor Raj 2022 Marlins key care Nanda's blue foil Independence Day Rodin blue foil laddie jr. I'll come all contoured of 2022 Marlins Josh Hader blue foil. Oh, yeah, Jordan pool You'll eat series bellinger to 999 Dodgers blue foil Kyle Lewis Mitch Hennigar to 2022 Mariners Josh Lowe rated rookie blue foil Josiah Gray nationals auto not numbered Blue foil bellinger Hey, Chris. What's up, man? It's a 100 Trevor story or Rockies 100 emoji a dullest Garcia.  That makes sense then Independence Day Melville meteor. It's trout there for the angels Salvador Perez That's going well it's a Every time I walk outside right now. I feel like I'm gonna collapse gold to it at 2022 possible MVP Brian Reynolds blue foil you'll eat series Chris Bryant Giants to 249 blue foil Ryan Zimmerman Randy, what's up, man? Now that the NBA finals over we can talk about what really matters NBA draft Johnny bench 2022 reds blue foil rated rookie Gavin sheets Evan Longoria jersey card Giants That looks like a raised Jersey Game war and doesn't specify but it looks blue to me Giants card there How Tucker blue foil got a wander rated rookie to 2022. There you go rays Xander blue foil What's that guys? The draft is next week guys. I don't even know the date Independence day with Merrifield blue foil Vladdie Pablo Sanchez Otto Get like the boys or whatever rated rookie Josh Lowe to 2022 ray is raised doing well Lily Sturgell Liberty Christian Yelich One per box on those there you go Brewers many machado to 349 hit list. I Thought I sound like almost that hit list the man is far from hit list. There's a lot of a lot of hits 23rd All right, so yeah less less than a week show hey out 46 For his 46 homers last year. There you go angels Kevin Smith rated rookie blue foil So we pulled that yellow chair reminded me what does everyone think of the Brewers City Connects I kind of dig them Read Detmer's Angels auto signature series blue foil Aaron Nola by Lewis mariners 2022 and a Goldie blue foil Yeah, they're pretty solid It's better than like 80% of the other ones Yeah, not too creative, but Sometimes that's a good thing I guess I Mean they're automatically better than the Giants ones. That's like automatically better than that Got to Brian Hayes independence day blue foil Kershaw Yeah, I think it's their colors that make it work Randy. I think you're right Gossman is that oh presidential but no not presidential Lightning or I don't know what it's called, but it's a 25 Lou Jays There do go blue foil. I like the Red Sox ones as a red as a red socks fan. I like the Red Sox ones They're not like the best, but I Like them more than I dislike them unleashed Mike Piazza the 999 for the That's east blue foil the same as just making sure sometimes they throw you a curveball and it's like Dodgers or something but due to 2022 Tigers Joe Ryan rated rookie blue foil Machado blue foil. I do agree though that like the hat for the Red Sox City Connect It's too much UCLA brew the hat like really does it got martin at 2022 Mets blue foil Shane McClanahan, maybe saw young McClanahan The angels ones are solid too nothing crazy with those rated rookie independence day Riley Adams Pops really stargile blue foil Vlad senior as an angel to 2022 Peter Lanzo blue foil Yeah, the Nats one is the best space city Houston is pretty solid too 249 Matt Olson a's blue foil contrarus cubs Overall, the ones they've done this year are a lot better than the ones last year. I think a Giants one is the worst to me 2022 Boba chef blue jays Blue foil cow Raleigh Mariners So do I have the cup final I'm gonna go abs because they're up one nothing I'm gonna take the team as of right now that needs less wins Red Sox Otto Jaren Duran Red Sox blue foil Koi Sigo to 100 Tatis Go Padres blue foil realm uto Phillies independence day Orias at Olson as an a Rockies ones. All right, the list Garcia to 2022 Rangers blue foil Trevor Hoffman I think though the most polarizing one would be the Marlins City Connect and I like the Marlins a lot Travis is a Bolts fan You got bomb squad saboteur Perez Royals of 999 JP Crawford blue foil Mookie to 2022 Dodgers Blue foil Louise Heel. Yeah, I like that Donneress is kind of quirky Emoji parallels and stuff Cardinals Dylan Carlson jersey card not numbered blue foil Mancini for the O's Kevin Smith to 2022 blue jays Tim Anderson blue foil Xander independence day blue foil wander Ray's having a nice break Camilo deval rated rookie Giants to 2022 Will Smith blue foil Liberty to Oscar Hernandez blue jays You got diamond marvels Mike trout to 999 nice trout blue foil Mitch Hannigan Angels having a very nice break Yeah, the deep axe. Yeah, let's let's a little serpent. This is pretty solid, but you're right The color scheme is what did it and to 310 Mike trout Angels with another numbered trout Casey Myers blue foil Yankees with a rookie auto here Louise Heel Not numbered blue foil Logan Gilbert Bucks to the 2022 twins blue foil Tom Sieber. Hope everyone's having a good day Got sterling and tribute coming up guys random team style dual case Still at 13 to go top sterling and tops tribute huge potential in that break 25 spots Five combo spots on the on the random team style Louise Heel to 2022 Yankees blue foil type France Nice Admiral. Yeah legends legends Arceus is definitely a fun game Independence Day Alcantara blue foil Shohei Otani. I hope to make another one. I just like really expand and like Pay more attention to detail with it bow flows to 2022 blue jays Logan Webb blue foil hit list for the Royals wet Merrifield to 999 blue foil Robbie Ray The tightest of pants Easy Sanchez to 2022 Marlins blue foil Matt Brash Mariners Rangers AJ Alexi auto got blue foil Brenda Woodruff Brewers Do you guys think Robbie Ray wears tight pants all the time or just when he's playing? That is Kyle from Waltham to 2022 Red Sox Blue foil here at Cole you can stallings independence day Bryce Harper That is Stan the man to 2022 Cardinals blue foil Josh Donaldson sweat pants until game day Liberty Robbie Ray speaking of there he is to 249 trending Louise Robert White Sox blue foil Frank Thomas Louise Robert White Sox to 338 And blue foil Camilo Boval raided rookie Gotta throw some heat that Philly Hans Krause autumn not numbered Blue foil schwindel for the Cubs Who owns the record for slowest pitch? Ever to be hit, right? Got Logan Gilbert Mariners of 2022 blue foil Akunya Mount castle independence day blue foil Pudra Rodriguez as a ranger Harper to 267 Phillies blue foil Jose Ramirez On leash Nuki Betson 999 Dodgers blue foil Willie McCovey Giants Got Cabrera to 2022 Marlins ever Cabrera blue foil Jose Siri Houston blue foil Crawford Fire emoji Ryan Zimmerman gnats to 75 Blue foil Brooks Robinson Pudra Rodriguez doesn't like doesn't like the nickname. I use a Hall of Famer You should just learn to live with it. I think Montas independence day blue foil Tyler O'Neill One of the best catchers to ever do it. It's 2022 Vlad jr. Blue Jays Franimal yeah comes beat the Braves. What a day end of the win streak Hit list Royals Mara field to 349 blue foil Karoo as a twin. Oh Schwindel Why like Why did he get upset? Snell the 2022 Padres blue foil Mike Balman. He's not a pitcher Jason Dominguez jersey card Yankees Numbered in blue foil Reese Hoskins Phillies You're just trying to finish the game with you know, you're losing While not using your actual pitchers and nobody getting hurt Rhino to 2022 Cubs blue foil story is rocky redemption That's blue foil Gilbert their first redemption one Soto Nationals Nice hit Nats Good stuff Monarchers on the auto one Soto Alex Pregman Astros to 500 blue foil gilito Yeah, I mean unless you're a cardinal Unless you're the cardinals and you don't want to use your position players to pitch all that much Independence Day devours blue foil and he'll cruise By your traded rookie Linda or to 2022 Mets blue foil glass now as a ray Which he has been for years now Hit list Pittsburgh Brian Reynolds to 999 Flexing yeah as a 2022 Giants blue foil Soto got Khalil Lee Khalil Lee on the Mets auto blue foil Corey Seager or he asked to 2022 for the Brewers blue foil Trey Turner Independence Day Adam ace blue foil Riley Adams You think the cut yeah, the Cubs want to solve the Cubs City Connect is definitely solid Hoskins to 400 Phillies blue foil Trevor Hoffman Padres Yeah, them and the Royals feel very similar to me like they're they're solid nothing like wow, but It didn't mess up, you know to 149 Bellinger Dodgers elite series blue foil to reexcubal 2022 Brian Reynolds Pirates Juan Soto blue foil. So again everybody personals are full for tonight To 100 Joey Vado Reds Nice blue foil is Melville Meteor Angels Mike Trout Dude tiktok Vado is that pretty cool Blue foil Meryl Kelly D-backs Joey Vado trying to become tiktok famous is like the Steve Busce me mean where he's like Hello there fellow kids or whatever he says It's 2022 Aaron Ashby Brewers blue foil Pedro Martinez Subsea hot Independence Day Shane Paz you'll raise Brandon Marsh blue foil 2022 Brooks Robinson Orioles Blue foil Will Smith Liberty to one not numbered Liberty box hit Reese Canair Padres rated rookie rated prospect Alexander Canario to 999 Cubs blue foil Louise Robert Andrew Vaughn Jersey card to 99 White Sox Blue foil Mancini. I'll to bed at 2022 Astros blue foil Chris Bryant as a giant Independence Day singer blue foil McCormick rated rookie Heasley for the Royals at 2022 blue foil Peter Lonzo trending Trey Mancini Orioles to 999 Blue foil Cedric Mullins one more box All right guys, let's get that sterling filled up sterling in tribute coming up soon Maybe for some caught auto bat barrel bat nob filthy patches biggest hit in what Independence Day Rosarino blue foil Devers Correa to 2022 Astros blue foil Nick Castellanos Dominators to Grum that's to 999 blue foil Polonco Twins stuff sand Independence Day Muller for the Braves blue foil Marta D backs Downtown is in Diamond Kings baseball To 344 Luis Garcia Houston blue foil Randy Rosarino Short prints in this would be crosstown rivals and whammy. I'm not sure if there's any more I forget Louise Robert to 100 there you go White Sox Andre Jackson blue foil to 2022 Jose Ramirez Cleveland's blue foil Carlos Correa Glad jr. Bomb squad to 999 blue J's blue foil Shohei Otani This is our first break this afternoon. Kenzie pulled some big stuff earlier It's 2022 soda for the Nats get a soda redemption earlier in this pretty good for a Don Russ got a pause rated Ricky blue foil for the Rays Nick Madrigal to 25 on the auto Cubs Blue foil Joe Musgrove Cy Young Canada Joe Musgrove man, he's he's killing had a really great outing again yesterday McGill rated rookie Mets to 2022 blue foil Andrew Vaughn Justin Upton Jersey Card Angels blue foil Ryan Sandberg Cubs Yeah, we should be pulling some big hits tonight Independence Day Charlie Morton Braves blue foil or Rosarino Rays Definitely pulling some big hits. We got dual case sterling and tribute coming up guys Let's get it filled up. It's down to 12. I believe yep down to 12 now Castillo to 2022 Reds blue foil Tommy Edmund Independence Day Glebert Torres Yankees blue foil Corbyn Burns Brewers Jaren Durand Red Sox rated Rookie to 2022 blue foil Mike trout Angels Yeah, we got a bunch going on later tonight to say Be fun. Yeah, we got sterling coming up soon to get absolute next Diamond marbles Max Scherzer to 999. There you go Mets are Says Queen's Club, this is a Mets card. Does he even say New York anywhere? I just met my dead blind. Oh there right there, New York right there Mets on that one Blue foil flexing it looks like he's wearing a dodgers. This is why I'm so confused independence day Strowman for the Cubs Buxton trending for the twins to 149 Blue foil Matt Chapman Jay Ram to 2022 Cleveland's blue foil AJ Alexi.  That was such a crazy card tops posted that It's like one of their highlights of like the week that it was pulled flex into 2022 Mariners blue foil Austin Hayes Definitely one of the biggest tribute cards Turner to 2022 Dodgers Colton Welker blue foil blue foil Kyle Seeger That is Giancarlo Stanton Lightning it's 25 Yankees and a blue foil Pedro Martinez Red Sox Pedro. All right, do a quick recap for you Relics first you adjust an upton angels still in Carlson Cardinals Jason Dominguez Yankees Andrew Vaughan White Sox Longoria for the Giants Autos Rodolfo Castro pirates Nick Madrigal Cubs Khalil Lee Mets redemption for the Nats Juan Soto Got Hans Kraus Phillies AJ Alexi Rangers Louise Healy Yankees Jaren Durand Red Sox Reed Detmer's angels and Josiah Gray nationals That's gonna do it. Thanks again everybody. We've got absolute coming up next