Hello everybody. Welcome today to this joint event hosted by New America's Open Technology Institute and Consumer Reports. We're excited to have to be hosting some great experts on the two panels that are coming up, but I will let our moderators introduce each of them to you. So we're here to talk more about labels. Some you might recognize and some that might be new to you. Consumers often rely on labels to help them choose products to identify information that is important to them and to make comparisons between the options that are available. Our first panel will be discussing existing labeling schemes in a variety of different industries. Our second panel will be discussing potential labels of the future and some really interesting projects going on in the tech space to revolutionize the labels that consumers see. On that note, I'm going to introduce our first moderator, Justin Hendricks, the editor of Tech Policy Press. Thank you, Andy. Thanks for New America for hosting this. So pleased to be here. My name is Justin Hendricks. I'm the CEO and editor of a site called Tech Policy Press, which looks at the intersection of technology and society. And so this is a great topic to be talking through today. We're going to talk a little bit about how some label models have been more successful than others. But across multiple industries, we see some best practices emerging and we're going to talk about how we can apply those in the technology space, which I'm excited about. We've got three great panelists today and I'm going to introduce each one of them. They're going to give a little bit of a background on who it is that they are and what expertise they bring to this topic and some of the things that they're thinking. And then we're going to have a bit of a discussion and hopefully bring in some of your questions as well from the floor as we move along. So really looking forward to this and thanks to our panelists for for joining us. And I'm going to just go down the list and invite you all to make an opening comment to start. And I'll start with with Brian with Brian Ronholm, who is director of food policy at consumer reports. Brian will tell you a little bit more about his background. We're not going to do lengthy bios here, but I will tell you this. If you if you look in detail at his background, you'll see that if you ever had an egg, or eating a catfish in this country, you have him to think for the fact that it's safe. So, Brian, welcome to the panel and perhaps give us a rundown of some first thoughts. And thanks, Justin. It's great to be with you all. Good afternoon. As Justin indicated, I'm Brian Ronholm. I'm director of food policy at consumer reports. And in my background, I served at the US Department of Agriculture as deputy undersecretary for food safety. Thus, that is the agency responsible for meat poultry. And as Justin said, catfish and processed egg products. I worked on Capitol Hill as a congressional staffer. And Justin, should I go ahead and start or do we want to do intros of the others? Absolutely. Let's go ahead and get some first thoughts from you and then I'll go to Quinta and then to Kyle next. So the issue I'm going to cover today is the nutrition facts label that we're all familiar with. And those labels, as you may know, first started appearing back in May 1994. And that was the result of the passage of the nutrition labeling and Education Act, the NLEA, which is signed into law on November 1990. And it took a few years for those rules to be written and go into effect. And the aim was really clear. It was to help consumers make better nutrition choices and encourage food companies to make healthier foods. Now, before this law was enacted, there was some information that was required on packaged foods, but it was limited and it was limited to nutrition information. So card only when there was a nutrition claim being made. Otherwise, there was no kind of expensive information available for consumers. So the enactment of this law obviously expanded the availability of this nutrition information. And it specifically required food packages to contain a detailed standardized nutrition facts panel that included information such as serving size, calories, grams of fat, saturated that carbs, fiber, sugars, protein, cholesterol, sodium, and certain vitamins and minerals. So since it first started appearing back in May 94, there has been some revisions. So trans fat was added in 2006 and then added sugars was included in 2016. And the purpose, of course, was to reflect current science, current data on these items. So the serving size also was updated so that it would reflect how much of that item people typically consume at one time. And the serving size question was probably been the most confusing for consumers to grapple with. So serving sizes on food labels are required to be based on the amount of a product that people typically consume during one occasion. And, you know, I'm sure during this time we've all encountered this situation where we pick up a food product, and we notice, hey, this bag of snacks is only 150 calories. This is great. I'm really in the mood for these. So I can just eat this whole package. And then when you read the label more clearly, you recognize that, oh, it's only 150 calories for one serving, but there's three servings in the bag. So really you're looking at 450 calories. And I think that was a source for a lot of confusion for a lot of consumers. And I'm sure, you know, the industry was quite pleased with that, that type of confusion as it led to more purchases and kind of misleading belief that certain products were healthier than they otherwise would be. So the update in the serving size information was meant to address this issue and make it more clear for consumers to understand how much they were consuming. And this amount for some items that people consume has significantly changed since the original law went into effect. So it was important to update this requirement to reflect current consumption habits. So now for packages that contain between two and three servings, companies are required to provide a dual column that displays the total amount of calories and nutrients per serving and per package. This is obviously meant to make it easier for people to understand the calorie count for the entire product package. So when you look at the product now it's like, okay, I really will be consuming 450 calories, not 150 calories. So, overall, as you can see the changes to the nutrition facts label the nutrition facts panel are meant to reflect current science. Another example of this is reflected in the most recent change in which the calorie count was made to be bigger and bolder on the label. And this was to reflect the rise in obesity between 2000 2018, which saw the obesity increase, almost 12% during that time. And added sugars was included on food labels in an attempt to decrease consumption of added sugars and bring it closer to recommendations that it be less than 10% of total calories. Recent data had shown that intake was about 13%. And there have been changes to vitamins and nutrients that are required to be listed again to reflect current habits and current science so vitamin D replaced vitamins A and C, because of concerns about vitamin D deficiency. Potassium was added for the same reason. And as we know potassium is important in the battle against high blood pressure so a lot of this data was was churned out and you know the changes to the label were meant to reflect that. And the percent daily value was another important addition to the nutrition facts panel because it tells you how much of your daily need for a particular nutrient is contained in one serving. Today overall the nutrition facts panel has been successful and has worked very well. It helps consumers get information about food products at stores very quickly and conveniently and empowers them to make better choices for their families. And a lot of times they get this information at the grocery stores. Not many folks have time to kind of study these study this information on websites before they go grocery shopping so to have it handy and convenient when they're there is certainly very helpful. And also the label, you know, it's been modified like I said over the years to reflect current data and current science and that makes it even more helpful for consumers. And that doesn't mean that changes can't be made and you know we can build on this progress by ensuring consumers continue to have access to to this information. And we do support changes specifically the food labeling modernization act that has been introduced in Congress, and that would require the establishment of a single standard for the package nutrition labeling system in a timely manner for all food products required to bear nutrition labeling. And we'd also like to strengthen the current law to target trends in marketing that confuse or mislead consumers. Specifically, we'd like to see new guidelines for the use of the word healthy and natural. You know those are terms that have yet to be defined by the agency so some some more clarity for consumers on that front would be very helpful. And we'd like to see warning symbols for foods high in the types of nutrients that should be limited or discouraged like sodium and added sugars, etc. So those are the issues that we track we track for consumer members and write about in consumer reports. Again, it's, you know, overall this, this program has been successful. It's very helpful to consumers. It's, it changes to data, but you know based on data and science and again we'd like to see improvements. You've also seen it extended to things like menu labeling. We'd like to see it extended to, you know, food delivery systems door dash Uber Eats, etc. to have that list. This type of nutrient information as well so that continues to be easily accessible for consumers. Thank you Brian, and there's so many different topics there that will come back to you from, you know how to weave in societal goals, the relationship between these labels legislation and policy. The pace of science and how that impacts things and, and this issue you brought up around confusion and how industry may in fact try to capitalize on confusion on occasion. So we'll kind of come back to some of those. Next up I'm going to introduce Dr Quinta Warren, who is associate director of sustainability policy also at consumer reports. If you look at the details of her bio, a lot of amazing things there as well has worked on accelerating entrepreneurial ventures in Africa, and has a PhD in chemical and bio molecular engineering got that one out this morning. So thank you, Dr Warren, give us a few thoughts from from yourself. Thank you Justin, and thank you for the opportunity to be here today. My name is Quinta Warren, and I am the associate director of sustainability policy of consumer reports. As Justin said I'm a chemical engineer by training, and I've worked in the energy sector for over a decade on multiple things including carbon management power generation and energy policy work with the Department of Energy and the Millennium Challenge Corporation. For consumer reports, I am helping to advocate for policies that benefit the consumer in many ways, including in terms of increased cost savings and reduced energy use. As an example, we're currently analyzing and providing public testimony on the proposed car rulings that have been put forth by the EPA and the Department of Transportation. So jumping into this particular topic on labeling sustainability in the energy sector, I would say primarily relates to reduce energy consumption and increased energy efficiency. So residential energy consumption in the US accounts for about 21% of national energy consumption. So 55% of that is from heating and cooling and about 45% of that from the heating water appliances, electronics and lighting. The point of all that is to say that we tend to use a lot of energy at home and reducing energy use will greatly benefit us in terms of savings and reducing any greenhouse gas emissions related to energy generation. This also in turn, so reducing energy use as I said relates also to lower energy bills for the consumer. There are several energy performance labels that I would like to mention today. The first is Energy Star, which uses rigorous criteria to denote energy performance and water usage. Managed by the EPA and the DOE. Energy Guide as well looks at yearly operating costs and for appliances and models that are also in a similar range as the appliance you're looking at. We have the window stickers on cars that primarily denote fuel economy and that help consumers that are buying cars to judge whether or not those cars match the fuel efficiency that they're looking for. And then I should also mention that consumer reports has the green choice designation or badge, which looks at which they give out to the top 20% of cars that have the cleanest emissions and also washing machines that are ranked in terms of water and energy efficiency. Another point I'd like to make is that consumers need good information in order to make decisions, and that's the fundamental for any well functioning market. The labels need to be trustworthy, they need to be accurate, they need to be easy to understand, but labels are just one piece of the puzzle. They can't replace strong consumer protections. So we could have a lot of really well labeled but bad choices and that would be the same as no choice at all. As an example, for fuel economy choices, we find that those are quite limited in the market, because about two thirds of all car models are actually within five miles per gallon of the model average. And for trucks, they're also nearly two thirds get within three miles per gallon of the model average. So even though we have these energy labels, we need to keep fighting to bump up the efficiency standards so that we can help to protect the consumer. So those are my opening thoughts. Thank you, Quinta we're going to go next to Kyle and Kyle Wines is a CEO of iFixit, and he is an expert in design for repair, everything from service documentation on through to thinking through, you know how to make some of these policies work that allow us to tinker with the things that we buy. And I think we have him to thank for the fact that we can repair our own tractors in some cases as well as that we can jailbreak tablets and cell phones. So Kyle, welcome and I understand you'll have a slide to show us as well. I do absolutely. Yeah, excited to be here and I mean as I think about the success of labels in general. I can't think of two more successful labels than the nutrition label and the energy star label. They impact consumer choices they impact are my purchasing all the time. I, we got a bottle of non alcoholic wine last week and it had a nutrition label on it and it was like what this is so interesting I've never seen the nutrition label on a bottle of wine before. And I kind of wish that that extended to all alcohol, it would be cool like how many calories are in this bottle of beer I have no idea. So that would be nice. So if you think about all the durable goods that we buy whether it's a smartphone or a vacuum we don't have information on how long they're going to last. So with something like a vacuum energy efficiency that vacuum might matter but what matters a lot more to consumers is, is it going to last for the warranty plus one day. Am I going to be replacing this vacuum in 13 months because you know some little clicky plastic piece broke, or is it going to be something that will last me for 10 or 15 years. And the appliances in, in general, you know they used to last it used to be by a refrigerator and you'd have it for 20 years. Now the average lifespan of appliances around seven years. And the UK consumer agency switched to the survey of consumers asking, like, how do you feel about the life expectancy of your products and 70% of consumers were dissatisfied with how long their durable goods will last. We need to look at increasing durability of these things and, and, and repairability is really the main factor because these products are increasingly complex there's different things that could break. Can you get parts when they break with vacuums usually it's the power switch that breaks it's like a little spring or a little plastic thing why are we throwing away an entire vacuum because of the $2 part. I fix it has been rating products based on repairability for over a decade. If you're not familiar with I fix it where the online free repair community for everything we help about 10 million consumers a month around the world learn how to fix things. And it's everything from vacuums to smartphones to you name it any product that you'd like. We have free repair guides, kind of like Wikipedia for fixing things. As part of our work we rate products on an I fix it scoring system. And this this slide that I'm showing you is the new French scoring system that is kind of an evolution of our work. I would assume that no one on this call has ever heard of this French repairability label before. But this label rolled out in France January one this is a mandatory label for five categories of products including washing machine smartphones laptops and a few other categories of products. And this score you can see I mean very user friendly you look oh it's got a light yellow label on the shelf okay. I've got an idea that maybe I'm going to have trouble getting parts. This label is mandatory if you go to apple.fr and you try to buy an iPhone you will see the iPhone gets a yellow label of five out of 10 on this scorecard. And this this stats I'm showing you here, Samsung commissioned a survey looking at what were consumer perceptions in France with this new label you know we're sitting here in October it's been on show since January. And you can see it's really had a market impact. It's important, you kind of step back and say well, what's the broad trajectory society what does it do to these products when consumers have the choice because right now consumers don't have the choice to pick. You go to buy a vacuum and there's $100 vacuum and there's a $200 vacuum, which one do I pick. If you knew that the $200 vacuum was going to last four times as long as the $100 one, you might be more inclined to buy that. But because consumers don't have that information, they buy the $100 vacuum, and that has driven prices down it's driven manufacturers to create cheaper and shorter lasting products. And we have this kind of vicious cycle where consumers want better products but don't know how to pick them manufacturers are not incentivized for creating better products. And so we have kind of a world of disposability. If we look at I fixed scores where we have rated products, electronic products for a pair ability over the last decade. Every single year the average score across the industry goes down things are getting less and less repairable. There is less parts available, it used to be for any vacuum you could get apart for from the manufacturer now it's increasingly challenging. So we're really optimistic that this French repairability label is going to change things around. This is not just France, Spain has announced that they're going to adopt this repairability label I've heard that Chile is looking at doing the same thing. And then the European Commission has said they, and that they're going to adopt something like this year applied. And so we have a number of US state houses looking to implement maybe a US state level of repairability label as we wait for Congress to do things. So there is a lot of momentum for this in the broader context of right to repair. So I would leave you with one thought if you're like, I wish this label was here now how do I pick I have to buy a vacuum tomorrow. The easiest thing to do is to search online if there's a thing you're thinking about buying, look and see if you can buy replacement parts for it. If there's no way to get a replacement part if there's no way to get that on switch for the vacuum, then it's a disposable product. And I think that we're excited we're discussing with consumer reports and other folks. How can we integrate product longevity, availability of parts availability of information into some of the existing consumers for cards that are out there. Thank you Kyle and while we wait for for for Congress to act is a loaded loaded phrase, which will maybe delve into a little bit. So let me just kind of pick up on a couple of things. And by the way I'll just urge our viewers if you have questions to share those in, and we'll make sure to try to address those but one of the things that it feels like it's sort of first thing to ask is, from your perspectives, what is different about trying to label tech versus trying to label food or, you know, energy sustainability or other things and I suppose I suppose sustainability leads certifications things of that nature, maybe closer to some tech products, but there might be some significant differences as well. What what are the unique challenges to tech as a category or sector of things to label. And maybe I don't know, Quint or Brian, if you've thought about this at all, but colleague and start with you. Yeah, there are some existing eco labels for electronics and I didn't talk about them honestly because I'm really unhappy with all of them. I would say what's different about tech is the tech industry is very savvy about this and there's a large amount of regulatory capture happening inside the existing green environmental labels. The common one the one that's used by the federal government is called EP and it's a family of technical standards. Initially, it was successful at at doing things like getting post consumer plastic into the process. But the manufacturers are very savvy and we're able to manipulate the process to completely water down those standards and make them meaningless, which is why we've had a decade of kind of the science being in that repairability and product life extension, that we needed to go and absolute stasis and no progress on this and any of the green standards. So I would say what's different about tech is that the lobbyists are really really good and have stymied progress. Brian from your perspective you talked about this a little bit the idea that industry can kind of get its sink its teeth in and create confusion and kind of capitalize on that. How do you avoid that in the process. That's a good point Justin and first of all to answer Kyle's question about the bottle of wine and nutrition information. The reason why alcohol doesn't have nutrition information is because it's regulated by different industry agency. So it's regulated by the alcohol and tobacco tax and trade bureau so and you want to talk about powerful lobbies that prevent certain progress being made. I think that's certainly the case. But yeah, I mean overall, you know the burden has always been on the consumer. Have you ever seen that before. Not on a wide bottle that's. Yeah, I mean, I think Justin that to Kyle's point the burden has always been on the consumer to get this type of information whether it's a food product whether it's tech whether it's automobiles etc. The industry council that they count on the consumer not having enough time and resources to really dive in depth into these products and get the information that's needed it you know I brought up this example of, you know that the serving size they certainly. It was nothing that they did but they explicitly but certainly something that they facilitated encouraged in terms of misleading on serving size so you know you know that's why we exist in this realm is to try to process all that information and provide consumers with really convenient up to date information so that it can make those better choices. And do anything from your end on this topic of industry engagement with the process and where that can go wrong, how it can go right. Yeah, I guess I'll start by saying, you know, the information that goes into the labels needs to be repeatable reproducible. And so that adds an added that gives an added layer of complexity to the process, because just because you can do something in the lab doesn't mean necessarily translate into the real world and yet you can do it under every single condition in the lab. So that's that's the first thing is that you know sometimes the information that are the tests that go into these labels are not 100% representative of the real world. And that can appear like a deception but it's really not it's more about reproducibility, at least from the standpoint of the, let's say the agencies that are responsible for this. So, consumers in real life might find for instance that their fuel efficiency is slightly different from what they would read on on a label, whose information comes from EPA tests, for instance, and slightly different because you know maybe you're in rush hour traffic all the time versus driving, you know, on a highway that's not congested. But having said that, yes, sometimes industry can be. It's the same as leading in the way that they present their products, because they understand how the tests are run. And sometimes they will take advantage of that, and they will beef up their numbers to look really good. And then the consumers become concerned because they read the labels and they see a number and when they drive the car in real life. And that's not really the average number that they're seeing an example of that is the scandal that we had with Volkswagen because they were manipulating the fuel efficiency tests. And, you know, fortunately we have organizations like consumer reports and other advocacy organizations that keep a close eye and try to make sure that, in fact, what we're seeing presented by by industry is what's really happening in this case then you know we have we have the opportunity to try to correct that by either, you know, going to EPA or, you know, filing suits or whatever we need to do to protect the consumer. So I don't know quite how to put this question but I'm kind of wondering about the interaction of the science with the with the labeling labeling process. And this may be a naive and incorrect assumption but on some level colleague kind of occurred to me coming into this that, you know, maybe some of this stuff whether it's you know the label on the on this particular panel talks about broadband and iot for instance, on some level maybe some of that stuff's moving a little faster than food science. Maybe that's wrong by the way, Ronald correct me. You know, has the kind of pacing of the science play into labeling and what we know about the products and how they work. Yeah we're always seeing new things I mean, I make a living selling tools, and we're always developing tools for the newest, the newest gizmos so yeah there's definitely a pace, but fundamentally, whether it's a automobile or a vacuum or a phone that you're going to be moving screws to open it right availability of information is the same across all products availability of parts is the same. So it's really more of an ecosystem challenge. Are you are you planning and providing for an ecosystem are you thinking about the product over the course of many years, a decade. If you can't get security updates for a phone, two years after it's made, then is there a plan was there ever a plan for that phone to last longer than two years. So I think that the it's it's new, but I don't think it's changing at a pace that that regulators can't can't match. Yeah, we'll be behind a little bit that's okay. So what about when a label goes wrong. How hard is it to kind of step back and fix things after the fact it's you know a lot of work can go into these but of course, you know, once it's in practice I'm sure that there are changes that need to be made on occasion and I'm sure occasionally, there are mistakes that may, you know, really impact the way consumers behave. Brian maybe I'll put that one to you. Yeah, I, as an interesting point, I think a lot of the examples that we've seen on the food side. In terms of labeling gone wrong is unrelated to nutrition nutrition facts so that I'm sure there's some cases now that I say that but just in terms of the label claims that are made. There's some of these outside of the nutrition facts panel, like healthy, natural, good for you, health, heart healthy, things like that. Where you've encountered products at stores like, is this really heart healthy and then come to find out later when there's a more in depth analysis conducted, it's, it may not be as as healthy as we would like or it's been misrepresented in that. And that certainly causes some consternation among consumers in that, can we really trust any of the labels. It's easy I think for consumers to make these sweeping generalizations that once they distrust the label encounter a situation where it's been proven then that impacts buying behavior for other products like well, you know, I don't believe in the heart healthy, you know, good for you, etc type labels so no I think that's an excellent point and that that's what kind of impact and incorrect label can have long term on consumer buying habits as you tend not to trust similar labels on other products. Quinta do you see this in your domain. Yeah, actually, I think it's just a normal. I don't think it's unusual for for most labels to evolve over time. I want to say that normally, there's good intention behind setting up the labels right it's to provide information that consumers can use to judge which products to buy, but the implementation of it is, you know, sometimes where the issues come in. I mean, I mentioned the EPA, because of energy star. So energy stars mandatory right for appliances, and there are certified tests that the EPA has set up in conjunction with DOE and. And so these things are in place but over time, we've had to go back to these agencies and say the test that you guys are running or not quite right. And here are the loopholes that you inadvertently allowed in these tests so I think there's there's always room for improvement and over time we found that generally the the agencies are responsive by it might take some time to fix but things can be refined. The labels can be refined and they do tend to get better and better over time. So it's a different issue if companies are being disingenuous and they're deliberately exploiting those loopholes but again, that's why it's important for consumer reports and organizations like us to keep paying attention and trying to make sure that we close those loopholes. Helen our pre conversation we talked a little bit about the idea of you know regulatory capture. You know how and you just mentioned you of course how the active the lobbyists are on behalf of the tech industry. What can advocates do to kind of get around that or to, you know, try to kind of keep things on the up and up. Well, if you want to get involved in electronic standards, you can join the technical committee there's actually no bar consumers can just join and participate. And then you can watch what I've been seeing. But I would say, I mean, one of the main challenges has been there's kind of no funding to participate in these processes from a consumer agency side of things. And so if, if consumer reports are I fix it doesn't engage like who else is going to represent consumers. And so we've seen over time, one of the strategies manufacturers have is to make the process as complicated and time consuming in terms of like literally years as possible, because who can dedicate you know two hours a week for the next three years to work on a on a standard. You kind of have to have someone paying you to do that. It's very hard to do that on a volunteer basis. So we don't have kind of systems and processes in place this is an area where I would say the EU is better set up for this, because the consumer agencies get government funding to engage in this kind of work. And a lot of the repairability research that we have done, we have a European subsidiary and we've been able to get a number of grants from the European Commission to do and kind of advance and push this work forward. And I think that's why you're seeing this label is is in place in France and coming, coming soon on Europe and you're also seeing discussion about you know requirements on security updates and standardized ports. And that's the kind of fundamental, I think research and so that work that has to happen to counter the manufacturers. Quinta Brian are the things that you admire about about the European approach. I'll let you go first. Thank you Frank I've not been following the European approach that closely. I mean just based on what Kyle is saying of course this is, it does seem promising I really like that I fixed it has been able to roll out this fixability index. If you would call it that because I do think that's something that's important. I do know that on the energy side, the European of the EU energy label. I've read reports in the past about how that that was a little confusing for consumers, because of how the information was presented. And I read with great pleasure that they've been paying attention to the complaints that people in the EU have had and they are revising those labels and I think they're rolling out the new labels as well so I can see from a distance they appear to be responsive to the consumers so that's great. Yeah on the food side my focus is primarily on the domestic side as well but I will say just in general European rules regulations tend to be more progressive more consumer focus, compared to the US. And the dynamic that you see in the US where the industry holds a lot of sway a lot of influence with policymakers, both on Capitol Hill and with the regulators on the, on the other side of the Atlantic that dynamic tends to be flipped. It relates to interactions and influence with, with policymakers there. And, you know, I certainly don't want to paint industry with a broad brush. I think, especially on the food side there certainly, there's been a more concerted effort over the past several years of outreach to consumer groups to build rapport to hold meetings to just kind of establish those relationships more for information seeking and information exchange so I think, ultimately, that's very productive and certainly the message from us to them is the more transparency that you can provide in these situations the better and the more trusting consumers can be. Great. So we do have a couple of questions that have come in from the audience. I want to thank you all for those and remind you there's a little time for that. In some of them are somewhat specific but I've got a couple of general ones. You know, right now I think Kyle this you kind of got onto this a little bit but what can we do to push for improved labels. And then how do we manage expectations clearly that last one from Nicholas saw. Anybody got a got a thought on that maybe one or one or the other of those questions what can we do to push for improve level labels and how do you manage expectations clearly. So we can start with just a couple thoughts on the second question how to manage expectations I think. I think the the honest lies on on the people developing the labels to make them as simple to read as easy to understand as possible. Right, you're not you're assuming that this is going out to everybody in the general public so they don't need to be a scientist to understand this they don't need to be, you know, over 21 to understand it you really want people who are going to be able to, at a glance, tell what the information is that they're looking for so. I mean that requires a lot of different things right maybe their skills that we use maybe their color schemes that that are easier to use but I'm not sure that I'm not sure where the question is coming the askers coming from when they say manage expectations. Because I think that the information should be presented and it should be presented in a way that's very easy to understand accessible for everyone. Can you pull my slide up again I'll show you some interesting things that the French did, and just kind of stepping back from the repair question looking at how do you design a label in general. They did a lot of behavioral research they actually created a bunch of prototypes of different versions of the label. So that you know they settled on this, both the number and a color system. And what they found is they were rolling the label out you have products on the shelf that didn't have the label next to products that did. The average product is coming in right around a five so the average product is yellow, and they found that a consumer looking at a product on the shelf that saw product that was yellow and another product next to it that didn't have the label was picking the product that didn't have the label because they're like well yellow is bad this one's probably going to be better. And over time as as they've gotten more of these things on the shelf we've seen, we've seen more but they did a good job of you know making the label aggressive and giving headroom giving opportunity for companies to improve. I would say you know what's how do we how do we get this to answer the first question how do we get this kind of thing going, going in the US consumers need to demand it policy makers need to start working on on policies along the slides we're working closely with consumer advocacy team on on right to repair laws you had 27 different US states introduced right to repair laws so far this year, it wouldn't touch the labeling yet it's kind of setting a baseline that then companies could improve upon. And, and you have a bill in Congress right now, the Federal Trade Commission has announced they're doing a big investigation into repair competition. So there's a lot of momentum there's a lot of opportunities to engage if you want to get involved you can drop me an email on Kyle I fix it, and I'm happy to connect you with the coalition, because we absolutely need help. There's a lot going on there's a lot of momentum right now but we haven't gotten anything across the finish line yet. I've got a couple of questions here that are that are sort of specific and they might be maybe slightly more for Kyle given that they're about electronics but anyone else can jump in. The first is we'll see you add replacement parts availability to product ratings. I can't answer that question. Okay, I just say we're having the conversation but I don't think anyone that consumer reports is going to. It would sure be a great thing to do though. Cool. Okay, well, that question will hang out there then one that will follow up on follow up on after fact I'm sure. And then a more specific question from an anonymous ask her what labels should smart devices such as TVs that require external services require. I have a perspective but do either of you want to weigh in first. It looks like that's in your wheelhouse Kyle. It's one of the things that's frustrating with a TV in particular is you cannot buy a TV on the market these days that is not a smart TV. And you may not realize this but the TV manufacturers are oftentimes losing money selling you a TV and they make up the money by selling your watching data. You know, like a tracking pixel on your TV and it used to be Nielsen ratings well now, you know, the TV manufacturers are selling what you're watching and and and then also injecting ads and everything else. So, I think there ought to be some disclosures around that. Personally, I will never ever give a TV my Wi Fi password just don't connect your TV to the internet period, because the lack of security updates for these things is major. This is the case across all smart appliances. If you go to LG's website they say that you should check for security updates for LG refrigerators every other month. So I'd like everyone call here to think about the last time you look for a security patch for your refrigerator. This is the case across all Internet of Things devices. So manufacturers should be disclosing like does the device have internet connectivity.  They're secure to connect to your network. And, and then in the case of something like, okay cool you built Netflix app into the TV great. How long are you going to support that for Netflix changes their login system three years from now is your TV going to be obsolete. And you have to replace a whole fully functional TV just because of software obsolescence. And then also of course parts and tools and that kind of thing. So I mean this and maybe just stay with us for a minute Brian Quinta if you have any kind of reflections on that from your perspective, looking at other sectors in particular but, you know, I think one thing that is different of course from maybe maybe most food products, and the types of devices that you just described is that, you know, once you purchase the thing and take it home you don't imagine generally that manufacturers going to, you know, silently slip by and change the dang thing. You know under your nose which seems to be what Kyle's describing so I don't know is that is there any kind of corollary in other labeling regimes to you know how to avoid the confusion that could come from that. I think the common thread in all of this is, you know, to Kyle's point earlier to just consumer engagement. Again, it just speaks to the burden that is placed on on consumers and in knowing what goes into these products food electronics, automobiles, etc. I mean some of this information that Kyle just alluded to. I think it certainly was a surprise to me, in some respect in terms of the smart TVs it's certainly going to impact my, you know, abilities or my reasons to to purchase certain TVs. Why, you know, isn't that information, you know, readily available to consumers and easy to process easy to digest so to speak, you know, from a food perspective. It just, you know that the burden is always going to be on the consumer to find out more information to, to get more transparency from industry and making sure that they're making the right choices and it's unfortunate because, you know, a lot of us just don't have the time to vote. To resources to to look into this information to find out all of the pertinent information. And it just makes that much more difficult and it makes all of us, you know, our work that much more important. Yeah, one thing I can add is in the past CR has what we do, you know, constant consumer survey research and past research shows that ads on TV tend to well ads in general tense emphasize for cars emotion over things like fuel economy. And in particular, the research I'm thinking about showed that only 15% of auto buyers reported learning fuel economy information from advertising, and then about 23% of all current vehicle owners. For them the window sticker was the only source of mileage information. So, you know, there it is. So, so one thing I wanted to kind of touch on as well, which which may, you know, come back to that on some level. Brian mentioned questionable terms and when those show up in labels things like heart healthy or organic that can be, you know, really ambiguous or you take an advantage of. I guess, to some extent I'll ask across the board, but maybe starting with Kyle, are there, are there terms like that in in the in the tech space generally with appliances and other technology products that you think of as the equivalent to organic or heart healthy ones that leave the door open for for lots of interpretation. Great question. I actually don't think so I know it's a major problem with food I can't really think of one. I mean, I would say it's just really about like length of updates with the really this is all lies by omission, rather than twisting words like I know that we see a lot in the food realm. Good question I will think about that. And Justin just to clarify real quick. I think the two, the two biggest labels that from the food perspective that tend to be misleading confusing etc is is healthy and natural. I just want to clarify that typically organic is doesn't really fall into that category just because there are some clear standards that manufacturers have to follow in order to be deemed organic so just want to make sure that that would put that information out there so we don't get in trouble. Don't want to don't want to introduce disinformation into the right to the minority complex topic. So, another question that is around a specific technology as Alexa and Google assistant are added to cars and other major products. They still work if we turn off Alexa and Google assistant so are we are we looking at a world where we can't tell Apple and Google know in the future and I guess this is also another question that gets to the complexity of the inter interoperable nature of some of these products and services or the, you know, importance I mean I have a problem like that with my car which you know, no longer seems to work with any modern Bluetooth devices for whatever reason even though it did perfectly well five years Yeah, this is very relevant then this is actually the topic of a ballot initiative that passed in Massachusetts last year that so you have when when a car is talking over the wireless cellular network to the manufacturer or to Alexa whatever that that's called telematics. And the manufacturers were increasingly moving the access to repair information from wired you know you plug in your mechanic would plug in a physical tool into the car. Now it's passing your your information on whether you need an oil change to the manufacturer directly over the cellular network. And the Massachusetts law says that manufacturers have to give consumers and independent repair shops access to bad information. You can also imagine maybe the ability to turn it off would be nice right do I want Ford and GM to know where my car is at all times. Maybe I do maybe I don't. I should have that choice. The same thing is happening with farmers, all these new modern tractors have cellular modems on them and they're communicating they're sending john dear the data the tillage data on your farm you want john dear to have that data they have a pretty cozy relationship with Monsanto who's selling me seeds maybe I don't want Monsanto to have that information. So we've been talking to farmers and teaching them how to remove the SIM card from their tractor. So I think these are all very relevant conversations. And you have to assume that the pace and speed of technological change is going to be much faster than the pace that you're going to replace your car refrigerator. The ability to kind of opt out of these internet connected ecosystems. And in the case of your car does your car have a headphone jack that sounds like a great way to bypass Bluetooth. These tried and true technologies are always going to be more resilient than whatever the current hot, shiny, you know, integration is to go and look for that headphone jack. So I've got a question here as well about, you know, items qualities features including included in labels and when they've been updated. This one I think really for Quinta and Brian. Are there new things or requirements you think should be added to update food and energy labeling systems at the moment, things that would really improve food or energy labeling systems right now. And Brian maybe I'll start with you. I think what what could really improve labeling for food are clear definitions for natural and healthy, like, like I indicated earlier. And also, you know, just going back to the slide that that Kyle showed just in terms of how the information was presented in the EU. I think similar for food would be very helpful to just a very clear upfront symbol front of pack that that gives some clarity some clear indication to a consumer of health of what have you that so that the consumer can just process the information immediately as opposed to trying to review the nutrition facts panel or you know go on a website to do more research there. I think those are the, are the, are the biggest items right now, you know, just, you know, more clarity on menu labeling to, you know, that's a law that went into effect this past decade and, and, you know, there's still some work that needs to be done there to kind of make that information more available and more clear to consumers. And again the food delivery apps that's that's another area where there's some obfuscation going on in terms of who is responsible for providing that information but it's clear that it has to be made available to the consumer. Yeah, that's an interesting point you know so many people these days are getting their food from either delivery or from, you know, services that send them ingredients already ready made. And, and you know that seems to bypass most of the types of labels that you're talking about. That's absolutely right and I think to Kyle's point too is that there's always going to be a little bit of a lag between, you know, what the existing technology is happening now and then, you know, consumer response and also more importantly how regulators respond. You know, and you've seen this play out with food labeling and that the nutrition facts panel was a combination of decades of work that went into it about the availability of this information and so you've seen this play out with other aspects of that as well. And I think that's an interesting point to anything from you. Yeah, you know, again, I agree with Kyle and Brian awareness, education of the consumers really important because you can put these labels out but do people really understand what they mean. You know, it can be easier, it's, it can be easy, because energy consumption energy efficiency those can be boiled down into how much money it takes to operate an appliance over a year for instance. So you can, you can put all that information but as long as you have the money, the cost, then the consumer is able to compare products and say well this cost me $200 a year this cost me $10 a year. This one looks better but maybe I want to spend less, you know, so that part is easy but there is also, there are also things like greenhouse gas emissions that are associated with products and so that's something that consumers are certainly very interested in but it's really complicated to get that data and to present it. Again, this opens up, you know, opportunities for companies to become to be disingenuous and how they present that information. So if I if I have a product in my hand a cell phone for instance or an appliance. Am I going to look at the greenhouse gas emissions that it took to manufacture it what about the greenhouse gas emissions that went into getting the materials that went into manufacturing it.  Do we consider, you know what happens when you discard the product. So I'm happy to say this is something that consumer reports is interested in and we're looking at. It's not an easy question to answer but it's something that we need to pay attention to. But if we put these labels in place then they just need to be done in a way that is unambiguous is easy to explain easy to understand and takes into account all the greenhouse gas emissions that go into making shipping and operating the product so that people can really make well informed decisions. We've had a wide ranging discussion about the possibilities with labels the challenges the opportunities, and I certainly know a lot more about them than I did when I started this as a mere consumer not as an expert like three of you so I appreciate it very much. Just really quickly, Brian Quinta Kyle is there a place where folks can find out more about your work or follow you. Sign up for our consumer reports food advocacy newsletter and the consumer reports advocacy website is separate from the main consumer reports website and attracts all of the policy issues that we follow the four policy makers. Great. Yeah, just to add to that I mean we work at the same company so nothing new there. But I will also say that we, you know a lot of our members are very interested in getting involved in the advocacy process. And so as we are following proposed rulings, we do give our members the opportunity to you to be a part of that process and just learning petitions or, you know we we provide information for them if they want to sign up and give public testimony as well so it's a really good idea to to sign up and follow our newsletter so that they can be alerted when those opportunities come up. And Kyle. Hey, I fix it calm and you can sign up for a newsletter there's lots going on. I'm Kay Wiens on Twitter, but also the repair coalition is repair.org. And if you want to get involved in right to repair and see some of the laws that are happening. We're expecting a lot of motion to happen on these right to repair laws starting in January when the state legislators get back in the session. There's a lot of momentum there's support from the Biden administration there's support from the Federal Trade Commission, and so everyone's going to be focused on the state houses this next year. Excellent. Well, I think we've reached the end of our hour it's time for me to turn things back over to Andy want to thank our panel thank you to Brian to Quinta to Kyle. Kyle has already got a drink at the ready there with the bottle of wine nearby so you all can congratulate yourselves for an excellent midday panel galleries for serving a wine did you know that I did nothing to worry about right absolutely. Okay, everybody Andy I'll turn it back over to you and thank you all so much. Thank you everyone it was great to hear about the work that you're doing. And I'm going to actually pass it on to our next panel, which is talking about some future labeling systems that are in the works. So, Jonathan Schwantis is a senior policy council at consumer reports, and he's going to kick off and introduce you to our second group. Great, thanks Andy and thanks to Justin for moderating the first panel into our excellent panelists who participated was actually quite pleased to hear some questions about connected cars and Alexa and Google assistant and smart TVs I actually do think it'd be nice to buy a dumb TV. All it is to say, we're going to take up some of those issues here in our second panel, a little bit about a little bit first about why I am my name is john Schwantis I'm a senior policy council and consumer reports Washington DC office. For better or worse I've spent my entire career in and around politics for 20 plus years. I started off and spent half of that in the Senate on the Senate Judiciary Committee, where I served on the anti trust subcommittee where I got fascinated with all of these tech and telecom issues at consumer reports, I am on the digital rights team, where my focus is on telecom and competition issues but we work on a host of issues including privacy Internet of Things algorithmic bias and artificial intelligence all of which I hope we will dig into here on panel to. So I kind of see that we're moving from old school, the nutrition facts energy star etc to new school. So what will consumer labeling look like in the digital world. What's the need, and how could labels help us make better and more informed decision at the point of purchase will probably touch on the politics which are unique which are unique to the digital marketplace or maybe not unique. And what industry is doing or failing to do to make better disclosures whether that's something as simple as being transparent about how much your broadband bill will be each month, you think that would be simple. It's not. And to what data about you is being collected and how it's being used and how long, for example, is a connected device you purchase on will be servicing updated. So let's kick off we're going to move to introductions of the panelists and then we'll tee up some opportunities here at the start for them to describe their work. I'm going to start with my friend Sarah Morris Sarah is the director at the new America's Open Technology Institute or OTI, where they have been working on a consumer label for broadband service aka a great excuse me, a broadband nutrition label for more than a decade. And I also know that OTI is collaborating with us here at CR on a sort of digital standard 2.0 I don't have any don't have much to do with that Sarah's but can you tell us a little bit more a little bit more about you and your work. Sarah thanks so much john it's great to be I'm really excited for this discussion, and I'm excited to be able to talk about both the broadband nutrition label and the digital standard but the broadband nutrition label, in particular, because it's been something that I've worked on throughout my long tenure at at OTI. So just by way of background, the Open Technology Institute is a program within new America that works at the intersection of technology and policy to ensure that every community has equitable access to digital technology and its benefits. We promote universal access to communications technologies that are both open and secure and we use a multi disciplinary approach that brings together advocates researchers organize organizers and technologists. This spans a multitude of issues, everything from broadband access to data policies around privacy and algorithmic accountability, government surveillance encryption, and lots of issues in between and adjacent to those that I mentioned. And access and adoption work. Our goal is to make sure that everyone throughout the country has access to internet that is both robust and affordable. And those two characteristics are important to us. As john knows because we've been long time partners and allies in this advocacy work. It's a complicated problem to solve I think if it wasn't complicated everyone knows how important internet access is and getting people if giving people connected were easy. Then we would just pull those levers and make the change and we are making progress I think but we we have a moment here to really lean into to to closing the digital divide. In advocacy work OTI has done a report periodically, called the cost of connectivity, and this report looks at the cost of internet we gather data on 760 internet plans across 28 cities in the US and around the world. And it started out as a comparative analysis to see you know where we stood visa be Europe and cities, the cost of internet and the service and cities compared to from the US to cities abroad. But what we found in our most recent report which we put out last summer in 2020 with, with less about the, it was about the comparative cost and quality of service, but it was really about the, the fact that internet in the US is not affordable for the vast majority for many for many millions of people. The average cost hovers around $70 a month and that's not even including the cost of internet after promotional rates expire and the rates go up, which we found to be as much as 2223 dollars per month on average. And so what this means is that, and the other the other thing we found in our research is that it's really hard research to do. It's it's not the information is not readily available. It's hard to compare services, because different companies make different types of information available. And all of this reinforced something that we had intuitively known all along, which is, which is we need some set of standardized user facing disclosures about the cost and quality characteristics of internet service, which is what we had proposed originally in 2009 with our broadband nutrition label. And I think we have an example of the nutrition label handy if our production team from New America can put it up on the screen. And so the idea here is that the broadband nutrition label would give would be internet subscribers the ability to compare apples to apples. And to know more granular detail about the cost of the service and the quality of the service which might otherwise be obscured things like promotional rates, data caps, other charges that aren't always apparent from the monthly fee. And so you would use the nutrition label to look across the available assuming there's more than one provider in your area which isn't always the case as we know that you could look across the available services and make a more informed decision about the internet that you're to which you're subscribing. And this is important. It's important because that's how an effective competitive environment works, but it's also important because often subscribers may come in under a promotional rate and then be an experienced significant sticker shock when that promotional rate runs out and not have clear recourse for how to navigate the information available to them to understand their work. And so we've been so thanks to the New America folks I think we can take down the label for now. So that's been our work around the broadband nutrition label and we think it's and we've seen, we'll talk to you think a little bit about the state of play in the broader conversation, but we've been really enthused to see a lot of interest in the broadband nutrition label again it's been out in the world since in some form or other since 2009. And but in the past year or so we've seen its inclusion in the fiber and clope a char bill the accessible and affordable internet for all act. The recently, the infrastructure package, which we hope will pass soon. We've seen it in an executive order from the President this summer on competition. And there is a standalone bill from representative Angie Craig out of Minnesota. So we're seeing a lot of interest among policymakers and I think the question now is how do we translate that into movement and momentum to make this type of label actually usable for for users. And you also asked about the digital standard which I'll just cover briefly here to the digital standard, not a label but it's a collaborative project that OTI has been working on with consumer reports it's a consumer reports. And it's really a framework to evaluate how technologies respect consumers interests and needs and so the way it works is it focuses on privacy security ownership and governance of connected products at the top level. And for each of those areas that lays out principles, then criteria, then indicators and finally procedures to evaluate and enable structure to enable structured evaluations by independent testing organizations researchers other product teams. So describing this I think it becomes apparent that there is an opportunity to take where the digital standard is now and to think about how it might be utilized not just by researchers and independent testers but by consumers by policymakers by companies themselves and I think that's where we kind of although it's outside of this conversation strictly about labeling and consumer focused focused labels in the technology space. There's a lot of opportunity there to think about how we connect the dots particularly as we heard from the last panel in this world of iot devices where not only is there a lack of information among consumers. There is this dynamic nature of the products themselves that are, you know, shifting over time. So I will stop there and pass it on back to you john introduce the other panelists. Great, thank you so much Sarah and as a fellow political junkie I'm excited to get into what is going on with our beloved label in Congress. And next up we have kasha shimolinsky. Tasha is the co founder of the data nutrition nutrition project an initiative that builds tools to mitigate bias in artificial intelligence. They are also a digital expert at McKinsey and company and an affiliate at the Birkman Klein Center at Harvard University. Previously, kasha held positions at the US digital service in the White House and the MIT media lab in Cambridge, Massachusetts. When not in front of a computer kasha can be found leading a feminist birdwatching event or cycling cycling uncomfortably long distance distances, but things I both love kasha the floor is yours. Thank you, and I'm honored to be on this panel this kind of two part panel all these great people I'm learning a ton actually. My name is kasha. I could talk about birdwatching the whole time but I won't. I am the co founder of a group called the data nutrition project as, as john said, and I've been a product manager and industry as well kind of simultaneously across a lot of different industries so consumer governments, I've been working in education, medical and fintech I'm basically quite old. And, you know, more more recently focusing a lot on the data nutrition project, which, as you mentioned is an organization a research organization that builds tools for more responsible AI. In particular I think relevant to this conversation nutrition label for data sets. So maybe it makes sense for you just to take a step back and tell you a bit about how it came to be. And then I'll pass the torch on here. Essentially, a bunch of us came together practitioners in the space, also artists and policymakers for a fellowship that was jointly held between the Berkman Klein Center at Harvard and MIT Media Lab in 2017. We physically got together back in the old days, and we, you know, sat in a room without masks and talked a lot about things and shared snacks and things like that. And the, you know, the, the target that we were given was to think about what we could do about the ethics and the governance of AI. It was a few years ago, so it was a little bit more of a nascent space. And all of us who are kind of practitioners around the table we decided to start with the data, because we knew the current situation which is unfortunately still the current situation, which is that there really aren't any data sets standards. So if, if I'm given a task, you know, if I'm a data scientist or maybe an engineer at a company a CEO comes to my desk one day says, I need to understand, you know, how to target people for this advertising campaign. We have some internal data. Can you go find some data demographic data something so that we can build a model get this out you're like oh sure that sounds like an interesting question. When's it due and the CEO says Tuesday, you know you're like oh shoot.  And so as a group sitting around in a room and MIT, we said, hey look why don't we start here when we think about the ethics and the governance of AI let's start with the data, because it's that classic problem right like garbage in garbage out whatever you build on the data ends up picking up the traces of that data in the final model. So we're really focused on the output of models, but not so much about how they got there. So if you are catching issues once the model has been deployed it's kind of too late, because if you're at a big organization they've put in a ton of people hours and money and resources it's changed hands a dozen times, and you get to the end of it and then it's remediation you're just trying to figure out how do we change the parameters or maybe switch the coefficients a little bit to make this a better model as opposed to, if you know what it is your model is going to eat. Before it even has started eating anything, then you can maybe change that diet so that it comes out different at the end. And that's kind of the nutrition analogy there. So we're sitting in this room and we thought, what if there was some way to tell what's in a data set before you eat it. Really, what if we had a nutrition label for a data set. And we knew the analogy would break down in some cases and I feel like the previous panel also had a lot of great things to say about the application of these kind of analog models to the digital and, you know, and we're going to touch on it here as well I know Sarah was kind of touching on that too. You know so so we thought well let's give it a try. And it was an academic fellowships we wrote a paper, and we produced a prototype with ProPublica on a medical data set. And then the fellowship ended. So we kept getting inquiries about the paper that we wrote and the prototype we built. And so we thought, you know, there's so much interest and we're still interested in trying to solve this why don't we just spin this out into a company. And so it turned into a nonprofit and we're incorporated as a research group, we put out a second version of the label in 21. Actually, I think there's an image if you want to put that up I can kind of talk to it for a second. There's a main difference here so this let me give you like a very quick tour. This is a nutrition label for a data set that basically is a bunch of images, 10s of 1000s of images I believe that does melanoma classification so this came from Memorial Sloan Kettering and also we worked with a doctor there dermatologist who helped us understand the data set. What's in it what's not how it should be used how it shouldn't. And the way that this differs from the first version of the label, which we published in 2018 is that it's really kind of centered around use case. So depending on what you're trying to do with the data we try to tell you some things about how you should or should not proceed. So the use cases on the right hand side, this data set would be useful for identifying melanoma, and to predict, you know the incidence of melanoma in a population actually that should not be a use case that's the type of there on the old draft. And then we have these kind of badges at a high level which are kind of binary or just multivariate but structured in a way to say this is what it is and this is what it isn't, and we have these alert counts for potential harms or bias. So you can take that down now don't want to clog up people screens but that's the that's the current version and we're actually working on a third version now that's going to make it even faster to create and more obvious to read. We're currently working with a number of tech companies, mostly on internal data sets a proprietary data sets there's an interesting conversation there about how public these become. And also through a grant at Harvard to put these on open data sets in their data verse, which is the Harvard repository. And just to kind of summarize right the goal here is that there are not really any data set standards across domains to do things like compare especially for usage and algorithmic systems or algorithmic systems builds. The goal is to drive a standard here. We believe the standard will drive better data usage, and also it will drive a culture change, because there will be an expectation of seeing that label as a data scientist, much like when I walk into my bakery down the street. And I see these like great pumpkin muffins that are now on sale because it's fall in New York and I want to know, oh wait, how what's actually in this muffin, I have an expectation of a label even though it's not on the muffin because I've seen that in other places where it drives the same kind of expectation for data sets. So happy to answer questions at some point but that's an overview of what I'm working on I'm, I look forward to hearing about everyone else's. Hi, I'm Sasha that's already fascinated. I'm excited to ask you some questions. But our third panelist is part is the mommy Naini part is a postdoctoral researcher at the University of Washington, where she investigates tools and methods to empower people to make informed decisions about technology. Part is obtained her PhD in computer science from Carnegie Mellon University there as part of her thesis, she led a series of projects on to design security and privacy nutrition labels for consumer smart devices. Part is a top venues in security and in the human computer, excuse me human computer interaction and covered by multiple outlets, including wired and the Wall Street channel artists, you're up. Awesome. Thank you very much Jonathan for the kind introduction and hi everyone. As you just heard, I'm proud to say mommy Naini from University of Washington. Okay, so let me tell you a bit about a research on IoT security and privacy labels. About four years ago together with my colleagues at the School of Computer Science at Carnegie Mellon University. We started a series of projects to design a usable and informative security and privacy label for consumer smart devices. The initially that interested in labeling smart devices. After we realized that many consumers currently are not able to make an informed purchase decision, mainly due to lack of readily available information about the security and privacy of smart devices at the time of purchase. This motivates us to design a nutrition like label for IoT devices that convey the most critical security and privacy information of smart devices in an understandable and usable format. We next invited a diverse group of privacy and security experts from industry academia government agencies and nonprofits to help us specify the most important information that consumers should know about when purchasing a smart device. And based on the experts input, we considered about 47 security privacy and general factors and design a layered label be two layers. Can you please have the label on the screen please. Awesome. Thank you. Okay, so the primary layer is the concise format of the label, which could be printed and attached the package of the smart device. And then there is a QR code and URL at the bottom that direct consumers to the secondary layer, where they can find more detailed information about the security and privacy practices of the smart device. After talking to experts within conducted series of studies with consumers of IoT devices to improve the usability and risk communication of our label. And the one that you see on the screen here is the final version of our label. In addition to this label, we created a specification document for users and device manufacturers to understand each field of this label. Moreover, we developed a tool to generate the labels basically user can fill out a fork for different sections of the label and then see the label being generated in real time. And we are now looking for device manufacturers to pilot our labels and real smart devices to carefully investigate the efficacy of such labels in informing IoT consumers purchase behavior. That was a very short summary of where we are with these IoT labels. Great. Thank you parties. Let's move on to questions and encourage the audience to go ahead and go ahead and ask questions. I'll get to as many as I can. Can actually go back to the questions from round one that would be great for this panel as well. But Sarah, I'm going to start with you and direct a question to you and this is again whenever I say it's a simple question it's not. It's a simple question. Why is it taking so long for the broadband nutrition label to happen. You know, assuming the biff the bipartisan infrastructure framework which is in the in the alphabet soup that is DC the biff is one of the dumber ones but anyway, assuming it gets passed, and I guess we can all assume but it will become law and actually be required, kind of a little bit about the issues. It's going to be a safe harbor kind of optional for ISPs. It will be required. But why is it taking so long. Well, I think part of it is we are that the fight to close the digital divide happens on multiple fronts right we are trying to improve the competitive landscape across the internet ecosystem more generally. We have, as consumer reports have and many other groups focused on subsidy programs, the lifeline program and more recently the emergency broadband benefit and then in the, in the biff, the affordable connectivity fund to really help immediately mitigate the high costs that our cost of connectivity report found at an individual level. And there's many other, you know, aspects of this work and so I think that, you know, and I think that while policymakers and the public more generally have long understood the importance of the internet in sort of an abstract sense. I think it's the pandemic and the way that that shifted how we engage with technology and our reliance on internet access has really laid bare. There's a need for internet that is both robust and affordable, and the, the harms that occur when, when disparate access exists. And so, there is this sort of urgency of the moment that has shown a spotlight on the problem that I think lead policy makers to look to concrete things that they can do to to really move the needle and we feel that the broadband nutrition label is one way to do that. Now it's part of the suite of things and I think it's important to note that transparency is important in the I'm sorry my cat is decided to join this panel and won't leave. Yeah, so the broadband nutrition label would be like a user focused mechanism for transparency we've also long urged the FCC to collect pricing data directly from internet service providers so that we have a better, more comprehensive understeer that the FCC at least has a better landscape around the price of internet, in addition to the sort of point of purchase or point of subscriber transparency that we believe is needed as well so there's lots of moving parts. And I think that's why we've seen I will I will note though that while I mentioned a number of bills where and the executive order where this issue is active. And we see a lot of progress on the broadband nutrition labels back in 2015 and 2016 and so if you'll indulge me for just a moment as I kind of because I think this is an often overlooked part of the nutrition label story broadband nutrition label story. As many people who have been following tech policy no in 2015 the FCC passed the open internet order, which did a whole bunch of things to help preserve net neutrality in the United States. It also included very robust transparency provisions, and it directed the consumer advisory committee at the FCC to create a safe harbor for compliance with those transparency provisions so the idea was that if a company didn't want to sort of like, read all the transparency rules and figure out a system of transparency themselves that they could just use this thing that was created by the consumer advisory committee which involves companies consumer representatives other digital rights groups. So has inputs from all and what came out of that consumer advisory committee process was actually a model nutrition label based on otis recommendations from 2009 and so we have a report contemporaneous that we put out contemporaneous with that process. And that gives our framework for how we think about broadband nutrition labels, but you know I do think this is an arc and we're at a pivotal moment where we could see major progress but it has involved a history and and varying degrees of interest from policymakers over time as well. Thank you Sarah and we can we can get into internet pricing, maybe a little bit later if we have time and kind of what industry tells us is a very rosy picture and what consumers are telling us we've done some work here at CR as well. As you know but, but really sort of why there is even the basic, why do we even need this label and the answer as well, you can ask the ISPs why but we as consumers know why. I want to turn it over to you next I was asked for your question. As someone who's not a data scientist, but it's fascinated by these issues, how I just want to better understand how labeling and transparency at your at the project you're on working and how does that translate to the world of artificial intelligence kind of like if we could see five years into the future. And where does labeling fit into that. Yeah, just the simple questions huh. Yeah, so I mean it's a really good question there's so many parts I'll just say some things and then if you want me to keep going in some direction you can let me know. You know I think that labeling as an act as we've kind of like heard from other people provides transparency and then through that choice. And a lot of the conversation about AI today is really not about choice I mean we're all kind of the, our data is being leveraged by algorithms that then make decisions for us. And so there's not often a way to say, Oh, I would like you to make decisions with my data or I'd like you to make a decision using AI for me versus not. It's usually just a binary of you get to use the service or you don't essentially. So I think that the labeling, you know is a, it's kind of, like I said before it's kind of a cultural change to say that we need to drive some kind of transparency into this process of making AI. And by doing so, I think consumers become more aware of what's actually happening, which sometimes is quite scary, just to become aware of what's actually going on in the world of AI and how things are being built. It also then you know drives change when there is real change, you know because people can, sorry, choice when there is real choice with people can say I'd like to choose this over that. And that's kind of why we as a group decided to focus on the data. So there are, there are a number of initiatives out there that look at the model and you know the algorithm itself or algorithmic system there's model cards, there's fact sheets there's a bunch of groups that are doing kind of similar things in terms of documentation. I think often the label outside of tech will be called a label and then inside of tech will be called a new form of documentation. And then there's kind of the work that we're doing which is focusing specifically on the data because that's where we feel people are actually empowered to make a choice. And when I say people I mean practitioners who are building who are building the algorithmic systems themselves. So, you know in my example from before someone's asked to go find a data set, you might go and Google it. And you might just find here are a few different data sets that I can use for this purpose I might find them, you know through census or I might find them on Kaggle, or might find them in some open repository hosted by university someplace. There are benchmarking data sets that people sometimes use. If you're in lending you might just decide to go use, you know, credit karma or lending trees data which is open. People are just going out there finding data. And at that moment when a data scientist deciding whether or not to use that data she has a choice right and and I think that that's that to us is the opportunity for a label is to say if there's a really quick way to compare what the data sets are that you're able to use on the internet, or even through a service or data repository these kinds of things. Then you can make better decisions about what you actually do with that data. And then, you know, when you have the data and you're building an algorithmic system on that data, you can say, All right, we need to think about representation and this data set so on the label it says this data set is going to be representative, with respect to skin tone, like on the melanoma data set, or, you know, this medical data was actually sampled from patients who are mostly based in Asia. Right. And so if you're going to go and build a model that is not going to be deployed in Asia, you need to think about whether it matters that all the data that's captured in that data set is actually built on a different population it's capturing a different population. It doesn't matter. Maybe it doesn't. So that's kind of, you know, where we've targeted our labeling technique and our initiative is at the data level, because again we believe that it kind of brings transparency and real choice around something that data scientists can actually do and are doing every day which is deciding which data to use to then build a model. So I guess I'll pause there for a second I think that's kind of our angle into how labeling ultimately impacts the quality of AI. Excellent answer Kasha and for the first time in my career I've been at CR from coming up on my fifth year anniversary I work very, for the first time in my career I work with data scientists and they're super careful about when we say it's a nationally representative survey for example. A quick follow up question if I may before I forget, with regards to your labeling at the data nutrition project, would those be readable by automated machines or is it just for data scientists just curious. That's a really good question and we've gone back and forth a number of times on how much of the information could be read, could be machine readable, but also could be created by a machine. So there are people in a camp that says this should be really easy to create. So when you create a data set you should be able to just hit a button and then a tool goes through it and then it creates a label and that's it. And then there are people totally on the other side and I think data sheets for data sets out of Microsoft is a good example of this where they're like, nothing should be done in automated fashion this is a conversation. Right and the most important things that you can learn about a data set you will never see in the data set it's who was not talked to who funded this. You know what was the time period over which this data was collected and where and what were the methods, right. Was there a quality or an ethical review on this. How did you clean the data. This is a huge question. And those are things that you will never see in an automated fashion or through a tool. So what we've done is, you know, what you would do and if you're just a person who's trying to bridge that which is somewhere in the middle. So some of the things in the in the label, you can think of the metadata basically for the label are somewhat more automatable. So it's just kind of like maybe a binary flag. This is about people. This is not about people. This includes information at individual level. This is just aggregated. So that those are things that you could compare read very quickly. Then there are some things that we do try to pull out that are more conversation starters, because our belief, our opinion is that in this industry, we need more people thinking and fewer machines doing. So instead of making instead of enabling a fully automated process here we actually want to force people to sit down and think because the machine can't tell you whether the fact that the data came from Asia to build a model in Europe. The machine can't tell you whether that's a problem. You really need to have a human sit down and say is that going to be a problem in this case or is that okay. So it's kind of a combination of both to your to answer your question quickly. No, I think it's one of the most fascinating issues with AI is how much human control, will there be moving forward and will it, you know, or will there be the singularity artists, let's move on to you. If I gave you a magic wand and I think we saw a little bit with the sample label you put up there but what would a label look like or what's the most important thing, if you could have this magic wand and create a label for connected devices and some of this from panel one and maybe we can get into that as well but let's just start with the general and maybe we can get specific with some other questions we're receiving with that more speak to your issues. Yeah, sure. Great question. So I think if I have this magic wand, I would just create the label as I showed, basically, but, but the label is just like a design, it's not. I mean it's really important to understand that the design is, is important definitely is important, but we spent a lot of time not just on the design of it, it was on the contents of this label. I think the content is really important it just we need to make sure that any sort of labeling for smart devices or for other types of products for data sets. It's really important to make sure that it's understandable to people and it conveys risk to them. So I think are really important, and we basically conduct a series of studies, both experts and consumers to make sure that they understand what they're looking at, and whether it actually would convey some sort of risk to them but they would just make them not purchase a device or to purchase a device better they can reason about privacy and security practices and that is really important that education part is really important. Basically, I think they're like specific privacy and security practices that is really important for smart devices to disclose for example, what type of data this device is collecting who the data is being shared with who the data is being sold to for how long the data is being kept the retention time. These are let's say for privacy practices and then on security side, whether the device receives security updates, what is the lifetime of the security office for how long the device receives secure updates that there is any vulnerability disclosure in the program and how people can find out about this program. So I think there are specific things that we can see on the on the documents on the security and privacy documents and standardization. And they're like some of the common privacy practices that I just mentioned, but in general whatever we put on the label we just need to make sure that it's understandable to people and it conveys risk to them basic I think this is the really the message that I want to convey. Yeah, just ask me like more detailed question if there's anything that you want me to talk more about on that. Thanks part is that's a great introduction and answer I think sitting here in 2021. There is a whole business model of surveillance capitalism that is reliant upon consumers not really knowing and not really caring to know. So it's difficult and that leads me to my next question kind of for the panel is where is industry when it comes to this debate. I think I have a general sense they certainly do with the ISPs and what Sarah introduced us to, but are there areas where they're aligned or are they just generally resistant or are there some cases where companies are alright. They're opposed to increase transparency and labeling I mean I know lobbying with OTI on the broadband nutrition label. I mean I just asked congressional staffers asked the cable lobbyist what they have to hide about not telling us what consumers are paying for internet service but but this is for the panel so anybody can take it. And let's just talk about the other side and where are they in this debate. I'm going to jump into you sort of petered up there john. Yeah, I mean, I think that the, the objections to the objections tend to focus on either this would be cumbersome, or for either because it's a lot of information to disclose arguably. Or it's the consumers won't use it like that people won't know how to use it and I think we talked on the previous they talked on the previous panel about the need to have sort of like social infrastructure in surrounding the roll out and the use of these types of labels because people need to make we need to make sure that the people have the right information and context to understand what's in them. And I think part of this is example of the feedback loop that they got from from actual consumers is a good reflection of that. And I think that it's, I think it's getting harder and harder to say that this type of disclosure isn't needed when researchers at an organization that focuses significantly on the cost of internet access and then work to close the digital divide has struggled over the course of a year, plus to put together this latest version of the report and you know we are experts in the space and devoting a lot of time to it and if it's challenging for us to find this information that it's challenging for for user internet people using the internet to as well. I also think that it's it makes it harder to suggest that the market is competitive. We don't know. It's a key factor in a competitive market is the product that's being offered and how much it costs and right now, there is, there's obscurity around both of those, those, those things Sarah. Yeah, um, it's a good question I feel like, you know, I think some of the, the opportunities that so okay when we say like tech companies, for example, there's a lot of different actors at a tech company. And I think there are a lot of different facets. So you can't just say, Google wants this or doesn't want this. I do think that there are a few different frameworks that are helpful for me at least one is kind of the top down bottom up. Right. So from like a top down perspective, we've talked to companies that really want to do something around data set transparency or a transparency because it mitigates risk.  You know, obviously the congressional hearings yesterday from our testimony yesterday from the ex Facebook PM, I think it's very in the, in the ether right now this this question of responsible tech and what big big companies are supposed to do about it. And then from kind of a bottom up perspective, a lot of data scientists don't have standards internally. And so even if it's just transparency or documentation internally, I think there's a greater and greater push to do something on the opposite side of that. You know, I think some of the challenges are systemic. So kind of kind of what Sarah said, this is cumbersome who's actually going to do this. There's no role or group or obvious fit for who would be responsible for this. Who would be the one who's kind of defining what's in it if it's going to be an internal standard. So there is kind of operational and organizational change that has to take place in order to support this and what I've seen before is that you have a champion internally who just really cares about data quality or AI ethics but they also have another job. You know, and unless you actually have somebody who is the resources in the time and the energy to do something and are supported by the organization it's really tough to drive that change. And then similarly kind of questions of well how do you justify that why do we really need it. And so unless you have consumers who are clamoring for it, which is hard and AI because consumers are kind of a few levels down from from the actual I mean it's hard to see it's hard to see the impact of the AI because you don't know what other decisions it could have made. So your consumers aren't really clamoring regulation isn't quite there yet. And so a lot of people say well we're doing fine anyway we're making tons of money so why would we need to implement this. So that's kind of what I see in terms of the opportunities but also the challenges. Thank you Kasia. Part of something you want to add in terms of sort of what what's it like working with device manufacturers. Yeah, sure. So the thing I say this, this IoT market is really the concept of market for lemons, basically. So the idea here is consumers really don't appreciate security and private good devices with good security and privacy. At least this is how we see this, but that is really not the case because they don't really have information about security and privacy so they cannot really appreciate a good device. In terms of security and privacy and so then manufacturers see that consumers do not really appreciate security and privacy appreciate the quality of the device so they would just spend their money and resources on those features on those aspects that would bring them money. And over time basically we're going to have devices on the market that are not amazing in terms of security and privacy and they're not transparent about their practices because that is really not cost effective for manufacturers. So this is a really huge issue. And I think a label actually would really help manufacturers to gain consumers trust and just for the market competition. I think this is really good for manufacturers to have. But I think I mentioned this in my introduction that we are really trying to find manufacturers to pilot the label and we still haven't found them. And it's also really important to know that it's not a binary thing that you should either adopt the whole label or just don't adopt anything basically don't just disclose any information it's even if a manufacturer comes and just say that okay they're like specific things that we think we can talk about it's a good start. We just need something like a company comes forward and just say that okay we would like to disclose this information to consumers it would like to help them make an informed purchase decision but I think there's a resistance from manufacturers to adopt the label or even parts of the label and I think And that is in the US in other countries are like some countries for example Finland and Singapore but that you see that we have IOT labels. So in the US this is a very new thing and we still have resistance and hopefully we are going to have less of that in the future. But this is my experience in trying to push the labels to manufacturers that we see that they're not exactly coming forward to adopt the label. And I think that's what you unknowingly gave me a great segue to my next question which is, where are the politics on here I mean in broadband nutrition label, Congress is forcing them to do it and sometimes you've got to pass a law. And this also kind of dovetails with a question I got from the audience how challenging is this politically in all the three just three different areas. I don't know if I should just take first stab at that or, but yeah that's for the panel, like where one of the politics around this, what I mean is the state legislatures as a Congress I know there have been some right to prepare laws that CR has worked on in the state level. But yeah, go ahead. So, in US there are like two major policy orders that I can think of in late March 2021 Congress and Senate reintroduce the cyber shield act. And this legislation creates a voluntarily cybersecurity certification program for consumer IoT devices and as part of this policy there will be an advisory board including diverse experts from industry academia government and consumer groups to specify a cybersecurity benchmark for smart devices manufacturers can then attach a cyber shield table to their products voluntarily certifying that their devices meet this benchmark. And I think the second one came very recently, May 2021, then a White House executive order was signed to improve the nation cybersecurity and protect the federal government's networks. So part of this executive order, the National Institute of Standards and Technology or NIST basically has been commissioned to work with some federal some appropriate federal agencies to set up a pilot program to really understand how the labeling for smart devices should look like which they are currently working on that basically. And that was like in the US, but as I just said, the discussions in the US are very new. It just got started, but in other countries such as Singapore and Finland, we have the labels for example in Singapore we have a star rating, the labels look like a four star rating. In Finland we have this, we have this mark security mark that for example if your device satisfies some requirements you your device can get a security mark to just show consumers that this device is secure to some sense basically. Yeah, so this is I think where we are with legislations in US and other countries. I hope to see more of them in the future. Tasha Sarah anything to add on the politics. I mean, very briefly because I'm really not a policy guy, I think, most likely will see policy shifts with an eye towards potential harms to humans. So I think there's going to be kind of you see already risk assessments or algorithmic impact assessments which say what are high risk and what are low risk. And those might start to be required, especially within use in government. So if you have a specific domains a medical financial. The big ones that have to do with humans. And you, there are government agencies that are building algorithms using data that help to say distribute resources, make decisions about citizens or non citizens as well. I think is is one opportunity to start to see some change. So you might have some kind of internal data or algorithmic standards that are required for use within government. And then I think, you know, on the kind of flip side of that policies or or initiatives around algorithmic accountability transparency, data set quality standards all of these that are required of big tech. So I think that's kind of down to the corner as well but obviously way more complicated in terms of lobbying and and and there's a huge push within the tech industry to kind of self regulate, as opposed to having regulation put on them. So that's kind of what I'm seeing is movement inside government, and then hopefully movement outside government, but with a lot of pushback. So in my time in politics, the, the self regulation narrative is usually a sign that they're worried that something's coming. That's so much to be regulated. Yeah. Sarah. Yeah, I think I've covered a little bit of the landscape already but just to kind of like put a bow on it. I think that the politics have shifted this is clearly something that has captured the interest of policymakers, including the the president, who very explicitly included of the broadband nutrition label and a mandate to the FCC to implement it in the competition executive order from this summer earlier earlier this year. And I think the inclusion of the broadband nutrition label in the infrastructure package is a sign of something that that I believe should be true which is that this should be a bipartisan, something that we want across both sides of the political aisle to see is, is hard to argue with. And we're talking about such basic transparency about what it is that we're expecting people to purchase. And, and so I'm hopeful that we're, we're hitting a point. I am sure that former former policy folks who have worked on this issue at OTI over the years are probably squirming, waiting for this to finally move past the finish line, because it really has been in the works even since before I joined 10 and a half years ago. But yeah I think we're really hitting the moment where this, this could actually happen and I think, as I've said before this isn't the end all to be all around the question of broadband affordability, or or access more broadly but this is I think a very concrete step that we could take. As soon as the infrastructure bill is passed to move the ball forward. No, there are a lot of challenges and failures in the broadband market having occupied that space with you Sarah for many years and a lot of it comes down to competition as well. We did a broadband survey in June that found that low and behold when there's competition in a market consumers pay on average seven to eight nine $10 less a month for the same service. I'm looking here I think we're kind of running out of questions, which is fine. But yeah, is there anything anybody wants to add before we close out. I guess a general question I might have partisan was kind of it came into the first panel.  What if you'd start disabling some of these things. Is it built into where the car will still drive, even if you're not connected 24 seven to Nissan connect for example, as a Nissan owner. Yeah, that's, that's a really great question. Yeah, I heard that from the first panel as well. So basically, on our label, for example, we have two fields on the label on the second layer of the label. One of them is saying functionality when offline and the second one is functionality with no data processing. And so the idea here is the manufacturer should basically disclose whether if the device is done basically if you turn off, whether you can basically turn off the how smarter devices that the intelligence of the device. And if you do so, whether the device still works and to some to what extent, the device still works. So, so we have that on the label and you really hope manufacturers would disclose that information but this is something that when we talk to consumers of smart devices they actually really like to see that thing on the label basically they were saying that. I think I mean I also heard this from the first panel that the TV is that you're like purchasing on the market right now, many of them are smart. And the question here is, can I turn this off. And if I do so, what functionality do I still get. So it is on the label and we hope that they disclose it but we'll see nothing is, nothing is forced basically. Thank you for this. No, I certainly do not have a knack for business but I want to believe that there's a market out there for dumb devices, because you can get any kind of accessory or like a PS4 PS5 to give you the smart TV connectivity. I just want a big beautiful screen that's dumb. But anyway, that's just me. If anything else I'm going to check the questions or no I think that's about it. So Sarah caution unless you have any closing comments otherwise I'm happy to thank OTI OTI for hosting us today. Certainly, a lot of our work both at OTI CR and with our panel today is educating consumers because a lot of sort of a huh I didn't know that moment so I think it was great listening today and learning and I really appreciate OTI for hosting this. If anyone else anybody else wants to raise your hand with one final thought otherwise I'm going to toss it back to Andy and we'll close it out here. So thank you everybody. My final thoughts just thank you for for having us and for engaging in this in this discussion. Enjoy the first panel as well so thanks again. Andy I'll pass the baton back to you. Thank you so much everyone for joining us today. I think of the events I've done this has had such a wide ranging group of issues and often we find people come to a New America event because they're an expert on one of the things being talked about. And there were so many today that I really hope that attendees may as well learn about stuff they'd never thought about before. If you'd like further information or to follow our work. All of the panelists have their Twitter handles listed on the invitation, as well as you could follow us at OTI or consumer reports at CR advocacy to keep up with sort of the work going on all of these issues as well as how you could get involved and advocate for better labels yourself.