Hi, good morning, everyone. I'm Southern. I'm from Mississippi, so I'm gonna say that again. Good morning, everyone. On behalf of the Millennials Initiative, I want to warmly welcome y'all to New America and today's event, the Millennial Public Policy Symposium. New voices and ideas on care, community, technology, and civic engagement. My name is Melody Frierson, and I have the pleasure of serving as not only the project manager of the Millennial Public Policy Fellowship, but also, as today's, if you need anything, let me help you, Representative. Today's symposium is the result of months of hard work and planning from our fellows and support from colleagues across our organization. To give a tiny bit of background, one year ago, we received nearly 500 applications from young people across the country and around the world, seeking to be a part of the Millennial Public Policy Fellowship program. Our goals for the fellowship was to bring together a truly diverse cohort of young folks to work within New America's renowned policy programs and also explain and also expand understandings of what a think tank looks like and how it operates. Our 10 Millennial Fellows here in this room today arrived at New America in late August 2017 and immediately impacted the organization and their respective programs. We are indebted to our New America colleagues in Better Life Lab, Family-Centered Social Policy, Education Policy Program, Political Reform, Resource Security, Public Interest Technology, the Cyber Security Initiative, and the Open Technology Institute, as well as our friends at the Institute for Policy Studies and the Poor People's Campaign for challenging, mentoring, and lifting up our Millennial Fellows. Before I hand things over to our next member of the welcoming committee, I want to do a quick bit of housekeeping. Today's event will be live-streamed and we welcome folks in this room and those watching from elsewhere to join us in conversation on Twitter with the hashtag Millennial Public Policy. Don't forget the second end and Millennial is tricky. All of today's sessions will take place in this room, our main event space. Restrooms are located outside and towards the front desk. Should you want to connect to Wi-Fi, the information is located within the front page of your program. Speaking of programs, in it you'll find today's schedule, short bios on all of our participants today, descriptions of each Millennial Fellows policy research projects, and excerpts from our blog, The Direct Message. Once we felt that we welcomed y'all sufficiently, we'll move right into our four distinct sessions designed by the Millennial Fellows. New perspectives on communities of care, the promises and perils of technology and big data, policy engagement and political activism, and to round out the day, expanding the table in a generational activism and policy change. In addition to engaging y'all through these sessions, we hope to use our breaks, lunch, and closing reception to connect with each of you and have deeper conversations on the important and timely issues discussed today. This symposium and its fellowship program would not be possible without the generous support of the City Foundation. Thank you for being committed to helping young people build an entrepreneurial mindset, acquire leadership, financial and workplace skills, and begin to engage in a formal economy through a first job. With all of that said, please join me in welcoming City Foundation program officer Julie Hodgson to the stage. Good morning, everyone. It's great to be here with you today. Thank you, Melody, for that warm welcome. I'm Julie Hodgson. I'm a program officer at the City Foundation, and I oversee a lot of our youth programming in the United States. And today's event is exciting for many, many reasons. But for those of us who've kind of worked behind the scenes, I would say it's especially exciting because it's really the culmination of a two-year effort led by New America with support from the City Foundation to invest in future policy leaders and to identify and source a new set of policy ideas. The City Foundation partnered with New America in 2016, I believe, to create the Millennial Public Policy Fellowship Initiative, which complements our Pathways to Progress Initiative. Pathways to Progress, for those of you who don't know, is the City Foundation's film-thropic commitment to support programs that help young people connect to job training programs and career readiness opportunities. It's a hundred million dollar film-thropic commitment connecting 500,000 youth to those types of opportunities by 2020. And so the Millennial Public Policy Fellowship was designed to identify a diverse set of young people who could engage in a healthy, robust public policy discourse as well as to support their professional development as critical thinkers, as policy entrepreneurs, and really just as leaders. And I think kind of what the Foundation found so unique about this is that it was giving a platform to Millennials to identify solutions to some of the country's most challenging issues. And you know, I think it makes a lot of sense, especially for Millennials and the issues that their generation is facing, that we would engage them in helping us come up with the answers since they're so close to these issues.  I was humbled to say the least by their intelligence. They are extremely thoughtful. They're engaged in some really interesting conversations, a lot of which we'll hear today. And I'm just thrilled to be here, thrilled to listen. I think they have a lot to tell us and I think that a lot of the solutions, you know, are within them and really excited to hear the conversation. So with that, I want to thank them because they're really kind of the driving force behind all of this for all of their effort and for embracing the fellowship. I mean, I think they've embraced it beyond our expectations over the last eight months. And then, of course, our partners at New America have been great in helping us bring this vision to life. So thank you for hosting us and for all of your work. And with that, I believe I will turn it over to Tyra, who's going to give another welcome. So thank you. Good morning, everyone. My name is Tyra Mariani, and I am the executive vice president here at New America. As I was looking at the lineup, I did kind of feel like the welcome committee. So for the third time, I will say welcome and particularly our special welcome to those of you that are visiting us from externally. I am really, really excited about the symposium that's happening today. As I was reflecting on the day and meetings that I had yesterday as well as today, I was reminded that New America was started almost 20 years ago by four individuals who were of millennial age. They were roughly young 30-somethings. And it is quite remarkable to both think about where the organization is now. And actually, in our leadership team meeting yesterday, we were starting to just recall some of the accomplishments over the 20 years. And so it feels really fitting that we're having this conversation today, both in the context in which we sit of what's happening in our country and our world, but also in the context of an organization that is about to celebrate 20 years in existence. And so I am personally excited not only because of the millennial cohort that I have engaged with, but also the millennials in general that in the room. For those of you that aren't familiar, New America is actually comprised of majority 20 and 30-somethings. In fact, 46 percent of our almost 200-person organization is made up of people that are 20-something years in age. And as an organization, we deeply believe in new voices, new ideas, and new tools. And I know that today's conversation will be no departure whatsoever from that. And I wondered what would come to fruition, what seeds would be planted, what would be our fruit that we will point back to today's conversation and say, it started here, there was an interesting idea that happened here among the millennials as well as just an intergenerational conversation, which is so, so very critical that it's because it started here that we can say that it bore fruit because of what happened today. So with that in mind, I will close with Reed's closing line and his letter, which is let's get on with the program. Have a great day. Okay. I'm Reed Kramer. I'm the director of the Millennials Initiative here at New America. And I guess I don't have to welcome you, but I can thank you for being here at the symposium. And with the support of many colleagues here at New America, this program's been organized, the symposium's been organized to facilitate a set of dynamic policy conversations. And I'd actually like to thank Melody Frierson, who really is the extremely able manager of this project. And also thank you for your work and our events and communications team who've helped get us to this day. And really a leadership of New America has been extremely supportive and a lot of our colleagues in the policy program. So it's been a pleasure to work with everybody and it really has been a group effort. I'm extremely grateful for Julie and her colleagues at the City Foundation, both for their financial support of this endeavor, this work, but also their real engagement around promoting pathways to progress for a diverse set of opportunity youth in the country, across the country, and also around the world. It's very important work and they've really been leaders in this field and it's been really grateful to engage with them. So the initiative was created to acknowledge that today's youth really are coming of age in a time of uncertainty. And there's a growing disconnect between the experience that they're living and the prevailing public policy that is either providing support or not providing support. And there's a misalignment between the social and economic conditions and it's creating a precariousness that is undermining the potential of an entire generation. So there is an imperative to develop a new set of ideas that can meet the moment that can create new pathways forward and ladders upward for the rising generations. And that was the thinking behind creating the fellowship opportunity. Good policy should be aligned with prevailing attitudes and behaviors and if that's true, then we need the engagement of those impacted to help drive the ideas generation process. And as Melody said, we designed the fellowship to identify this remarkable cohort of individuals who could contribute constructively to policy conversations and then also to support their growth and their professional development. So you'll see them today. They've organized this session. Their work is available on our blog, the DM, which is on our website, direct message for any of the non-millennials in the room. And there's also selected pieces of each of them in the symposium program. They've all been embedded in the different programs across the organization and working together, they've organized the symposium, really with the goals of elevating new voices and perspectives, addressing contemporary policy issues in cross-cutting ways. You know, they're all working on different things so we would come together as a group and bring some really unique perspectives to engage with each other. And then to begin to identify what some potential solutions could be to some of the most consequential issues of our time. And before we dig in further, I did want to consider what some of the limits and the value are in using a generational lens and then offer some kind of foundational data to ground our conversation. So the caveat to begin with is that defining and naming a generation is not a science, it's really more of an art, and we have to figure out how it can be helpful and where it can obfuscate insight. The Pew Research Center has been really helpful in kind of identifying a data-driven approach, so I've relied on a lot of their analysis. They define millennials as those born between 1981 and 1996, so the youngest are 21 and the oldest are 37, and that's obviously a pretty big spread that captures many different aspects of the life course. But it does give us a cohort to work with analytically, and using Pew's definition, we see that millennials are now 22% of the total population, 30% of potential voters, and 38% of working age adults, and they're clearly outpacing boomers more and more every day, and by 2025, they're going to comprise 75% of the labor force. They do share kind of historic and cultural experiences that are distinct from other generations. You can see there were early memories of some confusing social disruption with the Oklahoma City bombing, for instance, in 1995, the Columbine shooting in 1999, political disputes associated with sharpening political polarization with the impeachment of Bill Clinton and his acquittal in 1998. Millennials were between 5 and 20 when 9-11 occurred, and then they grew up during the wars in Iraq and Afghanistan. And certainly, for me, the Great Recession looms large, and particularly its aftermath, which has continued to have an impact. Millennials were between the ages of 12 and 27 during the 2008 election, when the force of the youth vote helped elect the first black president, and then 2016 was a pretty memorable election as well, a surprising result, to say the least. And over their lifetime, I'll also note that a lot of socioeconomic indicators have gotten dramatically better.  But if we look at the world with generalizations, we do miss some of the diverse experiences, and this generation is actually defined by its diversity. It's the most ethnically diverse generation in American history. 44% identify as something other than white, non-Hispanic. Minority shares are increasing across the board for all groups, especially Latinx, who 21% of millennials identify as Latino or Hispanic, and that's up from 9% for boomers. And given this diversity, we really have to be pretty skeptical of claims of universal experiences. Still, these are just demographic realities, and it's going to impact society as a whole. We've got older, whiter boomers that are kind of contrasted with millennials who are more diverse, and there's potentially a generational reckoning that could be unfolding, that's going to play out in policy debates, this cultural gap between the older, whiter society and the more diverse society. And millennials are really serving as this bridge to a much more diverse America. This is going to be the future of the country, and it will remake institutions and will remake the country. And if we acknowledge this diversity, we're also acknowledging that the experience of millennials of color is very distinct and actually quite perilous. Black, Latino, other millennials of color face a number of unique challenges. Unemployment is higher, living in poverty is more prevalent, wealth holdings are dramatically lower than for white counterparts, and violent crime more likely to be experienced, involvement in the criminal justice system. These are some of the issues that we're going to hear about today. But the diversity of the generation really should prompt us to look at where these disparities are needing to be examined more closely, looking for where policies not responding to current conditions. And what are those conditions? I'm going to just provide a couple of notes here of things that we can look at. The educational landscape has changed quite dramatically. There are more degrees and credentials, actually tripling since the 1960s, but we've also seen higher tuition and higher student debt. And our education policy program here in New America tracks a lot of these trends quite ably. They're producing a lot of great work. They document how students have taken on a lot more debt as tuition has gone up than their parents did, actually 300% more debt than their parents. And unfortunately, it doesn't always lead to a degree, which can be also pretty debilitating. So this debt and finances generally are changing behavior over the life course. We see marriage changing, millennials are less likely to marry. When they do, they marry older. Having children is less prevalent. The birth rate, not a lot of attention to this in our discourse, but it's actually reached a record low five years in a row in 2016. Lower birth rate. Since the recession, young adults are half as likely to own homes as young adults were in 1975. So previous generations, homeownerships, much less. And rents are going up at the same time. So more than in the past, let's see 15 years, the number of people that are spending more than half of their income on rent has gone up 50%. And this is really through no fault of their own. This is the economy that they've inherited. They've come of age in a stagnant wage economy. They're earning 20% less than boomers did at the same stage of life. And let's see. This is the net worth slide. Without the home equity to bolster the balance sheet, younger Americans are significantly behind older generations in terms of wealth accumulation. The median net worth is today 30% below its peak in 2007. But younger families, families headed by someone under the age 35, net worth is 41% lower today than it was in 1995.  So there's actually a millennial wealth gap that's emerging. And it should be quite alarming, given how wealth is so important for many aspects of stability. And when we combine the millennial wealth gap with the racial wealth gap, which is historic and it endures and it's growing, I think this is a terrible combination. Throughout US history, we've seen how every means of wealth accumulation, whether it's access to higher ed, home ownership, access to credit, it's been systematically denied to minorities. And therefore, we have these kind of net worth disparities that are up there 10 times wealth holdings of white households versus African-American households in recent figures. And so these are the new economic realities of the country. It's certainly complicating how young people are making decisions about their life, how they're assembling the building blocks of success and making the journey up the economic ladder much more difficult. And financial security has become the aspiration rather than the traditional features of the American dream that we talk about as a society, whether that's family, buying a home, moving into a community, settling down. So that's the state of play right now. And is this a problem? Is this a chance to respond? I think it's both. And increasingly, millennials are going to lead ships in public opinion that are kind of creating opportunities for large-scale policy change. We can look at this in a lot of different ways. As a cohort, they're skeptical of political parties. They want to be independent, but also, they lean liberal. They're open to big ideas. This is the spread of those that are voting. So this is, as a group, they're open to policy solution and big ideas. I think that's the point. The meta point I want to make to kind of close here is that all of this matters in a democracy, what people think about current affairs and the issues of the day. And the value, there's a value in elevating kind of new generational voices and perspectives in a search for policy solutions. So clearly, we've got some challenges in the political moment today. There's an undermining of norms that I think is making the system of governance much more challenging. It's adding some significant stress points. There's really value in thinking about the long game about what policies could work at scale over time. And I think we need to be vigilant in looking out for these solutions that can work at scale. We want to observe the world, ask questions, search for effective and durable policy solutions. So that's the task today. That's the spirit that we're convening this symposium in. We're going to stage a series of conversations that are going to really try to elevate some of these consequential issues of the day. And I'll just review some of the themes quickly. These are the featured sessions where we're going to ask some of these questions. We've got changes in the economy, evolving gender roles, changing expectations of families and employers. So what does this mean for how we care for each other? What does this mean for how we support care and educate our diverse populations? What's our collective responsibilities for care? So that the first panel is going to dig into that. Second, we're going to navigate looking at how big data and technological innovation has changed the landscape of how we live. And there's promise in peril ahead, which we're going to explore. How do we bounce these rights and social protections? And how do we implement technological solutions that can be transformative? And then third, there's political engagement that's required to make policy change. What's the civic space like? How do we govern it? Where policy and politics meet? What's the role of movement building and advocacy? How do we expand the table to get more voices in a meaningful exchange looking for effective policy change? So those are the questions today. Thanks for being here. Stick with us. And we are going to get on with the program. Thank you. So let's have Maika come up and bring on the next session. Thank you. Welcome to the first of several enlightening discussions we have planned for you today. My name is Maika Samson. And for the last eight months or so, I've had the privilege of introducing myself as the Millennial Public Policy Fellow in the Family-Centered Social Policy Program here at New America. This panel, New Perspectives on Communities of Care, is split into two roughly 30-minute sessions with a Q&A at the very end. In the first half of the panel, I'm joined by Elisa Dorana, the Senior Policy Analyst in the Better Life Lab, and Rodlyn Miller, the Millennial Public Policy Fellow in the Better Life Lab. In the second panel, Jenny, the Millennial Fellow from the Education Program, will lead a conversation with her colleagues Ernest Oswego and Abigail Swisher, members of the pre-K through 12th program and the higher ed teams respectively. So one of our goals for these conversations is to move past the wise, the justifications for care policy and to really get into the weeds of policy philosophy and in design. We hope that by now we're in agreement that people need paid family leave, that the patchwork of American public assistance programs aren't doing enough to meet the needs and aspirations of their constituents and that of course American public and private education systems are failing to foster equity. It's easy to whip up a populist frenzy over what's obviously wrong, but it's much harder to focus our attention to the myriad of historical, social, and technical considerations that must be taken into account if we were to ever get things right. Elisa and Rodlyn's work in the Better Life Lab exemplifies this commitment in depth to research in the field. They research and write about barriers to social and income equality, especially at the intersections of work, gender, and social policy. Before joining New America, Roslyn studied anthropology at Stanford University. Her studies focused on urban structures and their relationships to the public and private sectors. And before joining New America, Elisa's work spans the Department of Housing and Urban Development's promise zones initiatives. She worked in social services in the Washington metro area, and she did full bright research on social policy integration in the EU. Elisa, you recently offered an article that appeared in Slate on the Republican plan for paid family leave, and I think it really highlights some of the issues we'd like to talk through today. Specifically the messy things that happen when we come to a consensus on a good thing like paid family leave, but we have wildly different ideas about how to make that happen. And so to kick off our conversation, I wanted to ask the both of you if you could talk through some of the major points of this paid family leave policy proposal and some of the philosophical issues it raises in American approaches to care policy. Sure. So maybe to start, I don't know how familiar people are with paid family leave in the room, but there are currently two general proposals on the table. The family act, which was introduced by Senator Gillibrand and Representative Delaro, would create an insurance scheme so that people would pay into the scheme about a cup of coffee, the cost of a cup of coffee a day, and then when they need to take time off to have a child, to care for a loved one, or undergo cancer treatment, something like that, that they would have the ability to take that time and not lose their source of income. So that has been introduced several times. It's basically a Democrat only bill at this point. There has been interest from the administration, which is now crossed over into Marco Rubio's office to author a Republican plan. And so that plan has come together as sort of an extension of the social security system. So in the article that we authored last week, we were looking sort of at the merits of that bill. Surprisingly, or we're not so surprisingly, it's not particularly well thought out and would exacerbate funding issues within social security itself and also not be inclusive of the needs that most families have. So in the case of that policy, the Republicans would like to fold parental leave into our current social security system. So not even though the majority of people who take time off take time off for their own personal medical care, this would just be limited to new parents. And if new parents were to take time off pulling money out of their social security, they would have to delay retiring. So there are, well, there are multiple concerns, or I have multiple concerns about that, mainly that we already see a lot of discrimination of new parents, particularly women, the wage gap emerges when women are in their 20s and 30s. So that is one issue. And then the second is that the workers that probably need this the most who probably need to retire early because they are in very physically or emotionally demanding fields might not be able to if they take time off to have a child and then can't retire when they're no longer able to work. So yeah. Yeah, I completely agree with all of that. And I just want to add that right now we're at a moment where people are finally putting family values first in the sense that paid family leave has not always had a consensus about its value. Right now in American workplaces we expect people to adjust their lives, their families to the nine to five or to the shift work or to whatever it is that people are working in rather than making workplaces work for people. And that's a huge problem that I think paid family leave could be a cornerstone of fixing. And right now about 14 percent from 2016, that's kind of an old statistic, but about 14 percent of Americans have access to paid family leave, meaning that if you need to take six weeks off to give birth to your kid or to take care of a sick child or even an elder that doesn't have anyone else to take care of them, you have to sacrifice your salary, you have to sacrifice your position at work just to make sure that you can do those caretaking responsibilities. And that's not fair. And when we passed the Family Medical Leave Act that helped a little bit that provided unpaid leave for up to 12 weeks. And even not all Americans have access to that, not all companies are big enough. And yeah, why did we, you put policy forward and say this is important, this is valuable, but then not make sure that every worker has access to it or has the ability to take it.  So not to mention like issues around people who are disabled or like in the process of dying or there are many reasons why people need time to take off, but yeah, so the situation is fairly dire at the moment. I think part of the reason, well I don't want to speculate too much, but part of the reason why this proposed plan pulls from Social Security is over concerns about its cost. Frequently, the argument against developing robust safety net programs is that the cost will be enormous. And could you speak to that sort of scarcity framework and how it's framed other social policies? Sure, I mean Roslyn will also speak to this, but I think that we're in agreement that this is more a question of priorities and paid family leave would act as an insurance scheme, like people would pay money in and then they would have the right to take that money out when they need to take time off. So comparatively, it's, I mean it would function similar to like Social Security or Medicare, other social insurance programs like that. I think it's partially a question of values and whether we think that people in the US should live in dignity.  And at the same time, a lot of the people, we've been subsidizing a lot of the way that we live on low, the work of low income, mostly women and often immigrant women who do like a lot of the child care, elder care, the domestic care work in this country for minimum wage or around there. So yeah. Yeah, I mean in the end we set our budget and what we decide is a priority for our nation is what we're going to have money for. And the policies that specifically paid family leave and then the Family Act, sorry, specifically those policies are created, they're well thought, they have a budget scheme in mind and it's something, at least the Family Act, is something where employees and employers pay into and it's a social insurance fund. So there is money for it if we decide that we care about people's lives and beyond dignity, beyond need meetings, we're allowing people to have freedom to live a life that they choose in a way that they choose beyond just meeting the baseline need, everyone deserves happiness. I think we can clarify what we really mean by our values in the policy making process. I frequently think about David Elwood's original plan for welfare reform in the 90s, which was a lot more compassionate than the current cash assistance programs we have now. And I think a lot of his ideas sort of took on a life of their own and even though he did discuss a lot of values about equity and fairness to folks looking for assistance and trying to get back on their feet, the tagline became and welfare as we know it. Personal responsibility is a goal of ours, that's independence is a goal of ours, it's a shared American value of ours. And those values really coalesce into something quite different than what you would imagine them to be. So my question is how do we really be clear about our values and clear about what we hope to achieve and not let our ideas get lost in the legislative machine? So I think that framing this as a family issue, an issue that people contribute to, it is both a public good and that we want to invest in the health and future of our country and our well-being as it stands right now. It's also something that exists both in the U.S. and worldwide, so this is not some sort of radical idea we're not talking about. I mean there are many other policy issues that are a little bit more far-fetched, but this exists in California, it exists in Rhode Island, it exists in New Jersey and then there's new legislation in Washington state, Washington D.C. and a number of other places. So I think it's partially like how can we reframe this as like a way that Americans can support their own families and that also we contribute to child development, maternal health, which is worsening in the U.S. gender equality and other sorts of things. Frankly we need to start with culture change. We need to make it normal to value caretakers, to value domestic workers and shift workers in all of the people that make other work possible rather than sort of idolizing this one idea of success. Yes, by putting families first we can get on the same page with values, but even if we're on the same page if we do get a good paid family leave program we're not going to see equal utilization rates unless we address gender equality, unless we address wage disparities. Something you mentioned is the unintended consequence of the proposed plan that it would, it might incentivize employers to discriminate against young parents. What frameworks would you employ to avoid some of those unintended consequences? Usually I mean in our work usually what we try and think about is like how would this policy affect historically marginalized communities by gender, by race, at the intersection of those two things, definitely able people, that sort of thing. I mean what we know about the Family Medical Leave Act for example is that it did enable some mostly affluent, mostly white people to take time off when they need it, but only people that can afford to take unpaid leave. So in a way it is exacerbated inequality even though the idea behind it was to families take care of themselves. So in thinking about any sort of future paid leave scheme or other types of social policies, like thinking through what incentives could be included in the policy or like how this will affect different populations. One of the things that we have seen from other countries is that it's really important to try and get men in particular to take leave so often to help so that women are not still disproportionately shouldering the majority of the care work in this country, both in terms of paid leave and unpaid leave as well and unpaid work in their families. So something, making sure that language around paid leave is gender neutral, but that there's also an incentive for the second parent or caregiver in a family to take it and making sure that it's the pay is high enough so that our lowest income workers have the ability to take time off. When California first implemented their paid leave scheme, the amount of money that people would get if they took leave was just 55% of their wage. So if your minimum wage earner and you are only getting 55% of your income when you have a baby, it's still not really affordable to live in San Francisco on 55% of minimum wage. So bumping that up especially for low income workers is really, really important. And California recently did bump that wage replacement up by, I think, to about 70%. To about 70%. And in order to encourage men to take paid family leave in San Francisco, in San Francisco men get 100% wage replacement rates for taking paid family leave and that encouraged utilization rates as well. I think this also brings up the question of the drawbacks and benefits to targeted versus universal programs. On one hand, targeted programs can somehow like single out some of the most marginalized people in our society and really draw negative attention to them. But that's where we need to foster equity, right? So could you speak to some of the perils of that? The targeted approaches? Yeah. Yeah. Well, you can't just capture everybody with a targeted approach. And often you isolate the community that you're hoping to serve. And there is a stigma associated with it as well, especially if you're getting vouchers for things and waiting in line forever. For example, if you're, if you're trying to get food for your kids, and you have to wait in front of a food shelter for an hour early in the morning with people walking by in front of you, there's a certain stigma associated with that for trying to provide for your family. And we see this all the time in education as well with voucher, education voucher programs being offered for community colleges, but not for for your universities. What we're doing is we're saying that people are worth a certain value. And maybe the numbers show that that's where the impact is, but we're not centering the human, we're centering the statistic in that sense. So yeah. And then also universal approaches have a problem with ignoring marginalized communities. So I guess the big question is how do you balance both to make sure that a policy really does address the need that you wanted to rather and the people that you wanted to serve rather than your stereotype or perception of what a population needs. And I think that we can do that by including people in the policy making conversation and hiring the people with lived experience as well as in our research design, actually incorporating qualitative research. Yeah. No, I mean, I think that's great. And I remember hearing Virginia Eubanks recently talk about her new book. And one of the things she mentioned is like policy, if you're implementing a policy or research question and you would be unwilling to participate in that policy itself. So that's probably a good gut check as to like whether whether you're on the right track. But certainly, I think, including the communities you're trying to serve and the research and policy making process, making sure I I think that I generally have more of a preference for universal policies than targeted. That's the approach that the Better Life Lab takes and thinking about P.D., but making sure that we're yes, designing policies or research with communities and for communities that have been distributed. Yeah. And I think a historical approach to policy making is something that's often left out of the traditional policy making process. We tend to speak in euthymisms when we design policies like I was doing it earlier saying marginalized populations rather than talking about I'm getting more specific the way black women specifically were excluded from a lot of major social programs in the United States. And now that's a direct result of, you know, chattel slavery in the United States. That is usually considered sort of irrelevant to our design process when it comes to social policy, but it's completely irrelevant. And if you can speak to the ways that your work often tries to integrate history when proposing policies, I'd love to hear that. Yeah. I mean, with the case of the Republican paid leave plan and one of the issues with designing paid leave through social security is that a lot of the populations that we're talking about are cultural workers, domestic workers, like mostly like people of color, they're not a part of the social security, they were excluded from the Social Security Act, they're excluded from a lot of the labor protections in this country. So as we think through the retirement security, like unemployment, paid leave, whatever the policy is, I think that like looking at sort of, yeah, the construction of that policy, like where did it come from, who was included, who was not, how is that changing? I mean, right now, more people work it sort of in contract hourly work. And so even if they had been included in our safety net before, they're probably not included anymore now, given the changing structure of work itself. So, yeah. I'm curious about how this applies to your work as well with family-centered social policy. Yeah. Family-centered social policy absolutely loves historical approaches to policy building. And I think a lot of that is not just like a sort of historical research, but directly bringing in the people who are going to use these programs to the policymaking conversation. People who are going to benefit from these programs should have a say in how they're designed and how they operate, rather than sort of going to the technocrats and the people with fancy degrees calling out myself here. But including those folks at the table is really critical. Any other last words before we wrap up and go to our second panel? All right. So with that, I think I'm going to welcome the education team up to the stage. Thank you, everyone. Can you hear me? We're going to go ahead and jump right into our second conversation where I hope we continue to grapple with some of these same questions, but bring it into the education space. I've invited with me two of my colleagues from the Education Program at New America. They're working on two distinct projects that I think could serve well as case studies to begin to impact some of these questions around how do you intervene with care? How do you design policies with different communities in mind and ensure equity? So we have Abigail Swisher, who is with our K to 12 program, and her work revolves around college and career readiness. And today she's going to be talking about some of the work she's been doing in the last year on youth apprenticeship. And we also have Ernest Usuego, who is a program associate with our higher ed team, who does a lot of research and writing on a host of different issues. But today it's going to be talking about predictive analytics and higher education. So just to start off, can you guys give sort of a brief overview of what your work is? And we'll start with some of the positive aspects, the promise. If we do this well, who can benefit? And then also jump into some of the challenges and ethical concerns that are coming to the fore as you research this further. Okay, I can start. Yeah, so thank you, Jenny. And thank you all for being here. Like Jenny mentioned, the topic area that I'm currently working on in New America is predictive analytics and its use in higher education. Predictive analytics being the idea that institutions are taking historical data and then using it to make predictions about what really I should say forecast what student behaviors lead to success and learn more about what types of behaviors students make in general. I think the great promise of predictive analytics and in general the conversation about data and higher education and data use is that data used ethically and effectively has the potential to teach us more about student success than we've ever known. It can inform the way that we learn. It can inform the way that we intervene when we know that students are off course. And there are a number of other things, right, that having access to data can teach us. But the key in all of that right is ethical and effective use. I think if done properly, everyone, all students benefit from predictive analytics. I'll say all students, but like students at the institutions that are practicing this because it's still relatively new work. I actually think that first and foremost, students from historically underserved communities can benefit the most from predictive analytics. And I'm sure we'll get into that through this conversation. Thank you, Jenny. Thank you, Ernest. And thank you all for being here today. So as Jenny mentioned, my colleagues and I have embarked on this year-long project around the equity dimensions of youth apprenticeship. And if you aren't familiar with youth apprenticeship or if your idea of youth apprenticeship is still, you know, Benjamin Franklin slaving away in the workshop, youth apprenticeship has come a long way since our traditional conception of it, although it is a many centuries old program design. The idea being that it's a partnership between three institutions, the high school, a post secondary partner, and industry to provide students with paid on-the-job learning along with classroom learning in a post-secondary setting for post-secondary credit. And ideally, at the end of it, it should lead students, we believe, in a high quality program to a family sustaining wage job and a path forward in a specific industry. So functionally, what this looks like is a student either during the school day or after school, going to a work site, getting paid on-the-job mentorship in workplace. And either during the day or on the weekends, evenings, also going to a post-secondary institution to receive college level coursework in their industry area. At the end, they should get an industry-recognized credential or something that really validates the work experience that they've had. And as, you know, this program model has become more popular in the past few years, questions are really arising about who we see youth apprenticeship as for. There are a lot of potential benefits that it can accrue to a student. For instance, getting post-secondary credit early can reduce time and debt to a degree. It provides students a really stable pathway with built-in mentors to get to and through post-secondary into a family sustaining wage career. And it also means that for students who by necessity have to work while they are in high school, they're not working at McDonald's, they're not working at Subway, they're working in a workplace that will not only provide them with a living wage, but also move their careers forward. So that's, I think, a really tremendous potential. And so, you know, as we are thinking about who this is for, very often the discussion has come to this as a solution for students who we know traditionally struggle to get to and through college and into family sustaining wage careers. Specifically, very often this is put forward as a solution for low-income students and students of color. And as much as, you know, there is, I think, a tremendous potential benefit to that, we also have a pretty pernicious history as a nation around tracking and vocational education that has to be a part of the conversation today. And that was really the starting point of this project. It's a year-long sort of listening campaign across the country done with the support of the Annie E. Casey Foundation at the end of which we're hoping to have a better framework for what equitable program design would look like as these programs proliferate. Ken, Ernest, can you say a little bit more about what some of the challenges are for sort of the wrong turns that have been taken with your particular work? Because I think it's quite interesting and would lead us to the conversation. Yeah, certainly. So I think right now the challenge that institutions in particular are grappling with in relation to using predictive analytics is just this idea of practicing ethical and effective use while mitigating some of the effects and some of the kind of built-in negatives, unfortunately, that we know about using data. We live in, unfortunately, a time where we're still grappling with the effects of structural inequity. Earlier, Eliza mentioned the work of Virginia U-Banks. Kathy O'Neill also wrote a great book called Weapons of Math Destruction talking about the idea that currently algorithms while they happen to the benefit of maybe rich and the rich and wider communities, they're being sort of forced onto people from lower income and communities and communities of color. And I think what's key to understand there is we're grappling with this in this time. This idea that the rich get people and the poor get algorithms is just this idea that prior to algorithms, it's not like we were getting fantastic service from people either. This idea of grappling with whether or not we're using algorithms individually versus whether or not we're really thinking through the human ways in which algorithms can affect us, making sure that we're looking at how predictive analytics and data can have pitfalls related around tracking. All of that is key. I think that's the discussion right now that institutions are having. How do we use this data in a way that, and how do we make sure that we're not bringing to life all of the inequities that we know exist because we're using it? And part of this is interventions too, right? What does this data tell us about the way that our students are and how do we practice the most human part of this intervention in a way that really does highlight the needs of our historically underserved communities? I think what's interesting and what I found interesting about both of your work is that there seemed to be this very slippery slope between wanting to do good and having well intentioned policy and then tracking. There's a very thin line between this becoming sort of very paternalistic and reinforcing some of those deficit perspectives or stereotypes that we have certain communities. Abigail, you wrote an interesting piece about vocational schooling and how that might haunt this new sort of resurgence of apprenticeship. Can you speak a little bit more about that? Yeah, so I think your points will take in especially, you know, the anecdote that comes to mind was a time that we were meeting with the State Youth Apprenticeship Council in this past year and the college and career readiness person from the State Department of Education was talking about what she sees as the benefit of this program. And specifically to your point about paternalism, I'm thinking about what she said about who she sees program as forced. She told me this program is for the students who think they're going to college, but they're not. They don't have savings, they don't have the support, and we know that they're going to drop out within a semester, so we want them to go into this program instead. And that strikes me as the height of paternalizing however well intentioned it may be. I think very often when we were thinking about these programs that help guide students towards careers, we failed to think about their aspirations and that they have every right to the full amount of choice that students with the most privileged get in determining their career pathways and that as much as we have to balance the need to be practical and talk to students frankly about the challenges that they are going to face no matter what they choose for their career, I think that there's a fine line to be walked between that and shunting students onto the path because we believe certain things based on data about their opportunities. This is something we often don't get to in these conversations, but how do you avoid that? How do we fix that? Have you all seen in your work people that are designing or implementing some of these programs well that have been able to avoid some of those unintended consequences? What do those conversations look like? Yeah well so I mean that's sort of the million dollar question for this particular initiative and what we're trying to you know hear these programs are very new. They haven't undergone rigorous evaluation that tells us like demographically how they're serving students yet so it's an open question but I think like the first thing that people need to do when they're designing a program like this is to take care in what data they're collecting, how quickly they're reporting it before programs are scaled and also like actually know and be able you know to tell others who they believe their programs are for and that to me is the first step in sort of figuring out what are tremendously complex and multifaceted equity concerns. I think you know so far I've really only spoken about equity concerns as they pertain to long-income youth and youth of color but there are so many other like aspects to this issue related to you know English language learners to women who are disproportionately barred from certain apprenticeship fields to students with disabilities. It's it's complex so the short answer is we don't know yet. I think that's also the million dollar question with institutions grappling on how to use predictive analytics. It's just this idea that colleges are beginning to realize that you can have all the data in the world, you can have all the tools in the world, you can have the most complex algorithms but the way that we interpret that data as humans and the way that we intervene like the way that data into forms of interventions is critical to this work as well. I think an excellent example and probably the most talked about example in the field right now is Georgia State. They'll tell you that they worked with the Education Advisory Board to create this set of tools and algorithms that helped really pinpoint places where students were struggling some and help and help them discover right areas to intervene early and really touch on students but they will also tell you that before prior to having this whole set of predictive analytics their ratio of students to guidance counselors at the school was 700 to 1 and after implementation they took deliberate steps to also to also really step up the way that they were addressing like human needs with with humans you know decrease their ratio of to 300 to 1 which is still large but but it's much better right and really begin to target interventions really begin to use this data in a way that interfered sorry that informed their interventions and help them focus on like what to talk with students about leading from a wellness mindset instead of from a deficit mindset starting off to simple things like starting off emails as kind of nudges to students saying like we're proud of you and we understand that this is hard work here are the ways like here are the resources available and not just like warnings you know what I mean they're like hey you failed this test you're now like 25 20 you're like in this group of people who are likely to drop out of school and so you know I think that's part of that work as well it's happening and I think like schools are are across the board from all like walks of the higher education landscape are discovering that data is critical predictive analytics certainly help interpret that but the way that we as humans take that and then and and interact with our fellow humans is also critical I think that part of being able to have that forethought that was involved in being able to do something like that that was successful involves having and we spoke about this earlier diverse representation around the table when these programs are being designed who has been in designing these programs who has been at the table and what conversations have happened have people starting started using this historical lens in that work yeah I would say I would say that the number of the institutions that we've talked to who are really doing this work well are putting a lot of thought into who their personnel on this work is and who their team is so not just like which vendors outside vendors are you using are they producing the type of tool that you think can serve your student body and are you talking about them or talking with them about the effects of like negatives and historical data on your algorithms but also thinking at an institutional level from the student to the IT and IR institutional reporting departments who's involved in this work on the inside and how are our teams like working together with these institutions to identify potential issues that might arise identify unintended consequences that might arise as well I think it takes like a certain amount of bravery to be able to admit some acknowledge some of these problems at the onset and say these are the things that we know historically these are the problems that we know can arise and start building from there and I don't think that's something that happens very often at the go yeah um so I you know I wouldn't wish to speak for all initiatives across the country primarily um because it's it's a really diverse landscape and these are these are highly localized programs I will say that from what I have seen we have a lot a lot of work left to do to make sure that the like policy and programmatic planning level staff working on these programs represent the students that they are hoping to serve primarily my observation with these folks is that they are mostly mostly white and mostly male and I think that going forward that has like bound to affect how these programs will turn out in addition to thinking about you know the diversity of an inclusiveness of policy and programmatic level you know staffing you also have to think about how you're trading people to confront implicit bias which I think is is going to be one of the trickiest things to do with these programs um you know when we talk to people about it particularly when it comes to the level it's funny you mentioned counselors on the post-secondary level on the secondary level um implicit bias from counselors is one thing that people bring up uh again and again that like students are coming to them and saying like I know that the counselor recommended me for this because she doesn't think I can go to college um or you know on the flip side my teacher told me I shouldn't do this program because I need to go to a four-year college uh to succeed and so at all levels um there's there are issues both with inclusion and staffing um and confronting the implicit biases of the people who are already in the room think right it's probably safe to say in both of our fields and like people are coming to realize um exactly what was mentioned I think in our first the first half of this panel that uh you know if you're not talking to the people who are going to be affected by these things then you're not really doing this work as effectively as you can so they just flagged me and said that we have three minutes so maybe one person can answer this one um I want to go back to this question that arose earlier about you know the tension between universal and targeted approaches um have you seen that tension arise in your particular work um do you think one is worked more effectively than the other have any general thoughts around that I don't think I have a short answer to this question do you want me to go ahead yeah um so I think that in education in general um you know what we know like about universal versus targeted approaches is that we need targeted approaches for students um who are traditionally underserved in our school systems because they are getting less not more right now um than the average student across the board so we do need those targeted approaches and we also know that when we have universal approaches to interventions in education whatever they may be then in general the people with the most social capital the parents with the most social capital are the ones to immediately snap up those opportunities for their children um and I think about that a lot in the context of youth apprenticeship as it begins to funnel students towards high wage high growth family sustaining wage jobs um that have sort of a greater level of prestige than say you know plumbing has traditionally had um that a universal approach to these types of programs will mean that they're only utilized by the most privileged yeah I agree did you want to wrap up cool I can try to take a minute I go I completely agree uh Abigail yeah like one of the most frustrating things about uh policy education policy in general is I think that there is like this pointing towards um an approach that's all inclusive approach but but Abigail you're 100 percent right uh at the end of the day when we do that the people with the most social capital benefit the most um in predictive analytics it's it's of particular uh importance to focus on like structural and local interventions because different communities respond to different things different ways um and if you're creating once one uh once that fits all uh interventions for entire college communities where students are coming from different walks of life and backgrounds uh you're guaranteed to you know I mean rub rub people the wrong way and um having those adverse effects on those unintended consequences I think it also speaks to what my I can mention earlier about the scarcity framework and that if we really instill equity and that sometimes means that we're funneling resources extra resources to particular groups of people because they need it and I think that people are afraid that that means that there is less resources for them and so that's sort of a question we also have to grapple with right that there is not a finite amount of resources that we really have to focus these where they need to be focused and ensure that people don't get forgotten in the design of something um I want to thank you both for being here this was a short conversation but I hope one that sparks more conversations throughout the day throughout our breaks and in our reception um so I just want to thank you all for being here and listening to us and thank you both for coming on board and talking about the things that you're working on so thank you um really quick we're going to start a question and answer uh session it's going to be 10 minutes um we just want to let you know to be mindful of um waiting until you get a mic but we'll go ahead and start right now she can get a mic um oh sorry we wanted to get Elise up here sorry gotta gotta gotta and also in Mayaka Mayaka is up here and Roslyn is around also if you have any questions for her so so how I'm trying to find out how you divide determine the dividing line between the the millennials and the successive generation you know use 96 or 94 how how is that a year maybe able to determine to be the boundary line between the millennials and or you know I'm trying to think where how they determine like 81 to 96 or 81 to 94 how those how those are the boundaries between the millennials and the other generations I'll speak as the as a millennial I think that I don't know how they defined it I think they just really drew a line in the sand and I think that's why the conversation we've had is how much can we really focus on these these generational groups of people how much can these labels really define us um so question to that is I mean we don't know this question for Abigail um what role do you think for-profit institutions play in the apprenticeship model that a lot of young people are going into especially in areas where you see advertisements for you know these for-profit institutions that oftentimes pull resources out of people that are already cast-strapped and um you know does that impact apprenticeship models or should they be non-profit post-secondary institutions versus for-profit institutions I think that I could safely speak for my program when I say that ideally it should not be a for-profit institution providing the post-secondary I think that that to us wouldn't wouldn't be a high quality model but you know to speak more to that it was funny that you that you bring up for-profits in the context of this conversation because we we had a convening around youth apprenticeship here in this space last week and it's something that a participant brought up to us that youth apprenticeship actually has great potential for young people who are struggling to find their way into post-secondary institutions to avoid getting sort of um you know into the clutches of for-profit institutions the idea being that we make it easier for young people to um you know to access these programs and stay on um sort of a safely guardrailed path um through post-secondary so that when they leave high school they're not floundering and um and ending up you know sort of prey to those institutions which I thought was interesting and something I hadn't considered before. Any other questions? My question is for anyone from the first panel on the family leave I wonder if you think the change in landscape of work in terms of the shared economy has what impact that has on policy? A little bit of hard time hearing you could you repeat that just a little bit louder? In terms of the change in landscape of work and the rising of the shared economy like Uber and Airbnb what kind of impact that has on policies relating to families and equity? Yeah that's something that we're actually very concerned about um and interested in because a lot of uh a lot of our assumptions about paid family leave and other labor policies are um well we assume that people are going to like a nine to five job and that they have like healthcare through their employer but what we know is that more and more workers particularly millennial workers um they are working more hours unpredictable hours they probably don't have benefits um and this is true not just for people who like or like they work for Uber or Lyft but for other others as well so we're sort of trying to explore what the difference would be in terms of like a universal insurance model versus portable benefits like how do we capture the the most workers and also like our most vulnerable workers to make sure that they have um access to those policies it's not like a an exact science because a at this point we're still like in the process of developing those proposals um partially because this is a phenomenon that is perhaps more prevalent in the US than it is in other countries that um already have paid leave policies in place or already have more uh additional labor regulations around um health insurance paid leave and other policies so ideally um I think in an ideal world whatever the the paid leave policy that we implement federally which is probably not going to happen in the near term but ideally that that that policy would capture uh both like different types of worker years working different hours so it's not like underpaid family and medical leave right now you have to in order to qualify you have to work for an employer that has more than 50 employees you have to have been at that employer for over a year um so already like almost half the the labor force it does not qualify even just for unpaid leave so as we think about policies in the future we want to make sure that we're including part-time workers hourly workers contract workers um all the workers like agricultural domestic workers that aren't included in our current labor systems so yeah I have a question for aliza I was wondering how uh in your thoughts about and BLL's work on paid family leave you're thinking about families that are not necessarily traditional and that they don't involve a birth but could be adoption or foster parents or older siblings taking care of younger siblings anything like that yeah so we uh our approach to paid family leave like we think that it's really important to be as inclusive as possible and that and this is part of uh our criticism of the republican plan is that um families take lots of different forms so I it is important to recognize the the diversity in that structure and so making sure that um we like to talk about the concept of family as something of love blood or choice it you know it might be that the person who's caring for your child is your neighbor it might be a grandparent it might be you know a same-sex partner it like we want to make sure that um we're enabling people to care for those those structures so um we've worked with organizations like uh a better balance to sort of talk through um what that language should look like so that we include adopted parents foster parents um lgbt the lgbt community or even um let's say this is something that we've thought about a lot in in response to sort of police shootings for example that like often the people who end up caring for the children of people who have died may not be biologically related to those children they may not be qualified they might because of that under law not be qualified to certain benefits even though they have assumed um lovingly resumed responsibility for um these families that have you know uh been targeted by the police so that that is something yeah we're very much interested in and further developing yeah so my question is for Abigail and Ernest um i'm curious about what correlations can be made if any with the new youth apprentice movement and linking it in some way to school voucher program the school voucher program and i know years ago it was meant to really support those in the lower income strata and it's not really becoming a program that's really supporting that demographic anymore it's really been shift like the the focus has shifted and others have really kind of figured out how to make it work to their benefit who weren't necessarily of that demographic that was it it was intended to support and i'm curious with the youth apprentice model taking hold and also just this movement for many students uh millenials wanting to go to um more trade school uh and even um you know like non-traditional you know for you know for your program degree programs like where you think there's a correlation if at all um and if it's an issue that needs to be thought about now as a possible potential problem later and if so like what what can be done now to ensure that that doesn't happen yeah um i think that's a that's a fantastic question um and it's an interesting parallel one that i hadn't um i hadn't considered before but i think you know there's a real danger in that as these programs um become seen as as you know i mentioned earlier sort of more highly desirable um and um you know the type of education that people want to move into roles that they're interested in um and and how we how we guard against that i think is really an open question and i think jenny's framework of sort of targeted versus universal programs is is a one way to think about it but it's it's a really thorny problem and one with it we're you know we're excited to continue to work on regarding the earlier panel we we heard that a woman who's homeless has a family and has to wait on a long line or use a lot of time to get food for a family uh is there anything like amazon home delivery uh an equivalency that could be ride by the local jurisdiction uh that might make sense in a case like that there can't be too many cases like that but something and i wondered does this vary from jurisdiction to jurisdiction across the country it must i would think do you hear that clearly yeah so um your your question is uh related to you like how we provide food benefits basically and whether we can improve that or okay um i also don't know if rosalind would like to respond to that if she's in the audience but uh yes you're right yeah so i believe that was brought up in the context of stigma in accessing programs so uh that would be cool an amazon delivery system that brought food to people but you would have to make sure that you could get that to rural communities you you'd have to make sure that you there's so many different aspects to that and i guess in the context of this panel which is about care and education and creating those inclusive policies yes uh there are technological answers but uh we just need to make sure that we are considering the needs of the population when you're creating those technological answers and avoiding um creating extra stigma or consequences that you know we are unexpected but we we can check in on those by talking to the people that we are serving yeah i think i want to add because i think that is something that had been proposed earlier this year somebody said about um giving out meals like blue apron to communities i think i can't remember who's yeah um so i think you run the risk of again having a paternalistic strategy right like to dictate what the people what food people are going to get and then you you have to dictate the quantity of food and i think um that's a slippery slope and i don't i don't think that um that's necessary i think also like my concern is like why don't like why do we allow so many people to like endure like the humiliation of being economically insecure and impoverished which is not going to be solved by delivering food to their house like they probably need money for other things like medical care they probably need money you know to pay rent um so i think that you know thinking about like the whole picture of like a household budget their emotional uh and other needs like that is really really important and i think that um to respond i think to previous comments about targeted versus universal approaches a lot of what we think about is whether there's like a yes and approach so i'm thinking about paid leave we want it to apply to everyone we want it to um be gender neutral specifically since women have historically done more care work but making sure for example that like the the wage reimbursement rate is higher for lower income communities or that people who haven't done care work are encouraged to do so so like making sure that we that there's like additional supports for certain communities baked into the policy but that most people will have access is sort of the way that we we like to think about uh social policies yeah who has a mic so for the education folks it strikes me that a lot of what you are talking about or the the barriers to some of the high quality apprenticeship implementation comes down to mindset and low expectations and until replacing or raising the expectations of a workforce that has high expectations for students and young adults is a big task we've been undertaking for many many years now have you seen any um policy design or programs that have been able to make the shift to think about high quality apprenticeships and who needs the most sort of aside from getting in those adults who can think about it in that way of of the design that you talked about Abigail in the beginning have you seen any policy design that sort of helps to circumvent the adults with low expectation that still produces high quality outcomes for students um at the short answer being or are you studying it as part of this work yeah so that the latter we are we're studying it as part of this work um hoping to you know sort of over the course of years be able to lift up early success um I think like beyond you know as you speak to the mindsets of adults beyond that there also has to be some accountability to make sure that those mindsets um aren't infusing program design and so in these early stages what we're really looking for and and what we hope that people will do is collect and publish um very clear demographic data about who they are serving and how well it is serving those students after they complete their apprenticeship so that's what we're really hoping to see in these early stages thanks I have a question about the paid family leave um you've referenced other countries kind of throughout your commentary and I would just be curious to know what countries you look to to kind of emulate their policies and kind of what is it about their policies that that you think is particularly strong um and that could maybe be adapted here in the United States yeah so we um I think that there are a couple years ago actually we had an event called uh paid leave leap progging and sort of trying to understand like what mistakes we can sort of see in other countries and what are things that we can learn um we did an evaluation last summer looking sort of the landscape of like how much how long should paid leave ideally be in order to maximize um infant well-being and attachment uh maternal health gender equality and then business outcomes and our conclusion and this has been been reinforced by like public health officials out at UCLA is that uh we probably need uh at least six months um for both parents of whatever gender they are uh to enable all of those things especially for I mean for early attachment uh there's also interesting data looking at the fact that if leave is too short women won't take it if leave is too long it becomes very difficult for them to re-enter the labor market so there are countries that provide like three years of paid leave but then they have a lot of issues with women's labor force participation um and then it's interesting uh Iceland in particular we like always idealize Scandinavia but um like Denmark for example actually has an issue with like women's labor force participation because a lot of women will as the default just take paid family leave and the men will continue to work and that's when the wage gap grows and inequality around who's taking care of children or the elderly or you know the disabled grow um so what Iceland did was to create sort of like their their paid leave is gender neutral but they have what they call like a daddy quota or a bonus so if both partners take paid leave um then they will get additional time which helps them sort of pay for infant care so it's and it's really in a very short period of time bumped up the number of men taking leave and what they've also seen is that when men take leave um in the first like year of an infant's life it helps to equalize the amount of unpaid care work they do in the household either of the child or housework and that sort of thing so it's really uh in a very short time period like altered people's attitudes and perceptions of of care work just in that context so even within you know homogeneous Scandinavia there's like a lot of diversity in terms of who's taking leave who's providing care um and how that affects the society writ large so yeah looks like we're out of time everyone so I want to thank you all for your very thoughtful questions um it looks like we're going to take a short break um there seems to be some refreshments outside and then we will be back in 15 minutes thank you uh a seat nearby uh right oh my gosh it's 1145 exactly so we're right on schedule yeah let's give it a little settle down all right thank you thank you okay thanks hello good morning everyone oh good morning everyone my name is Braxton Bridgers and I'm a millennial public policy fellow working with the research security program here at new america on behalf of my cohort and program thank you for attending our spring symposium a quick bit of housekeeping before we begin directly after this session we will have lunch provided I'm really excited about that I don't know about you guys um so please feel free to grab some food and mix and mingle with us millennial fellows and new america staff my cohort and I are really excited to host discussions around some of the most pressing issues that our generation and future generations are expected to face one such issue is identifying the role of big data and technology in the policy making process whether it be in building government service platforms with the user center design or analyzing trends in user behavior on social media platforms to counter violent and extremist rhetoric with big data and technology have the ability to spark the creation of innovative and effective policies during this session our expert panelists will explore ways in which data and technology can be used to strengthen policy as well as examine the challenges associated with data and technology driven policy making we'll have conversations in three rounds our first discussion features millennial fellow and macomb and conversation with public interest technology and open technology institute fellow dupain goch emma and dupain will discuss the utility and having technologists at the policy making table drawing from dupain's experience as a technologist in the public policy space next millennial fellows bandy sing and new america international security fellow ivana who will discuss the challenges encountering violent extremism online and last but not least millennial fellow dylan rosine will host a conversation with new america cyber security fellow robert lord and public interest technology fellows sonia socar around the benefits and risk of health care data with a focus on patient privacy and security but before we dive into these important conversations cilia munoz vice president of public interest technology and public interest at new america will provide remarks regarding the role of data and technology and policy before joining new america cilia served on president obama's senior staff first as director of intergovernmental affairs for three years followed by five years as director of the domestic policy council prior to her work in government she served for 20 years at the national council of floraza now unidos us the nation's largest hispanic policy and advocacy organization where she was senior vice president for the office of research advocacy and legislation she received a macarthur fellowship in 2000 for her work on immigration and civil rights and serves on the boards of the open society and kresky foundations as well as the nonprofit united to protect democracy please join me in welcoming to the stage cilia munoz thank you very much brexton uh still morning right good morning everybody um i'm really excited to do this i'm really excited to be here in part because uh i'm pretty thrilled about the millennial fellows program here at new america emma colman who you will hear from in a moment is has been working in particular with the public interest tech team but as i have gotten to know the fellows and the work that they're doing across our components it's just a quite an extraordinary group of people um and more importantly an inspiring group of people and and i just am one of many across the institution that have been really inspired by their presence here and the work that we get to do together so thank you for that um and thank you all of you for being here so as you heard i'm a civil rights person i i spent 20 years in the civil rights movement before i went into government and um it's the combination of those experiences which brought me to doing public interest technology at new america and i'm just going to outline that for a minute to sort of set up the panels which are going to come um but that context is really relevant because i didn't i don't start out thinking of myself as a technologist i've discovered as i have learned about how this works and i learned about it from the experience of being in government at the domestic policy council i sat at the intersection between the sort of policy nerds which is what i am the policy team is working on a variety of issues solving a variety of different problems and this thing that we created the u.s digital service where we brought in hot shot technologists from the silicon valley we recruited them to do two-year tours of duty sort of peace course style to help us in the work of solving public problems and i got to help place these tech teams which seem to have magical powers because they knew stuff that i didn't feel like i knew um with policy nerds like me who like i didn't really get why we needed them to sit down with a product developer and engineer and when you got these different kinds of problem solvers sitting at a table together try to solve problems it was not just magical to sort of watch it happen but it was transformative and the insight that i picked up from that experience coming from the world that i come from is that the civil rights world that i come from the NGO world that many of us support that we are indeed part of here at new america we need the same capacity we need that's those those same transformative skills to solve the public problems that we try to solve so i came out of that experience feeling like the way in the future that we're going to be protecting voting rights the way that we're going to be solving homelessness the way we're going to be addressing income disparities in the future all of those are going to involve data and all of those are going to involve technology and in most cases we just haven't figured out how that's going to work yet but we must because we count on the institutions of civil society in this country to help us identify when disparities are happening and to help us identify what to do about them right that's what the civil rights world is about that's what it's for we pass statutes that give us the tools to hopefully get in front of discrimination before it happens but certainly to address it after it happens those tools are imperfect tools but they are sacred they're important the way we protect voting rights mostly in this country's through litigation on the voting rights act which was pat 50 years ago which has been weakened by the supreme court recently and it's not it was never a perfect tool it's not going to become a more perfect tool in the future we need additional tools and we live at a time in which technology is transforming everything about the way that we live and work it's trans it's right we're in this moment where we're in conversations about the future of work we are in conversations about the strength and the capacity of the platforms that probably everybody in this room uses to advance our connection to people but also to potentially undermine and corrode some of the institutions of our democracy everything is on the table everything is changing and the question that we're asking here at new america and with partners around the country is how do we make sure we are leveraging those tools to solve our public problems to make sure that they are in the hands of the institutions that we count on to identify when there's a problem and to help us steer towards solutions to those problems so just to give you you know one of the many negative examples that that are circulating we know for example that we can be perpetuating some of our biases some of the things which create inequalities lead to disparities through algorithms right that we may end up creating structures which perpetuate some of the things which have created inequities in this country right we again we count on civil rights institutions we count on the institutions of civil society to help us figure out when that's happening and to help us fix it help us get in front of it before it happens even better still the civil rights institutions largely are not living and working and operating in a tech environment they don't necessarily have those skill sets on their teams they're not necessarily they are thinking about the problems they're not necessarily leveraging the right skill sets to help get in front of them and to help us resolve them that's something that we want to change by building tech capacity into that work but I also want to make sure to give you a positive example because we think we we are starting to go down the road about thinking about this in terms of the potential dangers and those are very real it's important to understand what they are but it is also true that we can be leveraging these same tools to solve public problems so a modest example from from the city of New Orleans which leveraged data they figured out you can use data to figure out when a where and when a house fire is likely to happen in the city of New Orleans or anywhere in this country you can figure that out through zoning through the kinds of buildings that you have by past history of house fires there is data that you can leverage to figure out which parts of the city are more susceptible to have a house fire which is not only costly in terms of property but costs us lives and by using that data they transformed what fire departments usually across the country do which is that they do outreach and get people smoke detectors right so the typical way of doing outreach is to kind of get out there use access to media go out and speak to schools like hopefully try to find the people who need this information and that need smoke detectors and hopefully get them in the hands of the right folks but by using data the fire department was able to target neighborhoods where deadly house fires were more likely to happen and what did they do their outreach strategy was to go knock on those doors and help people install the smoke detectors and in the first quarter after instituting that policy they were able to prevent what could have been a deadly house fire they got the information the smoke detector worked they were able to stop the fire before people died so we can be leveraging data and technology affirmatively to get in front of problems before they happen as well there is as much positive potential as there is negative potential but the thing we have to do is make sure that there are technologists working in these fields working in city governments working for the federal government and working on the teams with NGOs there are lots of good people in the tech world who are who are smart there are problem solvers there are thinkers and they believe they have the solutions to potentially every problem and maybe they do but what I can tell you for sure is that they don't always fully understand the problems that they would be seeking to solve right there's lots of people who think you can build an app that can solve any problem I'm pretty sure that's not how we're going to solve our problems but I have witnessed what happens when you sit technologists down with the doers at the Department of Education to create policies that are transformative and tools that can have the potential to transform the way people access higher education in this country I've seen that happen and I have seen the light in the eyes of technologists who realize that how much meaning there is in public service and how extraordinarily rewarding it is to be tackling some of the big public problems that we face so what we are hoping to create here is a field of public interest technology so that you know somebody who is deciding to how they're going to pursue a career is interested in data is maybe interested in engineering can be thinking about getting that training so that they can go solve homelessness or getting that training so that they can be as addressing disparities in the healthcare system that's the world that we're trying to create here we're very I feel incredibly lucky to have been able to bring that passion to building a team of public interest technologists here at New America and a team that's working to build a field of public interest technology you're going to meet some members of that team including our millennial fellow Emma Coleman in some really fascinating and important discussions about how we leverage these tools what kind of public problems we need to be attacking and how we're going to be building this world together so I couldn't be more excited to have all of you here and on with the conversation thank you well thank you for that introduction Cecilia and thank you Braxton as well hi everyone my name is Emma Coleman and I'm the millennial fellow in the public interest technology program I'm joined today on stage by Dupain Ghosh who is both a public interest technology fellow and an open technology institute fellow as well and he focuses mainly on privacy security and civil rights policy so before Dupain came to New America he worked at both Facebook and at the White House as a technology policy advisor and his most recent work has been digital deceit examining the technologies behind precision propaganda and political disinformation on the internet in which he explored election meddling and disinformation campaigns across social media so thank you for being here Dupain yeah so I'd love to start off our conversation you've worked in the space for a long time why is there an urgency now to incorporate technologists into the policymaking process well I think that I think as Cecilia was just saying we are seeing technology really come to the fore across society when we are trying to read the latest news we turn to our phone when we try to hail a cab we turn to Uber or Lyft when we try to find that the nearest gas station were we're turning to an app again and so it has created so much opportunity economic opportunity across the board and has created the platform that we really turn to every day for every single every single thing that we try to do but I think at the same time as as Cecilia was also saying the use of technology across society has has really raised a lot of a lot of concerns and has raised a lot of really difficult tensions that we need to navigate in the way fold and so I think technologists have to increasingly be at the table both to figure out ways to leverage technology in the in the in the to its highest potential but also to to help remind us that there is a there is a darker side to and there's an underbelly that we we need to address and we need to really try to work around so that many of the tensions that we've seen thus far can can be avoided in the future and I know that you've worked in several capacities in terms of as an advisor on these types of things and now you're doing more research and writing what would be the best way to go about incorporating technologists is it to bring them into elected officials offices is it to teach elected officials how to even understand some of the more complicated uh elements of technological learning that we have now such as machine learning or algorithmic discrimination or anything like that I think it's all of it I think you you raise a couple of really great examples we have uh we have the tech congress initiative which which aims to actually place smart tech technologists into offices on the hill and we have several colleagues of ours that that are doing exactly that in leading offices on the hill that is important because we we saw just the other day a set of hearings that um you know I think I think members try their best to to understand all the issues at at hand but of course sometimes there are gaps and they're not technologists all the time in fact very few are our technologists it it helps to have a staffer who who is a technologist and who can raise these important issues in the right way and and can understand these issues and navigate around them so I think I think that the goal is really to bring that expertise closer to the decision making table now everybody has constraints of course members on the hill have constraints they have they have a they have a set budget and they can't hire everyone in the world and of course I'm sure they would love to have a civil rights expert and a technology expert and a health policy expert and and many of those are roles that are that are filled in in many offices but everybody has constraints so I think we just need to figure out ways to bring technology as close to the decision making process as possible especially given its really central role across business functions and our livelihoods and our our social life and I know now you're working on an anthology around machine learning to sort of educate policymakers could you talk a little bit more about that yeah um this is this is uh really about one of the issues that Cecilia had mentioned um that of algorithmic discrimination and algorithmic bias um and as a as a uh related issue um addressing ethics as artificial intelligence comes to the fore um algorithms are used across society in almost every every app that we use every um decision making process um that that the business world engages in whether it's uh determinations of credit worthiness or um routing a car through traffic most effectively or um making decisions about federal housing and and public policy um these are all decisions that are increasingly driven by algorithms and as artificial intelligence comes to the fore those algorithms are getting smarter and smarter and being trained on more and more hysterical data to drive um basically the corporate sector and the government whoever is housing those algorithms and trying to implement them um to to greater efficiency that is to greater profit um what we need to make sure though is of course that as those algorithms are being developed and implemented we stay grounded and we respect civil rights law we respect um ethical frameworks whatever the whatever the context might be um and and don't make decisions that can lead to enormous disparities uh across society in ways that can really really be damaging and hurtful to many many people um because that has happened in the past it it really needs to stop um and I think I think we need to bring ethics into the development of of algorithms in a in a much bigger way in a much more um concerted way starting from that decision making table um and so this in anthology is is really about trying to raise some of the biggest ideas uh across the country and around the world in this area and give experts a voice as to how artificial intelligence and algorithms raise ethical concerns in different spaces and what can be done about about those those issues and your other recent paper digital deceit uh which looks at the tools that are used to spread disinformation uh you suggest the the resulting ad tech policy should center on three main areas which is election law privacy regulations and consumer protection law when you have these global platforms like facebook and you're you're asking for these sorts of changes how do policy changes in one country affect such a broad problem does it need to be that technologists in across the world or in multiple countries are coming together to work on these policies yeah absolutely you're spot on in that analysis um these are global platforms they raise global societal tensions and so we need global collaboration to to address those problems um I think I think in this case that paper which I which I collaborated on with with Ben Scott a senior advisor at New America uh is our our goal is really to make one central point which is that the leading digital platforms of the day whether it's facebook or google or or twitter or any of the other leading internet companies um are are really premised on one function which serves which contributes to more than 95 percent of their their revenue uh which is targeted advertising um it's a space that it it's a space that doesn't really have much regulation right now in the in the United States we have um the federal trade commission which is charged to try to police it but they don't they don't have much authority in the space um and there aren't any other federal agencies that that have much authority and so the business community the industry um has almost grown like photosynthesis toward the light of um the practices that that can bring them highest profit margin and that whole business model is really premised on creating extremely compelling services that uh that are borderline addictive um uh collecting as much data as possible about individual people through those services and creating opaque algorithms to to try to target ads at those people and there's because there's no regulation in this space uh they've grown in this direction um in a in a huge way um for example facebook's facebook's revenues 10 years ago were three billion dollars and today they're 43 billion dollars um and uh around that time of a few years ago that is when they introduced all their uh tremendous ad targeting technologies um i think that the solution really lies in understanding that business model uh both here and around the world and trying to address it by listening to the listening to the global community's concerns and that's something that the industry is is trying to do more and more um but it really lies in global collaboration with with europe with japan and korea with india with brazil and argentina as they also consider data protection regimes um there is a big conversation that's starting now in in on capitol hill about a data protection regime here as well and i think moving forward this will really require global collaboration because regulation in the united states does not necessarily mean regulation in india or brazil and these are global platforms they're not just u.s companies anymore and as you mentioned facebook has seen incredible growth over the past 10 years that you know no one would have necessarily expected at the beginning how do we end up creating policies that are agile enough to adjust for how fast new services tend to come on the market i love this question because in the u.s we actually have a very rigid regulatory regime the two agencies that that oversee tech and telecom are the federal communications commission and the trade commission and it there is almost a waterfall effect of um you know depending on who is in in power democrats or republicans the regulatory regime is is either here or it's here and uh when the politics changes it moves back to step one when and then when it changes again it moves back to step two and there's really no certainty there's no consistency and we just really need more agile regulation um that can that can address the harms that and the benefits that uh that the that the sector um germinates and and and creates um i think it it will require more advocacy it'll require more events like the cambridge analytica news frankly it will it will require the public sentiment shifting um to understand that this is how the business model works and this is how it can create tensions across society um and as such we need to rethink the way that we oversee the industry um now that's not to say that that we need to implement strict and stringent regulations against everybody but but we do need to give the government a little bit more agency understanding how the sector works and uh and addressing it and how have you seen that change over time both during your time at facebook and during your time at the lighthouse honestly i haven't seen the regulatory regime change all that much um so i think the biggest example of an attempt at changing it was with the net neutrality uh announcement um briefly net neutrality is the idea of treating every bit that travels over the internet the same so you can't engage in paid prioritization or blocking or throttling of content you need transparency on the internet connection those are some of the elements of president obama's um idea for what net neutrality should look like um and what he tried to do or what he what he actually advocated was that um talcom providers should be reclassified uh under how the federal communications commission considers them providers of an information service versus a more utility like service and he he advocated that the the the federal communication communications chairman at the time uh tom wheeler actually proposed that they do get reclassified that's been that was brought back though that was brought back under the trump app cc pretty quickly um and uh that's that i think was the biggest attempt that we've seen at trying to legislate or regulate this there have been a lot of legislative efforts but they they haven't really moved forward through committee either so i see it as on the hill it's a lot of advocacy uh trying to um bring public sentiment um a little bit further um and in the administration you know i see i see attempts one way or the other to to try to have an impact on on regulation but i think we again we just need to think about the public sentiment and there needs to be better public education on how technology works to get us over that hump of political gridlock and how do you think the most recent uh big news stories around tech policy so net neutrality and then zuckerberg's testimony before congress have impacted public sentiment and then will in turn impact the regulatory side well uh we we just saw i think in the past day facebook's revenue grow by by 63 over the quarter so um you know i think i think public sentiment is is still pretty high in general it's it's almost like a filter bubble as sicilia was alluding to that there is a um there's a conversation that's happening amongst dc tech policy people but but there needs to be a broader sense of how things should change for the sector how the government should should be more agile in responding to it i think that the public sentiment has shifted a little bit but only a little bit in general because we might we might think that it's shifted a lot based on everything that we read in the new york times and everything that's that's being said on in different forums about the sector but i think the average american consumer still uses facebook every day and still will for the foreseeable future and and has a high affinity to all these platforms as a matter of fact i think it was just reported through a wall street journal poll last week that most americans think that we have enough regulation of the sector already um or that we have too much regulation of the sector so um so i think i think it'll require a little bit more um and as we sort of create these policies and move forward how do we create that collaboration between public sector and private sector between the government and between these large companies when so often they're demonized in the news for justifiable reasons but still in a way that we need to have them in this collaborative process yeah i mean they are demonized and news reports we have to remember that that uh you know not not to cast aspersions or anything but we do have to remember that the business model of tech implicates the business model of journalism and news and so there is a there's a hard tension between those two industries right now and so a lot of the reports that we might see might not always tell both sides of the story as as well as they could um many of them do but but many of them don't and so i think uh i think we just need uh a better representation of technologists and everybody in the sector uh sorry in the in the in the um in the universe of this conversation all the stakeholders at play we need them at the table um facebook actually uh facebook google twitter they do have they are global platforms and there are many many um minor and major implications of any public policy change that they might make um and we don't always see it unless unless we are you know working there or talking to to folks who are there it's it's often hard to see how a change here or a change in policy uh in in the company's privacy policy might implicate the global usership in ways that we might not have seen and once we you know hopefully have these evolved policies that better protect uh consumers what does the technologists place in enforcement look like well in enforcement we actually have a couple of good examples one um i'll just mention is the cto at the federal trade commission um we've had a a series of really brilliant people in that role from harvard professors to princeton professors to uh the chief technology uh technology officer um actually i think in this case it was it was not the cto the the title was actually the the principal technologist i might be wrong about that but um some some really amazing people have had that role and that is an important role because the federal trade commission which is headed by a chair men or chairwoman and has four other commissioners who all vote on um on a proceeding or a regulation moving forward or a an enforcement action for example um they are they're typically lawyers or or policy experts but but don't have a broad understanding of or a deeper understanding of technology and of course technology is defining a lot of the agency's priorities now um because of implications around privacy and security and communication and and so the principal technologist at the agency advises the commission and the and the chairperson um and that really helps the agency be incisive and understand industry practices to a much greater degree than they otherwise would would be able to and having that person internally really helps because it it allows them to trust um the recommendations that are coming forward as opposed to having a narrow influence actor telling the agency what it should or should not do um so i think i think that is one good example of how bringing a technologist into the into an agency or into an organization can drive a discussion as to what priorities it should take on well thank you so much for your insight uh and we'll welcome the next panel onto the stage thanks bye hi everyone thank you so much for being here my name is bandy and i'm a money a public policy fellow at new america's open technology institute i'm very excited to be joined here today by avana who avana is a new america fellow on the international security program and she's also the ceo and a partner at omelos omelos is a machine learning and data analytics firm that seeks to automate quantify and standardized approaches to countering violent extremism or cv e avana has done a lot of really awesome work around the world and it's kind of hard to condense at all but um just a couple of amazing points she's um interviewed a number of former extremists around the world from groups like hezbollah isis aky and the taliban and she's also participated in a number of de radicalization programs with neo nazis in scandinavia um so i'm really excited to be joined um with her today and today we're going to be talking about how can we improve the space of cv e um by including data and by using data um and to give you a little bit of background on sort of this conversation through the fellowship i've been writing a paper that looks at what are the challenges involved when it comes to evaluating um cv e approaches that are implemented by technology companies um and i found that a lot of it is related to a lack of metrics a lack of data and a lack of clear definitions in the space and as you know um companies like facebook and twitter and google have come under a lot of pressure um over the last couple of years to take down content and to do it um in a more quick manner and and this has sometimes translated into legislation so like at the beginning of 2018 uh germany instituted the network enforcement act which mandates that companies have to remove content um within 24 hours of it being flagged and they have to remove harmful content which includes extremist content otherwise they will face fines and so there's a lot of growing pressure but there's no real proof that these approaches work um and so this is sort of what we're going to be talking about today so um to kick it off ivana you've worked in the cv e space for a really long time and you've worked at this intersection of cv e data and technology can you talk a little bit about the challenges that you face when it comes to evaluating cv e programs um and how omelos has worked with us yeah so i'll start with uh the challenges i mean the first is it's really hard to measure the lack of something right because cv e comms left a boom right before stuff blows up and so with the pentagon where your mission is lethality you can say um a drone took out five uh bad guys and ten civilians and because of that x1z happened with the network but with cv e you don't really it's really hard to say oh yeah well this person because of the work that we did this person decided not to join isis or al-qaeda or a new nazi group unless you know it's a very um it's timed so well that you have managed to actually disrupt a plot that was happening and then you can count the plot right uh and so i think that's the first challenge and the second is also just it's the mentality in the space is government is not used to measuring anything it's and when they do they look at click like uh clickthrough rates they look at shares and likes i don't actually know what any of those things mean right because we don't know if they lead to any kind of change in behaviors they're just numbers and there's an interesting stat that 70 percent of people who share links on facebook with their friends we're on their news feed they don't actually read the uh the article they just read the title and then you and and that also causes other problems like fake news and all that stuff but that's a different conversation um so the way that we're looking at it at all malas is we said okay how do you actually measure the radicalization level of groups of people we really don't go down to the individual level because people change and people are fundamentally irrational especially when they're emotional and so we look at the group data so we're so what we're saying is right now the way that these things are measured is someone's walking around with a clipboard in Kabul or even within the Somali-American communities in the US and they're asking how do you feel about joining alshabab most people are not going to be like yes i really want to join alshabab right they're gonna they're they're going to tell you what they think you want to hear but that's basically lies and then we start thinking okay online people tend to be more truthful and they're probably a bit more radical online than they are in real life because there's a screen separating them from the online environment but that probably gives us a better data source than walking around with a clipboard or doing it by polling or any of the other options that we have so far so we start to look at okay where are the narratives that they're talking about can we actually start to match the online behaviors of people as compared to the online behaviors of non-violent extremists so these are people who you know we don't care about extremists we care about the people who are going to become violent extremists and so we started to use machine learning and create this really cool algorithm that gives you a score at the end and that's what we used to measure so when you're sending out a counter messaging campaign where when you want to look at the local narratives that people are saying about the US or about ISIS or any of the groups you can just sort of use our dashboard and organically track what they're saying online and when we talk about metrics and definitions in the space this is something that's come up in the conversation quite recently like over the last couple of years but companies and you know organizations that work in CVE haven't really implemented this quite enough so just as a couple of examples this week Facebook put up a blog post that provided an update on their CVE and their takedown work and in it they said that they were still working on defining their metrics and they're defining metrics for programs that they've run for years now that's kind of concerning that they don't actually know how to measure this similarly a couple of months ago some of the major internet platforms testified in DC about their CVE work and the Twitter representative was asked about metrics and he kind of just gave a vague answer and was like we're working on it so it kind of seems like you know they spent a lot of money spent a lot of resources to create these programs but haven't really paid any attention to what actually works so what do you think are the challenges that are associated with establishing metrics in the space like why why are there no metrics right now um it's just people don't care haven't really thought about it or are there actually hurdles to creating them or taking down content taking down content and CVE in general okay um I think for a lot of the tech giants it's not really in their interest they're basically meeting the threshold that the government has put in front of them and just speaking very frankly if you have watched the Facebook hearings with Mark Zuckerberg it's glaringly clear that our policymakers at the highest level don't understand technology they don't understand technology not even talking about AI which got flown around like I don't know what like they they did not understand it and the hearing I actually missed the entire point of why Mark Zuckerberg was called is like that was a national security threat and they made it into something completely different and so I think it's um technical illiteracy on behalf of the policymakers and because of that there the metrics that the metrics that they're giving to these tech companies um are just not sufficient and so it's not that they don't exist they just really suck amen to that and so when we think about um companies and governments operating unproven CVE programs can you talk a little bit about the consequences that this can have on people both online and offline I think the worst consequence is having a backlash uh meaning that the CVE programs typically are done with very good intentions they're implemented by people who really care um and yet there's a backlash effect that it actually further radicalizes the exact group of people that they're trying to help um DARPA did a pre a study on the effectiveness of the redirect method done by jigsaw which was google ideas and what they found was that this program which has been written up by every single newspaper um as this is the way forward and tackling online radicalization they found that there was a backlash effect and that it further radicalized people um who received the content through google but we don't really talk about that either right um and and you sort of see the same thing in places in pakistan and that's why a lot of successful CVE programs are not branded as CVE they're branded under community resilience or agricultural livelihood um or something that's completely different even though you know the actual mission of the program it it's to decrease the number of um push factors definitely and i mean when you talk about content takedowns like a lot of the criticism that you receive is that running an unproven content takedown strategy can actually censor um you know censor legitimate voices and that pushes them to be further marginalized and radical actually can i ask something to that so it's interesting because um we i actually sound the same stage with the uk home secretary who has been super gung-ho about taking down content and i asked a question i said you know taking down content is getting it's the topical way that we're trying to fix a problem but it doesn't fix the underlying problems of why people in the uk are actually joining these programs she didn't like the question very much um but i mean i think that's taking down content also has the unintended consequences that human rights human rights abuses were were were war crimes actually get scrubbed off the internet because from a from an ai perspective they look the same and so you know i have talked to the international criminal core and i talked to a lot of the nonprofits are gathering evidence against isis to try to prosecute them if a war tribunal actually happens and there and that's our biggest thing is you know youtube is taking down all the evidence so what do we actually have now to prosecute anyone yeah definitely um and so a number of these platforms also focus on the big groups so the islamic state al-qaeda but there's a lot of concern because obviously these are not the only extremist groups that exist and um you know some experts have said that by only focusing on these groups and only strategically dedicating your resources to these groups you're letting these smaller groups gain a presence online engage and then they can like in a world where there is no isis or no al-qaeda anymore they would become isis 2.0 and they would become isis 3.0 so can some neil nazis stuff have been taking had has been taken taken down um but yeah they're definitely missing the boat and even with al-qaeda with all this new attention turned on isis for the past couple of years what we have seen on our platform is that al-qaeda propaganda has risen significantly um telegram channels for isis have been taken down but al-qaeda in syria hattash any of their affiliates they're just it's like a wild west for them because they know that they're not going to be taken down yeah definitely um and so when we think about companies reporting data on their cv e um when it comes to content takedowns the real way that they do this is through transparency reports with your blog post updates um when we talk about counter narratives like you mentioned jigsaw's project there's not a lot of data disclosure that's going on in that and that realm how do you think companies can you know improve on that do you think they should issue transparency reports similarly for these kinds of programs or do you think that um you know companies shouldn't be the main sources of data and we should rely on things like omelas instead to um do that kind of evaluation i think the incentives the incentive structure is a little bit miss it's skewed right now it's not they you know you can say i've taken down 10 000 pieces of content or 10 000 accounts um but we don't really know what that means because it could also just mean that um isis immediately created another 15 000 right so what's the actual environment like and i think you do need someone objective who's not getting paid either by the government or by by the tech companies to basically be like a special auditor or like an independent odd odd yeah something like that um and then going back to jigsaw what's interesting to me is that the most sophisticated metric that we have that's widely accepted um within counter narratives until maybe a year or two ago was a click-through rate and jigsaw is it's internal think tank at google but if you're a client for google advertising like xpedia or anyone like that and you walked in to the client with the click-through rate they will literally laugh you out out of the room and so we have a lot of people who come from google advertising who work for my company and they specifically said if you don't have anything more sophisticated than you know the lifetime value over the average customer acquisition cost of a campaign like there's no way you're going to land that account and you will most likely be fired and so you have this very interesting juxtaposition of like the far of expectations and yeah okay great and so i think we're almost out of time so as a last question what do you think the next steps in this space are how can we use data to make cve more meaningful and who should be the real stakeholders who are doing this i actually want to see civil society step up a little bit we do some work with civil societies who who get money from the u.s government because they're more credible voices than the government itself and i really want to see that they get their skin in the game in terms of helping us come up with these metrics and also be a little bit more literate in talking about it because we do a we have to do a lot of education with them first um and then i think i mean in terms of the government the global engagement center as state just got a lot finally they got their money and so they now can actually do something and i want them to instead of leaving the monitoring and evaluation to sort of this last thing that needs to get done is you know when you're actually thinking about and giving giving vendors money to create the content is you actually start to talk about m&e like right there because it's really hard to run an m&e if you don't have a baseline in the beginning and i think that nuance um still needs to be further emphasized great um with that i think that's all the questions we have so thank you very much for all being here good afternoon everybody i think enough time has passed since cecilia is good morning that we can actually say good afternoon safely uh i have the somewhat and sometimes unenviable task of being the moderator of the session right before lunch uh but i am actually very excited to be joined by two colleagues today for a really exciting discussion on healthcare data and technology my name is dylan rosine i'm the millennial public policy fellow in the cyber security initiative here at new america and in my time here i've had the great fortune of working on a number of projects last fall i helped deliver a report to the united nations secretary general and his chief executives board on various normative entry points for the un and shaping norms around various frontier technologies and most recently in the topic of conversation today i've been working on developing a series of policy recommendations around uh healthcare cyber security and so i'm really excited to be joined by two folks today who are going to talk a little bit about that and generally about the promises and perils of healthcare data and technology so i'm joined by sonia sarkar here on my left uh sonia is a public interest technology fellow here at new america um she's also the former chief policy and engagement officer for the baltimore city health department and sonia's work is focused on identifying the role of technology in facilitating connections between various aspects of health sectors particularly through an equity lens and to sonia's left we have robert lord who is one of our cyber security initiative fellows here at new america and i might add has been really helpful and really leading the development of some of these policy recommendations that we're working on for the healthcare cyber security policy report which he may speak to a little bit in addition to that robert is a co-founder and the president of protennis which is an analytics platform that leverages artificial intelligence to detect data breaches in healthcare and he's a leading entrepreneur and thinker in the fields of artificial intelligence cyber security healthcare analytics and data privacy uh so very excited to be engaged in conversation with them really about one key question which we're going to be addressing on this panel today and that key question is how could the proliferation of healthcare data and technology both improve patient health outcomes while also exposing patients to new risks and what can we do about it so just for a brief background for those of you um who are just hearing about some of these some of these ideas today the emergence of new healthcare technologies has really grown and has been unprecedented in the past few years after the high tech act was passed by congress in 2009 we saw the rapid transition away from paper based records to electronic records in the healthcare system just to give you a sense of how fast that transition occurred in 2008 only nine percent of hospitals were using even the most basic electronic health record system but by 2015 that number had soared to 96 percent using certified EHR technologies so a really fast transition in addition we've seen connected medical devices growing at an unprecedented rate including things like infusion pumps and pacemakers which are connecting to each other to your cell phone and making it all very exciting for the possibilities that we can have with some of these new technologies so with that I want to turn to Sonia to talk a little bit more about some of those exciting developments and the possibilities that we can have in leveraging these health technologies really to deliver better patient health outcomes while in the clinic but also before patients even step foot in the clinic in the first place great thank you Dylan and I too am working through hunger so I understand what it's like to be to the panel right before lunch but I you know I really appreciate the conversation and one of the things that I'm very interested in as Dylan was referring to is how do we think about health as beyond just the clinic walls or the hospital walls how do we leverage technology to really address the fact that about 90 percent of what actually impacts our health outcomes doesn't take place through the traditional medical medical care system as we think about it but has to do with where we go to work and how we get around and what we eat and so one of the things that I'm really excited about and I get to play tech cheerleader today which is not a role that I'm often in is the ways in which we can leverage not only the EHR which we've seen you know a large proliferation and how that particular system is really being used you know both inside of the clinic but also to connect to systems outside of the clinic but also other types of technologies that enable us to identify what types of social needs a patient might have so you could imagine in a clinic in East Baltimore where there are significant health disparities being able to keep a patient out of the emergency room because it's been identified that ma