I'm Jamie Marisotis, President of the Lumina Foundation, and I want to welcome all of you to this event. We want to thank our colleagues from the New America Foundation as well as the Washington Monthly for the opportunity to hear a little bit about the latest version of the Washington Monthly's college rankings and some of the issues that have been raised in the rankings and yet again another terrific issue of the annual rankings guide. I want to begin by saying first that we're very proud to be partners with both New America where we have a terrific set of relationships as well as with the Washington Monthly magazine. The Washington Monthly, Paul and his colleagues have just done a great job in recent years in our view in helping to deepen and broaden the coverage of higher education issues in this country and I think everyone in this room would agree that improved coverage is really sorely needed. Lumina Foundation has actually been working with the Monthly for the last few years now in helping to launch the magazine's annual college guide issue through events like this and we're very happy to be a contributor to the Monthly's work. As I think all of you know by now, this is not your typical college guide. In fact, it takes an entirely new approach to identifying high quality institutions and we at Lumina Foundation feel that this approach is a huge improvement over the traditional rankings that we've seen from, let us say, other publications. Unlike those, the Monthly points students and families away from definitions of quality that have to do with things like status or reputation or institutional resources and toward those schools where genuine learning really matters, where efforts are geared towards helping students complete their programs and where productivity enhancements help students get what the magazine's editors I think aptly called the best bang for the buck. I think one line in the introductory essay says it best, our rankings aim to identify institutions that are acting on behalf of the true public interest. That line really resonates with me and with my colleagues at Lumina Foundation because it zeroes in on the very thing that drives our work. Our whole reason for being is rooted in the public interest at Lumina Foundation and our one goal is to improve the quality of public life by increasing high quality post-secondary credential attainment in this country. Many of you in this room know that we've been working for the last several years towards a specific goal. By the year 2025, we want 60% of Americans to hold a high quality post-secondary degree certificate or other credential up from roughly 40% today. This effort literally drives all of our work and not only has it compelled us to look at ways to serve more students so that we can actually get to that 60% goal, but it's also compelled us to help make sure that the system actually serves those students better to ensure that the quality standard is actually met. In other words, that pursuit of what we call goal 2025 has helped us look closely at what we mean by high quality to focus on this idea that genuine learning is what really matters and that's the yardstick that must be used in measuring the value that's added by a course, by a program of study, or by an entire institution. Without a doubt, the monthly college rankings hone in on the quality dimension as reflected in several of the categories that you see of institutions in the guide. I mentioned the best bang for the buck, the large enrollment national universities, the smaller liberal arts colleges, the important community college rankings previously never attempted before in rankings efforts, master's universities, baccalaureate colleges. The guide assesses institutions' performance in several specific areas grouped under these three important broad areas of concern, social mobility, research, and service. In short, we think that the monthly uses this finer, more nuanced approach, a high quality approach if you will, to determining high quality institutions. And that brings me, I think, to my final important point about this work and our reasons at Lumina for helping make it possible. We're proud to support the Washington Monthly because they're just as interested in high quality content as we are. As much as any media organization in business today, and far more than most in my view, the Washington Monthly is helping to change the conversation when it comes to coverage of higher education issues in the media. They're helping to create an appetite for higher education coverage with more depth and more relevance. And frankly, they've been doing it now for several years. They've done substantive pieces recently on competency-based education, on the perils of ballooning student debt, on the problems surrounding graduate medical education, and on the tenure system to name just a few. The issue being launched here today takes a detailed and enlightening look at the nation's community colleges, the good ones in a piece by my colleague Kevin Carey, and some that are not so good, one by Hailey Sweetland Edwards. Right, or Ann Kim talks about the alternative and growing world credentialing, where badges and other forms of educational currency are helping to redefine the system. And my colleague Jamal Abdullah-Lim, a journalist, returns to his alma mater, the University of Wisconsin-Milwaukee, to see why so few students like him actually make it to graduation. All of these pieces are thoughtful and thought-provoking, substantive, and relevant. They represent journalism that matters, because, as I said before, they're deeply rooted in the public interest. Again, this mirrors our interest at Lumina Foundation, and to serve the public interest, we're concerned that we're convinced that we must increase educational attainment among our citizens, particularly that population of 21st century students who represent our future, low-income students, students of color, first-generation students and adults. These are the rapidly growing segments of our population, and they represent a huge opportunity for this country. To make the most of this opportunity and to prosper as a nation, we need to improve higher education outcomes dramatically, and we need to do it soon. And to improve outcomes, what better place than to do so at the outset with the actual college choice process? So this issue of the Washington Monthly gives students and families an important, high-quality tool to help make that choice and to make it wisely. Thanks again to the Monthly and to our colleagues at New America. We're very excited about the work that you've done, and we're anxious for the dialogue to move forward. I'm looking forward to today's discussion. I want to thank all of you for being here, and now it's my privilege to introduce the editor-in-chief of the Washington Monthly, Paul Glastres. Paul? Thank you, Jamie, for that generous and great introduction to the issue, the new issue of the Washington Monthly, and to the issues we're going to be talking about today. We, Jamie and I, go back a ways. As I've mentioned in previous events, he and I were both started out our careers as interns at the Washington Monthly. And so we've been toiling in these vineyards for a good long time. And we really appreciate working with you guys, and with all of Lumina, you guys are always helpful and enlightening on these issues. We have a terrific panel, and I want to just jump right into it. A lot of, we're very honored to have this particular panel. They're, I think, friends of the magazine, fellow travelers in the reform effort that we've been writing about and trying to push forward. And the theme of today's discussion is, we call it the higher education's new caste system. A lot of evidence, up piling up these days, that not only is the nation experiencing historic levels of inequality, which is bothersome enough, but our levels of upward mobility are not what we used to think they were. We think of higher education as the one tool we can all agree on as a way to promote upward mobility. And yet, more and more evidence building up that higher education in some ways has become a driver of inequality. More and more lower income students are being channeled into open access campuses where the spending per pupil is about half what it is at select schools. Meanwhile, upper income white students are becoming more and more overrepresented in the three or 400 select schools where graduation rates are higher, lifetime earnings are higher. And so the very mechanism that we think of as vital to equal opportunity in this country is becoming something else indeed.  And we've been hammering away at trying to figure out why and what we can do about turning it around. That's what this, the issue is all about, and that's what this discussion will be all about. The issue comes out right after the administration put forth, as you all know, some very bold and interesting new proposals to rate colleges based on affordability and student success. Remarkably similar in general to the ratings we've been putting out the last couple of years, the rankings we've been putting out that Jamie mentioned, which we call best bang for the buck. So we'll talk a little bit about that. And so let me just introduce our panel. First to speak is James Quall. He's a deputy director of the White House Domestic Policy Council. And in that position, he took a lead role in developing the administration's new proposals to rate colleges by affordability and student success. This is, I think, his third White House assignment, having worked also in the Clinton administration where he and I worked together for a little while. He has also done two tours of duty at the Department of Education, most recently as deputy undersecretary of education. He's a veteran of several presidential campaigns, including being policy director of the 2012 Obama reelection effort. We are completely delighted to have James. And we are mindful that he's going to have to leave a little early, so we're going to let him go first. Next is Diana Natalicio. I am so sorry, I always get that wrong. We are extremely thrilled to have Diana. Since 1988, she's been the president of the University of Texas at El Paso. And during her long and distinguished tenure, the university has been one of the great success stories in American higher education. UTEP, I think is how we pronounce the acronym, has become a distinguished center of science and engineering with expanded research budget and doctoral programs growing from 1 in 1988 to now 20. Its enrollment has grown from about 15,000 to almost 23,000 in that period. And it's done all this while at the same time expanding the percentage of lower income and Latino students. And not only does the university graduate those students at a better than predicted rate, it has also managed to keep tuition relatively low. So not surprisingly, UTEP always does extraordinarily well in the annual Washington College rankings. I think coming in at number seven this year, way ahead of Princeton and Yale, I might add. Dr. Natasalio chairs the board of the American Council of Education. And I should mention she is also a distinguished linguist who grew up in my hometown of St. Louis and played, was a star baseball player and I'm guessing remains a Cardinal fan? I do. I thought so. Next we have Jamal Abdulalim. Jamal is a Spencer Foundation education journalism fellow and a freelance journalist specializing in higher education. As Jamie mentioned, his reported memoir essay in the current Washington Monthly is really one of the highlights of the issue and I urge you all to read it. Jamal has been a Washington correspondent for diverse issues in higher education and has written for Education Week, U.S. News, and of course the Washington Monthly. He's also a chess journalist and a recent chess champion, I believe. Maybe it was a, you're the number one journalist who plays chess, is that? I wish, just chess journalist of the year. Chess journalist of the year, well. But you played chess well, too, as well as writing about it. And he's a former staff writer for Youth Today in the Milwaukee Journal Sentinel. And batting cleanup today is Kevin Carey. He's the director of the education policy program here at New America and is the annual guest editor of the Washington Monthly College Guide. And I'd say clearly America's most eloquent, bracing and effective writer and researcher on the issue of higher education reform. It's an honor and a pleasure to work with him. Kevin has published articles on education topics in the New York Times, Slate, New Republic, other publications, and he writes a monthly column for the Chronicle of Higher Education. And his work's been featured in the Best American Legal Writing and he's frequently on CNN and C-SPAN and NPR, kind of the go-to guy. And he's currently working on a book on technology in the future of higher education. So thank you all for coming. We're delighted to hear from our panel and we'll start off with James. Thank you, Paul. It's a real pleasure to be here. New America has some of the most compelling and interesting thinkers on higher education, higher education policy. The Washington Monthly, particularly under Paul's leadership, has been a long time promoter of innovative ideas in higher education, how colleges can embrace their public mission more fully. And of course, the Lumina Foundation is pushing some of the most important and interesting ideas in higher education reform today. So I'm just sort of happy to be here at the cross current of three organizations. I respect a lot. I wanted to talk a little bit about the President's speech. Of course, the President gave a big speech on higher education a couple of weeks ago. What motivated him to give that speech and give a brief summary of the contents with a particular focus on the college ratings plan, which has some similarity to some of the ideas that will be talked about today. The President is giving a series of speeches on the future of the middle class.  And that's the struggles of the middle class. And we're an economy today where inequality continues to rise to levels near the highest since the Great Depression. Middle class family incomes have grown very little for the last couple of decades. And the cost of being in the middle class in terms of things like healthcare and housing and of course college have continued to rise. So the President is talking about different aspects of the problem but of course education has to be central to that challenge. It's hard to imagine a strategy to try and strengthen living standards and grow the middle class. It doesn't include a bigger investment in higher education and try and get more people, the education they need, more college graduates. And that includes not just four-year degrees but also associate degrees and certificates that have real value in the workplace. So if college is maybe more important than it ever has been before, it's also more expensive than it has been before. And if you look at the sticker price that is charged for tuition, for example, that's more than tripled over the last three decades while the median family income has gone up by only 16%. We've seen big growth in student debt especially over the last couple of years. It's now close to a trillion dollars depending on which estimate you look at. Recently we've seen more students struggling to repay their loans. We've seen signs that student debt is interfering with home ownership and other life choices for young graduates. So we have a situation where college tuition is continuing to rise. It's on an unsustainable path. It's one that is increasingly burdensome for the middle class and in some cases increasingly risky as students borrow to finance those college tuitions. There's good news also which is that across the country we can see examples of colleges and universities finding ways to both improve the quality of the education that they offer and to do so at a lower price. For example, Georgia Tech is now offering an online master's program in computer science at a small fraction of the cost but not a fraction of the rigor of its traditional on campus program. We've seen some of the researchers at Carnegie Mellon now at Stanford find ways to teach introductory statistics in a way that helps students master the material to the same degree of learning in a quarter less time. We've seen colleges and universities, four-year colleges develop partnerships with community colleges that make it easier to transfer all of your credits and earn a four-year degree at approximately half the cost. That sounds like a no brainer but actually only about 11% of students who begin at community colleges go on to earn a four-year degree. So there's a lot of work to do to make sure that those credits transfer and they transfer well. So the challenge that we have then is not to start from scratch but how do we accelerate this pace of this innovation, figure out what works, encourage more colleges to duplicate what works or find something even better, and how do we help students and their families navigate this higher education landscape in a way that they can find what works for them. So the president's strategy has three parts. The first is moving to pay colleges based upon performance and that includes not only the rating system, which I'll talk about in a second. It also includes a new race at the top for higher education which involves working with states to reinforce the work of leading states like Tennessee, Ohio, and Indiana to pay public colleges not solely based on enrollment but also based upon the results they get with students. It includes some new proposals to reform the Pell Grant program to encourage both students getting Pell Grants and the colleges they attend to make sure they're focused on academic progress and that students are making progress toward their degrees. The second big plank of the president's plan is promoting innovation and competition that includes getting new information out there on what works in higher education, how well colleges and universities are doing. It includes taking a look at whether there are regulatory barriers to innovation and should we do some experimental pilots to test new approaches to delivering higher education and whether there are federal rules that stand in the way. And it includes a modest amount of federal funding to encourage colleges to try new things to document what works and to expand the things that are working. The third part of his strategy is focused on student debt and there the proposal is to help students who have existing student debts manage their debt. He's proposed expanding the income based repayment plan so that every student who has student loans can limit those loan payments to 10% of their income. And that is a way that we can offer a guarantee to students and say, you know, we think college is a good investment and it will, we know that will pay off for the vast majority of students if you're one of the unlucky few, you will be able to afford to repay their loans. No matter what you'll be able to afford to repay your loans. So we think that's a very important message to send to people who are considering college. So I did want to spend a little bit more time on on the rating system. And there the president believes that we can do much more to help students understand which colleges are doing a good job offering a high value. And so he has asked Secretary Duncan to spend the next 18 months or so working to develop a system to rate colleges based upon the value that they offer students. The rating system is something that is going to be designed in collaboration with the higher education community with colleges with students with statistical experts. Hopefully many of the people here today. A couple of principles for what we want to get to. We want to look at questions of access. So who's doing a good job enrolling students from disadvantaged backgrounds, not like some college ranking systems that exist today and reward institutions for rejecting students. We think that's the wrong approach. We want to reward institutions for enrolling students, especially students from disadvantaged backgrounds and helping them succeed. A second important component of this is looking at affordability. What is the tuition they charge after scholarships? What are the student loan debts that students are left with? A third is what kind of outcomes are they achieving? Are they helping students graduate or transfer to another institution? Are they getting jobs after they graduate? Are they going on to graduate school? So these are the types of things that we're interested in looking at, knowing that developing the system is going to take a lot of refinement and a lot of input from the public at large. It is important to us that these ratings recognize in particular institutions that do a good job creating opportunity that enroll students that are hard to serve and help those students succeed. And that can be done both by building that directly into the criteria and considering that in the rankings. It can also be done by comparing institutions to other institutions that have a similar mission. It's also important to us that this isn't a system of punishment, that it's a system that works to help institutions improve. And so we're interested in looking at ways that recognize institutions that are making progress on these measurements. Unlike some of the existing systems, this isn't going to be a ranking. There isn't going to be an attempt to try and distinguish between the 23rd best university and the 24th best university. There's an interest instead in trying to give people a broad sense of which institutions are successfully delivering value and which ones aren't. So the goal is to have these rating systems in place by the time students are making college choices in 2015. Spend a couple of years giving colleges an opportunity to improve continuing to refine these measurements. And ultimately seek legislation that would give the secretary the authority to allocate financial aid partly based upon these ratings. So for example, a student going to a highly rated college could get a larger Pell Grant and that would give students an incentive to consider these ratings when they're choosing among colleges. So that's the plan. It is going to take a great deal of work and it's something that we're going to need a lot of help on. So interested in working with the community and figuring out how to make this work. I would just sort of close by saying that in the last couple of years we've seen a lot of progress in a similar problem, which is the growth of health care costs and we've now seen for the last couple of years at least health care costs growing much more slowly than they have historically. A lot of reasons for that, how permanent that trend is remains to be seen. But there is a strategy there that seems to be working and that strategy is moving from paying hospitals and doctors for the number of services they perform and to moving to pay them for the value they provide or how successfully they keep their patients healthy. And secondly, putting more data out there and there's been a real effort to make sure that patients have access to all the information we have on the type of work or the quality of work that hospitals and doctors do and that hospitals and doctors have access to all the information we have on how they can improve if they're interested in improving. So the strategy that the President has proposed for higher education, obviously the analogy is not perfect, but there are some similarities there in effort to make sure that colleges have both a financial tie to higher performance so that makes investments improving performance worthwhile and that students and colleges both have the information on how well colleges are performing and how they can succeed. So thank you. Thank you, James. And mindful that James has to leave early, I thought I would give you all a chance to, before we go into the rest of the panel, just ask a few questions. I know I have many about the administration's proposals and if I may, I'll actually start off with the questions with one question. And that is, and I think it's one that we'll explore with the rest of the panel, and that is, though I am deeply sympathetic to what you're doing, the hard question would be what confidence do you have that you can put out ratings with data that won't be gained by universities? There's no question that getting these measurements right is going to be hard work. And it's something that we're going to need, you know, the help of both folks here and at the Washington Monthly that have spent a lot of time thinking about it. Of course, there are ratings and experts across the country. I do think it's important to keep in mind that we're not talking about a ranking system where, you know, if you can gin up a few more applications to reject, you're going to leapfrog a couple of institutions. Instead, you're looking at, you know, a broad measure of the value that you're offering your offering students. And so you're not trying to make an artificial distinction among colleges that are offering similar levels of performance. And, you know, I think if we tie these measures to the right thing, then institutions trying to gain their performance will be a good thing. If we're rewarding colleges for enrolling and graduating more students from low income backgrounds, and they manage to do that, then that's a positive step, and we're moving in the right direction. So, you know, I think the premise of your question is that colleges will be paying attention. They'll be trying to improve their ratings, and, you know, that's very much our hope. Fair enough. I'm going to take just a couple of questions from the audience if we have a microphone ready, and then we'll move on. This gentleman right here. And please state your name and affiliation if you would. Thank you. Steve Goodman, Educational Consultant. And this is a follow-up to Paul's question. You mentioned about punishment. And to follow up on Paul's thought, what's wrong with punishing somebody for doing something that's not necessarily correct? So what would be so terrible about punishing some universities who did XYZ or didn't do XYZ?  I think we can talk about whether it makes sense to take away financial aid from colleges that fail to deliver value. But the goal of this is not punishment. We have a system of higher education where we need to dramatically increase our capacity. We need more students enrolling. We need those that enroll to stay long enough until they graduate. And we need, you know, colleges and universities across the board to raise their game in terms of focusing on a fundamental, their fundamental mission of offering a quality education at an affordable price. So, you know, the goal is not to punish institutions or to have failing institutions. The goal is to have institutions improve. The lady in the back right there in the first row, in that row, yeah. Thank you. Hi, I'm Leslie Baskerville, president and CEO of NAFIO, which is the umbrella organization of the historically black colleges and universities and predominantly black institutions. I'm intrigued by the rating system to the extent that it considers access, affordability and outcomes. I'm interested as to whether or not they will, it is envisioned that you'll equally rate access and affordability with outcomes. So, for example, would institutions that have disproportionate percentages of low income first generation students and are very affordable then have a third criterion of equal weight that would be graduation or is there some other measurement? Leslie, it's a great question and that is an important question. There are, it's not the only important question that we don't have an answer to yet. You know, I can't over emphasize the importance to the president, to Secretary Duncan, of recognizing those institutions that enroll students from disadvantaged backgrounds and help them succeed. That is something we need more of in higher education, we need to do a better job lifting up institutions that do that well and encouraging other institutions to emulate what they're doing that works. And so, you know, a core mission of this rating system is going to be finding those institutions and lifting them up. And whether that's done by building into the rating system an emphasis on measurements of access and success for students of disadvantaged backgrounds, whether it's done by comparing institutions with similar missions or both, you know, it's very important to us that those institutions are fully recognized and that the rating system encourages institutions to do more with students with disadvantaged backgrounds, not less. This gentleman in the second row here. Thank you. Being promoted to gentleman is very good. I'm Alan Sessoms. I'm with the American Association of State Colleges and Universities. I think it's a great idea and I'm intrigued by the idea of incentivizing the Pell. But one of the problems I've seen over time is that the students themselves don't necessarily have the tools to succeed in the early years. And a bigger Pell may not make that much of a difference. Somehow you need to couple, in my view, their preparation to the Pell. And you have to couple it to some extent to K-12. Because many of the institutions that I'm familiar with get students who are engaged and motivated and don't know anything. And that makes it kind of complicated. So how do you see maybe bridging that bit of a divide between, I wouldn't say K-12 but high school, which is not magical either, to incoming students, especially urban environments, who have some significant disadvantage just from an educational perspective? I agree with you 100 percent. And one of the things that Secretary Duncan has tried to do since the start of the administration is work with states that are interested in raising their standards and setting standards that are college and career ready. One of our major new education initiatives this year is high school reform, which even in these difficult budget times, the President proposed a new $300 million program to strengthen high schools by helping students take classes that are contextual so that they have some understanding of why they're learning what they're learning that offer them some work-based experiences that offer them an opportunity to take college level classes and potentially earn college credit. Of course one of the models is the PCAST Academy in New York where students graduate with an associate's degree, which of course makes college a four-year degree much more affordable. So I agree with you. There's work to be done along the educational spectrum and that starts with preschool, by the way, where the President is working to make preschool universally available. So we do need to make all of those investments. I agree with you 100 percent. All right, thank you. I know you all have more questions for James. I know I do, but we're going to move on and thank you, James, for that. We also have another panelist who has to leave a little early and that's Jamal. And Jamal, I think we'll go next with you so that you can make your train. And so, ladies and gentlemen, Jamal. I'm also parked on 116th Street at Columbia University, so I want to make sure I don't get a parking ticket. I'm not so thrilled about going back to the university, though, Columbia.  So I'm having a bit of a midlife crisis. It's like I don't fit into the college population. I can't blend in anymore, so. In any event, it's an honor to be here. If I wasn't here on the panel, I would be in the audience covering this event as I've done for the past three years. And as a journalist, I've always left with a great story. And so I feel a great burden trying to match the things that I've written about here as a panelist. I also want to thank the Lumina Foundation for the work they've done in getting us to rethink how higher education is run, particularly how well higher education serves minority students, first generation college students and students of lesser means. And so it's with that in mind that I wanted to turn our attention to the piece that I wrote for the current edition, the college rankings edition of Washington Monthly. Before I discuss the piece, I just wanted to say a bit about an earlier piece that prompted me to write this piece. Back in January, my colleague and co-panelist Kevin Carey wrote a piece called The Next Affirmative Action. And a major premise of the piece was that if you want to help minority college students that we should strive to make higher education more accountable. And in the piece, Kevin cited a few examples of universities that weren't doing so well when it came to graduating black students. And I was surprised when I saw my alma mater, the University of Wisconsin-Milwaukee, on the list. I went to the school from 1991, graduated in 1996, so I guess I was on the five-year plan. So I guess I made it within the federal timeframe to be counted as a graduate. There were times when I thought I wouldn't, and I touched on that in the piece. But it really struck me that the graduation rate for black students at UWM was 19%. And that statistic kind of struck me for a number of reasons, but one of them was that, you know, I just didn't know that it was that it was that low. And so I thought it was kind of a departure point for a deeper analysis as to the why. You know, I almost failed statistics when I was at UWM. So I've never been really a quantitative person, but I'm into the, you know, the qualitative types. Maybe that's why I'm a journalist. And so I wanted to kind of get into what was behind this, what was going on. And I figured by being familiar with the campus, you know, knowing some people that were there. In fact, I was surprised when I returned. I guess it's 17 or so years later, many of the people that were there when I was there were still in their positions doing their thing. And so I just wanted to get a deeper sense as to what was behind the figures. And so I went there kind of having a sense based on my own experiences as to what was going on. You know, I struggled. I was first I placed into remedial math due to, I want to say kind of this ill preparation at the K-12 level, you know, wasn't really challenged to take challenging math courses that would prepare me well for the college that was required, for the algebra that was required at the college level. And so I placed into remedial math and then when it came time to pass college level math, I struggled with that as well. But there were some particular reasons that took place. One was many of the instructors, the algebra instructors at the University of Wisconsin of Milwaukee, they're foreign nationals and sometimes they speak with just a really thick accent. So if you can think about the challenge of trying to learn a subject that you're already struggling with from someone who might be struggling with the language themselves, you can see how maybe there's a problem there. And when I went to do the interviews, I realized that that wasn't just my perception. There were many students who said, yeah, we have a very difficult time understanding just what the instructor is saying. And so I actually got through math not at UWM but at MATC, which is a technical college, the local community college in Milwaukee. And that was how I was able to pass the math requirement. So I kind of went to UWM doing this story with that theory in mind that this math was probably causing a lot of the trouble as it relates to the low graduation rates. And so there was a moment when I struck what I call considered pay dirt as a journalist. I'm in a particular advising room and someone there mentions, hey, we've got these file cabinets for black students that dropped out of UWM. They almost made it. And I'm thinking to myself as a journalist, wow, what are the chances of me hearing about something like that? Here I am doing a story on the black graduation rate. You mean to tell me they got file cabinets on black students that didn't make it? And I'm thinking to myself what I wouldn't give to just see a few transcripts. And so I figured out a way to do that without violating FERPA, of course. But just to kind of get a sampling of the transcripts, because I thought they would yield kind of some good insight into what was going on. In other words, if you examine the transcripts of students who didn't make it, maybe you can get a sense as to why. And so based on the experience I had, struggling with math, I knew to ask the question, the person I had to review the transcripts. I always had to look at whether or not the student had satisfied the university math requirement. And in three of the five cases that were, or transcripts that were randomly sampled, so it was unscientific. But in three of the five cases, the math requirement had not been met. And so that shows that, hey, this is one of the issues. Maybe if the institution took this a little more seriously, if they looked at the fact that students weren't able to meet the math requirement, and if that's the only thing that's serving as a stumbling block between them and a degree, maybe they could dramatically improve their graduation rate. Unfortunately, one of the most disappointing aspects of the trip, or my return journey there to investigate what was going on, when I spoke with the people in the math department there and asked, well, what are the pass rates for college algebra for remedial math? They didn't have the pass rates. And so I was hoping to get breakdowns by race and ethnicity and stuff like that. They didn't have them at all. And so it's clear anyone that's been following the college success, access and success movement has heard about the importance of if you want to solve a problem, you have to at least understand the scope and nature of the problem and measure it. So that wasn't taking place. So that kind of gave me a sense that, hey, this is something, this is an area where they can improve. It's not as if the university is oblivious to the problem. They do have different initiatives in place, such as Summer Bridge, which enhances students' chances of getting through math. The problem is not all students take the Summer Bridge. So it's one thing to have a program. It's another thing to have a program and implement it well. I myself, I didn't go through the Summer Bridge, knowing what I know now as a journalist who covers our education, I probably would have done that. So it's a matter of whether or not the university adequately conveys the importance of taking advantage of certain things. And in some cases, quite frankly, mandating that students take certain things in order to dramatically enhance their chances of graduating. One other thing I want to say about the institution is that, you know, I had a chance to meet the Chancellor here in Washington, D.C. for an alumni event of the D.C. chapter of alumni from the University of Wisconsin, Milwaukee. And he mentioned something that I thought was quite interesting. He said that the university, when they set out to launch an entrepreneurship program, what they did was they dispatched different university personnel to go to, I believe it was Babson College that had, it was consistently rated as number one in entrepreneurship and U.S. News and World Report in their rankings. And so clearly the administration understands the importance of studying success, successful models and incorporating those models in order, and emulating those things in order to get better results. And so one of the things that I say in the piece is that when it comes to improving math, they should probably do the same thing. You know, there's models out there that work. We mentioned Statway and Quantway. And these are programs that have been implemented by the Carnegie. I'm mixing them up with the county. There's two Carnegie's. It's the one that advances teaching. This is dramatically improved completion rates in college algebra because it makes college algebra more practical. You know, the number one complaint we all hear about algebra is, when am I ever going to use this? So one of the things they do is take a more practical approach to algebra and statistics. So really it's a matter of having the willingness to look for things that work and to incorporate those things. I do want to say just one thing, and this is actually something I stated in the piece when it comes to judging a college or university strictly by graduation rates. You know, I'm personally kind of resistant to that. And part of the reason why is because of the success stories that I've seen come out of the University of Wisconsin-Milwaukee, Milton Coleman, long-time editor at the Washington Post, as well as some others. I don't tout myself as a success story per se, but I did graduate and got a couple journalism awards. You know, I kind of don't like to judge an institution strictly by those things, but at the same time, I find that there is a lot of value in looking at an institution and if the graduation rate is low, then asking the question why. So if there's anything that I think my return visit to my alma mater teachers is that, you know, if you go to an institution and ask that question, you know, it's one thing to say the rates are low, but if you ask the question as to why and you have some, you know, some on-the-ground intelligence that, you know, can lead you to some answers, I think there's tremendous value in that. Thank you. Well, good morning. Paul asked me to talk a little bit about the University of Texas at El Paso and the work that we've done there. And so I prepared some remarks because I'm a lot more efficient that way. So I joined UTEP in 1971 as a faculty member in linguistics and I immediately resonated both to the unique setting in a binational metropolitan area of two and a half million people on the U.S.-Mexico border and to the important role that public higher education could play in the future prosperity and quality of life of that historically undereducated region. Little did I know at that time, however, that my life work would become so deeply focused on increasing opportunities for all young people there and enabling their engagement in the same enhanced educational experiences that are offered to their peers in more affluent settings knowing that they have every right to expect nothing less. By the time I became president in 1988, I sticked two things. I understood that UTEP was at a critical crossroads, that my vision of UTEP's huge potential wasn't universally shared either on or off the campus and that my most pressing challenge would be to address UTEP's profound identity crisis. UTEP's focus had historically been regional, starting with our establishment in 1914 as the Texas State School of Mines to support nearby mining operations in the United States and Mexico. But by the 1960s and like many other regional institutions we'd become convinced that the single path toward greater glory in higher education would require emulating prestigious research universities, however different their settings, constituencies, and missions. As a wannabe research university, we perceived our low income Hispanic students and our border location as liabilities to be avoided or overcome. Some faculty found humor in placing Harvard on the border bumper stickers on their cars. My goal as a new president was to shift our collective focus from self-deprecation toward a greater appreciation of who we were and whom we served to convert such perceived liabilities as our large Hispanic population into assets and to achieve far greater authenticity in our institutional aspirations and efforts to achieve them. Articulating this bold vision and building the confidence and courage to aggressively pursue it have required innovation, agility, and resilience, but mostly it's required a steadfast confidence in the talent in our community. Data were essential, as Abdul just said. We had to learn more about our region and our responsibility for its human and economic development. We studied data on the young people UTEP served and those we didn't. And we began to identify and celebrate our distinctive features all toward a goal of transforming UTEP into an institution better aligned with and responsive to the needs of the surrounding region and its residents. What we saw in our self-examination wasn't always pleasing, particularly troubling to many of us was that although nearly all of UTEP students were from the region, a disproportionate share were graduates of a relatively small number of more affluent and more Anglo high schools. UTEP didn't look like El Paso. Large numbers of young Hispanics in our community were being denied access to social mobility through education. They were written off as not being college material and their talent was being squandered. We began to understand better UTEP's critically important public university responsibility for both the dreams and aspirations of all these young people and the future prosperity and quality of life of our underdeveloped border setting. Our data also reminded us that educational institutions in the region from pre-K to post-secondary formed a closed loop. More than 80% of UTEP students are graduates of El Paso area high schools and two-thirds of the teachers in those schools are graduates of UTEP. Despite our mutuality of interests, we endlessly blamed each other for our frustrations. UTEP faculty blamed the schools for not sending us better prepared students and threatened to raise admission standards and the schools blamed UTEP for not sending them better prepared teachers. Clearly, the conversation had to shift toward accepting our shared responsibility to increase educational opportunities for all young people and leverage each other's contributions to enhancing educational attainment. To that end, in 1991, we formed the El Paso Collaborative for Academic Excellence which was a partnership that includes all nine school districts in our county, UTEP, the El Paso Community College, and civic leaders working together toward a goal of raising educational aspirations and attainment across our entire region. Our focus was once again on data, sharing them, analyzing them, and ensuring that they, not our biases and assumptions, would drive our efforts to align curricula, assessments, and standards along the entire pre-K through 16 educational pathway. This vertically integrated commitment to systemic reform and its reliance on data have served us very well for the past 20 years, creating a robust longitudinal data set, bringing together K through 16 faculty and math and other disciplines to align expectations and best practices, developing a robust range of dual credit courses and early college high schools, and perhaps most importantly building trust among all the key stakeholders across the region. The outcomes tell the story best. In 1991, El Paso's high school graduation rates were among the lowest of all major metro areas in Texas, and there was an unacceptable disparity between the rates of Hispanic and Anglo graduates. Today, El Paso area high schools have the highest overall graduation rates of all Texas major metro areas, and the gap between Hispanic and Anglo rates has narrowed. El Paso County districts now also ranked first in Texas in the percentage of high school graduates who complete the state's advanced curriculum, and first in the number of low income high school graduates who enroll in post-secondary education. As a result, enrollments at both UTEP and the community college have also grown significantly over the past decade. In UTEP's case by nearly 50%, which means that more El Pasoans are aspiring to and being successfully prepared for post-secondary attainment. Especially encouraging is the decline in the number of UTEP's entering students who require developmental education. In math, college readiness has improved from 29% to 83% of entering students. We are also especially proud that with 77% Hispanic enrollment today, UTEP's student demographics now mirror those of El Paso and that UTEP is recruiting the region's most accomplished high school graduates, most of whom are also Hispanic. More than 60% of the top 10% graduates of area high schools who attend a public university in Texas now choose to enroll at UTEP. While successfully building a smooth pathway to UTEP for students from area schools, we've also been devoting substantial effort to understanding better how we can promote greater student access and success on our campus and eliminate our own barriers to students' progress toward degree completion. We've identified four dimensions of access, aspirational, academic, financial, and participatory, and determined through systematic data analysis how best to approach each of these. Our aspirational access efforts are best captured in the extensive pre-K through 16 collaboration that I've just described. Improving financial access has required gaining a far deeper understanding of the impact of our students' often severe financial constraints. Nearly half of them report a family income of $20,000 a year or less, and responding with efficiencies, tight tuition management, and robust creative financial support, including externally funded scholarships, grants, and on-campus employment. We're proud that in the most recent U.S. Department of Education ranking at $2,543 per year, UTEP's net price is the lowest of all U.S. research universities. Academic success has improved through streamlined degree program requirements and degree plan data analytics that inform advising and ensure availability of required courses. Participatory access has increased greatly through student demand data-driven class scheduling to offer courses at times and places, including online, that enable the enrollment of our many students who have major family, employment, and military obligations. We've also learned more about strategies that are effective with our student population and those that aren't through a constant cycle of development, monitoring, and evaluation of specific interventions and their outcomes. A toolkit that's filled with ideas that work in other settings will not likely be useful at UTEP without very thoughtful recalibration. Our analyses have also enabled us to challenge one-size-fits-all policies and such metrics as graduation rates, which mislead rather than inform. And I'll be delighted, more than delighted, to talk further about graduation rates which fail to account for 70% of UTEP's graduates. In short, our data have liberated us from distracting and often irrelevant debates, made us stronger and more confident advocates for the students we serve, and far more successful in creating conditions for them to achieve their goals. As a result, degree completions have grown dramatically at UTEP with an 85% increase in undergraduate degrees awarded over the past decade, more than 80% of them to Latinos. And UTEP consistently ranks among the top three universities nationally and the number of Hispanic graduates in nearly every discipline. Embedded in all our efforts at fostering student access and success has been an equally firm commitment to excellence. Without excellence, access becomes a promise to be broken. UTEP diplomas must enable our students to compete successfully with their peers from across the globe. Considering UTEP's historic STEM strengths, our students' financial constraints, and state funding trends, we concluded 20 years ago that building competitive research capacity through strategic growth in externally funded research was the most promising pathway to creating the enhanced climate of excellence that would be critical to achieving our commitment to student access and social mobility, so its access and excellence. We're pursuing excellence UTEP's way for and with our undergraduate students, not in spite of them, as is evident in the commitment of UTEP researchers who enthusiastically support undergraduate students as members of their research teams. And in the major investments we've made to create a large number of on-campus jobs that are tied to undergraduate research opportunities well-aligned with students' academic goals. Outcomes again tell the story. UTEP's annual research expenditures have increased steadily from $5 million in 1988 to $79 million last year, and UTEP now ranks fourth among Texas public universities in federal grant funding. Our annual budget has grown from $67 million in 1988 to more than $400 million today, and it hasn't been thanks to the Texas legislature, I might add. We've expanded the single doctoral program in geology we offered in 1988 to today's 20 doctoral programs enrolling more than 700 students. And as expected, this activity has greatly enhanced the campus climate for and raised the aspirations of UTEP's undergraduate students. A growing number of the 55% of our 23,000 students who are first in their families to attend college now confidently express plans to continue on to graduate and professional schools very successfully. UTEP ranks seventh among U.S. universities in the number of Hispanic bachelor's graduates who successfully complete doctoral degrees. The entire UTEP team has worked hard over the past 25 years to serve as a transformative resource for the surrounding region, and especially for those students who bring us their dreams and aspirations. In the process, UTEP has undergone a major transformation, too. The success of our talented and hardworking students has enabled us to enthusiastically embrace our institutional identity and our mission. We're now comfortably and proudly authentic in our UTEP skin and joyful knowing that we've made huge progress in validating our success by becoming the best UTEP we can be. Not Harvard on the border, but bordering Harvard in Washington Monthly's latest ranking. Thank you. Well, that's a hard act to follow. You know, I've lived here in Washington for 12 years, and it's a strange place to be in a lot of ways. And one thing that I came to realize, and it took me longer to figure this out than perhaps it should have, if you do this kind of work, you have the privilege of being able to come and talk to people about interesting things for a living.  And not only only ever talking to college graduates, but only talking to a certain kind of college graduate, people who made a very easy and successful transition to college after high school, who don't even know what the word remedial means. People who graduated in four years, and in almost all cases, went to a selective college and university. And that personal experience of higher education, that very, very atypical experience, not just of being an American, most Americans are not college graduates, but being a particular kind of college graduate has a very profound effect on how people understand higher education and as such what kind of public policies they think we should have about higher education. And I think if there's been one thing that's been tied together, all of the great work that the Washington Monthly has done in covering post-secondary education over the last decade, it has been on this stubborn insistence on pulling people's attention away from that very narrow distorted vision of higher learning toward the reality of higher education in America. Higher education in America is the University of Milwaukee at Wisconsin and the University of Texas at El Paso. Higher education in America is the 45% of all students who start college at community colleges. We don't have a community college panelist here today, but it's a big part of the issue, so I'd like to take just a few minutes to talk about some of the great reporting and data that's in there. For the third time, we published a list of America's best community colleges, very much with the intention of highlighting the plain but chronically overlooked fact that there are best community colleges, and the idea of variance in quality applies just as much to our two-year institutions as it does to our four-year institutions. This year, I think we actually have an improved process and metric for identifying the best community colleges. It's a combination of a survey of best educational practices and a number of measures, pardon me, student outcomes that include both graduation rates and also a ratio of graduates to enrollees, which makes sure that 70% that gets left out of the traditional graduation rate metrics are reflected. We also for the first time had an article somewhat provocatively called America's Worst Community Colleges, and I think we have to be honest and recognize that if we believe in excellence in a sector, we also have to believe on some level in the lack of excellence. There's a terrific reported piece by Hailey Sweetland Edwards, who's here in the audience today, looking at the system or lack thereof of two-year institutions in the San Francisco Bay Area. Some of you may have been following the news around the possible de-accreditation of the City College of San Francisco, which it turns out is really only one of a number of institutions that by all of the data we were able to put together and all of the measures we were able to accumulate is really falling short of what we ought to aspire for two-year institutions as a society and what students need. And as Hailey really I think does a fantastic job of pointing out in her article, it really is a range of responsibility for these failures that goes right up into the governmental level. What she found were institutions that were essentially paralyzed by a system of finance and governance that makes excellence all but impossible. And so there isn't any. And this is a part of the country that we think of is in many ways the apex of the American vision for higher education. You say higher education in San Francisco, you think Stanford, you think Berkeley, you think Silicon Valley, you think this virtuous combination of excellence and economic activity. And yet the vast, vast majority of actual college students in that part of the country and every part of the country are nowhere near institutions like that. There are 90,000 students enrolled in the City College of San Francisco that's in the midst of an existential crisis right now and have been subject to just a combination of governmental apathy and a brutal fiscal environment over the last five years. So all of this is, you know, I think very much to the magazine's credit that it has, you know, insistently, year after year said, we are going to write about the reality of higher education in America. What I think is interesting and exciting is that we're starting to see, I think, some of those same, that same perspective and those same priorities starting to emerge in the way people are talking about what public policy should look like particularly here in Washington, D.C. Despite this, you know, Washington, D.C. is essentially the city of, if not the 1%, the 5% when it comes to, particularly if you add political and social capital into your sort of sense of the influence and things that people bring. It's very clear, I think, that if we're going to meet the economic and human capital goals that we have for this country, if we're going to remain a vibrant, decent, competitive society, that we need more people to have the benefit of higher learning than have that now. And I'm just struck by when I hear success stories like that of President Natalicio when you said, we saw our students' liabilities. And that is, I think, the reality of in far, far too many places in its reflective of this distorted perspective on what excellence means. That excellence can only be an institution that looks like the places where the 5% came from. That is a powerful, powerful headwind to change into progress. Great leaders with concerted effort can overcome those headwinds and UTEP is a fantastic success story and we're thrilled to have President Natalicio here and to see them highly ranked on the Washington monthly rankings. But we should acknowledge that it is an accept story, that there is an exceptionality to that kind of success story. We can't rely on great college presidents having 25 years to work. If we want every college and university in America to be able to move ahead toward the goals we need. We need to take those headwinds away or we need other forces pushing people in the direction that we need to push them. And in the end I think that's what I see not to sort of speak on behalf of the administration but that's what I see personally reflected in the kind of actions that the administration is taking and the goals that embody this idea of creating new ratings that reflect the values of access, the values of affordability and the values of success in the context of institutions and what their missions and what their students are. The fact of the matter is one of the outcomes of this distorted vision of excellence in higher education in higher education. It's driven a lot of institutions to orient themselves in a way that has pushed the cost of higher education farther and farther above the means of students and families to pay. We know state disinvestment is a big part of that equation but it is not the only part of that equation. Even if our state legislators were as responsible as we wish they were there is still a very, very large underlying trend of increased spending and the federal government has been essentially complicit in that in the sense that it has simply up until now chosen to put money into our higher education system with no judgment as to what's being done with it. Personally, I'm not one of the people who thinks that that's been the primary cause of increasing prices and lack of opportunity in higher education and it hasn't done anything to solve that problem. It certainly hasn't done anything to give countervailing forces to institutional leaders that are pushing against those headwinds that sees the students we need to serve as liabilities. Change in higher education will happen one institution at a time on some level. We are not going to take control of the industry through government regulation. It has to be choices that are made voluntarily and with enthusiasm in partnership between institutional leaders and faculty and stakeholders at the state and local level. What the federal government can do is give those people strong reasons to act. It can give them reasons to push against these countervailing forces that are still going to be with us for a while and I think a combination of practical financial incentives but perhaps even more importantly the values that sit behind them publicly endorsed and publicly stated really can have that effect. I think that we are moving into in a lot of ways an exciting very positive time in terms of the evolution of how we think about higher learning in America but we definitely have a long way to go still. Thank you Kevin and thank you President Natalicio and I'm sorry that James left before he could hear your story because as he said the administration is beginning a conversation I hope that you get a chance to converse with him and we'll make sure the transcript of this gets to James because it's a very important story. I want to open up the discussion and to do so since we've lost half our crew I'm going to ask Jamie Marisotis to come on up and pitch in here to engage with you all and I want to begin the questioning by picking up what Diana said which is she doesn't trust graduate graduation data we use graduation data in our measure so obviously I'm interested in knowing why we've made this horrible mistake or how we can improve it so do you want to expand a little bit on that? Well the issue that I have with graduation rates is of long standing a lot of people here have heard me talk about this but many people think that our graduation rate which lags far behind our degree completions, degrees awarded in terms of growth is because our students are working class and because they go to school part time or because they have to drop out for a semester or something like that the truth is that the real issue with the graduation rate is that the calculation itself doesn't capture 70% of the people who graduate from the university because they didn't start at UTEP as first time full time freshmen and so if you don't get in the denominator you can't be in the numerator and if you're not in the numerator then you don't count as a graduate in the graduation rate and so my worry is that while on a policy level we're encouraging institutions to to urge the enrollment of transfer students to make that possible facilitate all of that and yet in the graduation rate calculation you get absolutely no credit for the work that you do the investment that you make in those students graduation. Now there are options that are people are working on the six associations have been working on this new SAM measure which is an effort to try to capture a little bit more of how institutions contribute consecutively to a student's progress toward a degree and I think there's a lot of important work that can be done there and so I commend that and I hope more public universities are going to get involved in volunteering it's a voluntary metric and I hope public institutions will do that because I think public institutions are the ones that suffer the most from this calculation of graduation rates which tends to look at cohort enrollment in a fall semester full time and that usually is about private institutions and public universities don't typically do that but I'm I'm very concerned that this graduation rate is so deeply etched in people's heads that it is one of the sources of this kind of pushback that we feel we feel we're swimming upstream in the legislature whenever I testify I talk about the number of degree completions I talk about degrees awarded everybody applaud and then they say but doctor what about your graduation rate and so then I have to explain about graduation rates and most people believe that graduation rates are actually degrees awarded and unfortunately that is obviously not the case and so from a marketing perspective private institutions like graduation rates they make them look very good public universities of course suffer in the process so it's something I think we need to work on because the calculation itself is problematic Kevin Jamie you want to respond? We have some tricky tricky technical challenges in front of us to put together federal college ratings that will pass muster Accurate graduation rates is not one of them it is hard to figure out how much a student has learned it is not hard to figure out whether they do or do not have a diploma it's very easy to find that out President Natalie shows absolutely right that our current federal graduation rates are limited and their utility in accurately depicting the success of institutions declines for institutions that have diverse populations and are not enrolling you know overwhelmingly traditional cohort of first time full time students that is a totally solvable problem I have personally spent I can't tell you how many hours in my career on panels and technical review commissions and you know very very kind of in depth discussions trying to figure out better ways of doing this the only real barrier to better college graduation rates is colleges themselves when we actually get down to suggesting better systems the objections always come from within the higher education community well that would be more complicated there would be more administrative cost to us to you know calculating in other ways there have been suggestions you know the easy way to do this is of course to have a student you know record data system here under the auspices of the National Center for Education Statistics would actually be much cheaper for institutions it would save them money every single person in the system would be in the denominator in the numerator we could have graduation rates that are exactly accurate that take transfers into account that look at whatever timeframe you want is inappropriate the only reason we do not have a student you know record system would have a better graduation rate and this frankly is a case where we have divergence of interests inside the higher education community basically between private and public institutions if it was just up to public institutions would have a student you know record system would have better graduation rates it is the private institutions that have opposed it so that's a problem we have to solve so oh yeah I'm going to follow that great just to add to the fact that both Diana and Kevin are offering here which is part of the problem with graduation rates is our obsession with single measures right one of the challenges we have in public policy Diana's point is a very good illustration of this when you speak with policy makers at the federal or state level they say yeah yeah yeah but then they want to know what's the one thing and the problem is we're talking about complex systems for which I believe graduation rates are an entirely legitimate measure but should not be the sole measure used to make decisions and that's where the complexity comes in terms of making effective public policy and where I think the administration has made a wise choice here in giving itself some time to figure out how to construct these ratings because I think what they're going to get into is essentially a balanced approach a set of weights to be used among a variety of variables in order to be able to use so given a different illustration I see sitting in the front row here Mark Schneider college measures so college measures is an attempt to try to shed some light on the important issue of wages of recent graduates now wages of recent graduates is an incredibly important issue and what they've done is groundbreaking anyone who makes a decision based solely on wages of recent graduates is probably making a mistake but to say that those data aren't valid because the recent graduates aren't reflective of the lifetime of their experience belies the fact that what consumers want is high quality jobs and wages early in their experience in terms of getting out of college they've made an investment they want to see that investment pay off so seeing these things like wages like graduation rates like some of these other measures as part of a sort of diverse ecosystem I think is very important but public policy really struggles to have those kinds of conversations great thank you so let's open it up to the audience and is right here Mark we'll get the immortalize you in video and sound here I've already been introduced to Mark Schneider I'm the president of college measures and we're working with states to put into the public domain wage outcomes with the generous support of the Lumina Foundation so actually I want to ask you Jamie you started going down the road and I want to just push it a little bit further so if the US news puts out its ranking the Washington Monthly puts out its ranking you have weights you know you could fight about the weights and you could say oh that's terrible that's good that's bad weighting these variables are right these variables are wrong but when the government gets involved in this right and they have to start talking about the weights on a wide variety of measures many of them have to be determined by some model based models that are regression based or multivariate based and then some of these results may be knife edge you change the variables you fight over the variables the things change so I have no problem with the Washington Monthly doing that I have no problem with the US news doing it I'm going to do this myself so everybody does this but when you move from this information to a regulatory component where you're tying outcomes and real money to it I wonder if there really is a path forward when we move from this informational component to a regulatory component yeah I mean I don't see much of a difference to be honest with you whether the Washington Monthly does it or take in your case college measures you're doing it presumably state policymakers are making decisions based on your data whether government is directly doing it determining the weights and measures or whether you're doing it you know in effect you may have some independence from them but in effect they're sort of using you as a contractor even if you're not being paid by them if you know what I'm saying so I think the the difference is there's not much of a difference in terms of the two I think the complexity at the federal level is that this has to do primarily with the allocation of resources to students as opposed to the allocation of resources to institutions and there I think you need to take some time to figure out what are the unintended effects of some of these decisions you might make based on the allocation of student based resources but I don't I don't really have a problem with government getting into this and as you know there was a great piece yesterday by Ben Miller and inside higher ed in which Ben said you know let's how about let's talk about the bottom 10% maybe we could start there with a good example of there must be some agreement that when we come up with this system which is going to have a lot of input from a lot of different people that in fact you could make some decisions based on the truly terrible as opposed to trying to figure out sorting in the middle or what most of the ranking systems except for the Washington Monthly Do which is to quibble over who's at the top those aren't the important things the important thing is to figure out how you make sure that those that are truly at the bottom have some incentive to do a heck of a lot better Kevin? I think one the administration has made a distinction between a a system of rating and a system of ranking and when again I think this is all to be determined when I hear rating what I hear is something like the Carnegie classification where you're putting people into broad categories there's a ranking that sits underneath it but on some level you're not being quite as precise about it any kind of ranking rating system can be partially empirical but there's an element of judgment into the weights that you put into it I think the only thing you can do is to be as empirical as possible be as transparent as possible about what the judgments that you're putting in there and if what you're really doing is just making the rating system trying to look at some dual perspective on excellence and terribleness or whatever to some extent that I think some of the ambiguity or the judgment is accounted for in the fact that you're just creating categories rather than extremely fine distinctions. Just quickly to add to that you know this is one of the things I've studied ranking systems in other parts of the world another great American export ranking systems and this ranking takes place in other parts of the world and what we've learned in other countries is that they've figured out how to do this better they've taken a sort of monthly approach and said okay well what are the outcomes that we care the most about and what do we actually want to do with those outcomes so in Germany they have a rating system not really a ranking system in which institutions and actually programs within institutions are sorted based into these categories so it's a yellow light green light red light sort of system as opposed to an ordinal ranking that's an interesting idea and I think helps to create some consumer sense not of is 23 better than 24 but which are the ones that really are struggling which are the ones that need a lot of help which are the ones that are doing pretty well and which are the ones that are at the top it's not you know not a bad idea I think something we can learn from. Yeah and the monthly will go ahead and rank then take the pressure off the federal government yeah this later right here. I'm Kathleen Connell and I want to endorse what the president said regarding graduation rates versus degrees I served as a controller of California for eight years and we audited the schools and in fact if you look at your list today eight out of the ten UC schools are ranked among the top and the two that are not are the ones that have transfer students and whose majority of students come from transfer students and that was the intent of the UC system so they're not included in the top ten and yet they're just as good as the other eight that are the same thing is true with your Cal State University you've got six out of the 23 and those six are the ones that don't take transfer students and the other ones are the ones that are feeder colleges as we call it so it's a perfect example of what you're saying president I wanted to follow up on what you say though about STEM education because that is the area that we're focused on now in preparing middle and high school kids in the STEM areas and doing boot camps for them as they go into the college that bridge program is so critical you obviously had success in your collaboration with the school districts can you kind of expand on what you did there and what direct impact that's had on the quality of your graduates well we we obviously are in a fairly circumscribed geographic area so we have school districts a community college in a university and we're relatively isolated from other regions and so it's kind of a hot house laboratory situation or at least that's the way I perceived it initially and we began as I said to understand the reciprocity that existed between the school districts and the university the community college everybody working together and so we reached out we did a lot of data analysis looking at what students were actually what courses they were taking in high school to this day we provide every single high school from which we draw students students enroll at UTEP from a broad range of high schools in our county we give them data every single year on the performance of their graduates in our math courses so that we can track and they can track their students as they go to the community college and onto the university they know exactly which of the preparatory courses whether it's pre-calculus or whatever it might be the efficacy of those courses or not and they begin then to be able to do the kind of analytics that I think the districts deserve the data to do and so they're working very hard always on trying to integrate the information that we provide them to be honest I think data as I say earlier data are the key people spend so much time talking about stuff and expressing opinions about the schools and all kinds of things but they often don't have the data and so we focused a lot on data the collaborative that exists until today we still meet every couple of months the board the superintendents the president of the community college myself business leaders we still meet now and share data we don't we don't talk about a lot of broad issues we look at data sets and we try to understand what the issues are we then are able to draw conclusions from those data about first of all the gaps in the pathway between pre-k and 16 but also we can derive certain kinds of policy concerns so for example early college high school students dual credit students we have early college high school students who are so accelerated they complete their associate's degree by the end of the junior year in high school okay so they've concurrently been enrolled in the community college and in the school they complete their associate's degree by the end of their junior year and want to continue on at UTEP but they're not eligible for federal financial aid because they haven't finished high school okay so those are the kinds of disconnects that you can identify once you begin to track students in the way that we have and so it's that vertical integration that's really worked very very well for us I think I think it's just extremely important that institutions regional institutions like ours spend a good deal more time on understanding the schools and the school districts and the community college that they work with we really haven't done that for the most part across the country and then we have to worry about just what you talked about the unintended consequences of any kind of use of a metric like graduation rates there are a couple of possibilities one is discouraging transfer students if you're going to have to invest in them without credit then why would you do that so then people come up with a separate transfer graduation rate but nobody pays any attention to that so it doesn't matter the other unintended consequence of course is not to admit at risk students we made a calculated decision that at risk at UTEP meant something totally different from what most people think about and it has much more to do with their personal baggage if you will all of the issues that they deal with financial issues and health issues and all kinds of things that relate to being poor in America and so we spend a great deal of time consciously seeking out so-called at risk students because we know that whatever metric it might be SAT, ACT whatever it might be likely won't predict success at UTEP and so metrics matter but you gotta challenge them at all cost and it's very very hard to do people know that I never give up I just don't give up and you gotta communicate that to people that I'm coming back you said no today but tomorrow I'll be back and for 25 years I've been coming back and so they begin to get to know that but I do think there's a lesson there and that is that if you don't believe the talent is everywhere that it crosses all boundaries socioeconomic, ethnic, gender race, geographic if you don't really believe that then you're probably not going to do a great job of providing leadership for regional public universities because you have to believe that many of the metrics that you work with aren't going to work for you they're just not and you can't trust them. Kevin, what Diana said sparked something in me and I sense it sparked the same thought in you so I- Sure, well we'll see another issue that I think should be put on the table in terms of how institutions react to incentives and provide opportunity is the way colleges universities provide their own financial aid to students there's a great piece also here in the college guide from my colleague Stephen Burt who's a senior policy analyst here at the New America Foundation where he tells the story of the as he calls it the Meridate Arms Race and traces actually of the state of Ohio goes all the way back to the 1970s and shows how a lot of the first the private but now increasingly the public colleges and universities in Ohio got caught up in this competition among themselves to give more and more of their financial aid to wealthy students instead of to low income students and what it really is is a story of in a lot of ways good people trying to do right on behalf of their to understand exactly what's going on but feel compelled in order to act on the financial best interest of the college to keep the doors open to kind of keep things going to basically shift more and more of their financial aid away from low income students and toward students of means people who need at least and what's happened now over the last five or ten years is that the public colleges and universities who could sort of ignore that for a while because public subsidies allowed them to do that are now getting in on this merited game with gusto this is another example of where we need strong leadership and in some ways real dollars on the table at the federal level to provide a countervailing force so these leaders can make different kinds of choices. So it was not exactly the point but very close so there's another story in this issue that I want to recommend to you that Diana said that this focus on the students that you have and this interplay between the university and its feeder schools what you're seeing for a lot of the flagship public universities out there is something completely different they are chasing the dollars and increasingly those dollars are coming from very capable very eager foreign national undergraduates whose numbers have soared at some of our flagship universities and you can understand why they're doing it these are young people whose families can provide full ride payment that holds those huge budget holes that have opened up especially since the recession but what it also means is that and this is sort of a theme of ours throughout the issue you're seeing our flagship public universities becoming more and more like private universities where they're chasing the dollars they're less and less focused on the needs of the students especially the students of modest means in their own states and these are universities that have been built up over decades and in some cases more than a century of public subsidy and now they're becoming places where wealthy students from Saudi Arabia and especially China are sending their kids so the disconnect between the mission of the public university as originally conceived and where they're finding themselves forced to be is growing greater and greater it's sort of something that's very uncomfortable to talk about but I think we felt absolutely that it ought to be on the table so let's ask some more questions the gentleman in the back there James Sang I have a question a methodological question about the use of universities and colleges as the basic unit of aggregation have you looked in your numbers at how for example at a particular university different schools spread out on your rankings how much variation is there within a particular university or college given your ranking system that's a great question and I think one of the fundamental struggles in thinking about quality and higher education is that the college has always traditionally been the unit because the college is sort of a circle that we draw where there's clear distinctions in terms of who's in charge and where the money is and whether you're enrolled or not enrolled but what we found I think over the last 10 or 15 years in particular you think about something like the national survey of student engagement which is a very very I think interesting and thoughtful way to think about quality and higher education and we use its sister survey CESI for the community college rankings is that the numbers tend to all average out inside of the institutions and it's because there's so much diversity inside of colleges and universities in terms of not just schools but even programs within schools one of the things that I think is really interesting is that happening here at the federal level is that the gainful employment regulations that we're put in place as an attempt to kind of get our arms around essentially the for-profit higher education industry and is still subject to litigation and negotiation today but those gainful employment regulations are programmatic measures of success not institution level measures but programmatic measures of success and you can right now get on the US Department of Education's website and get a huge spreadsheet of programs within institutions and what you see is what you'd expect to see huge differences in different programs inside of the same institutions because really it's at the program level that we're making actual choices about what to teach people and whether to do a good job or not to do a good job and so we're gathering now important student outcome data in this case the earnings data that Jamie was talking about at the programmatic level personally is an iteration going forward and we run into some tricky problems particularly when we talk about money because the college is the sort of has the money but I really think that we need to move toward a program level vision of what quality and higher education is and we need to extend that to things like learning and success and other things in fact I think in this increasingly democratized system of learning where you can gain access to high quality learning in lots of different places I think this idea that the institution is the most important unit of analysis is going to decline and I think consumers are saying they want outcomes they want result employers are certainly saying this government's now weighing in and I think what they want is the results as they relate to the students and what the students have learned and what the students are able to do unless the sort of reputational factors that we've seen traditionally in these ranking systems monthly not withstanding and in fact it's interesting to go back to the example I mentioned in Germany the ranking done in Germany is done by an organization the acronym is CHE because I don't speak German and the CHE ranking is essentially programmatic in nature they do do some aggregations up to the institution but it's almost secondary to the program level ranking that they do because that's what consumers in Germany are really interested in they're interested in the best engineering programs they're interested in the best science program what have you as opposed to the main unit of analysis being the institution the question is how rapidly will we move towards that kind of model in the US institution is deeply ingrained in our higher education culture it is very difficult for us to get past which sports team we root for which alumni association we're members of etc but I think that that change towards this more student outcome driven student focused system and movement away from the institution based system is inexorable it's going to happen great my friend Ben will ask you right there is in the middle row there I'll throw out just one on the foreign students coming in and I guess I would suggest two things I mean first this is really no different than what happens at a place like UVA where there's a lot of controversy not over foreign students necessarily but over state students so there's a constant balancing out with the legislature you can't have too many out of state students yeah they pay full freight but you're somehow undermining the Virginia character of the institution I think there's a kind of zero sum aspect to this you know why can't a place like that get bigger I know Kevin's argued many times that you know why is Harvard not getting bigger there's so much demand for it and I know there's been some research on PhD programs when you look at foreign students this is going back to the 80s and 90s a number of PhD student programs that have had huge demand from foreign students you know in science engineering they haven't rejected a lot of American students they've become larger because there's demand so it just strikes me and I have not yet read the entire piece it looks like a thoughtful piece and I look forward to reading the whole thing but I guess I would suggest that there's more different ways of thinking about this than simply those rich Chinese or those rich Saudis coming along and squeezing out the deserving Americans I don't think that's a useful way to think about this yeah I think you know the the piece that we wrote about the foreign students was largely kind of instigated by if you look on I think you all have a copy but if you look on page 56 there's a graph here and it shows the number of F1 visas granted from the State Department for students by nationality and what you see is actually as back in 2008 not that long ago in 2008 there were about as many Chinese students as there were Korean students F1 visas issued both around 60,000 the number of Korean students to stay the same last year in 2012 for Chinese students it was 190,000 so just an explosion of students coming to the United States in very recent like just the last five years now is that how should we think about that I think Ben is absolutely right we shouldn't we don't want to be kind of too narrow or nationalistic in thinking about these things that said I think a couple of things are clear the motivation is entirely financial this is being done 100% by institutions as a way of balancing their books there is some zero sum in this to it we can argue about whether these institutions should be bigger or smaller they are on some level place bound there is a fixed number of students that are enrolled they are a taxpayer supported institutions the article actually focuses on Purdue University a public university in Indiana that's basically seen essentially a one-for-one substitution of foreign students for students from the state of Indiana the argument that institutions make is that this is all good because it's enhancing the diversity of the campus this is a way to kind of give people interaction with people from other cultures and there's some great quotes in there from some of the Chinese students themselves who basically said yeah I basically came here was segregated into a community of people like me for four years and then I left and I never had much of an interaction with anyone else at all yeah I think the author of the piece Paul Stevens very good journalist had a quote from one of the students saying well one thing I got in my four years at Purdue is my Chinese improved because I was talking to a lot of Chinese students from different parts of China but he never sort of made friends with any American so I think we're trying to provide some clarity to motivation and and also this is a trend that seems to be accelerating and we ought to think about what that means so let me put two or three or four questions together because we've got about seven minutes left and I want to get you all out at noon as we promised so do we have other questions or there's this lady here and keep a nice brief question and try to answer them all at the same time I'm Peggy Ochowski I'm a congressional correspondent with a Hispanic outlook I also have a grant from the Fund for Investigative Journalism and I'm doing it on foreign students and the thing is I've done a number of stories on really impressive increasing numbers of Hispanic kids going into engineering they need support in grad school they're having a hard time getting it because basically most the areas of support like readerships, lectureships are pretty much owned by foreign students who cannot work off campus and they have to work on campus so they're finding it challenging to find support in grad schools I'm very curious what if you have foreign students in your grad schools if you have any idea about the clustering and about especially the domination of foreign students in all the support readerships, TA's lectureships, research assistantships that kind of stuff I guess Dr. Natalicio you should go ahead and just answer that one just quickly we have a very low percentage of foreign students overall the majority of our international students are Mexican and they cross the border every day and so we really have focused heavily on encouraging our undergraduates as I mentioned to go on to graduate school some of them go on to graduate school others go elsewhere and that's been a very high priority first generation low income students typically see a bachelor's degree as a destination we wanted to be a milestone toward greater accomplishment because so many of them are really able to do it our engineering program is very mature it's been part of our history for a hundred years and so we are a very large national producer of engineers who are Hispanic and we're getting a lot of corporate support trying to generate the kind of interest that I think many recruiters from particularly defense industries have in recruiting US students and US Hispanic students in particular so we've tried to work on the financial side of that to ensure that we're not simply bringing in large numbers of international students you'll find that our percentage is quite low thank you and there was a lady in the back whose hand was up there you go hi I'm Sharon Webb Einstein fellow at the department of energy in the first talk it was mentioned that the rankings from the Lumina foundation were based on social mobility research and service and I'm not sure I've heard about the service and research aspect and wondered if you could comment on that sure the research aspect of it is primarily based on federal research spending federal so and just a basic financial measure that a lot of institutions themselves use to benchmark themselves against their peers in terms of the overall level of research output the service measure is actually fairly complicated there are a number of different data points that go into it it includes percentage of students who go into the Peace Corps percentage of students who go into ROTC and a series of measures from a survey that is done by the it's the Corporation for National Service looks at how often service is incorporated into the both courses taught the percentage of students on work study who are in whose work study role is service oriented so I think there's five or six different measures in there and they're all average together and the research numbers also include numbers of Ph.D.'s also I believe right what it's actually to pick up on something that present not each of said it's the for each institution it's the percentage of undergraduates who go on to get Ph.D.'s so we're we're judging them by whether they're preparing students to go into successful careers as scholars and researchers and we have one more question back here I'm really happy that I'm able to tie up with the final question but early on you guys had said that the particular criteria you'd use was an outcome measure had to do with recent college grads I have a question about the extent to which we want to extend that to much more long term to be able to trace what happens to people at longer paces in the workforce I think to some extent related to the question raised on programs and schools if we're able to trace what happens long term in the workforce then we're better able to say what happens which degrees are successful, which types of programs are successful, what types of programs we ought to be funding so that's my question have you looked at not only addressing recent college grads but what happens later on tracking them later on yeah I think we're still in the beginning stages of gathering employment data and I think part of the reason they've been looking at recent college graduates one closer less distance from the action itself of going to college and also you ought to pay your loans back in the short term so even if we can make some probably fair and nuanced assumptions about how different kinds of programs may have both short and long term effects on people's economic potential and earnings, the majority of all students are undergraduates are borrowing money now under federal programs and you have a limited window and opportunity to defer that debt and so because the focus has been the earnings data has been collected really for a fairly narrow regulatory purpose to try to identify programs that are just way out of whack in terms of what they cost and what they return in the labor market it's made sense to focus on the short term but I think you're absolutely right we should explore this question of the economic gains to college which is important from a variety of perspectives including longitudinally over a long period of time yeah the irony is Nicole is one of the best economists in the country she knows this that we have data that is about lifetime earnings and we've got very good data that is emerging data now about recent graduates and we've got very little in the middle and I think being able to develop better systems you know philanthropic organizations can certainly be a part of that is really important you know what's working against us is time right we need to invest in long term collection of information over a period of years because I think one of the things that we know about our labor force is that there's a higher level of mobility than there's ever been before so being able to rely on sort of old assumptions about labor patterns is probably wrong and figuring out what happens to people's wages and earnings over time beyond the first few years or taking into account just the lifetime earnings would be really valuable but that's literally going to take us time and that's something that I think is worth doing so before we end I just wanted to thank you New America for hosting this and of course Jamie and the Lumina Foundation I also want to say a special thanks to the Kresge Foundation that is going to be supporting the dissemination of our college guide and among other things we're very grateful for that and I want to say a special thanks to my colleagues at the Washington Monthly this is an issue that takes about a year to put together in a couple of weeks when we're no longer exhausted we'll begin talking about stories for next year and it takes a lot of work we've got and I just want to begin by thanking my partner in crime Dianne Strauss Tucker our publisher and my editors Ann and Phil and Hailey and Ryan and Daniel and our business staff, Ambie, Carl, Claire so I think it's been a great panel I'm very very thrilled we could get such thoughtful and quality people up here I hope you enjoyed it and thank you very very much for coming