I also wanted to make sure that I thank Chairman Rockefeller for letting us use the Senate Commerce Committee hearing room. It's quite an honor to have an event. They don't allow many outside groups to do events like this, but I think they thought this was such an important topic. And particularly thank John Branscombe, the Chairman, Senior Counsel, and Stephanie Gammesh. I hope I'm not mispronouncing your name. The hearing clerk of the committee who just did a fantastic job in making sure that this was all set up the way it normally is not set up, which is facing this way. So let me just, again, as I said, you all have everyone's bios, so I won't go into detail. But in order of speaking, we're going to go right down the road. There's going to be brief opening remarks because I'm sure there's lots of interactive discussion and then hopefully we'll have time for a few questions out from you all before we end a couple of minutes before one. And then please, as I said before, please try to get out into the hallway so don't start conversations in here at the end, but you can obviously be in the hallway and take your, if you can, take your box with you so that we can get them thrown out right away since they have to turn this around for a hearing very quickly, which wasn't scheduled at the time that they allowed us to do this event. So first up would be Larry Krever, who is the Vice President for Legal and Government Affairs at Sprint. And then Eric Graham, C-SPIRE, Senior Vice President for Strategic Relations, who was good enough to come into town for this. Joan Marsh, is Vice President of Federal Regulatory for AT&T. And then finally, Matt Wood, Policy Director for Free Press, a non-profit media reform and consumer advocacy group that I'm sure you all know about here in Washington. So as I said, there'll be short opening remarks and then we'll get some good discussion. Larry? Thank you, Michael. And I hope everyone's enjoying lunch. We'll try to talk a little bit while you all want to eat. I think I'm going to start with something that was mentioned earlier today. And as I think it kind of brings to the key issue here, it was suggested that well Sprint has a lot of spectrum. And if spectrum is what matters, then why is Sprint's market share lower than some other carriers? What's the reason for that? And I'll tell you what the reason for that is. The reason for that is spectrum matters. Different frequencies matter. What kind of spectrum you have matters a lot. And as Mr. Stravitz pointed out from the back of the room in a dryer panel, the coverage of the United States, a huge geographically diverse company with high density and low density markets, but people all over the place. Competitive carriers generally, those without low band spectrum, cover approximately half or so of the amount of coverage the carriers with the low band spectrum are able to achieve. I mean, the fact is very simple. Low band spectrum goes further. It goes inside buildings better. And that means your capital cost, the cost to build a network, are lower. Not only the initial cost of building sites, you don't need as many sites, but then the costs of operating and maintaining those sites, which are extraordinary, including, of course, backhaul, which in most cases gets paid, of course, to the Bell companies, to their landline affiliates. So as an economic matter in terms of sustaining long-term downstream robust competition, you do need to have a mix of spectrum bands. And the way a carrier, ideally, can structure its network is to use high, medium, and low band spectrum, taking advantages of the strengths of each, trying to minimize the weaknesses of each, but build a network along those lines. Now, that, of course, is hard to do because the spectrum was never set out and allocated that way at the beginning of our industry. Spectrum has become available in different bands, not necessarily because it made perfect engineering sense, but because it was politically possible, or as a regulatory matter possible, to open one band after another. And when most of the industry was at 850 or 800 or even 1.9, it mattered far less than it does today when we now operate commercial terrestrial wireless broadband services from 700 through 2.5, and hopefully soon 600 megahertz and possibly additional spectrum in between as well. So spectrum really does matter, and the characteristics of spectrum matter. So that brings you to the point. You have two carriers which today have 80% of all the available low band spectrum. As previous panelists have talked about, that's concentration. So now we have an auction coming up of what is likely to be the last large tranche of low band spectrum in the foreseeable future. So if your goal is to promote long-term downstream competition in the wireless industry, then you don't want to see all of that spectrum basically acquired by those two carriers. You want to see a broader distribution of that spectrum. Those carriers certainly would get on and acquire some, and I expect they will, but you want to see other carriers as many as possible bid for and obtain low band spectrum because of its critical nature to building an economic structure into your network that enables you to sustain competition over the long haul and provide the services that customers want. Now, the last point I'll make in this introduction is that some have argued, well, we're really not as concerned with coverage now. We're concerned more with capacity. And other spectrum bands, higher spectrum bands, are very good at providing capacity. So yes, capacity is important, but if you don't have a signal, then your phone is a paperweight. So coverage is still very important. Capacity is important. We have Sprinter working on something we call Sprint Spark to meet our capacity needs, but here's the key. We're not just living in a capacity world. We're living in a broadband, ubiquity world. People expect their phones to work everywhere. Inside the Empire State Building, in the middle of a field, out when you're working in the field as a technician or as a delivery person. And of course, at home, inside your house. And in order to achieve that, and again, achieve it in a manner that enables you to have competitive costs to carry us low-band spectrum, you've got to have low-band spectrum. So I think that's the reason why this issue is before us. That's the reason why the commission is considering how to structure this auction to try to make the percentages as high as possible that multiple carriers will get low-band spectrum. That as many carriers as possible will get low-band spectrum. And that's why this is such a critical issue. But unlike some of the others on the panel, it's probably worth me taking just a couple of minutes to give you a sense of who C-SPIRE is. We're formerly cellular south, and we are the largest privately held wireless operator in the U.S. And we are, I guess, the sixth largest operator overall now. So we've got a little bit less than a million subscribers, and we operate in southwestern Tennessee through Mississippi, along through coastal Alabama and into the Florida Panhandle. We've been around for 25 years. And we trace our origins to some of the rural landline companies in the state of Mississippi from decades ago. And so because we've been in the industry this long, we have an assortment of spectrum, from cellular and PCS to a little bit of AWS. And we acquired 700 megahertz spectrum in the auction back in 2008. Given where we sit in the industry in terms of subscribers, we're obviously not the big guys. But we do share a few things with them. We share certain things with the smaller operators as well. But it gives us sort of a rod-defense position on a lot of these issues. And hopefully maybe a balanced perspective in some ways. I do want to take just a minute to add to what Larry just said about spectrum making a difference. And low band spectrum in particular, making a difference. I ran through the list of our spectrum holdings. And if I were to show you a map, we're privately held and I won't do this. But if I were to show you a map with pinpoints of all of our customers, you could take that map. And without knowing anything about our spectrum portfolio, you could tell me where our cellular spectrum was located. And you could tell me where our PCS or other spectrum was located just by the concentration of customers we have in that area. And because the lower band spectrum gives you that much more of an advantage over the higher band spectrum, it's stark. And there's no denying it. It's indisputable. We heard previous panelists talk a little bit about the concentration of the industry over the years. And we really have reached this sort of tipping point where we have to make a decision now, or the commission and the administration has to make a decision, of how this industry will be regulated. And before the AT&T, T-Mobile takeover fell apart, I think things were headed down the path of hyper-regulation. We were going down the path of ultimately a duopoly and the government was going to have to step in and regulate things the way they did to the airline industry until the 80s. When that blew up, it gave me hope that maybe we've got an opportunity now to reintroduce competition as the regulator for the industry. It's not something that'll happen overnight, and it will take years and years moving down that path to get there. But we really do have that choice. On the one hand, do we want hyper-regulation? Or on the other hand, do we want competition to regulate the industry? Now, C-Spires vote is for competition to regulate the industry. That's what we would prefer. And I think most operators would tell you that same thing. But how are we going to have competition to regulate the industry? It's technically true that if there are two operators, then yes, there's likely competition among a duopoly. If you have three or four operators, yes, there's probably competition there. But in individual markets, as you look at the number of competitors and the number of competitors who can effectively offer competition, that's where you determine on sort of that local level, is there true competition? Our industry, just given the nature of the capital that's required to enter, given the fairly easy ability to restrict access to the inputs to the business, is one that is particularly prone to those with the largest and the greatest market power being able to stamp out competition. It can be done through refusal of roaming agreements. It can be done through device exclusivity. It can be done through the foreclosure value of purchasing all the spectrum licenses that are available. There are any number of ways. In some instances, it can be done through backhaul. We heard somebody mention special access earlier. There are necessary inputs to operate this business. All of them are capital intensive. And it's the nature of the industry now that those with the greatest market power can, in some respects, determine the success or failure of their competition. Ultimately, the spectrum, it's yours and it's mine and belongs to people outside of the Beltway. This is a public resource. It's held in trust by the government, managed by the FCC. And this is going to be something unpopular, I'm about to say, in some circles. But we hold a spectrum license, which means we have the right to use that spectrum.  That spectrum still belongs to the people. And as operators on that spectrum, we have a public interest obligation that requires us to operate that spectrum in a manner that's consistent with the greater good for the public. At C-SPIRE, our underlying policy principle, our foundation principle here, is we believe customers should be able to purchase the device they want, choose the network provider that best suits their needs, be able to access compatible networks wherever they go, and then be able to have access to the lawful content that they're choosing. Any policy position we take is gonna come back to those four bedrock principles that form our foundation and anything you hear me say today is gonna come back to that. You'll be able to trace it back to that. As it relates to the 600 megahertz auction specifically, this is our last best chance for low-band spectrum. It would take longer than we have today to run through the reasons that 700 megahertz auction was not as successful as it could have been. It was a bifurcated auction separated by six years of one auction followed by another. Various license sizes that didn't all necessarily fit together well. The spectrum wasn't clearly fungible. There are any number of things. Some licenses were so large that in reality, only three or four people could participate in the bidding on those licenses. But what matters the most is that now, as we look at 600 megahertz, we can understand what did not work at 700 megahertz. We can take what did work at PCS. And if you wanna throw in some examples from some outlier auctions where there were smaller bands of spectrum available, we can do that too. But we have an opportunity to get it right this time. And that's gonna involve a lot of things. The band plan certainly being one of those. We can't afford for the US to have an outlier band plan like we have in 700 megahertz. Nobody else in the world is going with the US band plan. Everybody else is going with another one. Who does it benefit when the US has an outlier band plan? Whoever has the most market power. Everybody else is at a disadvantage. So we've got a chance to get it right with 600 megahertz auction. And I know we'll get into more of that in just a minute. I've already gone longer than I wanted to, but I look forward to hearing what the rest of the panelists have to say and any of your questions. I'm Julie Marsh. All right, thank you. Thank you, Michael. And thank you for inviting me to this debate. So many things to respond to. Where do I start? But let me try to keep my remarks brief as well so we can get into a dialogue. Much has been said about AT&T's low band spectrum holdings. So how did we get here? There are some popular themes for how AT&T got here. One is that we dominated the 700 megahertz auction. We did not. Of the six blocks available at auction, AT&T bid on one. One block and only one block. We won less than a nationwide footprint. We lost to other bidders in major markets. We lost to other bidders in suburban markets. We lost to other bidders in rural markets. The other popular theme is that nobody can possibly compete with AT&T at auction. It may not be able to compete with us at 600 megahertz auction unless there are limitations put on us. Again, if you look at the facts, that is not true. As I said in 700 megahertz, we won one block. There were over 200 applicants for that auction. There were over 100 different bidders, including carriers and participants of all size. If you look at the AWS auction, you see the same facts. An open auction that permitted everybody to participate. There were over 200 applicants, over 100 bidders. In that auction, T-Mobile emerged as the largest winner of spectrum. They won more than AT&T. They won more than Verizon. And they won more than the cable companies who formed a consortium to buy spectrum. I think these auctions indicate that with open auctions, you can see full participation from carriers of all size and multiplicity of bidders. Now let's look at some other facts about low band holdings. Again, the 700 megahertz auction, as we all know T-Mobile and Sprint did not participate. You cannot win spectrum when you do not come to auction to do so. I don't criticize that. That is a legitimate business judgment as to whether a carrier wants to participate at auction or not. But you must bear the consequences of those decisions and not seek to have the government address them later on. Small carriers were shut out. Again, not true. Many small carriers won spectrum at 700 megahertz auction. In fact, Eric just indicated that his company did. The fact that folks are not focusing on is that many of those small carriers that won 700 megahertz auction, 700 megahertz license at auction, have subsequently sold 700 megahertz licenses on the secondary market. I don't criticize that either. If you are not going to deploy your spectrum, you should sell it. AT&T participates very actively in the secondary market. But I will take issue with a carrier who chose to sell their low-band holdings on the secondary market and then come to the government and say, we don't have low-band holdings. Therefore, you should provide us advantages in the next low-band auction. In fact, I could provide a very, very long list of carriers who have sold their holdings in the secondary market. And in fact, that secondary market is still active. Verizon holds a significant amount of A-block spectrum in 700 megahertz that's been on the market for two years. There are now some rumors about who may be interested in that. But the fact of the matter is the secondary market is still trading on low-band holdings. And any carrier who wants low-band holdings can go there to acquire them. So second, are low-band holdings necessary to compete? I certainly think they are not. We could talk a lot about this, but I would point as evidence most recently to T-Mobile's last two quarters of performance. They are competing aggressively in the marketplace and successfully in the marketplace. In the third quarter, they had more ads than did AT&T. There is fierce competition out there. My hat's off to T-Mobile for what they're doing out there. Analysts have indicated they're wreaking havoc in the marketplace. But again, not an indication that their spectrum holdings or the lack of low-band holdings in the portfolio are preventing them from competing effectively and aggressively in the industry. Where do we go from here? I think for a lot of reasons that we can probably can discuss further two in questions, AT&T will impose any low-band limits. I don't think the limits are appropriate or necessary. I think such limits will protect some carriers from business decisions that they've made in the past. I think such limits will punish other carriers for doing exactly what the government has wanted us to do, which is to buy and deploy spectrum for the benefit of providing broadband services to the public. We will also oppose any auction limits that seek to limit or disadvantage our bidding while they favor some other bidders' activities. Again, I think that those are unfair limits necessary. We have seen open and fair auctions can produce a multiplicity of winners and ensure carriers of all sizes and participants of all types can win spectrum. We appreciate, however, that some feel the FCC must ensure that not a single bidder can run the table. I think that that has been discussed as well, but the proper way to do that is through an auction rule of general applicability that applies equally to all bidders. We would take a serious look at any such proposal if it was made. No such proposal like that has been made yet. The proposals that have been made would disadvantage AT&T and Verizon to the advantage of other participants and would seek to put different limits of different types of limits on us, some limits so severe that they could effectively act as an exclusion when not limiting other players. And in fact, I think that the proposal most prominently discussed is T-Mobile's dynamic market rule. And while that would put pretty significant limits on AT&T and Verizon, it would, in fact, let T-Mobile and Sprint and other carriers run the table if they so choose. I think that there would be possible consequences, negative consequences for any rule. And I think George's comments point that out. There was such a similar rule apparently in Thailand, and you saw that it suppressed bidding activity and suppressed revenue. We always have to be concerned about that. But we feel that any rule, if any rule is going to be adopted, has to be a nondiscriminatory rule. And it has to apply equally and fairly to all bidders in the auction. Finally, I just want to talk a little bit about the consequences of a proposal where you set rules and then do divestitures after the fact. And it was indicated that that would be a disaster. The fact of the matter is that's exactly what the FCC did in the 700 megahertz auction. After the auction was over, against their spectrum screen, they did a review to determine if there would be any negative competitive impacts. The fact is they did not issue licenses until that review was complete. As a result of that review, Verizon did make some voluntary divestitures and then they made issued licenses. We got our licenses later than everybody else by about three or four months, because that review was underway. But they accomplished that review. It was a process that was manageable. And the licenses were not issued and then tied up in litigation for years. The fact of the matter is they held, they withheld the licensing until that review was completed. So we do think that that is a workable process as long as the spectrum aggregation rules are clear out front. We cannot change the rules during the auction or after the auction. Everybody must go to auction understanding what the rules are and bid appropriately, understanding that post auction there may be a review in terms of whether there'd be negative competitive impact because of the acquisitions. So I will stop my comments there and yield to my colleague, Mitt. I've just been elected, apparently. I have to say too, it's so interesting to sit next to Joan and hear her extolling the benefits of T-Mobile Spheres competition in the market based on the history there. I thought Kathleen would like that. As Michael said, it's an honor to be in this room and it's actually kind of a comfort to be facing in this direction too, rather than in the other direction. But thank you all for coming and I will also try to be brief as I know we're past lunch and past all of the cell phone crinkling and getting ready to wrap this up. I want to have some more dialogue here and hear some of the earlier panelists' responses to some of Joan's points. I'll just, I'll say a couple things and start with the plain words of the statute. Those haven't really been in dispute today. I think the rhetoric has been toned down from where it was, at least by most people, a few months ago, but the statute says you cannot participate any person from, or prevent any person from participating in the auction as I think a lot of people have made clear nobody's talking about keeping anybody out of the auction. The statute also says that the FCC retains the power to promote competition with spectrum aggregation limits of general applicability. So just to echo some of the things that Joan said, we are all in favor at Free Press and with a lot of our public interest allies of general applicability of non-discrimination. Another interesting word to hear coming out of certain companies' mouths, of general applicability that means, we have actually not proposed intra-oction limits in our comments in the docket. We said in a different spectrum aggregation docket that we would like to see caps across the board that would apply to all different carriers that would look at their existing holdings and apply in the context of mergers and acquisitions and also in auctions. So obviously there's not complete agreement about that, about the right way to get it, but most people are in complete agreement that the FCC has a need to use that power to promote competition with some kind of general, generally applicable spectrum aggregation limits. It's not rigging or ending auctions, it's not doing anything that would dampen competition in the long run. I think Tom said competition isn't controversial. I think it has become kind of controversial or at least confused in some people's presentations of it because we're not here to promote competition in auctions. That is not the end game of the FCC or of DOJ or of the statute. The end game and the goal is to promote competition in the wireless market because that's what benefits consumers and if we have any staffers here, you're a constituents. That's what this is all about. It's about promoting competition because that lowers prices, improves service quality, improves choices and gives those people out in the real world a chance at better service, at better prices. Now we're not here to talk about how does auction design work and obviously that's something that's very important and feeds into that process, but that's not the end game of the process. So the question then becomes, do we need any sort of ex-anti-regulation? Do we need to have some kinds of rules in this auction or at the FCC in general about spectrum aggregation and concentration or are things working well enough? I think there's a little bit of a bait and switch sometimes when it comes to antitrust, not just in spectrum in another context, you'll hear people say, well, we don't really need the FCC to do anything. We have antitrust to take care of it. I can't do better than Congressman Pickering did and Chairman Hunt in talking about the problems of unwinding concentration after it occurs or unwinding any other sorts of harms of concentration after they occur, but I would also say, let's look at how the market is functioning today. Do we really have effective competition or not? Or do we need some kind of ex-anti-solution as the FCC can do to promote competition to this kind of excessive concentration in AT&T and Verizon having maybe too much spectrum to have really meaningful competition? I think the answer there is clearly yes. Mark Cooper put up a bunch of stats that suggested there was market power being displayed in all these different indicators and the response to that was a little bit of, well, let's forget the merits and let's talk about principles, but I think that the merits show that there are problems in the marketplace and that we see all these sorts of harmful parallel pricing and parallel conducts being undertaken by AT&T and Verizon. So if you look at the mobile share prices, prices are basically the same across both carriers, across the board as you go up the ladder of the number of gigabytes they're selling you per month. These are innovations in a sense. They're innovations that they're often mirrored by the two companies with the most market power and they're basically doing the same things and almost identically matching each other's prices as they go up. So other speakers have said this, but yes, you get some competition when you have two. Is it really effective competition? Is it really meaningful competition for consumers? Is it competition on price and not just simply on the bells and whistles? I think the data shows that it's not always effective competition. It's not actually lowering people's final bottom line and the bill they're paying every month and that's why the FCC is fully justified. DOJ is fully justified in saying we need to look at this in advance and again, not just to promote auction competition but to promote actual competition in the wireless marketplace going forward. So I think I'll stop there. Happy to take questions and have Michael have us trade off up here. I'd be remiss if I didn't mention unlicensed as well, which is also part of this auction. That's not really the focus of today, but we have called for the proper mix of auctions and unlicensed spectrum and shared spectrum going forward. Kind of to Tom Powers point about the PCAST report. It's not about one single solution to solving our broadband needs going forward. It's about getting the mix right. And so that's also part of this auction. It's something that we and of course OTI have advocated on but the focus today is getting the auction part right. And I think it's clear the way the FCC needs to proceed to do that. Thanks to all of you for being so concise. And I guess I have a couple of questions in mind, but is there any immediate kind of quick immediate rejoined or anything you've heard that for many of the other panelists that you wanted to, I don't know respond to, so I don't want to necessarily anticipate what's the hottest point of contention here, Larry. I'll just make a couple of quick points. Joan mentioned of course the fact that there is low band spectrum that has apparently been on the market or at least some have reported it to be on the market and mentioned the Verizon A band spectrum. That's six by six megahertz. That's not a lot of spectrum. That's great and I hope somebody buys it. If in fact it's available, I don't really know, but it's six megahertz. In the 600 megahertz proceeding, we're talking about potentially quite a lot of spectrum, maybe 84 megahertz, maybe more, divided into five by five megahertz blocks in all likelihood and the ability to assemble multiple blocks. Now when we're in an environment and a data demand from customers situation where people are, where carriers are building 10 by 10, 20 by 20 and even larger blocks of spectrum to try to provide a mobile experience that is what the customer is demanding. Six by six is not necessarily to be on end all. So I just think you have to keep that point in perspective when you're looking at, is there a proper role for the FCC to structure this auction so that there's more potential for competitors, more competitors to get low band spectrum. The second issue is the 700 megahertz auction and the comment which we hear constantly that some carriers didn't participate. Well, Sprint did participate as part of Spectrum Co, but more importantly, I think is the world has dramatically changed since that auction was held. At that time, less than 10% of the marketplace was buying smartphones today, it's 70%. The nature of communications using mobile communications changed dramatically from voice centric to data centric to films and everything else, massively different usage. And if this were another industry where you weren't using a scarce resource that is as we've discussed many times today owned by the people and you have to get a license for it, if this was vacuum cleaners, well, who cares, right? But here you have a resource which is scarce, you can't make more of it. It is administered by the government by half of the people under a law that says promote competition. Therefore, it is always appropriate looking at releasing more Spectrum or repurposing or reallocating Spectrum to see whether there needs to be a structure to that release in auction in cases today that promotes competition. And so the fact that certain things were the case five or 10 or seven years ago isn't necessarily relevant to the proper public policy decisions today. So I would just make those couple of points to perhaps more fully bring out those comments. Any? Just to vote off that for one second. Everything about the lower 700 megahertz A block is of course it's been under this cloud of interoperability dispute and contention for quite a long time now. So maybe we're finally reaching the end of that and that will sell better, but because of the interference claims and engineering issues that were argued about at the FCC for so long, I think that's obviously a fair test, but it's not the only kind of test. We need to run to talk about how well the secondary market is working for sub one Spectrum. One thing I wanna go back to is Joan brought up the issue of think of whether whether there should be upfront limits on the upfront limits on what you can acquire, whether it would be, for example, in the incentive auction, what it would be, just a share, for example, the FCC floated the possibility in the rulemaking, there could be a one third limit of the Spectrum in the incentive auction or alternatively, we talked a lot today about there could be a one third or other limit on the total amount of Spectrum below one gigahertz if you wanna focus on the unique propagation characteristics. But in any event, this issue of having an upfront limit that would kind of put a ceiling on the amount and it would be market by market, presumably, that's a good thing for someone to explain that you could bid on in this auction or instead just allow the auction to proceed on an open basis and then have the FCC apply a Spectrum screen after the fact with the possibility of the vestitures. So I know you're an opposite side to that but it would be good to have some of the reasons, some of the trade-offs there are the reasons why upfront versus after the fact is good or bad from your perspective. I'll take a stab at that because our perspective, obviously would be different than the four largest operators and after the fact decision on what licenses we're able to retain after an auction, it would have us question whether we participate at all. For an operator our size, we go to our bank group to secure capital so that we can participate in an auction like this. And once we secure that capital, we begin paying interest on that capital. And going to an auction, we have to assemble a footprint of spectrum licenses that makes sense for us to operate and we can't just cherry pick areas around the country and know that we've got a business plan that'll work there. We have a limited footprint, a limited geographic footprint. And we have to make bidding decisions that are rational with that footprint, expanding that footprint, overlaying that footprint, whatever the strategy may be. But we have to know, going in on day one, that what we end up with in the auction are licenses that we will actually keep. If not, for somebody our size and smaller, the whole business plan can fall apart. So the idea that we would go through, we would compete, we would bid as if it were a game of monopoly and then at the end, we'll all sort it out so everything's even. Doesn't work when we're using bankers money and we're paying interest on that, keeping us from doing other things with that money. And then not knowing until sometime after the auction, did we actually win or did we in fact, think we won and we lost? So let me address each of those because they're very different. The after the fact approach is exactly what the FCC has been doing for years and it's worked effectively. They do it in all their transactions. They use their spectrum aggregation tool to measure the competitive impact of the spectrum aggregation being undertaken and they make decisions based on that tool. And as I indicated, it's the tool they use in the 700 megahertz auction. We do think it needs to be updated. There's a lot of spectrum out there that it currently is not accounted for in that tool, the S-band that Dish has. All of the clear wire spectrum that Sprint has now aggregated and is using to deploy broadband services. So the tool definitely needs to be updated but it's worked, it's worked effectively and we think it can work in the auction construct and in fact it did in the 700 megahertz auction. Upfront limits, you mentioned two flavors of upfront limits. One that would put a limit on total low band holdings. Understand very clearly that depending on how that limit is defined, it could act as an exclusion. For a carrier like AT&T that has low band holdings because we participated in the 700 megahertz auction because we have worked aggressively in the secondary market to acquire additional 700 megahertz holdings to rationalize and fill out our footprint. If that limit is set too low, it could potentially exclude us from bidding in multiple markets because we would have no ability per that limit to hang on to those licenses. And I think Verizon is indicated in a filing that they are in the same place. Now an upfront limit that is an intra auction limit that would put a limit on how much any one carrier or bidder could acquire at auction. Again, we would be concerned about the consequences of that in terms of whether it would suppress bidding activity and suppress revenue but ultimately if that's what you're concerned about in the auction is ensuring that no one bidder obtains all the licenses, that is the better way to go. They recently did that in New Zealand and I did a blog about that a couple weeks ago. That is the fairer and more nondiscriminatory way to approach the challenge. Well, I think you can get there a couple of different ways. Whether you do it as a cap or limit for below one gigahertz or you do it in the auction, obviously there are different incidents of which way you do it. I agree with General on one thing. I think the rules need to be clear up front. I think the commission has to establish clear rules so that you can bid and you can obtain capital to bid understanding what you're bidding for and what you're gonna be able to keep. I think the point from C-SPIRE is exactly right. Many carriers can't go out and fund an auction and then worry about whether they can keep it or not. Now, as Joan has suggested, the spectrum screen is outmoded and broken and it needs to be fixed. I'm not sure that that's the right way to solve this. I don't think it is. I don't think after the fact divestitures work. If it's more complicated than it's been in the past and it drags on for an extended period of time, some carriers may have access to the spectrum, others don't, and it's critical when you're obtaining spectrum that you be able to deploy it in a competitively rational timeframe. If you're sitting there still trying to see if you can keep it three or four years out, your disadvantage is huge and that makes no sense. That's not good public policy. So I do think we need clear rules. I think that's what the commission's attempting to do here. I think once we have clear rules, then you enforce them in the auction and everybody should understand what they are going in. I do agree that no carrier can be excluded from the auction. I think the statute's clear on that. So in setting whichever approach you use, it does have to allow all carriers to win something. The question is how you deviate up, given the fact that two carriers today have 80% of an essential scarce resource that is essential to downstream long-term competition. I want to add one. Oh, go ahead, Eric. I want to add one thing, because it occurred to me as Joan and Larry were talking about the pre-auction limits versus divestment spectrum afterwards. I've run the auction for C-SPIRE. So real time as the rounds are going, I'm leading the team that's making those decisions on bids. And one thing we have to consider is whether to stay in the auction. I mean, it's a basic question and it may just be assumed that once you enter the auction, you're in, but that's not the case. And so envision a scenario where there's no pre-auction limit, but maybe spectrum will be divested afterwards. It's entirely possible, if not likely, that a competitor in that auction is acquiring spectrum I'm interested in, yet they'll have to divest it later. And because they've acquired spectrum that I would otherwise be interested in, I can't form the footprint that I need in order to make the business plan to go, so I withdraw from the auction. Well, now I'm out of the auction, otherwise I would have been a participant and the spectrum is freed up later because the bidder who got it ultimately was not allowed to keep it. That's a reality that you can face with a post-auction divestiture that if you could avoid, if you have a pre-auction limit on the amount of those licenses you could acquire. Well, they may decide they don't want to sell it to you. Right. Yeah, that's right. Exactly. Oh, man.  I want to get to some of the questions if we can, but just to say, you can have clear pre-auction rules that still apply after the auction ends, and I think the key word there is clear. I mean, certain kinds of rules are not particularly clear if you have a spectrum cap that may or may, you know, can only be exceeded by waiver. That's different from having a spectrum screen that may or may not trigger divestitures. So I think, you know, it's about getting the policy right, too. We can say we want to have clear rules in advance, but unless those rules are actually clear and actually non-discriminatory and actually promote competition, then, you know, it's not really giving us anything to have it written down in advance, even if that's better than just figuring it out after the auction runs. Mm-hmm. Since we just have five or so minutes, want to see if there's questions out here in the, for many of you. We'll think about it. I'll ask a quick one in the meantime. It may be quick, I hope, but, you know, this came up way at the beginning of the event. There's obviously been a lot of, a lot of the publicity has been about, wow, how much money is this going to raise? And particularly for FirstNet, because, you know, the statute provides that, you know, the first seven billion in that revenue from a number of auctions combined, right? Not just in the TV incentive auction, but some other auctions would go to FirstNet to fund the Interoperable Public Safety Network. So I was wondering how likely do you think it is that this debate has been largely about the sub one gigahertz spectrum, the TV band 600 megahertz spectrum, but how likely is it that, I guess, A, these other auctions will occur first at this point, and B, that they will raise enough money to take care of FirstNet so that whatever policy decisions are made about the incentive auctions, they can be made kind of without that overhang of first responders, you know, whether or not they're going to have their, you know, the seed money for their network. Well, we know the H block is going to go forward. I think the chairman made that very clear last week, and we have some sense of how much revenue it might raise because of the reserve price. And the AWS three, I think is still a big open question because we don't, we have not nailed down the pairing and how much revenue can be raised at that auction. I think we'll be very dependent upon whether we can nail down the appropriate pairing and be able to auction that as viable paired spectrum. So certainly there's an opportunity to sort of start seeding the FirstNet pot with those two auctions, but we certainly don't think it will get done and revenue will continue to be a very important goal for the 600 megahertz auction to meet all the statutory goals identified. And that pairing depends what largely on what DOD is willing to do on 1755. So 1755 to 1780 is the target. There's been a lot of progress. The industry is working jointly to try to work with DOD to find a way in which we can establish a clear path toward them clearing enough of that to create a viable pair. But there's a lot of challenges and hurdles and getting that accomplished within the statutory timeframe identified by Congress. But the reserve on the H block is what, 1.5 million? Is that right? So that's 10 megahertz. AWS three, if it's paired correctly, it'll be 50 megahertz. So I mean, you can't guarantee these things, but it does seem like that would fetch a much higher price than that. And H block is going forward unless something catastrophic happens. And I can't help but think AWS three will happen before this whole incentive auction process is all figured out. And the fourth auction, 1695 to 1710. Is that likely to happen before the TB and auctions? Yeah. Also difficult to tell. I do think though the AWS three has the potential to raise potentially all of the first net money, but the key is that John said it exactly right. The key is the pairing and whether DOD agrees to move off enough of that spectrum or with limited exclusion zones to the point that carriers will see this as a really valuable paired band. It's certainly in the right place in the spectrum. It's good spectrum and bid its true value. So if that happens and it is what 50 megahertz, that could be raised a lot of money, but it's dependent on those contingencies. I think those options are important for also the implications on any restrictions on the 600 megahertz auction. As Larry said, it's a lot of spectrum. AWS three is potentially 50 megahertz. We don't know who's gonna buy that. We don't know who's gonna add that to their spectrum for portfolio. And we don't know how that's going to impact their decision about whether and how they participate at 600. So again, identifying limitations right now on an auction when we don't know what the future looks like and we don't even know who's going to show up is a very dangerous game and then you have to have certain revenue requirements met to make this auction succeed. The auction cannot close if at a minimum, we don't raise enough money to pay the broadcasters for what they're prepared to surrender and other statutory goals. And in that same vein, as you look at the 600 megahertz auction structure, I think it's critical that we conduct that reverse auction first, figure out what spectrum is available and how much it's gonna cost before you begin the forward auction. I don't even think you can craft a band plan until you know what spectrum is available. So as we talk about how the forward auction is going to work, where operators will bid against each other for these licenses, we don't know how much spectrum is available, which would I think have a major bearing on the size of a pre-oction limit on bidding. A third could be too much, a third could be way too little. You gotta figure out what's available to bid on before you determine the rules of the auction forbidding on that asset. Yeah, that's a great point. A third of 40 is gonna be a big difference than a third of 80. Yes. And depending on band plan, you may have no paired spectrum at 40 or you may have three blocks of unpaired spectrum. You may only have supplemental downlink. And the differences among the band plans, the three major band plans that are out there right now are very pronounced depending on how much spectrum is actually cleared. So trying to figure out the band plan when you don't know what you're gonna get is certainly complicated and just adds another level of complexity to which is the best way to go. Well, I think without burning questions and because of the hearing that's coming up, we'll end it there. I wanna first thank the panel. Thank you.


Hey everybody, I think we'll get going in just a second. I wanted to mention real quick Amanda one of the organizers came in and mentioned that the fire marshal has been shutting stuff down if too many people are standing So let's pack this house. We're psyched to have everybody here, but uh Beware Yeah, it looks like a couple up here Yeah, seriously, thank you so much all for being here five o'clock on Wednesday everybody's ready for a beer and you're here with us Yeah, thank you Didn't say we're providing it That's why we're gonna go quick Let's get going So welcome. This is agile with a lower case a art of collaborative PM Just a little housekeeping to get out of the way real quick I'm sure you guys have heard this all day, which is great because it's super important Make sure you join us for a contribution sprints on Friday all the information is up there Also all on the website Devs UX everybody's welcome come help make triple great Greater So I'm Nick Schweitzer I'm the development director at elevated third I manage our development team and place sort of a solutions architect lead dev role on most of our projects And I'm Kylie for Sunita. I'm one of the account managers at elevated third. So I manage internal and client teams and Our company is elevated third. So we're a digital agency based in Denver that helps organizations solve their technology And marketing problems. All right, so let's get into it. Why are we here? Before we jump into the content, let's do a quick survey. How many project managers in the room? nice devs design Other slash sales business Anyone do an agile right now or some version of it And also awesome right on well, we're really excited to have you guys here with us talking about it So again, thanks for joining us So we're here. We want to help cure your post-discovery hangover I'm sure everybody's been in this situation You get the client you work through discovery You got a brand new exciting project and you're super excited about all the great features you're going to implement and you haven't quite thought about implementation and Teams ready to go and you don't want to lose that momentum that you've gained through the discovery process So just to give you guys a little insight into what mean what we mean when we say a discovery process for us that's essentially a few weeks to a few months that we spend with our clients just determining what they are who they need what their requirements are and Like I said a couple weeks to a couple months But we'll typically make that a really collaborative process So, you know like we'll talk about through implementation. We like to involve our whole team. So that's tech design strategy will always have sales in there to sort of pass all that information along and We'll get that gift off the screen. That's distracting. There's a lot of gifts. So I apologize. I went gift crazy But we are making a few assumptions just sort of about where we're at because like I mentioned We are going to focus more on the implementation process So, you know, like I said, we do our discovery process up front And we really put a lot of effort into making sure that's a cross-functional team involved in that So, you know cross-functional is a buzzword. What does it mean for us? It means we've got at least one person from each department and They're actively contributing to the project. So you're not just sitting in meetings not doing anything We're actually working together having collaborative meetings and producing documents and deliverables for the client together Second you're coming out of discovery with a pretty solid set of documentation That's ready to go and implementation and that doesn't mean it's all complete and totally finished We'll we've got a whole section about documentation. So we'll definitely get there But those are just kind of our two big assumptions coming out of the discovery process Awesome. So that's why we're here now. Let me tell you about where we were So we saw the lady on the roller coaster. Everyone's coming off of this great post discovery high they're looking for some direction and What we found out that was happening with our projects is that we'd come off of discovery and that momentum would start to lag We'd start to plateau. We were getting our work done, but it wasn't necessarily the easiest way We were finding that that momentum was starting to not decline, but just hit a plateau So we weren't making as much progress as we should be making or as we wanted to be making So what we really were looking for was a way to find a clear direction for a more unified approach So get the team all working together in a unified way that was efficient and maximized Maximized our budget and our dollars and just produced better work So we thought to ourselves there has to be an easier way How can we satisfy client contracts and budget needs but still keeping a key focus on collaboration and a priority on team involvement? All right, so before we get into the specifics Why is the a lower case for us the a is lower case because we are not practitioners of a specific methodology We're not scrum enthusiasts. We don't love Kanban We just think agile has a lot of great stuff to offer and we really appreciate the core values And I think we try to bring that into a process that works for our team at an agency And you know, of course, that's not the same for everybody everywhere So, you know kind of that same point I we take agile very literally it's it's quick and well coordinated in movement And and that's really at the core of what we like to do in our process, you know Regardless of what we're producing who we're meeting with what the hours look like we wanted we want to do it fast fail fast fail often and create great software And and so to do that we take the best parts of agile and we modify them to fit our needs and We we like I said we like those core concepts But that does tend to evolve a little bit across projects which we can talk about a little more when we'll we'll get to sort of More specifically how we implement those values and make those a part of our process and and we found that you know It does tend to scale pretty well Whether you're talking about sort of a small implementation team of a couple people or maybe up to a larger team with a more enterprise level budget So the analogy I personally really like is the moneyball analogy We talk about this around the office all the time and the general idea is you don't want to blow your whole payroll on a superstar Right like you don't need a superstar in one role to have a successful project You just need to make sure your whole teams invested Understands their role and their responsibilities and expectations on the project and how they personally can make an impact and know when to step up and when it's their time to sort of shine on that project and And what that creates when you have the right people on the team is is a really great self-organizing atmosphere right where um you just get really great organic ideas and and Yeah, everybody comes together to make something really nice So now that you kind of have an idea of the platform that we're rooted in and what we're striving to do with that platform Where do we start? So just basically Nick kind of alluded to it earlier We really looked at the agile manifesto when we were talking about implementation and ways to make it better And those core values of them of agile manifesto Really resonated with us so we took them more as their literal meeting but then Bent them to fit our process and our needs for our team and then keep evolving them for each project not every client's the same Not every project's the same not every budget's the same So it's just taking those ideas and those core values and then molding them to fit your needs So starting with the first one meetings and communication. We're in a communication industry all we do is communicate all day long But let's focus on to What that is keep it simple so taking individuals and in and interactions over the processes and tools So a lot of people are familiar with agile in this room as we previously noted So we'll be using words and phrases like sprint product owners retrospective So sprint being a span of time to implement for that implement implementation phase Which we use two weeks friends, but there are projects where we've used three week sprints because that's what we needed But that's essentially the sprint meeting and then a product owner So having that client liaison and that key person who is your client lead and Incorporating them into the process as a product owner so that they're owning and are just as invested in the project as you are And your entire team is and then finally a retrospective So this is just a meeting to get your core internal team together and that project team together to talk about What worked well what didn't work well? And what can we do to make it better and more efficient because as Nick talked about with the flock of birds image It's really about creating that self-organizing and self-managing team so that if one person can't be in a meeting The meeting doesn't fall apart. It needs to keep running. It needs to get efficient So that's really the focus of of what we're talking about here So Communication tools consistency again rooted in keeping it simple and keeping your tools simple find what works for you When your team and stick with it it doesn't need to be complex You don't want to waste five minutes of a 15-minute stand-up figuring out who's calling in what's the number? I don't have a WebEx. Who's what's the screen share? It was time. It's inefficient It's just annoying so find what works for you find what works for your team and make it predictable I do things such as even have an entire meeting room keep that same meeting room for the entire project because You don't want to waste time or keep it just keep it easy for people like if they know where they need to be at 9 a.m. They just go there. It's easy. It's simple. Nobody wants to be in meetings But they're necessary and it gets them in and out and on on with their day and Then communication is streamlined and direct so something we also do and enforce on the beginning stages of our project is Just have a set of expectations. This is straight talk. We're honest with each other Everyone has a clear understanding of the expectations for the project and the meeting rules In these meetings, we don't want to spark notes of your entire day like just give us the facts and let's move on Just be honest and direct with each other And then having the right people in the right room No one wants to be in a meeting that they don't add any value to and then likewise your budget doesn't want that person in The meeting either so have the right people in the right room at the right time Use those stand-ups to discuss just the key things for the day So each person gets two minutes talk about what you're working on what you're working on tomorrow and any barriers You're facing if you can get those three things down with a room of five people You can be in and out of your stand-up within 15 minutes or less Again eluding back to retrospective So that's the time when you want to air your grievances or air What things that you felt it could go better for the next future phase? And that's again what you want to talk about when you're when we're looking back on our past sprints How can we evolve this sprint and make it better? We always want to keep striving to be better. So Right people right room right time live demos, I think that these are really scary, but they're really necessary so They're important for internal and client teams Internal teams because you don't want your internal team to be in the dark and this goes for a deaf design and strategy It's good to have these live shares and it keeps everyone in the same room on the same page so it's a great forum for Developers to call out any concerns with UX or design features or Also, it helps the design team and the account team stay on track and on pulse with where we're at with the project And nothing is more scary in the world as an account person than going into a client live demo without seeing the work before It's like going into a dark scary room and not knowing what's going to happen in eternal lights So you don't want that so it's good to keep all of the team on the same page And on track with where we're at For client teams so having these live demos for your client team is awesome And then going back to the internal team as you're sharing with the client The internal team knows what to expect and they can set that expectation and that precedence up Right off the bat for the client. So if there is a bug we can say hey, here's where we're at We are here. We know there's a bug here. This is the expectation. So it doesn't set up any ambiguity For that for that product owner it also keeps our product owner involved in the entire process and it keeps them invested it allows them to Draw any concerns that they have or as they're talking about it with their internal team bring any additional outside concerns in and then If that creates additional features, we're able to add those features into our sprint and prioritize as needed So it keeps everyone on the same page, which is great And then also these live demos for the client team also involve Get the client more involved which allows them to be more familiar when we could get to the testing phase So they're already familiar with your product and and what it's going to do and how it's going to function So finally take away here from communication meetings are boring, but games are fun So what we like to do is gamify our meeting So for our internal stand-ups we set the expectation up front that if you're late for the meeting or if you're veering off topic Two things that are that run meetings over you owe a dollar. So it's fun to keep Keep track of this. It also keeps the chemo count accountable So it keeps yourself accountable because you don't want to pay a dollar and keep everything off track And it keeps your internal team accountable for you So if you walk in third like two minutes late, you know, you're gonna owe a dollar and your team is going to call you out for it You can also do something fun with the money at the end right now all of our projects only have like six dollars in them And so maybe we'll split a 40 or something, but But it's fun to just keep it. It's good to have a low low dollar amount All right, so Next up we'll talk about planning and scoping and and agile value we thought we really connected with here was customer collaboration over contract negotiation, right because Contract negotiation is stressful and intense and lots of paperwork, but collaboration is great You know, everybody's involved and and you're all working together to produce something that everyone agrees is the right solution for the problem So, you know, this is something we like to involve the client and like assuming they're comfortable with that and and you know Want to be there. It's I think a huge benefit to the project to pull the client in for planning and scoping whether that's you know Your sprint planning meetings every couple weeks or just sort of the the initial sprint planning where you group features across your sprints Super important and I think you know, it's also really key to remember in these situations You know, most people aren't super familiar with agile methodology software development Like it's it's really important to be a coach and and really teach them to be a good product owner Just like the always lovable coach Taylor And and second don't set sail when the boat isn't built So collaboration is great, but don't let that replace having everything defined So it's really important that through the planning and scoping phases the internal and client teams are aligned on what the features are What the budget is what the user story priorities are and what the user stories actually are Before you actually start building and you know, of course these things are going to change through the process But it's important to have that common baseline So everyone's speaking the same language as you move into implementation on the project Additionally you need a good way to manage that stuff a budget and sprints aren't going to do you any good if you aren't keeping track of them And you know, I think even more important than a specific tool is just make sure your tools make you more efficient Similar to what Kylie talked about with communication like if the tools are getting in the way there You're probably not using them right. Maybe it's not the right tool for the job And I think also loop your client slash product owner Into those tools as much as they want to be you know, and I mean that of course across projects That's going to be varied levels of involvement But you know, I think really the takeaway is like there's you know, there's nothing to hide there And we want to be organized and on point and sort of share that expectation with everyone on the implementation team So for us those tools kind of the two categories that we really needed to fill are basically budget and hours management And then task and sprint management and we have two tools that we really like for those One's called Maven link for budget and hours management and a sauna for task and sprint management We won't get into those two in depth, but if anyone's curious like we're more than happy to talk about those afterwards So try this In your planning and scoping share budget progress and current risk with your client in a weekly meeting, you know And I mean I think this Isn't something that may be super natural to be checking in on this all the time But you know, it's important and this really helps prevent a lot of surprises And I think in terms of how you actually do this it can take a lot of different forms But really you know like like we've been saying over and over and over kind of pounded India the whole time It's just like make sure everybody's on the same page and do it early and do it often so next up is documentation and I think this is This is the piece that I think for us tends to have sort of the most flexibility Just in the sense that you know being in the agency world and sort of having clients and being accountable for Deliverables and hours. It's it's tough to always sort of say. Oh, yeah, just look at the style guide trust us It's gonna look right, you know So it kind of that that piece will evolve a little bit sort of depending on the client and project needs And and the agile value we really connected with here was was working software over comprehensive documentation So, you know, obviously this one is absolutely a compromise between those two things But I think we just sort of always keep that in the back of our mind that in the end We're working to produce working software does the things we're handing over sort of serve that end goal And you know like I mentioned, it's it's a balance, right his shared understanding is great, but shared understanding is not always easy, right and you have to make sure to really Hammer that home and and sometimes the best way to do that is documentation and and to write it down and and you know Like we mentioned, it's it's okay for that to change across projects. I think that that's something that You know, you can almost factor into the planning and scoping phase, right when you're planning out your sprints and deciding on your features You can sort of build those deliverables into that process so that you know Everyone involved can essentially look at your timeline and your milestones and see what those deliverables are going to be And if you need more you can add them if you need less you can take them away And then when it comes to actually Producing those documents. We find it's really important to prioritize living documents that are maintained and useful over throw away Which you know, I think sounds kind of obvious and seemed obvious to us But in implementation, it's a little bit tougher, right because you want to make sure Sort of everything you're producing in that discovery process that we mentioned earlier can carry you all the way through the Implementation phase and that you know, you're not sort of building a comp and then Titling it v1 and putting in a folder called old for everyone to find in two weeks and get really confused what it actually means So for us just to talk a little bit about I guess sort of our foundational documents and This is definitely not an exhaustive list of every single thing that we produce over the course of a project I think just sort of in our experience This is what we found are kind of like the key documents to get the team from discovery all the way through implementation so the first being user stories which are great to prioritize functionality and Just sort of start to get people on the same page about what are the user roles? what are the different features and You know those those can take a lot of different forms that you can you know if it's new you can go as low-fi as a spreadsheet If you have a more complex larger projects, you know a tool like Jira or Trello might be right for you I think that's just kind of finding the right size because it's definitely not a one-size-fits-all situation and then We have a risk mitigation doc as well Which is super helpful because this is essentially where we keep track of any current risks to the budget and scope of the project So I think what's really key about this is you don't want to frame it as a negative thing. It's not a scary document It's just a way to keep track of this is sort of how our project has evolved and you know a lot of times We'll have hours at the end of the project and we can squeeze those features in and and you know Those essentially can become icebox items, which we'll talk a little more about Later if you don't get to them before lunch Build spec this is just kind of what we call it at our company This is essentially a content architecture plan so this is really key especially in the Drupal world because we do a ton with very structured content and It's a way bigger headache to change field names and change architecture Once you've built it in Drupal and built functionality on top of it Then it is to just define that all on a spreadsheet and show it to the team and know okay Like I know this is what an event's gonna look like um Feature estimates which you know this is essentially a feature list of all the major features, you know some Practitioners of Agile might call these epics And you know, I mean I think this can take a different form sort of depending on each project But we'll typically use feature estimates as sort of a comprehensive feature list and then also an estimate for implementation that turns into sort of like smaller Defined tasks that we can move into our task manager and then essentially have estimates associated with each one of those tasks and then finally I Kind of group these all into one category just because it sort of ends up being some number of deliverables Wireframes obviously very important to define the look and feel and those are really great to produce in tandem with the build spec Because you obviously don't want to put anything on the page. That's not represented in Drupal And then style guide and designs like we said it gets kind of ambiguous What will actually deliver for that sometimes and a lot of times we're able to take the approach of? Sort of I guess MVP design if you will where we can produce a style guide and sort of a component list and then you know Oftentimes there are just maybe less technical people or less involved people on the product owner side Who just need to need to see like full page comps and you know at that point We've got all the components together in a style guide and we can just kind of build those into page comps and then give them something to look at So try this Like we mentioned if you haven't done a lot with user stories Like full disclosure. We're fairly new to them and it's been a huge huge help to our process and you know We just started using a spreadsheet because we just knew we needed something because having something was better than having nothing And and in the spirit of maintaining a living document Turn this into your UAT document when it's time to test you can essentially just repurpose everything So that way you're testing all the same user stories You've been talking about through the whole process and we'll talk a little more about that when we get to the UAT phase Cool, thanks So our next value that we really focus on is balancing the big picture with the little details Which is our interpretation of responding to change over following a plan and just really being agile With our agile plan. So when you create a plan, don't be discouraged because it's going to change It's always going to change estimates are going to change the project is going to change and evolve And that doesn't necessarily mean a bad thing, but it is unchartered territory. So I really I really focus on with our internal teams The role of the team leader and that team leader can be an account manager project manager the development director But somebody on that project team needs to be the leader and they need to keep the whole team in perspective and keep the team Positive when things change when we start to enter unchartered territory So as that team leader, it's your role to be the lighthouse in the storm as we navigate those rough waters Whether that be different technology a change in priority Timelines move whatever it is It's your goal to keep to stay above water and to keep the team driving in More most efficient way in order to create that success And then this is kind of goes back to the communication slide But it's just having the right people in the right room So set aside time on specific problems with the key people that need to be involved The entire team doesn't need necessarily need to be involved to tackle a problem Maybe it only needs to be design and development getting together to make sure the wireframes and the feature list are right Maybe it's a developer or two developers getting together to work through a problem or to test something But it's really just focusing on setting aside that key time to make those connections and to move forward on that That issue that you're trying to resolve So try this so it's really simple But I think this is something that is gets easily overlooked and again being that team leader It's your it's your job to manage the team So including an agenda and a goal in every single meeting Keeps keeps everyone on track and on target. So for stand-up. What are you working on today? What are you working on tomorrow? What what issues do you have and then also for the client team because there could be a lot of team members on the Product owner side that need to be involved in meetings So making sure there's a clear outline of what the objective is and what you need to get out of that meeting is Listed down that way everybody can come prepared to the meeting and they know what needs to be Done in the meeting in order to to make it a successful meeting Nobody wants a meeting about a meeting or wants to walk out of a meeting being like we didn't accomplish anything So including something as simple as an agenda and a goal really keeps the team focused Internally and externally if something comes out of that meeting such as a stand-up that requires additional discussion Schedule a separate time with those key people, but it keeps only the relevant people in that conversation and Doesn't waste anybody else's time which your budget will thank you for later All right, so my personal favorite part of the process user acceptance testing So this is very important if you guys haven't seen the Two unit test zero integration test meme you should Google it. It's awesome. This is my favorite But the whole idea right is that we're testing after every sprint We're doing live demos and and everybody knows how each individual feature works and remembers that oh, yeah We tested this and it worked after sprint to but why is it broken now? It's because you need to do integration tests right you're building this big complex system You need to make sure that it actually works together Which I think on the surface sounds very simple, but in practice. I think is much harder Because I'm sure everybody in here has been like and let's just push dev into uat a little bit It's okay. We can keep building features while we're testing like no problem Which is okay right like we want to respond to change and and we want to get those features in But we just got to make sure that this testing still happens because we don't want all our paper towels blown out of the bin when we launch So must haves here Implementation team and client team should be performing the same tests If you're not on the same page there Then that's going to be a really confusing bug fix process that will likely lead to a lot of scope creep and Also the uat document shouldn't include functionality that's outside of the current approved user stories So Actually, I'm gonna go back and talk about this a little more so like we talked about I just want to talk a little more about actually producing that uat document Because I think like for us that was kind of a daunting thing to figure out like how do we produce this giant document of all these tests to Make sure we're actually covering the whole systems functionality and then I was when we're like, oh, we already have that We have the user stories And in our user stories We also included a permissions grid, which you know for some sites Maybe it's literally just anonymous authenticated and administrator, but still all important roles to test You know maybe for a more complex set you've got five or six roles that are logging in and actually interacting and producing content And you can break those all out if you're using a spreadsheet break them out into different tabs of the spreadsheet And then you know try this pull people in from around your company or Friends or whoever wants to volunteer their time But probably people that work with you that maybe aren't familiar with the project right because the whole idea is you Want to build this system that works for people who are not developers, but are your users, right? And then take those people assign them user roles put them in the shoes of someone who's actually going to be using your website and Have them do all your testing for you You know, I think you you get a lot of really interesting feedback, and you know, it's really not as scary as it sounds All right, so you've done discovery You've done implementation you did testing now you've launched Yay, you're excited and now you have a giant post launch feature list things that didn't get done it within your sprints Or didn't get done by launch whether they were new priorities or just additional things that you thought of along the way that weren't included in the Implementation plan Well, then you put them in the icebox, but the icebox if you think about it. It's kind of gross. It's a little scary It's freezer burned. It's forgotten. It's ignored forever Well, we like to rebrand it as The refrigerator because refrigerator the food in there is perishable You need to be gone through you need to use it and you need to use it quickly within a reasonable amount of time So keep track of your features and address them within a reasonable amount of time setting up Release cycles with your client are important because it essentially this will keep building on your product creating a better product And it creates a better client relationship because you're continuing to grow and build their product to meet their business needs Which leads into here doesn't mean your process has to end. So this implementation process can keep going It's a cyclical process. So you can keep creating mini sprints and maybe you do a three-month release cycle That keeps those new features keeps everything fresh You keep the whole process going maybe at a smaller scale now since it's not as large as it was But what this really does is it just creates a better relationship like I mentioned because you're helping solve your clients business problems And as you continue to do that you continue to grow your own internal business, too So you're helping create solutions that fit the needs of the product and your your client So that creates great software and a happy client and then this quote is from the agile manifesto Which is really true to the value that we want to harness which is the highest priority is to satisfy the customer through early and continuous delivery of valuable software That's all I got for you today if you have any questions Please give us feedback. We'd love to hear what everyone thinks and we'll post the slides on our note on the Baltimore Drupal con site by today tomorrow. Yeah, and if you have any questions, let us know Yeah, yeah, we're happy to answer questions and you get you out of here a little early to get a beer or Stick around for an hour and talk process. Whatever Yeah, I'll repeat the question for the sake of the recording He's just asking if we use the same technology to share sort of all that information with our clients and with our internal team and Kylie looks like you had an answer to that The answer is yes, so we do share We'll share that same user story document Which is a Google Drive Slot or a Google sheets and then we'll also share some form or of permission set with the tool that we use Which is Maven link so we'll share that with our client and they have the ability to to post questions They don't necessarily have all access to financials, but I share that with them on our status call So I can do a screen share we're completely transparent with where we're at with the burn down And it's also nice to see like oh these features are running a little hot Which means that we might have to start looking into cutting a priority three or the opposite We are doing really good right here. We're being really efficient Which means we might be able to bring up some of those those priority three features and Then we also for the UATU UAT phase we use a sauna for task management so we'll create a separate a sauna project for our product owner and then as they go through their testing document of The user stories they can add tasks into there So then I would be the owner of managing that communication from the product owner project to our internal project Thanks for the question Yeah, yeah, that's that's a great question and you know we've had that response just talking to People around about it before and you know, I mean I think for us. It was a really good low-fi way to get into it You know we use a sauna which is pretty unstructured for task management and you know up to this point Google Sheets has been you know really great. It tracks revisions. You can share it with people And you know, I mean it's worked well I think we will you know get to the day where we outgrow that But you know up to this point. It's worked well for us Do you have a preference just curious? Oh Yeah Do you prefer a more structure? Yeah, absolutely revision history and Google Sheets is amazing Go back for years Yeah, cool. Yeah, um, that's that's a great question and just something we talk about a lot I mean we do as a company work time and materials So, you know, we'll essentially like if we are dealing with sort of more of a defined budget on the client side I think you know at that point We essentially have to push the flexibility into the features and how we manage those right so I mean I think that's where you know It just takes that really close management to watch your hours per feature and really make sure You know when you're building something you're tracking hours against that and so, you know Like how we've specifically addressed just tracking hours to the right place is in our a sauna task We've got fields that essentially Match to Features in Maven link, which is where all the time for those gets built to so we can see pretty granular Burndowns of hours and and I mean I think that's where talking early and often is really important because you know We are working with a real budget and you know, I mean that that's totally understandable You know, I mean we get it budgets are real But like I think if we're talking a lot and really I guess being agile on on the Development and design and sort of implementation side then I think we can Produce something that still works with sort of that flexible approach Flexibilities just in the work rather than the budget Yeah, I think just to add on to that a little bit when we get into that implementation phase part of not building the not Shipping setting sale before the boat is bill is just making sure that the estimate range that we're working with and our user story Priority list matches up to that that budget range that our client has so if we're Providing everything from a one two and three priority, but it can only be x amount of dollars Then we we essentially take that stress off of us and provide it back to our client saying this is great We want to do all this but you need you only have this much so Reprioritize and we can work together figure it out But it's just really we push back on setting that priority and and making that fit within what they can get And then as Nick stated as the project goes on that's when we can pull in those other priority threes Whether we're burning like pretty low or where we're at Yeah, not not much honestly the stories themselves stay the same So you know you don't want to be changing functionality really we just add some more administrative stuff, which you know likes each user story We'll typically I can't remember. I think we will keep priority and then we essentially add a field for Untested unresolved issues or testing complete to each one And then just sort of like test dates to track when it was actually test so other than that It's the same and like I mentioned we'll break it out into tabs per user role Which was more just to fit how we were structuring testing But you know, I mean if it's just a couple people testing and it works better for them to maybe just like you work from the top Down I'll work from the bottom up. Maybe that's the way to go. So yeah, the idea is minimal effort there for sure Yeah Yeah, that's a really good question, I don't think I have a good answer for it But I would love to do some research and get back to you about it. Yeah Yeah, absolutely No Yeah No, we were actually pretty fortunate our account director is a certified scrum master So she came and kind of helped train our account team in that so we've been rolling it out since then But we've just been reading documentation also just educating ourselves and then rolling it out to our teams Yeah, I couldn't point you towards anything specific, but you know, honestly Lots of there's lots of doing an experience and you know, I think this is my Sixth year at Drupal con and just kind of coming in talking to people and hearing other Agile presentations and sort of the the pains that other people were running into and honestly, I think for us It it felt like it came pretty naturally. I totally don't mean that as a cop-out answer But I think it should feel relatively natural once you're in it Yeah Nice well, well, thank you again for sticking around we appreciate it and have a great rest of the conference I