I guess we can start now. OK. Hey, everybody. Hi. I can't see you, so I'm going to assume the room is totally packed, and that's awesome. There's more people here than we're here to see trees, right? Yeah. Good job. There you go. If you want to come down to the front and get cozy, feel free. But if you'd rather check email, that's OK. So my name is Angie Webchick Byron, and this is Greg Hayrock or Dunlap. And we're here today to talk about scaling the Drupal community. So to talk a little bit about how the community has evolved over the years, sort of the point that we're at now, some of the challenges that we face as a community, as we grow larger, and sort of hopefully a healthy discussion on what we should do about some of this kind of stuff. Because we're not coming here with solutions per se, but more to just create an avenue where we can discuss some of the things that have been coming up in the community recently. OK. So there's a lot of parallels between Drupal and the stages that a startup company goes through. So if we imagine Drupal as a startup company that started 10 years ago, right? So you start up, and you're kind of small and scrappy, right? And you're very engineering focused. And it's very exciting.  And everybody's really focused on the same problem. You need to build something exciting and get it done. And it's a cool time. And everybody sleeps on the floor and eats nothing but beer and pizza all day long. And it's really exciting and fun.  And they're all really excited too. And that's great. But then a funny thing happens as you start to do that. And that's like people start using the stuff that you're building. Oh my god. And so now all of a sudden it's funny because you could plot all sorts of graphs about Drupal, but they would all look exactly like this. And so what happens then is that the project grows and more and more people come in. And one of the things that starts to happen as that happens is that obviously the way that you work has to change. There's more people involved. And your focus starts to shift a little bit as well. But the other thing is that things start to slow down as well. It can't be as exciting and quick as it was when you were 10 people locked in a room. And this is also the point in a startup company usually when management starts showing up. And so now you've got people to report to. And you've got HR staff and all of this kind of stuff. And this is where you start turning into what appears to be a real corporate atmosphere. Another thing that can happen also in the growth part is that as those management people start coming in, a lot of times they don't come from the ranks of the people who have been working with the company for a long time. Because nerds like me aren't necessarily the best people to be managing your HR department and stuff like that.  But they also cause a separation between the leadership and the people who have been doing the work, which wasn't there when you were all sitting around drinking beer and eating pizza. And then what you do is you get to the success phase. And it's funny because people start companies because they want to achieve success. But it so often happens that the people who start the companies get to the success phase and they're irritated and pissed off and bitter. And it's because they don't really think in their mind about what success looks like when they start up. And so now we've got a situation where we've got the White House running on Drupal and we've got 1,000 contributors. And the global Drupal community is hundreds of millions of dollars in revenue through across everything. But it means that our culture has changed. And the way that we work together and work as a group has changed. And so things will move a lot more slowly, for instance. One of the things that we introduced in Drupal 7 is the automated testing on patches. And you weren't able to get your patch committed unless it passed all the tests. Now, that's a good thing for the stability of the project. But it also means that it makes it harder and slower to get your patches committed. Another thing that happens is that organizing large changes and architectural changes takes a lot more planning. You spend more time in meetings. You have to worry about your feature set and how it compares with your competition and what markets you want to grow into and all of this crazy kind of stuff. I mean, Dries had that slide in his session earlier about how companies go from being engineering focused to marketing focused. And that's really what's happening to us as well. And another thing that happens out of that is that you spend a lot more time fighting fires rather than growing and building what you're doing. And we see all that with what we're trying to focus on now with keeping our bug counts low and things like that. So it's really interesting to look at the parallels and the way that we've grown and the way that corporations have been growing for dozens of years and see what we can't find in terms of that from parallels. And so as we were talking about this and talking about this presentation, we sort of found two ways in which the Drupal community is scaling. And one is how we scale our resources, how we grow the number of people and allow more people to contribute, but also how we scale our community, how we keep what's important to our community in the face of this massive growth and infusion of capital into Drupal basically. And so I will hand this off to Angie to take on the next bit here. Sure, so when we talk about, you know, the first side of this is like resources and by that we mean like people weaning, like tools, things like that. So let's talk a little bit about the structure of the community and basically who's in charge around here which kind of looks a little like this. Now I'm just kidding. If we look at the evolution of the community structure over the years, it sort of starts to look like this. So in 1999, you had Drees, you know. This was Drees' project. He built it in his dorm room in Antwerp and he kind of called all the shots and kind of did his own thing. When he open sourced it in 2001, it was him and a couple of his friends. It was Steven Wittens and Chiartin Jansen and they were sort of equal level. It was like they all sort of ran the project together and there were a few different people who also contributed patches via email and that kind of a thing. But it was very like small group of people and everyone was kind of like focused around one specific area. Sort of parallel to what Greg was talking about.  By 2005 when Drupal had hit sort of its growth phase, we started to see a transition as sort of more of a hierarchy forming. So Drees and Steven Wittens both kind of escalated to sort of this Uber project lead, project management kind of thing. And then beneath them they had branch maintainers so Neil Drum and Gerhard Killsranter that maintained one specific version of Drupal. And then below that you started to get a much bigger pool of people who were contributing patches. And this was all in the interest of sort of like making sure that people could scale out. We also appointed people like a security team lead and a documentation team lead because that all used to be done by the three guys at the top or whatever and now it was sort of expanding out. And now if you look at Drupal today, you see a few different things going on. So one is that Drees has sort of like become like the grand pooba overseer of the entire project. He's sort of like in charge of the whole thing. He's the project lead. He is the president of the Drupal Association and you can see his dot keeps getting bigger and bigger. He's also kept the branch maintainer paradigm. So there's me for Drupal 7, there's Gabbo for Drupal 6. But now we've also started appointing additional leads below that. So there's the Drupal 8 initiative leads. There's people and maintainers dot texts that are sort of given a larger voice than others in the project. And we've also seen like say the documentation team starting to scale out and appoint two leads instead of just one. We've seen the infrastructure team expand out from one person to an entire team managing that.  But it's not obvious from the outside of the community who those people are until you accidentally mouth off to Earl Miles and his HQ when you get smacked in the face, you know, that kind of thing. And you also see people kind of pulling apart and some people going off into silos a little bit. And you see people sort of, you know, there might be this group of people that works on core and this group of people that works on contrib and this group of people that doesn't use the forums ever and this group of people that only uses the forums sort of, you know, as almost as like a manner of sanity, people are starting to branch off into their own little groups because they just can't take it anymore with that many people all crowding in at once. So this is sort of how the evolution of the community structure is going and we've introduced a couple of things to sort of like formalize this stuff more.  This was a big step for the project for a couple of reasons. One is that every release prior to Drupal 8, the way that core development has been done is basically, you know, Dree's post up post and says, please, please reply here with what you want to work on for this release. And everyone would say like, I'm going to fix forums or I'm going to fix performance or I'm going to work on this crazy new idea I have and it would kind of go like that. And then people would do like half of that stuff or none of it at all. But you know, it was sort of like driven by the people who wanted to do stuff. And we still have that, like that's still, the duocracy model is still fundamentally part of our community and that's what we do. But it's interesting because this was the first time that the project lead actually said, no, these are things that we should actually really focus on. And not only that, but I'm going to actually appoint people to really pay attention to those things. So we have yet to see how this is going to play out because this whole initiative thing is still pretty new. But it's interesting because it's more of a top down direction setting than we've had traditionally in the past. The nice thing though is that when we do a survey of all of the Drupal people in the world, they come back and they tell us, yeah, that's the stuff we want. Oh, and also this stuff, which is good. It sort of validates the direction that we're going. But it's interesting that the core development process itself has gotten a lot more sort of directed or at least attempted to be directed and we'll have to see how that goes. Another thing that we've done a lot of work with scaling is on infrastructure. This is a graph of the number of committers that we have on Drupal.org before and after that get migration. People didn't like CVS, it turns out. Yeah. But that's an example of a change that we made to our tools that enabled lots of more people to get involved. And so looking for items like that, that this is a barrier to contribution, let's knock it out of the way to allow more people in. But it's also going to mean more work on sort of the output filtering, right? Because let's say everyone can create a module. Woo, everyone creates modules and suddenly we go from 10,000 modules to 50,000 modules. Well, you're going to have this problem where there's like a thousand voting modules and you still need to find the one that works for you. So we also need to do better tools of like searching and ratings on projects and things like that. Better ways for people to find what they're looking for as we scale this up. But this is all related to tools and this is all Drupal.org stuff that we can do. We're giving a core conversation about that tomorrow, by the way, if you want to learn more about that. At what time? I have no idea. It's in your mobile application and stuff. The second side of this is scaling the culture of the project, right? When it's 10 people around pizza and beers, it's really easy to keep the same culture and everybody to have similar values and things that they look at. But in Drupal, it's become harder and harder to sort of maintain that culture. It's really great when we have these Drupal cons because it exposes a lot of new people to the people who've been around for a while and I think a lot of that rubs off. But it's very challenging because in our virtual interactions, we see this siloing problem sort of happening. We sort of have two very distinct, separate places where people go to talk. We have the issue queue, which is where developers go to talk to each other and we have the forums, which is where users go to talk to each other. And although there's no physical reason people can't post to both, developers sort of pull back because they can't take it and there's all these new people that don't know what they're talking about and haven't learned the community norms and so they withdraw just as a matter of self-protection because they just can't deal with the noise. And then what you find is newbies trying to support each other without the guidance of the people who brought Drupal to the place where it is now. It's very dangerous. Another thing that I think is very dangerous is about two years ago, we split the IRC channel. So it used to be that if you went to the Pound Drupal IRC channel, you would come to the contributor channel and it would just be people talking back and forth about all the patches they were doing and these initiatives that they're working on and this module and that module and the other module. And so new users would be sort of thrust into this buzzing beehive of activity around all these smart, dedicated people in Drupal. And we got a lot of crossover of new users to contributors that way because IRC is really the only medium we have now that nobody uses the forums on the dev side to do that sort of informal chatter and sort of the way you make friends and stuff like that. We made the decision because we were sick and tired of telling people that it wasn't a support channel to just be like, ah, fine, make it a support channel and we'll go into Drupal Contribute and do our thing there. And that was a necessary step in scaling the people of our community because our developers can't handle the concentration level that they need to maintain if they've got someone asking support questions every three lines. But it's very dangerous from a scaling our culture point of view because there's a whole generation of Drupal users right now that don't remember a time when Pound Drupal was the place where stuff happened. And in fact, when Drupal 7 was released, when I was rolling the final thing and tagging it and putting it up, you know, Pound Drupal Contribute was just a million miles a second of people like posting WebChick++ and Crel++ and all this, they're just people super excited and it was just scrolling faster than my IRC client could even keep up with it. And then in Pound Drupal, it was like nothing. It was like, anyone there? I can't figure out nodes. Anyone? And it was just, it was like a very stark contrast to how the Drupal 6 release happened which was with everybody in Pound Drupal throwing a huge party and stuff like that. So we've made that split. We've made that split where we have users and we have developers just like every other project out there and I'm worried about the sort of cultural impact that has because if the people who hold the cultural values in the community aren't there to mentor the new people coming in, you know, it's sort of a dangerous road to go down because new people coming in start to feel like Drupal.org is a place where they take and consume and borrow from and not a place where they participate. We can also take a look here at the evolution of the Drupal ecosystem in terms of maintaining that culture.  They used Drupal because they had a website to build for their church or for some person that was paying them for it or whatever, but they were all independent people. There was no such thing as a Drupal company back in 2001. No one had ever heard of such a thing. But when we hit our growth phase around 2005, there were Drupal companies. There was Civic Space, there was Bright, there was Civic Action, some of these bigger ones. You know, and they all sort of started to gather things and you know, freelancers sort of banded together and started to form coalitions and things like that and they're still a pretty healthy freelancer hobbyist group but people started to actually earn a living off of Drupal. And we saw that in Dries's slides, how the number of people who earned their living from 2008 to 2011 even had skyrocketed so far. But this is when that sort of started to happen. Then in 2007, a certain company that starts with an A and may not have several millions of dollars in VC funding started up and that sort of changed the ecosystem because all of a sudden there was like one company that was like on very different footing than the rest of the companies. The rest of the companies just sort of been like, you know, self-starters and things like that and Acquia had VC funding. So they were naturally able to like, you know, put a lot of resources behind Drupal and it was to the benefit of Drupal but it definitely changed the dynamic dramatically from what had been happening before. And if you take a look at today, you start to see things like those big dots have gotten a lot bigger. You know, Drupal now is like, you know, it's a go-to where everyone, you know, for the most part in the IT world has heard of Drupal now. Like we've made it, we've succeeded. You know, we've passed that threshold. But we can see that the number of freelancers and hobbyists is dwindling quite a bit as people sort of find ways to get paid to do what they would do in their spare time and that's good. You know, it's a good thing to be able to do what you love and get paid for it but it changes people's motivations a little bit and it changes their values a little bit as well. We also start to see things like Acquia acquiring different companies. We're not the only ones doing that by the way. There's lots of people who are acquiring companies but, you know, we have a bigger target on our backs because we're big but the funny thing about this picture though is that like, you know, there are shops like Capgemini, you know, some of these agencies that are way more true of people. Accenture. Yeah, Accenture than Acquia ever is and, you know, like, so this is just a very, not a very complete picture but this is what the community sort of perceives and the community gets really worried about what happens when that big dot gets this big and all of the circles are inside of it, you know? But that's not what we want. I mean, Acquia doesn't want that. We don't want that as a community. That would be really bad if there's only one choice of who to go to for any sort of project. We want there to be lots of big circles and everybody working together and that sort of thing. You know, but in order to do that, you know, we all need to sort of band together and, you know, contribute and make sure that that sort of happens. So before you change the slide. Okay. Another thing that I wanted to point out here is that all of these big companies have, all of these companies as the community has grown, all of the developers and the companies are starting to come at it with different goals in mind for what they want to get out of Drupal and that's caused a lot of conflict too because a lot of times these goals are in opposition to each other and they need to like kind of work, it's very tough in the community to work that out right now and so the community is facing a lot of problems with that right now. I think for instance, you know, the stuff that goes for really high end sites versus the stuff that goes for sites that you're hosting and stuff like this, you know, we're still, I think we're still trying to kind of find out where Drupal stands and what market it belongs in and stuff like that and that's causing a lot of conflict right now as well I think as these companies grow and start to kind of focus on different aspects and things like that. Jeff Eaton gave a talk about that earlier if anybody saw that. If you didn't, you can catch it online. That's right. Yay! Yay! The other thing I just want to point out here is part of the reason AQUI is doing this AQUI positions thing isn't to be evil and isn't to consume the lifeblood of Drupal because that doesn't make any sense from either a money making or a professional or a community health or any other sort of perspective. What it is is that things like security and like migrations are really critical to us achieving our goal of like, you know, dominating this legacy platform business that Dries was talking about and if we absorb it into an organization that has the resources to turn this into training curriculum and has the ability to pump resources into these tools to make them better for the community, we feel that that's a good thing for Drupal and we feel that's a good thing for, you know, for the entire ecosystem. So, AQUI is not the Death Star, sorry, conspiracy theorist. Dries is not Darth Vader. Yes, unfortunately. So, one of the other things that happens with this, with all of these big companies coming in and I think it's been a problem for a while but nobody's really come out and said it so I'm gonna come out and say it is that whenever you start bringing money into the picture, people start thinking that your motivations are changing and so if I as a loan developer or hobbyist come in and put something into Drupal, people can look at it and say, oh, he's a loan hobbyist and he just wants to see Drupal be improved but if I as a representative of a company that has a certain focus on Drupal, like enterprise systems or e-commerce or whatever come in and push a change in, then it starts, people start seeing that through different colored glasses. They're trying to, they start seeing it for, well is this something that's going to benefit him? Is that the whole reason that this developer is doing this change is just so they can get a leg up for their company is are they just trying to sort of enrich themselves on the backs of all of this open source advocates and Drupal lovers and stuff like that? And I think a lot of that has been going on recently with regard to a lot of the companies that have been going on is that you see sort of a lot of these pushes that are good for Drupal but people view them through the eyes of differing motivations and I don't think that's really helpful for the project at all but it's going to continue to be a problem as more of our top contributors become less hobbyist. I mean, of our top 30 contributors, was there anybody who wasn't working full time on Drupal at this point? Yeah, I'd have to look but probably not too many. And I mean some of those are just working for consultants and they're getting their clients sites up but I mean if you're in a niche market like when I was at Palantir they had a real niche around doing museum websites for instance and if you're in a niche market and you're pushing and you're pushing for instance, Larry is really passionate about the fact that field API should work with remote data systems. Well, one of the reasons he's so passionate about that is because we've had to do so many clients who have digital asset managed systems, their museum clients and that's one of the reasons that we want to get that in. We do think it's good for the Drupal project but a lot of people, we're nerds and we think about stuff way too much and our mind goes and we see ghosts everywhere and so I think it's a problem that we need to look at and face up to. But another thing that happens with the money is that we're based on the culture of a duocracy and we've traditionally always been based on the culture of a duocracy but when you're funded externally it makes you much more able to do than other people. Like for instance, right now my company node one is giving me half of my time at work to work on my initiative and that allows me a lot more of an ability to focus on Drupal than a lot of other people might not than probably any of the other initiative owners at this point and so for companies like Acquia who have determined that it's important to do something like improving the usability of Drupal or for a company like Commerce Guys who are also venture capital funded to improve e-commerce and things like this, they are able to put much more resources into getting the things that they are important and so it kind of unbalances what we have traditionally thought of a one developer to one developer to one developer duocracy in a lot of different ways. And it's something that we always, I think it's really important for us to keep in mind as our culture shifts and as we scale and grow not just into being a developer focused thing but into being a more professional marketing sales broad based organization group of people. Yes, you can change your slides. And so a really important thing to come out of that I think is that, you know, because I don't know anybody in the community who I've met who is looking to contribute to enriching themselves specifically. I think a lot of these companies may have different visions about what's best for Drupal in the long run and that's healthy and should be discussed as we try and plan our future but I don't think anybody has really this goal of taking over Drupal or using it to make them into Bill Gates or anything like this. And I think that one thing that we can all do is step back and sort of trust each other and assume good faith on all of our work and all of our commits and all of the things that we're working on. I think that will go a long way towards reducing a lot of the tension that we see in the community today. I mean, one of the things to keep in mind is that you don't dominate the world by making church and community websites, right? And I think one of the things that we all want is for Drupal to spread really widely. But in order for Drupal to spread really widely, we are going to eventually, if it happens, get to that success phase that we saw earliest in the thing. We're gonna get to the point where our Drupal is a stable, boring, slow, not boring. It's a stable company. It will work more slowly. It will do different things. But I think that most of the people who are really passionate about Drupal are also really passionate about open source.  Am I still changing the world? You know, and I think you kind of are because, you know, one of the things that we're spreading is not just Drupal. We're getting that crack for open source in the door at these places.  And you know, they may have a core of people who are really passionate about open source who are really passionate about working in a different way. And if we can just get that crack in the door and get those people really going about Drupal and about open source in general, that can start to spread. It's like every intranet that we do for one of these big companies or every little side community site like that is one more site that's going towards the good side or whatever, you know. But I mean, empowering users and democratizing the internet isn't just for the small guys. It's like, you know, it's important for us to grow and to get into these things. And I do think it's just as important as changing the world in the more traditional sense that I think that we're all used to talking about. Well, plus the beauty of open source is that when you fix a bug for Sony or Walmart or whoever, you're fixing that bug for Greenpeace and Amnesty International and everyone else who uses Drupal too. So there is an enriching thing that comes in. You sort of feel like Robin Hood, you know. It's like, ooh, I'm getting paid by some heartless evil corporation, but I get to help out like the people like don't pay my money too, so that's pretty cool. No, that's absolutely true and it's a great point to bring up. But, you know, again, as we see all these things happen, we need to, you know, I'm not saying that any company that's big and has a ton of money shouldn't be kept in, shouldn't be, you know, we shouldn't assure that they're behaving as good community stewards and stuff like that, but I think that in general, we need to trust each other's motivations, yeah. So the bottom line here is if you are feeling frustrated with Drupal right now and you feel like you don't like the direction it's going, you think that what the core developers are prioritizing is not what you'd like to be prioritized or you like what they're doing but you want it to happen faster or anything like that. The key thing here is you need to get involved because at the end of the day, you know, the people who actually do stuff are the ones who are going to make the change in Drupal.  And I don't think that's good. I don't think that's good for Drupal to have very few voices. I think we need wide range of voices and a lot of people helping to drive the project but it really requires documentation help, patching help, it requires testing help, user experience help, project management help, all of this kind of stuff that we really need in order to make sure that all voices in the project are being heard in the direction of Drupal. So please get involved and help us and like make your voice heard in the most direct way possible, which is actually like driving the direction of the project yourself. And with that, I think we'll open it up for questions or discussions. I'm not sure if this is what you guys were hoping to get out of this sort of, you know, presentation, but we kind of felt like there was a lot of community sort of issues that were going around the internets and it was good to have a forum for discussing those. So I can't see very well, but. Yes, go ahead. Oh, okay, here, I'll travel with the microphone to the questions. I want to raise a point, a discussion we had on Drupal.org. Thing is, I think if you want to scale the community for beginners, for newcomers to get involved and to get in doing stuff and Drupal the software or Drupal.org the website or whatever, they need to understand Drupal. And for this, I think one of the main thing we need is as all open source projects, a good documentation. And I'm not gonna please you maybe, but for me, like the new books that just got out, the definitive guide to Drupal 7. For me, it should have been in the Drupal.org documentation. Because for newcomers to get involved, they need to, I mean, basically they need to understand if it's, well, I don't know if you see my point, but what can we do about helping others? Because to help newcomers, only old players can help the newcomers at this point. And it's better to fix problems and to answer questions before they're asked. And for this, documentation is the core, is the core I think. I think that's something, I mean, obviously the documentation issue has been an issue for a long time. I know that Dries has set up what he's calling gates so that one of the things that's gonna happen for any commit to happen in Drupal.org is it's gonna have to have documentation along with it. And Jennifer Hodgson, who's one of the documentation leads right now, has been extremely passionate about making sure that that happens properly. But I also think there's, and there's been a lot of talk around the documentation on Drupal.org now and especially like our handbook pages and stuff like that, how we can make that more efficient and how we can keep them up to date more because they're really unmanageable right now. And I know that there's a core conversation at some point about addressing that sometime this week. If that's something that you're interested in getting involved in, they're always looking for help around that. I don't disagree with what you're saying, but I do think that the community is aware of it and starting to finally take a serious look at it. Yeah, and I guess one other thing I would say to that is, you know, when I got started with Drupal, I applied for Google Summer of Code and I got accepted and I was like, oh crap, now I gotta learn Drupal, right? And back then there were no books. There was nothing, there was a couple of pages in the handbook that were written for developers and I was totally stuck and I had no idea how to do it. So the way I tackled that problem was I got involved in the community, I hung out in the support channels and I tried to answer other people's questions and most of the time failed. But once in a while I would answer one, I would feel so totally badass. But the first thing I would do when I answered that is I would write the documentation myself because I was at exactly the point where I knew what was confusing and I had just figured out the answer to my problem. Because the issue happens when someone like one of the top 50 core contributors tries to write documentation for someone new to the project, it might as well be Klingon, they're not speaking the same language because they'll start using jargon that they're familiar with and other people aren't familiar with. One of the things that I don't think a lot of new people understand is that you can write, anybody can write documentation and the more new people we have doing that, the better our documentation is gonna be. I think you find our documentation lacking because a lot of times people don't start writing documentation so they're so far up the Drupal learning curve that they forget what it was like to be new and they start writing it for themselves and not for other people. So don't be shy to wait in there because you could write a page and it could be completely wrong. Let's say you just gave it your best shot. Entities are something, I don't know, something like this and this and this and this. Because you have to figure it out either way to get your site done. So just type up a documentation page and then go on IRC or post the documentation to you and say, hey, could somebody take a look at this and see if that's right? And probably someone will say, ah, that's almost right, but not quite and they'll edit it and they'll fix whatever's wrong with it and then you learn and then they didn't have to write an entire documentation page by themselves. So that is a much more sustainable way to improve documentation than asking people to not get paid for a tremendous amount of work and instead do it for free on Drupal.org. You know what I mean? People have their own motivations, their own capabilities. It's not really fair to ask that of them. But I agree, ideally our documentation is that good and you can just read it online. But that only happens if all of us help. I get to a point completely. I already wrote some Drupal.org pages and answered some support requests on the Drupal.org. But what I'm saying is, let me find my words. There is the handbook pages like and there are like end user documentations how to create a view, how to configure this module and everything. And then we have the API documentation which will hopefully get even better than it is today thanks to the Drupal 8 gates. And this is pretty cool but what we lack is in between I think. Like, okay now I can use through the interface like to do basic stuff. How to get in? This is, I think this is what lacks because the API documentation is like pretty good. Like per function, we have like the good comments but how to help people grog the whole Drupal thing or at least just like the whole theming API or the whole form API or the whole, I don't know, feeds API for a country module or views or anything. This is where we need to do stuff. It's interesting because when I was working on the initiative I was looking at a lot of other open source CMS projects and one of the ones that I looked at was Plone and their documentation is essentially, they don't have API which is terrible because API is a huge benefit for us but on the other hand what they have is essentially the equivalent of using Drupal and learning Drupal module development and that's their documentation and that's really great but on the other hand I have no idea how they organized a team to write that and keep it up to date between versions because that seems like a Herculean task. I mean I could barely bring myself to write two chapters for the book that I worked on and I got paid to do it and so it's like the process of organizing that and finding a place for it on Drupal.org I think is really important but I also think it's a very difficult thing to round up in the community. Yeah and I guess what I would say to that is there's a documentation spread on Friday, go. Say this is an area that I feel is lacking. I know what I found confusing and maybe you can partner up with someone and actually get a start on what you're looking for because we need people like you to provide that perspective. So go to the documentation spread participate. It's all about participation. It's a duocracy, right? Huge effort, sorry, as Greg said it's a huge effort to organize a documentation like a book like The Definitive Guide to Drupal 7 or to yes, sorry, and the module development. So this is what needs. It is but it doesn't get done if we just complain about it and don't help. So we should help. We should all band together and make sure that that gets written. That's okay but we need to, yeah. I want to address the question about trust and sort of this question commercial corporate versus social advocacy use of Drupal. So I come from the academic world and there's in scientific research in particular there's an extremely well developed and successful and long standing system there where anything that you have community resources for meaning government money that you once you've published it, it's now open source. People for instance cloning genes or making reagents or whatever you have to make that available to community. That's just part of the system. It works extremely well. So you have to publish it. That means that you document how to use it and you write a method section and the papers and so forth and it gets reviewed by people who understand how to read it. And this is no more technically demanding or it's equally technically demanding to Drupal. Drupal needs something like this and the people should be, companies should be expected to commit back whatever they've done in terms of Drupal development because they are making use of a community resource that doesn't have the sort of grant status and money exchange type of thing like federal grants do but it's exactly the same thing. It's a public venture where a lot of people have put effort into it and so forth to just organize for it differently. And I think that I've had this idea of academic open source for some time. I think that the open source communities Drupal in particular could learn a lot from how academia runs and that would get at solving these questions because companies would be forced to provide back to the community in a very reasonable way. Well it's difficult to force anyone to do anything but I would say like the good companies and the smart companies in the Drupal sphere do give back their code because they understand the maintenance benefit of that. There's no reason for this to be volunteered. It should be required and it's required in the academic community. But who is gonna require that because the code is GPL and you can do whatever you want with it. It's a matter of a company making the strategic decision that says. Yeah but I mean these licenses are evolving so you've got creative commons where people are expected to acknowledge use of something, they have free use of it but they at least have to say that they got it from someone else and they can't just ignore that. If you put a creative commons at the bottom of your page of content that's the expectation and if you don't do that. People in the scientific community can ignore these laws and generally it's not like the government's on their door but if they don't provide the clones eventually people complain and stuff that gets back to them. But there's no reason. I think we exert similar social pressure.   You know what I mean? It's like then do something about it. You know what I mean kind of thing. But I think that the people in that bubble ecosphere they get that. You know what I mean? The chapter threes and the lullabots and the aqueas and the node ones and those people they understand that and they do give significant amounts of time back. The smaller shops that sort of heard you can make a lot of money with Drupal and then go in there and say ooh let's make money with Drupal I think there's a cultural thing there that we can stand to to sort of increase the knowledge of. But there's some significant like just no brainer benefit you give from giving back. I mean you get Carmen the community. You raise the profile of your company especially if like if you're the company that figured out mobile for Drupal. I mean that's got you set for the next 10 years probably of client work you know. You know my favorite example of this is actually sorry I'll stop. Is Sony funded the five star module. So they funded the creation of the five star module. And then one of their direct competitors Warner Brothers turned around and used that same module on their site which you know from one perspective you could look at it and say well that's terrible they just paid for a bunch of development that their competitor got for free. But on the other hand then Warner Brothers gave a couple of patches that fix a bugs added a new feature to it. And I think they even ported it to Drupal 6 or stuff like that. I mean so this is all stuff that people who are educated in the Drupal community value or open source community values even really understand. But I think that there's no way to mandate that from on a high it sort of has to be something that we individually sort of share that culture with other people. One of one of the biggest ways to force that is that I think it's almost impossible to hire a significant Drupal developer if your company doesn't give back anymore. I mean I wouldn't go to work for a company that didn't. That's one of the biggest social pressures that we can apply as a community is just don't go to work for companies that don't do that. But I agree with Angie that I think the GPL as it stands is an important part of who we are and what we do. And Drupal would not have grown without it. There's no doubt about it. And you can't just, I just don't think that's a realistic model. We talk a lot about getting people involved and getting them to start contributing and to get companies largest in order to contribute. But after a certain threshold, it's extremely expensive to contribute, right? So it uses a lot of time to wrap your head around things, to actively participate in the issue queue, to hack on core, to really participate in an initiative actively is a full time job. So that's really limited to people who have no life at all or for people who, like Angie, unfortunately, or for people who are fortunate enough to work for a company that is able to sponsor a full or a half time job, you know, job placement. So what that basically does though is it causes a situation in which the only people who are actively doing these, you know, working on core and doing this stuff are more increasingly working for big companies. And there's a sort of a threshold for participation if I happen to own a company that wants to say small, it's almost impossible for us to do both things. Do you think we're creating a situation where the only people who are actively working on major initiatives are the big companies? Yeah, that's definitely a huge risk, right? So, you know, the idea that, yeah, it's fine for Aquia to give, you know, two engineers full time for a couple of weeks or Drupal 7, because they have whatever 800 engineer, no, they don't, you know, they've got like 12 engineers and so they can take a couple off of the pile if they feel that that's important. But it's a lot harder for a four person business to do something like that and make that kind of big initiative. And it is hard with this duocracy model, it creates inequality, right? It creates a huge inequality because even though we all want the same thing, we all want Drupal to achieve world domination and our goals and our values are very similar. One person who has more resources than another person is automatically at a more of an advantage to get their stuff done because this person's basically reduced to just like yelling really loudly when things aren't going right, you know? It's like, stop, please don't do that, that's terrible. But they can't actually invest the time to actually code the alternate solution that would get us out of that rut. We ran into that a couple of times in Drupal 7 actually, like the Field API and some of the UX stuff and things like that.  It's like, yeah, it's one of your passionate things because I know a lot of small shops, they want to give back just as much as an aqueo or one of the larger people, but it's significantly challenging. And I don't think we as a community have ever really figured that out because, you know, you have this body, the Drupal Association, who could potentially set up a grant system to go and fund core, but then that gets really dangerous into now the Drupal Association is driving the direction of core, which you also don't want. So it's one of those things that I think we really need to talk more about because it's a, you know, we, none of us, including aqueo, want to get into the situation where aqueo is coding core by itself. You know, that's a terrible situation. We want, you know, people representing small businesses in Minnesota, you know, to have an input in the direction of core and to be able to direct that. So yeah, we need a better model. I just don't know that we've kind of figured out what that is yet, but I completely absolutely see the danger there and we need to figure something out. Do you have any comments on that? No, I think you've covered it. Okay, I guess you're... We'll come back to you. Sorry, sorry, I stole your question. So the money discussion comes up a lot and I actually need some education in the other direction because I have yet to really understand it that don't we want to take this application out in the world and apply it to problem after problem and solution after solution until 10% of the web is using it to solve business problems, to solve real life problems that to me, the people in the community not who are also who are building it and contributing and of course, I 100% agree is essential or it doesn't exist, but also there's all these people that are out there taking Drupal and building the custom theme and writing the modules and applying it to solve all kinds of complicated problems with this tool and charge for the services involved in doing that and I don't understand why that's perceived as a negative thing. I would think it's the point. I don't see it as a negative thing. I think you're right. I mean, I think the problem comes with, change and growth and scaling are hard for any company and it's hard for us too and I think a lot of the people who came into Drupal came from this sort of hippie non-profit area and I just think that there's been a real tough adjustment and this change has happened so fast. I mean, in 2004, the global group Drupal revenue was nothing. It's like seven years that we've gone to this. It's unbelievable and I think that it's just been a really tough for people to adjust to this, especially people who were there at the beginning and to figure out how to wrap their minds around this new world. I mean, a lot of those people are used to startups because a lot of people just jump from startup to startup and they're really used to small, tiny companies where everybody knows each other and now suddenly they're in this community and they feel really passionate about it but it's changed almost out from under their feet and I don't think necessarily that anyone thinks that we shouldn't have a sustainable community that people shouldn't get paid or any of those things. I just think people, I just think the changes happen so swiftly and so massively that people are still trying to figure out sort of where we are and where we're going and what this means to them kind of thing, you know? I mean, would you agree? Yeah, I've talked a lot already so I'll leave it at that. Thanks, I just wanted to make a comment back on that last question. Sorry, I grew up in a small town. My name's Tom. I'm a colleague of the NGs at Aquia. I grew up in a small town in a small business across the river from Angie in Wisconsin. Plugged for that state. Anyway, we've got a new initiative at Aquia that we think we can involve some small companies. We really believe very strongly that small companies are the heart and soul of the ecosystem around Drupal. It's where the creativity has come from. It's where Drupal's been built from. This effort effectively, we call it a large-scale Drupal. It is focused on some of the difficult problems that are really hard to address in Drupal. And we're working with organizations that are willing to fund that. And effectively, what it'll end up with is grant-like situations for smaller organizations where we'll act kind of as a project manager and a go-between to try and find people to contribute on these various efforts. We're actually hiring a program manager to work in the office of the CTO together with Angie and Dries and Moe Schweitzman. And I think it's an exciting step. It's A-step, it's not the answer, but it's an A-step to try and keep, you know, that innovation and the growth in the small companies. Thanks. About the Aquia CTO office, don't you think that if too many top contributors are hired by Aquia, is going to be an issue in the discussion because if everyone is coming from the same company, it's going to be bad for Drupal because the people that are active for the project are going to come from the same company. What is Aquia trying to do is very good because they are getting things done, but if too many people are going to the same company, the balance between the people is not going to be the same. We will have some Aquia guys maybe deciding some stuff internally and then discussing. The key point is that everything is happening on Drupal.org right now, but isn't going to change a little bit if too many key contributors are going to a Aquia CTO office. I apologize because I didn't totally understand, but so you were saying isn't there danger in Aquia continuing to consume all of the top talent in the Drupal project and doesn't that basically harm Drupal in a roundabout way by taking out like the power from out of its feet kind of thing? Is that the gist of what you asked? Yeah, I mean, it's something that every time we hire someone at Aquia, there's always an email chain that goes around expressing that exact same concern. It's like is this the hire that pushes it beyond the point where Drupal is sustainable? But I'd say two things to that. One is that it takes two people to change companies, right? Nobody extracted me from Lullabot like kicking and screaming and strangling me and dragging me behind a truck. I mean I made that decision and I weighed that decision very heavily but I made that decision and I'm happy with that decision. You know, Mosch and Greg sold their companies, they made that decision. I think that there's two sides to that. One is that when we, and Tom is sitting right there and he's my boss, but I watch what I say here. But when we hire somebody though, the end goal there is sort of two parts. We think about, okay, we obviously think about how is that gonna help Aquia make more money because hey, we wanna make money, we're a business, right? But we also think about how is that gonna help the Drupal community be sustainable? It's actually not good for the Drupal project that there's only one person in the community that knows anything about migration. That's bad, that's terrible. That's a horrible situation to be in. We would love there to be 50 curves and then we could all choose from it and hopefully this vacuum will cause other people to come up. But in the meantime, what Aquia's trying to do there is pump more resources into the migration module because we can put some engineering talent behind that that we have that most of this company doesn't have. We can do things like create training curriculum around migration and start training Drupal shops and Drupal professionals on how to do migration stuff with tools and suddenly we start scaling the amount of people who can do migration work which is critical to be capitalizing on that opportunity that Dries talked about. So I would say like from the top down that concern about the impact on the community has or I'm sorry, the impact on the community that a new hire or a new acquisition has is always at the very forefront of the thinking there. But I would also say that I think it's wrong to deny someone from furthering their employment because there's a concern about the community thing. My wife is the most patient person on this earth and she put up with me spending all of my nights and weekends for the last three years on Drupal 7 but at a certain point that's gotta stop and if there's an opportunity to get paid full time to work on doing the same stuff I was doing anyway, I'm gonna take it, you know, I'm gonna take it and I think it's a good thing for the project because we need someone who can think about this stuff full time with the kind of resources that Acquia has behind it, but it's absolutely something that we think about every single time because we're very sensitive about that picture of the ecosystem and we wanna make sure that it stays diverse and it's not just the death star absorbing everything that's packed because that's not good for anyone. I would also add one point, I think that, well I think Angie's completely right about this being a two-sided coin and that people wanna go to work for places where they can fulfill their dreams and stuff but to some extent that's up to all of us too because like when I was looking for a new job from Palantir, one of the most important things for me was I thought I'm a very highly in demand skill set right now and I should A, not let this go to waste because it may never happen again in my life and B, I don't wanna just jump to another development shop or another consulting shop. I'd like to find something really cool and unique to do, right? So while it took me a year, I did. I found a company that would bring me to Europe to live full time and was willing to dedicate to me the resources to work on the stuff for Drupal that I thought was important. There is no reason why any Drupal developer in the world should not insist on the same thing. All of us are in such high demand right now. I feel like I wanna be like Norm Aray standing on the table with that huge sign, right? There's no reason for anybody in the Drupal universe to not be doing exactly what they want to right now and one way to do that is to not do stuff for companies that aren't letting you contribute for companies that do things unsustainably to insist because you're the resource driving this, not them, that you get what you want and deserve being a high demand job set right now. And then that sort of gets to Michael's point about we exert social pressure, which effectively raises the bar for making sure that people contribute back. So the short answer is that it's good but we don't need only Acrea doing that. We need other companies hiring more people dedicating some time to Drupal core. We'll take it. It's really echoing in here, so it's hard to, yeah. Not just, sorry, if I can just completely change track for a second, I've been lucky enough over the past year to be brought into a company who were working on ASP.net and change them completely over to Drupal, which has been a great opportunity. But I find that it's almost the project managers and the bosses and the client managers and those kind of people who don't understand Drupal enough to understand why it's so important to allow the developers to commit back to the system or to give back into the community or whatever the case may be. Is there anything out there within the community that can help me turn them around and convince them? On lullabot.com there's an article about, I forget what it's called, but it's basically about that. It's about the benefits that you get from contributing back. That thing's raising his hand, so he might... Oh, I can't see that at all, but... Oh, okay, never mind. Oh, okay. Yeah, so I wrote something a few years back about that. We could really use though an actual white paper or something about this is open source for people who don't understand it and this is why you give back the actual financial benefits, the actual karmatic benefits or whatever. We don't really have that resource that I'm aware of, but it's definitely something we could really use. Maybe you could come to the documentation spin as well or do we have that too, Tom? Yeah, okay, you're right, opensource.com has resources like that, that's true. It's not Drupal specific, but it would be, it would help them understand the ecosystem a little bit and why contributing to open source is good. I actually think we're out of time. Can we just answer Jeff Eaton talk? Well, okay. We can cut, we have 15 minutes of coffee break. There is a five o'clock session and it's 4.50, so if we can do it quickly. And we actually have to do it at that session though. No, we are just 5.30. Ah, no, it's okay. Okay, one more question. Basically, the company I work for scaled up really rapidly along with the rest of the Drupal community and one of the things we're facing now is a concern that there is such a thing as too big, too big to preserve the kind of individual agency and trust and quality that we really want and also keep the kind of atmosphere and community that we want to be in. Are you finding that same thing? Are you having those same conversations in the Drupal community as well? Yeah, so I didn't catch all of that, but basically the concern is that there is such a thing as too big of a community, right? And has the Drupal community have discussions about that? And yes, there's this constant tension between developers and new users that sort of drives people into silos. There's also like at the same time paradoxically, a drive to get more diverse people involved like Dries talks about marketing people and product managers and people like that that are currently not in the community and we want to have more of them in. In terms of dealing with the scaling there, what we tend to do is siphon people off to groups.drupal.org which are special interest groups. So there's like a group for education and there's a group for social networking and there's a group for this, that and the other thing where people of like mind tend to congregate in their own sort of sub community almost but part of the larger sphere. So we have certain commons areas like the issue queue where we all congregate, everybody, the big messy pile of all of the Drupal people and then we also have sort of focused quiet rooms almost where people who are working on a specific initiative or a specific area can sort of congregate as well. So that's kind of how we've handled that and it seems to work to some extent but I imagine we will get to that point where I maybe somewhat argue where they're already where there are just too many voices in solving each individual issue that it just takes too long. And then the way to answer that might be to delegate even more responsibility further down the chain or something like that but the good news is we have a lot of tools for dealing with that stuff and when we don't have tools, it's Drupal so we can just make them. So yeah. I'm so optimistic. You make it sound so easy. Good night, everybody. Now, did you want to say one more thing? No, I think I'm... Okay, I'm sorry guys. We need to wrap up because we're five minutes over but anyway, thanks for coming.


So this talk is called, this time it's personalized, preparing your site for effective personalization. I'm gonna try to talk fast, I'm sorry about that. But we have a lot of information to get through. So the agenda today is, what is personalization, why should I care, planning for it, sustaining your strategy, implementation options and conclusions, and just really quickly because it's like you're talking to a few unconscious or for image action media, Vancouver, Canada, and I can tell you more about music, you know. Yeah. Oh, okay. I'm a walker, so it's hard for me to... Maybe you can grab this one. Oh yeah, maybe later. Perfect. So first step is, why personalization? Probably not a brand new topic for everybody, but it's important to know. So how to engage users with the urgent needs within a single site, a big challenge for any kind of site, but especially sites like higher education sites, public sector sites, anyone with lots of different types of users. How to improve content discoverability in sites with hundreds of thousands of pages. So how can we push the key content up to users and make them feel like the site is more for them that's tailored to their needs. And also how to accommodate users who may have longer decision cycles. So if someone's coming back six months from now or if someone's coming back three months from now after they need more time to think through a decision, how do we kind of hit them at the right moment? So that's where personalization hopefully comes in. So this is maybe a little bit of an older school example, but this is a standard page right here just to give you an example of what personalization looks like just in case you want a mental picture. So on a standard page, we can magically provide them with a message, it looks like you're coming from China, you may be interested in our programs for international students. But let's raise inversion as we can say, hey, the software's done, here's the programs for international students. And then potentially TMI, hello Tom, we're helping students succeed and try to succeed myself. That's sort of getting to the uncanny valley of creepiness when we're starting to talk to people as if we've broken the fourth wall. But that gives you an idea of some of the things you can do with personalization, many, many things you can do. But the idea is to get more and more engagement. So the potential outcomes are to give your users what they're looking for at the right time in the desired format, improve retention and conversion and increase user satisfaction. So the research on this is really clear. There's tons of surveys, big surveys with 20,000 data points, saying that personalization, people who commit personalization and do it well, see enormous increases in sales and engagement, e-consultancy and did a survey on that HubSpot, looked at 30,000 of their users, did a very broad survey that found 42% of people who personalized calls to action, or sorry, 42% higher conversion rate for people who personalized calls to action. So amazing stuff. So why isn't everyone doing it? How many people are doing it, by the way? Okay, a few people, three, four, that's great. Limited time and budgets, competing priorities, technology barriers, what problems should I use? Not when we restart, how do I implement it? What do I implement? How do I manage success? And how do I sell it to my team and make them feel like spending this money is worth more time? So what I'd like to talk about is not necessarily how to do personalization in depth and drupal. You want to talk about how do you get personalization moving within your organization? How can you plan for it and effectively get started on that initiative? So effective personalization, like anything worth doing requires planning. There are incremental steps we can take that set us up for success. First, the strategy and the perspective we should be right that we can layer in technology. There's a great article in here. This is linked to from the bottom and I can share the slide deck. Where Steven, Steven, you talks about kind of the golden, where? Yeah, the golden basics of personalization. So the key cornerstone to technology, content, analytics and data. But if you start with the technology without the content, the analytics and the data, you'll a lot of time end up spending your wheels and not making the best use of the technology. So I'm talking fast, so I'm losing my moisture quickly as well. So before beginning with the technology, we have to step back and start thinking about what are some ways that we can actually get our users to engage with the site? How can we understand our users? And make sure we have the right content, the right place before we can start the technology. So we need to understand the audience and the context. What are the business needs? Who are the users, what do they want? What key content will help them achieve the goals? And what friction, what sort of anxieties or pain points are holding the back from achieving those goals? So some great activities to help you prepare for personalization are kind of the traditional activities you'd associate with the kind of discovery project or a kind of user engagement project. So identify organizational user goals and not just talking about the goal and being one responsive site, but a little bit deeper. What are some of our key segments? Maybe you have five key segments, maybe you have 20 key segments. What are the value propositions that drive your users and make them feel motivation and attachment to your content? Developing personas, so I think a lot of us probably don't know what these owners are, but I'll show you a couple of examples. So personifying your users, helping to ensure that the site is matching returning users, sort of incoming students, joining students, all that kind of stuff. Mapping user journeys and also gathering user feedback. So the great news is, as you prepare all these things, your personalization can actually adjust this information. Once you develop personas, you can plug what personas into whatever sort of personalization you use and it can actually make use of that to build up much more sophisticated data models and gauge your users much more effectively. So you're not doing anything, this is what you should be doing when you're getting ready for a personalization effort. So the first step is identifying business and user goals. So establishing organization context, identifying overlap between institutional needs, so what we've got is the aid institution and what the users need, identifying the gap there. So who are our users, what benefits can we offer them and I'm probably not being picked up very well by the mic, hello there. And how can we reach them? So in order to get that data moving within the organization, a key activity that we do, a lot of people do sort of onsites, they do a lot of kind of stakeholder engagement activities. We like to get the actual users in the room together. So this is an example of actual users of an actual product thing in a room together talking about what are the sort of key things that they need from the product and how do they use it and what's the context of the use, all that kind of stuff. Super amazing, great data, everybody gets excited, tons of opportunities for buy-in and enlightenment. Enlightenment always ensues, which is always great when you can guarantee epiphany in an activity. So these are some of the outcomes from an activity like this. All the personas that are sort of the key personas for the product or the site, their priority, which ones are most important. And then value propositions, what do users actually respond to, what messages make them feel like they want to move forward, what messages leave them cold. Pretty powerful stuff. The second part is identifying and developing user personas. So how many people here have personas on your site? Okay, about third, okay, that's pretty good. So the important part of developing user personas is just basing them in some sort of research or data, not just coming up with them from your head, which can be a great starting point, but establishing who are the key audience sites that want to connect to the site. And you want to connect with them. What are their goals and motivations and what makes them sort of stay up at night? What are they anxious about? Is it the price of your product? Is it the difficulty of choosing among many different options, all those different things. So this is another example of an onsite we've done with these are all real users, not actors who play users within a workshop. Developing user journeys and personas together collaboratively, super, super amazing activity. And out of that, we get things like this. So we get these great personas. I'm gonna try, I'm gonna try. We get these great personas that are actually based on some sort of actual analysis data talking to users. We know we have a little more confidence that these are the actual, the right information, which then we can sort of have our personalization engine consume this really sort of great segmentation data. The third part is mapping user journeys. So for each individual persona, whether it's a prospective student or a new customer, whatever it might be, identifying pathways to achieving their goals. How will they get there? What's it getting in the way? What content will they need to be created in order to support this journey? So again, this is something we really like to do with users as well, like to get them in a room together, whenever possible, or if not possible, you get stakeholders together and look at your analytics and I'll tell you two ways you can do that. Mapping out the personas, user journeys. How do they get to your site? What do they want to know when they get there? And not just mapping out, not just mapping out the actual path they're taking, but also what's the key content? What do we need them to see at what order in that journey? And that really helps us understand how we personalize this site. So we identify that some magic is in the blue area. Identifying the opportunity. What will get them to keep going? What will get them to keep moving in the journey? And finally, gathering user feedback. Sounds easy, it's not so hard. Validating hypotheses we've generated, identifying issues with the current site. What is blocking users right now? Is our proposed direction correct? Is this the right design, the right content? So this is a great slide. From a group called UX guys in Toronto. Anyone heard of those? They're cool. I think they still exist. This is basically a quadrant. Oh, it's like docking so fast. I can't even keep hydrated. So this is a quadrant that basically, so with all the different usability activities you can engage in, this is fantastic stuff. And how you can try and do it, different techniques from different quadrants to achieve a much more holistic, triangulated data picture. It's been passive. So within our project, we usually try to engage four or five of these items. We try to choose from different projects so that we can get a really convincing package of data. So this is just vital stuff. There's all kinds of different types of things you can do within this picture. But I strongly encourage you to check this out and think about what are some ways that you could combine these data sources, not just using Google Analytics, but actually getting outside of that. Whether it's unmoderated testing or focus groups, whatever you wanna do, lots of different possibilities. And the link is in the bottom of the slide there. So out of all of this effort, and sort of very kind of traditional sort of UX activity, hopefully the outcomes are, you understand what users value and what they don't. What does the sequence and priority they want to receive information and content in? How do they want to receive information, video, chart, testimonial, so on? And how do we get our team to buy in? Our colleagues, our bosses, our sponsors, showing this data, showing this, even getting them into the workshop will make a huge difference to getting them on board and excited. We see that time and time and time again.  Well, once you've thought about the content, the data and the analytics, personas, sort of raw data that you've collected and the content you're developing, the fourth part is obviously you need something somewhere to put it. So in order to get ready for automation, you need the right platform. So with a framework of content, meaningful data, and at least we can now start layering in technology. A couple of use cases, for example, I assume a user from China, we might want to show them international program information, but we might want to also emphasize academic reputation because that's very important. Generally for students from Asia, reputation is a key driver in a value proposition. We might also want to emphasize safety. Visitor to a nursing program, we might want to display testimonials featuring a nursing student on the about page rather than a generic about page. So there's lots of things that can do that. Some of them are easy, some of them are harder. Some of the options in Drupal include Aquialift, so that's available for Drupal 8, which is a very robust product. There's also contributed modules. Some of the challenges, some of the contributed modules, and you may have noticed some in Drupal 8, but right now personalization and personalization are D7, only I believe. So that might be, if that's not correct, let me know. And also custom solutions, which we've done a number of times where we've tied together geolocation data, taxonomy to kind of pull together, sort of help users self-identify and drive them to custom pathways based on that data. So those are Drupal options. They're also great, best-of-breed products, like Evergauge, it's a fantastic product out there. So a lot of times, it could be the case that the best option for you is the third party option. That does everything you need to really, really well. So Lyft is out there, Evergauge, as well as Qubit, there's many others out there. But check them out, there's some fantastic tools that will kind of take in all this research and data that you've collected and actually allow you to automate it in a sort of powerful way. So here's an example of a Qubit screen. So it might be hard to read this, but it says new customer and non-farm users. So that's one of your personas. It's ingested at persona, and starting to recommend content based on that persona data. Finally, start your persona activities, you're moving forward, how do you keep it going? Resources are always limited. How do we include personalization into our existing process to make it habitual and supportive? So the first step is start small. Start with one thing, start with one change. Measure the effectiveness of that, make another change. Report back on your findings to your peers, to your upward stakeholders. And that really helps sort of get by and motivation, get those quick wins, and focus on activities with the highest ROI. Be careful of over-reliance on sort of just looking at what behavior, try to understand why people are doing things and how you can make changes to really hit that, hit the mark with your users, and focusing effort on optimization over variety. So not changing content all the time. Freeze your homepage, make your homepage great, and focus on that. So lots of things you can do to make your data more actionable. Do customer research. Set up a user customer advisory board or focus group. Set up a booth and offer a pastry. Who doesn't like pastries? There are some. And focus as much as you can on collecting quality data, which is often more actionable as it makes us understand why people want what they want, not just what they want. Once you have a personalization engine in place, all these things become, some of these things become a lot easier. There are often survey tools built into these products and lots of different ways you can get. I get your data, sort of enrich your data. All right. So in conclusion, personalization over and over again is shown to have an enormous ROI. It's one of the highest ROI communities you can do. So if you're wondering how can I move the needle, this is a huge way you can do that. Outcomes are improved big time by understanding users and their motivations. If you just install, you know, lift and start using it, you're not gonna probably get as much value as if you start doing some research activities before you get there. The best idea is often talking to users and not looking at graphs and funnels. So having those conversations, understanding what users actually want is really critical. And then finally, when you're beginning personalization, start small and be sure to measure. So better test one or two changes and measure that and to make a lot of changes and feel attractive. I think we have, whoa, you got time. Look at that, seven minutes. I talked too fast. Which one was that? Was it this, was it the UX guys one? Yeah, yeah, absolutely. Was it this one? Oh yeah, that was great, that's cool. Hey, I'm Frank Curie. I'm curious what your feedback it sounds like you may have worked on a few of these projects. A couple, we've mostly done the custom. We haven't yet done one of the personalization engine. I'm wondering if you found certain platforms, certain site types are easier or harder versus others say like news versus fashion or something. Right, right. Answer is the best of my ability. And then let me know if that answers your question. I think, so we've implemented it for a B2B site. It fits super naturally into the way that B2B site naturally is architected because that site is asking users, are you like a CEO or are you implementer? So that kind of data just is naturally being pulled into the site all the time. So that was a great sort of progression to fit super well with the way that they're thinking about the site already. I can also see a higher education site. How many of you are on higher ed? Oh, wow, okay, I'll talk. Higher ed sites are great for that. A lot of times you have things like a persona menu where you have different sort of ways of targeting your users where you're collecting that data all the time. So that can be another site that's just a natural next step for that site. Does that answer your question? Okay, great. So I had a question on the different options, custom, third-party, or Aquialift. I know the lift pricing and it's... Right, I wonder if that worked out well. For those of you who don't know, it's five figures, I think. It's expensive, it's expensive. So that might roll out for different groups that you're working with, but then so what is the benchmark for the free options that engage? Or if you've worked on custom solutions about how much time and money was that? Can you share that? I can definitely share that. So that's a great question. We thought about that a lot because we didn't want to implement something because there's also the recurring cost. Because once you have it, you need to continue to pay for the license. We chose to go custom because of that very reason. We felt that we were able to deliver something within sort of 20 to 30 hours worth of effort that was actually valuable and worth doing. So in that case it paid off. It was the right sort of investment. And we only need to really set up a few of those things once. We didn't feel we needed to spend another 5,000 or 10,000 the next year. So that's why we were custom. So I believe Lyft was previously in Drupal 7 based on the personalized module. But since then it's detached from that because that module has not been ported to Drupal 8. So right now, this is where I don't know. I tried to look around. I tried to poke around and try to find some Drupal 8 options. I couldn't find any that were in any kind of shape. Lyft is really the only thing I could find that was Drupal 8 ready. But obviously a lot of the third party options have incredible capabilities. Again, if the budget is not ready for that kind of option, then Drupal 8 may not be quite there yet. Yeah, definitely. I wonder what it could do. I wonder when this talk ends, I can put my slides up and I'll add them to the webpage of this talk. And then you can see that. Yeah, because it is hard to read some of these at the bottom. But anyway, I'll tell you what you do. You can go to Google, targetmarketing.com and Google Steven Yu. He's the guru guy. Steven Yu, why you? And he has about 10 articles in a series that are just like gold. Amazing. Is that it? Okay, thanks everybody. Thanks for your time. You did it. All right.