Folks, I would like to hear what your interests are, but I was just telling some of the other folks that Kat and I co-teach, and we're in Norway the next two weeks teaching a course for undergrads called Design for Non-Human Clients, and so we're asking our students to identify a plant or animal and then make some sort of app or social network for that plant or animal. And so that was part of the inspiration. It's also something we've been thinking about for a few years. But yeah, I would love to hear about your guys' ideas for incorporating other organisms besides humans in our cybernetic systems. A round of introductions will be good. So my name is Kat Kramer, and I am an artist. I co-run two organizations, Zach. One is called the Center for Genomic Gastronomy and the other is called Co-Climate, and they equally deal with food and climate change and technology and people's relationship to those things. My name's Erin. I'm a user experience architect and co-founder of an open source software company called NONE, and I'm mostly designed for humans, but I really like this idea of designing for other things that aren't humans, and I've always been really interested in smart home technology and sensors that I could potentially in the future tie into my house plants. And then someday if I had a house with a yard and chickens and beehives and things, that would be cool too, but I'm kind of limited right now to an apartment with house plants. But so ideas for the future, I'd love to grab them. Hi. I'm Jesse. I make stuff. Some physical stuff, historically mostly software. I've built software for non-humans, but it was software for other software, not for things that live and grow on their own. I'm Irv. I'm a professor, and one of my main areas of interest is the impact of what we do on human health and the environment, and so this topic really fascinates me. I'm Dane. I'm a student of my professor. I study computer science, and I was just curious about what this was all about. Hi. I'm Kevin. I'm a PhD student at Harvard, and I work on folding robots, or robots that fold themselves up, and I'm interested in how robots can be integrated into our lives in harmonious ways. I'm Jess. I'm a web developer and also a owner of two parents, and an iguana, and I constantly try to find ways to use technology and all kinds of other things, and other computers to improve their life and improve their environment, so any further ideas I can do or share what I've already learned is awesome. I'm Tim. I write about architecture and design, and so I came here because I'm really interested in cyborgs at the scale of landscapes. One of the groups I do a lot of work with is called the Dredge Research Collaborative, and we are interested in all the technologies around erosion control and prevention and that kind of thing, and how that reshapes at very large scales the planet. I'm Zach. I'm an artist and a professor, and Kat mentioned our group, a collaborative design program in Portland, Oregon, and the next two weeks the students are looking at this intersection of technology, the environment, and humankind. So I think it might be interesting if people have maybe some examples they've seen of prototypes or imagined projects that are really inspired by. I know that we have a definitely a hardware hacker actually making stuff for pets, but one of the things that got me thinking about this a long time ago now is the Center for Tactical Magic, which is a really interesting art group that proposed to put volumes, and if the old growth force were logged, the Insects Editor would fire rockets at loggers, and it's really an absurdist crazy idea, but that was really inspired by that notion of arming non-humans with a technological system, not because I actually want anyone to have more armament at all, but just as a sort of provocation. And so that's one of the things that got me really interested in the topic. Like, was that eight or years ago now? That's one of their proposals. Juan, I was trying to basically just create an optimal and comfortable environment, sort of trying to do, I know with Boston, it gets really cold in the winter, and occasionally you have even lost heat sometimes due to oil issues and lots of roommates, so I sort of ended up using a combination of various heating utilities, heated perches, heating panels, to create a birth-state sort of heated environment for them, so you know they're always comfortable that way. I've utilized various things via YouTube and Internet Radio to help keep them entertained during the day while I'm at work or circus, so there's always something going on, and as I mentioned briefly earlier when I was on vacation one time, I actually also have them set up on a views online stream so I can check on them, see what they're doing, and if I notice that they're getting very anxious, noisy, and upset, I actually recorded a video of myself on my laptop, sending calming words, uploaded to YouTube, promoted into my home machine from vacation, played the YouTube video for them, and was able to sort of interact with them that way and calm them down and keep them happy, and anything I really do to help sort of improve their environment to give them more fun, more entertainment, and more comfort is really what I'm trying to do. So I have a question, and this is the question that occurred to me when I first saw the topic, and then when you were talking about what you were talking about just now, do you think it's possible to design for the environment without anthropomorphizing? Because I don't know that a tree wants to be protected from a logger, maybe it does, but how do we know that? And is it somewhat presumptuous of us to think that it does? I think it's such a hard question in this space. One of the techniques we've been using in our classes is letting students do that first and foremost. It's like, okay, to try that as a technique if they're not really disrupting the non-human. We use a lot of the metaphors of client relationships, so we have students go out and find a non-human eat lunch with it, understand its nutrient cycle and its waste stream. But another way in is if we can never know what a non-human wants or needs as humans, what do we need in relation to that? So we use ecosystem services as one way of relating to non-humans to understand the services that they might provide for human kinds. And then since we want those, to give them things that make those better for us. But this is like a big deep ecology question, right? Like how do we get beyond these anthropomorphic things? Yeah, I think another thing that's interesting is we've been throwing around a little bit planetary sculpture.  We are sculpting the planet, but kind of unconsciously. And so the idea is, well, what if we were a little bit more thoughtful about the things that we do to change the planet? And so by opening up this idea of designing for non-humans, it's not necessarily negating the human. It's including that in the space, but being a bit more considerate of everything else as well. So taking into consideration ant or actor network theory and how every act has an effect on everything else. And yeah, as you know, designing consciously for the world around us. First thing that comes to mind is home design. And I know a lot of homes, at least in the Alston Brighton area, will have issues with birds, finding little holes to nest into a roost into. And oftentimes, learners have to find a way to basically kick the guys out, which isn't cool, but they're nice. They get rid of bugs and all other important things in the environment that I could go into, but no one right now. But, you know, homes designed specifically to have areas for, you know, other animals in the area, squirrels, birds too. We know that they're going to be living in here, we'll design it specifically, maybe get some use out of it, use Sizer for local plant and wildlife that we can use and work from that and sort of designing a housing system that rather than try to block all this out incorporates into a more intelligent. I just thought of something sort of along the not necessarily considering the environment that we put things in. A friend of mine who is an architect and very environmentally conscious was tracking. They at one point changed the street lights that they used the lights in the street lights to make them very, very much brighter at night than they were. And there was a single street light that happened to be right under a tree on his street. And he noticed after they made the switch in the light bulbs or whatever they did there to really increase the brightness at night that that tree in the spot that was all around that light post all of the leaves very quickly died. And that whole part of the tree died right there because it was so bright at night after they switched out the light bulb that the tree just couldn't cope with that anymore. And he was very distressed about that. And I thought that was really interesting because that's not something that occurs to me that often, but you could definitely see how we change the technology in something like a street light and it has immediate effect on the vegetation right around it. There was actualists for his client. And that was a kind of curious client to work with because how do you design for something you inherently can't see? Or actually, there's also what's the movement in the UK? The I think, yeah, there's I can't remember the specific name, but there is an organization that's encouraging darkness and they have actually created sort of national parks, dark zones, so that you can see the stars and and yeah, there's sort of a bit of a movement on that front as well. So would you consider, for example, terraforming Mars as as an example of what we're talking about here, totally changing an environment, which right now is arguably pretty dead. But you know, sending technology out to completely change the surface of a planet. Would that would that be a client? Would Mars be a client? We we've sort of like, I think cases, introduction, talk, looked back and said, you know, we in some ways we've always been cyborgs, any technology tools. In some ways, we've always been terraforming like the part of like the human ecology ideas that were animals within this system with other animals and organisms. And so before we even go to Mars, maybe way of thinking about it is we already have these augmented ecosystems that we exist within like the light and that we already had this tech that was affecting light. It was only when it really affected it and killed it that we were more conscious of it. So a little bit earlier, you were speculating what you thought this this group might be talking about. And you had some really interesting ideas about pipelines and measuring or understanding the surface of the ocean or the the ocean's bed or something. I wondered if you could elaborate on that idea a little bit more. So what you're remembering is so we have all these cables that are really good and transmitting lots of data. And then we all have all of the previous cables that were laid down there. It's really expensive to bring those cables back up. And so what scientists are beginning to do is you can run signals along those like old copper cables and depending on what is happening with the stations in between, you can get information about the situation with the cables. You can use that to measure movement under of the ocean bed and gather other environmental data. And so we have accidentally wired up certain channels with substantial amount of sensors, right? They weren't meant to be sensors. They were meant to be conduits and now they've become sensors. So I was going to mention that I had gone to Ireland a couple years ago and one of the things that was really interesting there was looking at the history on the ecology. So the entire idea of having peat bogs to harvest as something to burn. Those peat bogs came about because people settled on the island and cut down all the trees. And so there was this, like what we think of as the ecology now is something that humans had already terraformed. And so it's not something that is, I think that there's a tendency to think of it as something new. Like we are going to go out and we are going to terraform as opposed to we have already been terraforming for a long, long time. I think it's one metaphor that might be useful is there's a lot of user interaction coming out of computing that then bled over into the rest of the design world like, oh yeah, we should ask users what they want or observe them using things. And there's maybe an interesting backtracking with non-human cyborgs and augmented ecologies like, oh yeah, we're always already affecting our environment with our extended capabilities with technologies. So we should do that more thoughtfully. And yeah, so it's a bit like this. And it's amazing that, what's the name of the, is there a project or consortium around this cable sensing? Yeah, I don't remember. Well, someone out there will, or we'll find the Twitter. Yeah. I think partly that's more valuable as the backwards looking way as we move forward. So it's not like a break. Ooh, it's meek. I think also in the, if we're going to do the cyborg thing, right, there's a marked difference in the world of erosion control between a kind of 1960s heroic infrastructure, engineering, hard lines between water and land, like armored put down concrete, what those sort of approaches and beginnings of new approaches that deal with and enhanced feedback loops, right, that recognize that feedback loops exist. And so one famous project in the Netherlands, Netherlands, by the way, if you're like at all into erosion, like a country that is largely undersea water is a really good country to look at. So they did a thing called a sand engine where, so the problem with sand and beaches is that you put the sand there and it probably doesn't want to be there. And so then it gets eroded away. And so current technology largely means you just truck out more sand. What they did instead is they put us, they did a bunch of math and figured out a way to put a sandbar that's sort of a very odd shape such that when currents hit it, it would redistribute the sand along the shore. And so it would require them to do the replenishment far less often, right. And so they designed a system that for a certain period of time worked with entropy instead of trying to hold entropy off. So talking, you know, just jumping off what you're talking about using using nature to control nature, right. I think that's a really important idea. And you know, again, it goes to the deep ecology thing, whose businesses that have us to control nature, maybe what's happening to the planet right now is nature's way of saying bye-bye. You know, you've had your time here and now we want to get back to where things were before. But assuming that we want to make it habitable for us, then probably the things that are going to be most successful are where we work in harmony with nature, in communion with nature instead of trying to, you know, do our technological things to prevent nature from doing what is natural. So I came in late. I don't know if Christopher Alexander has already been brought up. But he, so he was originally known for the Timeless Way of Building, which is a pattern language for building buildings that are livable and usable. And he went on, his current work is on neighborhoods. And how do you build sustaining, self-sustaining neighborhoods? How do you grow self-sustaining neighborhoods? I think that the idea of a pattern language has a lot of power for ecological engineering on a pragmatic scale. I was also struck about sort of some of the quantified self things. You were asking about how do we know what nonhumans want? How do we know what humans want? So I don't like, I hate putting tech first, but I think there's a promise of people prototyping different new ways and need not involve electricity or digital technology. But new tools for measuring and assessing and checking in and asking organisms and things other than human, kind of what they want or how they're doing or, you know, wait, they're amplifying things tough. There's really specific ways in like environmental engineering to measure things and ask for, is this going to be successful or not based on our needs? And I think actually, as artists, there's, we're excited about those messy metaphors. I think it's totally great to like make some tool to ask the plants how you feel. And we have lots of plants at Tweet, you know, sharks that have RFID on them.  I think also anthropomorphizing is a good empathy building mechanism, whereas you wouldn't necessarily consider it at all beforehand. If you're suddenly like, well, you know, how does that, you know, should I care about that tree? Should I care about chopping it down to make room for my house or whatever it is? It's like, well, if you think about it as a client, for example, you might want to keep it alive, right? If you think the base, the base premise is things want to stay alive. Maybe they don't always, but you know, so then building for that and to do, oh, it wants to stay alive and it wants to be healthy and it wants to be and it wants to thrive. And so, yeah. Isn't the the decision to pick one or another thing as a client already kind of incredibly fraught? Like every every tree that is there is a bush, right? Like a forest is the way trees in forest work is by choking out all the other plants that are trying to grow in that forest. Recently, I was doing a little bit of research and the thing is, in some parts of the United States where the fire departments get really good and prevent forest fires, they actually do have to do some controlled burning, because oftentimes with trees in large forest, they need to occasionally have a good burn down, get that nutrients back in the soil so new things can grow. And trees are going to want that. But for the rest of the forest, it's important for it to happen. And also toss in, you know, even for other animals, if they want something, it doesn't necessarily mean it should happen. My birds want all my ramen noodles. It isn't good for them. I'm not going to assess any like that is not what they need as a client, even though that's what they want. So it's also a matter of figure what, not necessarily what it needs or wants, but what is, in a sense, best for it going forward, both for your needs and its needs. There's sort of the assumption there that preservation or the status quo is our goal or is and I think that that's especially in ecology, that's never the case. Everything is it's about where do we go next rather than how do we keep things alive? Everything's going to die. So I happen to know a bit about some transformations that have happened in the field of chemistry in the past 15 years where a term called green chemistry has emerged, which is based on some people sitting around and coming up with so-called 12 principles of green chemistry, 12 concepts that if you really care about doing chemistry in ways that are safer for human health and the environment, you will you will respect these 12 concepts. And I think maybe something like that would be a good outcome from the folks that are thinking about this. You know, what what would the eight or the 12 or the 16 principles of designing for non-human organisms be? I mean, I think the point is well taken, right? That's in you the second you start designing, you're including and excluding certain things. That's that's the difference between academia and intervening in the world. I think and that's not a critique. They have their places but what this might these kinds of lenses might add is just making those decisions at different scales with more consideration and like focusing on an organism or species mix or a landscape really, you know, you need to take those all into consideration. But partly I think in design because if you were going to do a full systems analysis, there's rarely time or money for that. Having some really well thought out heuristics or rules of thumb that you're talking about could be really helpful because so often when we talk to like industrial design students or students who want to make stuff, even just talk about ending end of life issues with their materials. You know, it was not considered so bringing in some other clients that have a say that aren't just there, the company could be quite interesting to have the voice of the table. Yeah. So but yeah, there's always you're always excluding things the second you start being like this and not this. I think one one person that's been kind of inspiring for us in this space is Natalie Jeremy Jenko. I didn't know this artist. Case showed her project from like 20 years ago. That was the the trip wire was the network traffic line. But in the last sort of 10 years she's transitioned to doing stuff called the environmental health clinic and sort of building biotechnologies that connect humans and other organisms. And so I know one of her projects was in the Hudson River with fish and sort of visualizing right their patterns, but also creating these gummy worms that could be humans key or the fish could eat that has she leading agents that process the mercury. So a food for both organisms that you could sort of share amongst each other. That was a super low tech solution. And there was kind of fancy high tech, blanky light thing. You were nodding in in a confusion or approval or disapproval. So it was I know her and she did the inner species cookbook. Was she in collaborate because there's also an architecture firm called the Living in New York who also did a thing with the Hudson River and fish. And I'm trying to remember if that was the same thing or if they each. And it was like so their thing was like they made a floating grid and when there were fish coming by lights would turn on so you could like you could like have a sense that there are fish at all because there's a lot of misunderstanding in New York about how poisoned or unpoisoned the Hudson is. Things are pretty OK now. They also did one in Korea. They made a they made a like mini map of the city and then they put sensors in all the different parts of the city to see how badly the air was polluted and then the map would glow and so they fool around with these kind of the idea of the feedback loop being about making it consciously making invisible systems or actors visible to people in a kind of playful cartoonish fun. Art way. So I'm curious what are some of the things that your students have done in the past that really blew you away. Well we've done it with graduate students of Portland as well. There was one project where one of the students was really interested in space junk and in order to to clean up space junk and thinking about all of the zombie satellites and sort of other aspects. She proposed putting a trash collection of the moon sort of launch out and funded by rich people and then that could start to fund the rest of the space junk cleanup. But so the idea was to have a manned mission to the moon to go and sort of look at the the leftovers including possible contamination. Yeah there's an auction. The idea was to auction off the space junk is like high end art sculpture and so then you know make it super desirable. So you'd have billionaires you know trying to buy these sculptures and bidding against each other to fund the space junk cleanup. And there's a right there's apparently a ring of human feces and urine around the planet from the early manned space flights when they eject of the waste. We should the internet's out there. I've been told it's true. But so I think the part of the idea too was like you know scooping up some of that goodness and and bring it back. But she said she had a really amazing diagram of how this was all going to happen. And the darkness guy was also quite interesting because he started the project not knowing that there already was a pretty strong movement around these. I think they're called darkness parks. But it's about reducing light pollution and having sort of zoned areas sort of like a fair day cage like people actively want to seek out spaces where they can't get. They want to discourage certain signals and have other ones. Right. So you want to discourage light and actually see the stars and to think about the scale that that that's required to do that is pretty phenomenal because light pollution travels pretty far. So it actually creates a sort of interesting no go zones dark sky initiative. That's pretty good. Just to build. I've had the microphone a lot actually. So. So if somebody else wants to please take it from me right now. OK. So I think whenever we're I think this conversation if I'm interpreting it correctly it's trying to understand the world around us better. Is that the point of enabling non human things to have something that we can understand. So to me this is an effort to be to enjoy our world more. That's that's what I'm really seeing it as is just a yeah being you know in the world very like present Zen you know kind of idea. I think that's really what I'm getting out of this. Well that's not really even just us enjoying the world but allowing other things to co-enjoy it with us. Oftentimes previously it's been us enjoying it at the expense and at the cost. And you know we've been augmenting ourselves for a while not only not even just hybridically but with tools since day one and they're being able to sort of give that extra ability and give that extra power to non humans so that you know they're also allowed to enjoy the world in a moon at the same fashion but in a similarly more empowering way that we have is you know I think also a part of the goals. I'm guessing. So I again I don't know if it was mentioned before I came in but the companion animal manifesto is amazing and also by Donna Haraway and it sounds like it would be like one of those things from academic feminism that but it really is and it's amazing and great. The companion animal manifesto about our relationship with domestic and cated animals in really cool and fun ways that and my poor cat now has like the automated laser pointer and you know it's because the poor cat is bored because she has no mice to no work to do so but yeah I think that there's a lot there about how do they enjoy themselves. I think domesticated pets and animals aside I think a lot of the rest of the organisms around the world would prefer that we completely stay out of their way and not put wires and things on them. And like if I kind of think that it's a really interesting question but I think it's also important to acknowledge that with just satellite imaging almost every organism on earth is surveyed by us already at some resolution or not. So there's not I think it's not it's an interesting debate about how much more to intervene in natural systems. But I think it's also important to acknowledge the level of surveillance and the level of data capture that we already do around organisms and think about which of that we should actively ignore and keep out of our systems almost like we do with privacy for humans. But also to yeah we're doing it at a huge huge scale with all kinds of sensing and it doesn't need to be invasive or it doesn't need to be putting wires on things. But yeah it's happening. And even if you just consider the built environment a form of technology setting urban boundary growths and that kind of thing helps them to helps you know some perception of but there's organisms living with us everywhere. I mean on our bodies in our bodies in in an urban environment especially there's I can't remember his name there's a guy in New York who does kind of tours looking at nature in New York and then there's Mark Dion who set up a hut in Madison Square Gardens to watch to watch the nature that passed into catalog it and yeah there's it's also not an us and them necessarily but more of a we conversation. And also if you go look at YouTube you can see the literal hoops that squirrels will jump through to participate in our experiments as well like you and they in fact you prefer that sometimes to food that is easy to get because it is interesting and fun and like other animals it doesn't have to be in us doing things to them thing necessarily unless we conceptualize it that way it can be participatory or my favorite design for non-humans yogurt right we have a whole gut microbiome that's not human it's just like live in there inside of us and we often try to control it in certain ways and more and more we're trying to do it thoughtfully and doing pretty bad job of it so those are the rules of thumb and heuristics that actually maybe some traditional technologies were quite good at this and yogurt would be interesting right communication medium between you and your gut microbiome but I so I'm really interested in this conversation later as well about whether these metaphors work for you or not because I because I think you have a really good point we we don't want to be sticking wires on stuff I'm not interested in that


So it's a 545. So that means that we're gonna start right on time, right? This is the custom of the land, I understand. Everything runs like a German train system on time. Actually, I'm gonna kind of hang out and talk and ask some questions, and then we're gonna dive into it if some people come. So my name's Andy Kouharski. I'm gonna say all this stuff again. I'm gonna talk, this is a business talk. I'm gonna try to slow down. So hand, do we have some folks who are running businesses? Okay, so you guys are running businesses, and where is everybody from? Texas, whoa, Texas, that's Denmark, Denmark? Okay, is this, where are you from? Okay, so this is the, okay, if anybody's, everybody from South America, you cannot sit on that side. Welcome, welcome. I was just in Korea, a couple, who do you work for in Korea? Which company? Open, OpenF? Great, and where, Brazil? US, you gotta go on that side, Ron. Mm-hmm. Oh, okay, okay, you're, Peru, Peru. Colombia. Colombia. California. California, come on, you guys are getting it wrong. Brazil. Okay, we're gonna start in one minute. Okay, so, I guess I might be being recorded right now. I am, okay. So my talk is going to be about measuring everything, and it's a personal journey of many, many mistakes that we have made in my company, and so I'll share those with you, and it's an evolving journey, so I am here, of course, to learn from you guys, and hopefully I can share some things that will help you avoid and make some of the mistakes that we've made. All right, so I am, my name's Andrew Kuharski. I am the founder and president of Chromat Source. Those are my kids, and I think they are making a snowman, wondering where their dad is, because I'm always on the road, so I do travel, and I do Drupal stuff, and I run Chromat Source. I've been doing this, well, I've been in consulting and manager and running business for about 20 years now, and as you can probably tell from the previous picture, I am from Chicago, and in Chicago we are Chromat Source. We are a Drupal development shop. Founded in 2003, we've been doing Drupal for about since 2008, so that's like seven years now. We offer 24-7 support, and we focus on custom development, and this is important for the talk, so we do custom development. We also provide DevOps services, so we have DevOps philosophy, so we have support services, system administrators that work with developers and our support team, and we provide support to our clients, and that is a separate service line. So I like measuring things, and that's really why this was a fun talk for me. So I was just in my hotel, and I realized that up to today I took 4,878 steps. I use Fitbit, and I'm happy to report that right now I am up to 6,767 steps. If I keep on walking, it's gonna help me. So I like this stuff, I like statistics, I look at numbers, so I see how much I sleep, and just because I look at it, I know how well rested I am and how much I more sleep I need. Not really, but I also like competing with some friends, so everybody has some Fitbit right now. I'm pretty good doing pretty good right now, but maybe if you have a Fitbit friend me, and we'll compete. Another measure, I have this scale at home, and in the last 2.8 years, I've measured, I've weighed myself 864 times. Luckily, as you see, this trend is going down. It's good. I hit a milestone right after Christmas, the heaviest I've ever been, but it's trending down, right? That's important. Obviously I like numbers and I like measurement, but why is that, what does that have to do with business? Well, I put something in the beginning of this talk about numbers, which inspired I think the DA to accept this and vote for it, but I think it's true. Data is important, I like this saying, and God, we trust all those bring data, and the second one is really important, and I think that what gets measured gets attention and gets results. I truly believe that. So some more words about metrics, right? So besides the personal inspiration we get from these metrics that help us motivate us to lose weight or walk more in business, they tell us about what we're doing, right? So that could be used for marketing. Are we committing enough? Are we delivering on time? Do we have less bugs? You know, we have what, 56 critical defects in Drupal? Well, that tells us something, right? We still don't have an update, but it's less than 149, right? So that's good. I mean, it's a little bit of marketing, but it's good information to share with your shareholders that gives them a sense that it's not just your opinion, but you're backing your information by some fact. Okay, well, 56 critical tickets, that means that it's going down, that's good, we're on the right path. And it's a good way of communicate, but I think the most important part of it is, and I've noticed this around my teams, and I think others, is that what gets measured gets results and it gets attention from your team. So if you're running a business or a team and you're looking at measuring some instance that you wanna improve, for example, your team will notice that you're doing it and you're doing it on a regular basis, and it'll hopefully get the results that you're looking for by measuring that. So, before we start looking at more numbers, I think your CEO from Thayer, Thayer, Thayer, Thayer, did a really nice job on the earlier presentation. I encourage you to see it online about setting vision and starting with why and having a passion for what they're doing. And I think that measuring all things also starts with this thing. You start a business or you start a venture and you think about why, why are we doing this, why is this important? And that drives to the mission and that starts, Dries talked about this at the Acquia Summit yesterday and talked about, well, at Acquia, we started writing down what are we about? So he's got it down to five sentences, I think he said. And then from that, you get goals and from goals you get, okay, you break that down to objectives, what are we going to do next, what's next? And from that, you get KPIs, right? So we're the key performance indicators and then you break that down further into metrics. So that's the kind of exercise that we like to go through that I think is important. So for example, my management team on a weekly basis, we have a set of numbers and it's not too many, it's just few numbers and we thought about it really hard at the beginning of the year. What do we want to report on? And those numbers, they may be responsive before the developers or the other teams that generate those, but it helps us drive the business and it starts with the top. So what do I get evaluated on? I'm a CEO of a small company, we're about 30, 40 people depending on how you measure, maybe up to 50. My primary way that I get evaluated is financial performance of the company. So that is revenue and net profit. Okay, well what about all these other things? What about giving back to the community? What about client satisfaction, which is really important? What about your retention? Are your folks happy? I believe that it's my job to make sure that we have good financial performance so that we can achieve all these other things and anybody else doesn't have to worry about those things. Once we have that footing, we can do things to increase client satisfaction and we can do things. We have the time for developers to go to camps and we can do all of those things. So my primary job is very simple, right? I get measured in hard numbers. Did we hit our profit objective? Did we hit our revenue objective? Yes or no, right? Luckily, I can't get fired, maybe not yet, but so it makes it a little bit easier, but it's key, right? There's also a concept of a balanced scorecard. So that's a concept of measuring a lot of other things besides that one number and that is also important and we look back and then we take a look at it from an annual performance basis. So we look at those few hard numbers for our team and for myself and then we take a look at a balanced scorecard. Okay, well, now that you've hit this number or maybe now that you're having a problem hitting this number, let's take a look at these other activities and other parts of what we're doing to see whether we're hitting that. So we look at financial perspective and we look at financial perspective for all of our key members. We look at customer perspective. So this is very, very important to us and the second part of my presentation is really mostly about customer perspective, internal perspective. So are we creating people that are growing themselves? Are they enjoying what they're doing? Are we doing good things? Are we growing professionally? That's one of my goals as well, right? So I get measured on financial performance. My personal goal is to grow as a CEO and become a better leader. So those other things are important but they're not as clean cut as did you hit your number? So just a little bit of balanced perspective here. Oh, Karen, I didn't know you were gonna be here. So this is Karen. Just a question. How am I doing in terms of not this side? Am I doing good in terms of speaking too fast or good? You guys are okay? Perfect. Uh-oh. Okay, so I'll let you know. Yeah, I walk around, I'm pretty good with that. All right, so further. So this is Karen. She's one of our project managers. She just so happens to be sitting right here. Last time, last time I gave this talk, she was not, she was another continent, so sorry, Karen. So Karen has some measurable objectives in terms of what we're looking for and that is project profitability, right? So we share project profitability with all of our project managers. So we have certain goals. At the end of the year we review where we're hitting, whether we're hitting those goals. Client satisfaction. So there's the financial objective and there's client satisfaction. And we would almost put that, I should have put that one as number one. Our net promoter score, we're gonna explain what that is, is number one, our clients happy. Team growth, revenue contribution.  How are we doing? And is she supporting us in terms of marketing? And by the way, so here's a plug. Karen has a session tomorrow on women and technology. Really highly recommended, right? So we try to measure these things in real time when we have real numbers associated with that. So COBS is one of our solutions architects. He's a senior development on a way to essay. And so what are our essays get measured on? Estimation of projects. Are those projects accurately estimated, right? We do a lot of fixed bid or RFP responses. But even during sprints, right? Are you able to achieve the things that you're planning during the agile process? Those are important for us. So are we, are those estimates realistic and are they being hit? Correct, architectural solutions. That's a KPI. So how do we measure that? We measure that by defect counts. We take a look at QA duration, QA process. Are those things bouncing around a lot? And then we have a number of quality of code delivered. We were looking at number of defects in warranty period. We take a look at the feedback that our developers are providing. We have a, it's a, before you commit, you have to get reviewed, right, Dougie? So we take a look at all these things and try to get numbers around that. So again, Doug, you weren't Dougie, not a developer, didn't expect you to be here, but just using as an example, don't take it personally. Uh, so what are your developers get estimate? Yes, what are your, what are your key measures? So I'm gonna ask for the internet audience that's going to happen afterwards. I'm going to pass my, what are your developers, get what are your measures that you use to evaluate developers? Okay, so right now we are using the, how can I say this in English? Let me check. It's like we are measuring our performance through the pipeline of our trial process. So we are trying to measure quality against bugs and the code that is delivered. We also measure the estimation like you guys and also the time of, we have to get the result of the value of our delivery. And so it's something like if we understand our expectations of our clients. So I think it's like a really major indicator of the quality we are trying to deliver. I wanna try some Q and A. I'm actually really curious on how you achieve those, you know, key measures, right? Cause that's sometimes hard to measure. How do we measure? Well, that's a lot of things we measure. It depends on the company, the maturity level the company gives for the project. So because we work with CMMI maturity levels, so I've worked in projects of five maturity level and three and it carries a lot of weight with that maturity level. So we do code reviews depending. It's almost like a standard already. So we agree with the client the number of bugs that we go with releases. So for us, it's like first internal standards and then agreements with the client. Yeah, CMMI is a really interesting model. It's been around for years and years. It's really cool. Anybody else here to? So it seems like we're pretty aligned with what we're looking at. And the one thing that we also keep track, like administratively, that's an administrative part of us is are we submitting the things that our developers hate to do, time sheets. Keep track of whether those are complete on a weekly basis from everybody in the entire company. So we also do support, right? So we measure a lot of things around support. Some of this stuff that you see on this slide in the next one is borrowed from a call center. We take a look at, we have SLA, so service level agreements with our clients around SLA. So we talked about whether we're hitting those. So response, right? So in a typical call center when you submit a ticket and measure it against whether you hit that response, you have a call, a massive call center, so abandon rate, people, they get through the issue. So the same thing here, do people actually stick with asking that question, issue, how long does it, do they wait until they get to somebody or whether they can submit it? So another call center thing is average speed to answer. We also take a look at, do we answer that response to the particular problem? Response time, and again, another call back, those are really call center issues, but they do apply to a support the way we do support. Of course, resolution, right? So just because we have an SLA that says within this critical of an issue, you have to answer it with this many hours, there's also a resolution, right? That's even more important. Just, I mean, it's nice to have somebody say, yeah, I got it, I know your problem, but it's nicer to say, okay, I understand your problem and here's your solution, right? So we take a look at that, whether we are getting those issues closed within the time we promise our clients. And then, of course, it's really important to see whether we have tickets that are reopened and that also gets tracked to the person who's closing those tickets and the people managing those. So there's lots and lots of other support metrics so we want to provide high client satisfaction around support, that's a really big part of our business. So we take a look at things like planned or unplanned time, we take a look at whether we, so we trend that over time, we try to see differences. So we know, and this is what was very interesting, this topic of queue, wait queue and how much time is, or how busy your team is. So can we answer, do we know, we try to predict how many, how much unplanned work we're gonna get on a weekly basis and then go back to some of this. We don't always, we don't keep track of all of these things but just to give you an idea of all the different measures that can be taken in a call center. So of course in project planning, we use JIRA so we're starting to employ more and more type of measurements in terms of our projects. So, and we do both agile and fixed bid. So we take a look at burn down rate, whether we have our budget being used versus the number of features that are being completed. We take a look at open bugs, so all of these things and we just recently adopted JIRA so it's making it pretty easy for us to do. Take a look at, you know, we're in sprints, are we ahead of budget, under budget. And again, going back to time reporting is pretty key. Actually, it's not only important to measure those on a micro scale, on a sprint or an issue or an estimation, but then to kind of pull up and take a look at it. I don't know if you guys had this experience before, we came to a project where somebody just wanted, a client of ours just wanted a little bit of help and this is a large project they've been working on for over a year. And we said, sure, we'll take a look at it and we took a look at it and we sent one of our project managers to kind of do a deep dive and they used JIRA and they said, we just need a little bit of help because we want to finish it in a month. And we said, all right, let's take a look at this. So we looked at all their take hits and we said, all right, so are these estimates correct? Yes, all, you know, these hundred or thousand issues there or whatever number of issues they're outstanding, all these estimates are correct. And we said, well, that's interesting. Are you increasing your team? No, no, no, we just want a little bit of help. So we said, all right, and you're communicating that you're gonna get done in a month to your stakeholders. Yes, we are. Well, if you're not increasing your team and your estimates are correct, that means that you have six months that you can complete those estimates with your current team. And it was like, whoa, they're stakeholders totally blindsided by this, right? So it's good to roll that stuff up. I'll just look at it at a micro level. Yes, your question. Are you using JIRA with sprints for SERVS desk for support teams? So that's a very good question. It is a little bit outside of what I'm talking about here, but yes, we are using JIRA service desk and we are actually, we just transitioned there. So I don't have a full result, set of results there. Yeah, so this is a great discussion because there's a lot of measurement around support. So for example, JIRA has a service desk. JIRA is a tool, if you guys haven't heard of it, it's a great tool. It's really a bit like Atlation. It has a lot of plugins, including Get Bucket and all those other things. And we've adopted it recently. One of the nice premises around it is that it keeps track of the SLA. So if you establish SLA, service level agreements with your clients, you can say, okay, a high level priority issue, I'm gonna address it in, let's say, five hours or 10 hours. Whatever you agree to do, whatever your capacity is. And then you can see how quickly you need to add on, react to that issue before it crosses the SLA, right? So the question is, how do we use it? And so on, I can have that conversation with you a little bit later. Actually, my support director is running this and I'm not that familiar with it, but I can tell you what I know afterwards. So in terms of project reporting, of course we take a look at big projects. We analyze issues, how many are high, medium, low, and this is one of those metrics that's really key in making that happen. And you can report back to your stakeholders what a likelihood is something that's gonna succeed or not. So these are all great and we implement them in our daily lives and we also do a lot of retrospectives so we can learn from those things, how much time we spend on projects, how much, you know, where do we spend our projects? So we look at like this, we did this a while ago, but this is a pretty highly focused development project. We actually developed for like 57%, the rest of it was spent in other activities, right? And this was like a development project. So retrospectives are important for future ability to estimate and lessons learned and be able to do a better job on future projects. Also take a look at capacity planning, looking at how much burn down you have per developer. This is another small project as you can see this is pretty old, but retrospective estimation. So keeping track of time and what your time is spent on is important for us. So we were doing all of these things and we felt like there's something that's still missing, an important part that we wanted to capture. And this was around, you know, quality is really hard to measure and client satisfaction we thought was pretty hard to measure. So we thought about this and we started doing some research and we really, really found some great things. And I'm gonna talk to you about this for the rest of this session and why we made this a key part of what we do on an everyday basis. So we made a decision late last year to make focus on less, focus on key numbers but less numbers and make a company goal focus on one number. So we started doing some more research around client satisfaction and how we can measure that and how we can do better. And some really interesting things came up when I was looking at this. So I just wanna share those with you in terms of for those of you who are running your businesses and companies, right? So I always, while I didn't love it because customer complaints usually come at a time like right before I'm giving a presentation at DrupalCon or right when I traveled for 20 hours and I just wanna hit the pillow but I need to hit that. But those are really valuable. So I did not like them, right? Why?  A client complaint has by some pretty decent research here many other clients would never bother to say that. So the client raises their hand and says, hey, I have a problem with X, Y and Z especially if it's tied to a process we try to stop everything as you guys stop the line and try to figure out what that is because behind that there's many silent voices and then that means that we need to address it immediately. Another interesting thing, 2% increase in customer retention is the same as decreasing costs by 10%. And I wanna talk a little bit more about that but that's great, right? So if you're running a business or you're running a team or you're responsible for a P&L making your clients happier is basically like cutting costs. So how many of you guys had problems flying here or had some issues? Yeah, right? You talk to, did you tell people about that?  Like everybody, I complain about travel all the time. Talk to Dries, he's like, oh, it's airport. So unhappy clients will tell many, many, many more people about a bad experience. But happy clients may not. So another goal that we have is not to just make happy clients, make really happy clients, make help them, make them feel like they need to tell your story. The other thing that we also are starting to focus on is that, and this can be like a tough thing in engineering, right? Because we deliver a project on time, we deliver it on budget, and we deliver everything that the client asked for and they're still not happy. What? Well, there's an emotional part of interacting with another human being that needs to be satisfied as well. And for us, a lot of the feedback that we receive was really important, didn't really have to do around the areas of improvement. Not many of them had to do around code quality. It was perceived very high, the value was perceived high. It's really about that emotional, it's about that emotional connection. And so it may be difficult for engineering and a software team that's kind of, and especially me who loves numbers, how do you make that emotional connection? So it's key, but it really does matter. Oh, and by the way, notice this one, right? So you can charge more if you provide better service. Like they will not walk away if you charge more. They'll be happier because they want better quality service. And I don't know how much money or time you guys spend on marketing, but it costs and we haven't done this, we should do this, but I wonder if it's six or seven times, it costs a lot of time and money to acquire a new customer. If you really break it down, right? The amount of marketing, blog writing, your website, we have a marketing person, basically all they're in charge of requiring new customers, right? So how many new customers do you get divided by the salary? Then how much time do you spend on the phone, negotiating a contract, all of those things, a lot of money. Keeping that customer just gotta do a good job, right? So it really does make a lot of sense. The last thing, another fun fact I found during my research is that, and I think Seth Godin wrote this cool book and that says all marketers are liars. Well, they're not liars, they should be telling a good story, but people tend to believe people they know. This is the blue line here is the percent that folks trust completely. That number one here is recommendations from the people that you know, right? So if you tell me that Lufthansa or Austrian air or whatever it is that you flew here is terrible, I'll probably avoid it, right? But if I read it that they're great on a flyer, I'll probably not believe it as much as I believe you, but I don't know you. So anyway, word of mouth, right? Your customers keep them happy, they'll give you more business. So I hope I've convinced you that this is really important. So as a business, we really made a decision to focus on this, but this talk is about measurement, not client satisfaction. So how do you measure client satisfaction? What are we doing to do that? So who's here heard of NPS, Net Promoter Score? So from me, I've told you I weigh myself five times a day, I measure my sleep, walk, whatever. I like to have a number around that. So somebody found a way to basically measure client satisfaction. And this is, so this is Frederick Raicheld, I think that's how you pronounce his name. And he wrote this up in Harvard Business Review more than 10 years ago. He basically did research on surveys in which surveys are most effective. And he found that one company in particular was doing a one-question survey, and they were using that survey to apply their practices, they were more successful than these other companies. How many times do you get a survey that goes on for pages and pages and pages, right?  And second of all, you just kind of, how do you do the analysis on it? I'm just gonna do the time check. Okay, so he found that asking this one question provided so much value to this company that they were ahead of the competitors. So he then took this question at the time and applied to the companies who were growing and companies who were staying steady and companies that were losing market share. And based on this one question of their clients, he asked for permission, he noticed that the companies who have a higher score in this one question were in a growth category, right? So clients were getting high points for that one question, we're also growing. So there's a correlation between having great customer service and through answering this question and business growth. So that's basically an NPS. So the question is how likely is it that you would recommend our company a product or service to a friend or a colleague? That's essentially that question. And you include one more text box for some input. Anyway, so, but that has to tie to a number, right? So we take a look at the number of the people who answer nine or 10. That means they're extremely happy, not just like an eight or a seven. How likely are you to recommend somebody else to see this talk on a scale of one to 10? An eight, right? A nine or a 10, that means I did a great job. Five or six, like you're just being kind to me. It sucks, basically, right?  That's basically what it means. So what it does is the net promoter score basically throws out the seven and eight. The seven and eight don't count. It takes the nines and the tens and subtracts all the people who gave you a zero through a six and it gives you a net promoter score. So that one score, it's actually a really widely adopted measure. There are many companies now that are building businesses, online businesses that help you measure this stuff. And it gives you an idea of how well you do from a customer client satisfaction perspective. So basically we started adopting this and we really started thinking about our philosophy of, yes, we measure a lot of things in terms of building our projects. We're developers or project managers. We provide value, we try to partner up with our clients. We have to provide value to our employees. But in the end, we wanted to have a number around how satisfied are our clients? And it was, we gave a similar talk to an all-hands team meeting. And very quickly, everybody kind of saw the value of this and we started talking about how happy are our clients? How are they responding? And we're tying the NPS. None of you guys remember back. The net promoter score is tied to our project manager's perspective or evaluation. So most of the field folks have an NPS score. Of course, I have an NPS score as the entire company reflects our ability to deliver client satisfaction, but it drives down to everyone who interacts with our clients. And we're looking at taking this, we can't overpole our clients, but we're looking at having interactive measures and a reset. And we're looking to obviously continually improve on that number. We want to generate more promoters. We want to generate fans. So we're going to tell other potential clients or other potential employees or other folks how great we do. And that's started to become part of our culture. What's our NPS? What are we doing to provide better customer satisfaction? So we started looking. Yes, question. I'm not a manager or anything. I'm a developer, but my question is simply this statistic that you take, it seems like something that you can only ask once to a client. Or is it something you can repeat every year? How do you manage existing clients? If your client base is small, but you have a lot of existing projects and continual projects, how do you measure this? If it's something that seems like you can only ask once. Yeah, that's a great question. So there's a small industry around best practices for not promoter score polling. Yes, you're right. First of all, statistically speaking, your sample size has to be really big to get a true, true NPS. We don't have that luxury. We have 50 cell projects, let's say going on, or a hundred projects going on every time. And then you multiply that by the number of people that we interact with. You don't get that grade of a statistical number. And the other question, a very, very good question, is how often do you poll? So we're looking at resetting that. According to best practices for NPS, you're not supposed to poll more than six times every six months. So if you keep on polling your clients, they'll just get tired of it.  Thank you very much, right? Or I don't care. So every six months, the other thing that we're doing, so because we tied this number to our team members' evaluations, right? So if Karen gets a really high number, we're gonna try to tap into that system, that secret and see what she's doing better than some of the other folks that may not have that high NPS. So we have a concept of continual polling. So as we bring new projects on, or we have new folks that are coming on board, or we have new experiences, we incrementally poll. Now we understand that could, again, a statistical variance doesn't give us that much, but it's better than nothing. So we started looking at understanding better on how we can deliver on that quality, on a high score. So we take a look at four areas of our service, and that is, what is that perfect service? We've been told perfection, nobody's perfect. Thanks break, we were redundancy, nothing's really perfect. So we're trying to basically, we understand that, yet we try to provide a service that is reasonable within a foreseeable framework of instances. So we try to set better expectations during the sales process. We have a re-onboarding during the project manager introduction, if you have a new project, and then we do certain toll gates during, or we try to do certain toll gates during projects to see whether we're still delivering. So delivering in a caring, friendly manner. So maybe as engineers, we may not be so great at this always, and I guess I'm speaking to my former self as a former developer many, many, many years ago. But first of all, there's another, there's an emotional intelligence and business course on Coursera that's really, really good. It basically talks about your attitude and how you interact with people, and people can really tell, basically the biggest takeaway from this course that I took is that people can really tell when you're bullshitting them. When you're not really happy to deal with that client, they know they don't like them, right? They really do know, and one of the things that's interesting about me is I really love my clients. I love all of our clients. I'm humbled by the fact they choose to spend thousands of thousands of dollars in time and bet on the fact that we're gonna get them a promotion, get them what they want, help them achieve their business goals, whatever it is. So we talk about partnering up with our clients and seeing what they're trying to achieve, and hopefully also buying into that, like really getting in what their needs are, whether they're a fact they're totally disorganized and maybe, well, all of our clients are super organized, perfect communication, always show up on time, they know what they want always, right? So we're just here to help them get this but in a really caring matter. And so we say that we wanna partner up with our, we wanna become partners with our clients. We really do. And it's really important that we also emotionally get involved and care about that. And if we show that, that's really a big part of the way. So in a timely fashion, this is also another interesting thing, right? In a timely fashion, may not necessarily mean what? To a client, what it means to you. So obviously there's some expectations to be setting, right? There are no projects that can be completed tomorrow. So you have to set those expectations but you also have to take a look at what we're doing and how we're doing it and see whether that's timely. So that's another key part and backed by an effective problem resolution. So are we empowering our teams to enable them to solve that problem? If something happens, do we have a communications process and in a process where we can mobilize and solve that problem? So now we said there's no, the perfect service is within normal operating circumstances. When something bad happens, how do you react? And from our experience, some of our best clients that are most that have great retention and we always wanna make our client life first, we call them life first, we want them to be with us forever and ever, we had major problems. I can't even tell you how many screw ups we had. Like major, major issues and we basically stopped what we're doing and said, yeah, we totally screwed up and here's what we're gonna do to fix it. And we mobilize and we get things done whether we delete files, whether we lose a backup, whatever it may be, and we communicate and fix it and I see that an effective problem resolution problem, problem resolution process is in place and they can trust us not to abandon them when we own a problem and they've been great. So we try to deliver value. We take a look at the timelines and what we're trying to do, what the client's trying to achieve, not just what they're telling us. We try to break down different types of project by teams and different types of project by project managers so that we can deliver to what the client's actually expecting. Establishing a relationship is really important, not just the project managers, but the technical level, we establish relationship between those folks from account managers and myself. I try to have a relationship with most of our stakeholders. We do focus on automation and testing, continuous deployment. That helps us in terms of setting expectations and it decreases the time in the deployment. So in terms of that timeline, we try to destroy silos so have our system engineering team work with our developers, be able to quickly respond and power teams so whether it's giving credit or whether it's spending more time, our teams are empowered to make those connections with their clients and get that client satisfaction higher. Culture, so this is tough, right? Because Drupal developers are really hard to find. And some Drupal developers think that, or some developers, some folks, some people just think that they're always right and if they can embrace that culture that says, no, we're really here for the client because that's really what drives our business and what we're trying to important, I think it's better to say goodbye. If they don't have a good culture fit, it may hurt you in the long run to have that person than not. And I borrowed this from Scott Meze. He's at Pantheon. He runs their support department. We also try to do this. So we always say that a project risk, there's a greater project risk that we don't have clarity. So complexity is not the risk. It's a lack of clarity, whether we're accurately responding to whatever we're trying to do, executing correctly and of course having empathy for the client. So those are some of the things that we're doing. We measure a lot of different things throughout our different types of project and support and our personnel and time and where time goes. But ultimately one of the things that we're focusing and it's starting to work really well is that client satisfaction over net promoter score. So having that mindset, are we doing things that are gonna increase that?  So shoes is our theme, first part of the year. And what are our clients thinking? Because we're gonna ask them to see how likely it is they're gonna recommend us. So that concludes my talk on measurement. I hope that was enlightening and helpful. I have already posted these slides on SlideShare. So I think that's my Twitter handle. I'm gonna tweet that as soon as we're done. And I wanted to open up the floor to questions. I'd love to hear how you guys are doing and somebody give me a time check by the way. I'm sure I have, I had 45 minutes I understand. So I've 45 minutes talking in 15 minute question Q&A, right? I'm actually, so in terms of time, I would give my, I'm just two minutes over. That's pretty accurate. Okay, so Q&A and I think we can go, have they been letting? Okay, so we have like 15 minutes for Q&A. I have a question. This is regarding some figures that you presented but I didn't want to interrupt you. I noticed that, for instance, you presented that you measure the time that, I mean, the meeting time that you have, for instance. So also I noticed that you show that you work under sprints. So I am assuming that you work under an ashy approach. So how do you tie those kind of metrics if you are working under an ashy approach? I mean, it seems to me that it's pretty tough, you know. I don't know if I'm explaining myself. Yeah, I totally understand. So we have basically three types of projects, right? And it's pretty, it's important for us to recognize that they're different. And so we have support and that's a completely different department just around support. And we do measure time there because we have a different, we don't have an all-you-can-eat type model or basically a gym membership model where you said sell 2,000 memberships, especially after Christmas and New Year's, right? Like everybody's like, yeah, I'm gonna go to a gym in January. They sell 2,000 memberships and it only holds 200 people. So if everybody showed up, there would be screwed. So we don't do all you can eat. We actually, we set time aside and we measure and we charge our clients for everything because we provide value, we believe we provide value in every meeting. So we keep track of time there. Just so, and then yes, on development project, which is a different department. Agile is a little bit different. We take a look at it. We try to check once in a while and make sure those projects don't exceed kind of what we expect. But when the first time we did, when we first time looked at a fixed bid project and we took a look at the time of the meetings, it was like, wow, that was huge.  So if you show that to a client and Agile we have a problem, for example, and we say, look, this is how much time you're spending in times and this is costing you, it's like, okay, we have some clients that just really only wanna have 15 minute meetings with us. We're like, great. Did I answer your question? Talking about the measurement and about the amounts of project management, for example, you need or an amount of meetings you need. Do you show these numbers to the clients only afterwards after the project or do you have checks in between where you say, okay, we are not on track on budget and this is why? So Karen can tell you about this for hours and hours and hours. She had an interesting project just recently. We depends. So first of all, our philosophy is to be completely transparent. So all of our clients have full access to our time tracking and Jira. Every one of our team members is required to put in their time at the end of the day. We have a call out when it's not done at the end of the week.  And yeah, if we share, they have visibility into that. I don't have metrics on how many clients actually do look at that. Like I said before, when we pointed it out to this one client or like when we have overages on support they wanna get a detailed list, like everything is in there. That has to be communicated internally also, right? Because we share comments and so I make sure that those comments are respectful and so did I answer your question? Barley, so when we have showed it to, so our clients are aware of that. Obviously you're working in this business, you understand that this time is there. We haven't made it a practice of saying this is how much time you're spending in meetings. Like we don't break it down in those areas but maybe we'll try it. Sure. Anybody else? You had some questions before? In your opinion, what is the importance to have one account manager and technical account manager? Because in my company we just have account manager and sometimes we have problems because you don't have technical account manager to serve this customer in this time, right? Yeah, that's a really good question, right? So we have to take a look at it in context in terms of context for what kind of project it is and the size of the overall client. So for example, there are clients that I have a monthly check-in with them because they are a very key strategic client. We have recently introduced a concept of account managers and project managers. So we have account managers who check in on the emotional health of the client, the project progress, and have an escape valve. Because if you're the account manager or if you're the project manager and you're basically in the trenches, right? Sometimes if you have a conversation with the stakeholders or with the clients, project manager, they may tell you something that they may not tell your project manager it's just a different relationship. I think the other part of the question was you have your project managers who may not be so technical and then do you have solutions architects or developers sit in meetings where sometimes they're completely non-relevant, right? But sometimes you really, really want that technical resource on that and I don't have a great answer for you on that. I know this problem exists. I know that there's a need. Obviously we need to solve that. On one hand, we don't want to waste people's times and our client's money. On one hand, sometimes it's really valuable and it really gets that, when we talked about time to solution and time to resolution, especially on a support side, it really helps that. So we've seen a number of things. We have a 24-7 support team so we have teams like kind of around the world and we noticed that on certain client interactions it does help to have that technical team on these in the same time zone so that was really helpful and we're trying to basically listen to our clients and when we sense like there's a technical person needed, do it. Well, we don't have like a hard rule that says, every two weeks you have a half an hour of the technical solution architect, not yet, not yet. Sure, my pleasure. Any other questions? Yeah, sure. I love questions. So you said you do both like new projects and support projects and you have two different departments for that. So at some point there's a break. So at some point the project is launched and when does it go actually to support? And like, is it a matter of size? Can it go back to new project at a certain size of a new project coming up, like of a new development? How do you feel like that as you said it's about building a personal relationship so do these relationships break? Do you build up new ones? Do you leave them the same? Many questions, sorry. No, I mean these are really, really good questions, right? These are very, very important questions that we have asked ourselves and are still asking ourselves. You know, we try to have a hard definition around what goes to development and what goes to support. So we actually have very complimentary skills in terms of the technical capability on our development team and our support team. We have great developers and solutions architect in development and we have great developers and solutions architect in our support team. The approach is a little bit different and we have really big what you could consider continuous development projects and a support team. The way we internally define it right now is when we have a project that has a start and an end date is in development. When we have a project that has maybe a monthly budget or has a backlog of features and security updates and some other things, then we move them to support. That's how we differentiate it. At first we thought it was just gonna be the size but then we noticed that our support team is really well equipped to have those ongoing meetings without an end date and also we managed the security stuff and sometimes internal infrastructure and DevOps and we build it up for our clients and train them under support and they have other developers internally and we have other dev shops that are working with them and our support team is really kind of well trained and equipped to deal with that. Whereas our developers are just like we're gonna hammer this away and we're gonna do a great job and we're gonna crush this project and deliver on time on budget and it's gonna be great. So maybe that's kind of the same difference. Then the other question was how do we hand that off in a relationship? That's really key, that's a really big concern because we talked about we try to build great relationships so sometimes the account managers help a little bit so they help bridge that kind of loss of a friend of a relationship, don't worry it's gonna be okay, I'll introduce you to your new project manager that just as great as dealing. So the account manager helps a little bit. We do have a process where we hand off but really we do have to break and we announce this and say look, the project's gonna be delivered here, there's a warranty period and of course we're gonna be available for some questions and we're gonna do our best to do a transition knowledge transfer but this is going over to support. Hi, I'm a developer. So I started my company as a product manager and you said earlier that there is this emotional connection and I think I'm having that problem so how do you develop that emotional connection with the customer? I mean, I think it's very interesting. Well I think emotional connection is most related to how much you want to do for your client so yes, it has to do by putting yourself in the client's shoes and trying to think about how to make him happy but mostly it's based on communication because when communications won't go wrong then things escalate wrong very quickly. Like if you didn't tell him that his budget is almost gone or you didn't tell him that maintenance is going to be at a certain hour and his site is going to be down or things like that when you know that but sometimes you don't tell it at the right time then it becomes a big deal and that affects your relationship a lot. So first thing to work on, communication and really try to keep like a right outlook of life when you talk to them really. Yeah, so to me most of the time it's really easy like I really love our clients and I care about them and I want them to succeed.  Well they want to raise, they want their projects to succeed. So I really try to put myself in their shoes and see how and I think about it all the time and that's kind of how I interact with them. I talk to my clients all the time. I had an hour and a half conversation with a client of mine about some very personal things Saturday night while being totally late for dinner that I planned for months in advance. Like I just, you know, we interact very openly and honestly and I feel that's my approach. I don't know whether that works for everybody else but I think it's really important and I think in the end it shows that you care and also when you make mistakes and when we make mistakes and we really work on it like, you know, we dumped some AWS servers that were unrecoverable and I got a call from a client. I said, look, I was on vacation, like here's the deal. I'm on vacation, the door's about to close, I'm with my family, client, baby.  How can we help you? That helped, like that was a really big investment for what seemed to me not a big price and they're a lifer, I hope. There's no guarantees in life but I think so. Okay, I actually think I'm out of time. Gracias para invitarme, gracias para tus preguntas. And hasta luego.


Going on everyone Sam here with late in sports cards rip in 23 bowman draft baseball jumbo 8 box case number 65 random team style There's a look again at your random results as always you can find these on the website under the results tab as well as link in chat It's good luck everybody Let us do it to you this This got a bunch of people checking out right now, thank you all Again, 2020 bowman jumbo baseball 19 to go I don't think it has been no I don't there's no no word of it at least the Brady No, I haven't seen it. I haven't seen it. I'm out there somewhere. I have a feeling we would Yeah If it did come out by now feeling we would have seen it would have known right sit a Brady auto Come on Yeah, I mean now that you mentioned it cheesy I guess now we know we'll just pull easy She's he's got it. She's he's got it. I'm dipping it to Joey Bart. I Don't even want this you know Joey Is Andrew Wolters to 199 this is all papers donated right first base auto zap for in So Travis Travis. Let's do a baby first of salt ebon travel with the Giants and brewers in this pretty decent teams actually giant very good. We're not that either A Carson Rock of fortnight Jay says the high offer at $2. Okay, can anybody beat $2. He says the high offer the Tom Brady 12 of 50 that may or may not be 199 Chase Davis not sure He was pretty convincing the Zanzanatella cheesy said was it pulled? He said I'll have It's trying to throw us off No offers. That's the gifting of Joey. Are you doing Alex? Let me And that smile on Joe's face again worth it all priceless First your fracture Ottawa. I'm gonna take on it to 499 I'm seeing them are with the hair out First grand Taylor I have here in there by getting new braids for the yawks.  Oh man Theoretically, yeah, it's possible that if somebody like buys a box and goes home and opens it and Never does anything with it marion boy. Oh, it's definitely possible. Todd. It's Patrick Rylan. Seth Keener There's a lot of times that'll happen with like hiring basketball stuff It'll it'll go to collectors who do not have Instagram social media like that They they will not let you know about it. They will not let you know they have it that for and TJ Walton That's like the the the Luca in 2018 we it's never surface, but we don't know if it's still out there or not It's never surface. It's a good point Danny. It's like first tell you I won the lottery, but there will be signs Danny's got seven spots in this one there. It's every Sunday fun day. It's like Maybe Danny has got Daniel F. Coming through. Thank you. Thank you Daniel F. Closing us out on Sunday fun day taking lots of 24 All right, then There's Kemp Aldermen to 499 there will be signs and Christian Campbell plane to 125 Zach Levinson I don't know if I mean I was selling it. I was certainly telling but if I wasn't I don't know that I would yeah I don't know if I'd want all the harassment. Yeah, kind of just chill with it. You know, Andrew Lindsey Tom see first refractor Jonah Cox for a training camp To T with two spots and 10 box Tuesday basketball, thank you, buddy I believe that is down to under 20 to go as well now That is down to 18 to go very nice got to get knock out these last two baseball breaks everybody 19 on 2020 Bowman or Joey No, it cheesies on That's rough man. You lost like your biggest support TJ why? Why I know that's what I was gonna say watch it be in the loose. We've been Today first base out of Jake the Leo. It's happening Vaughan. You take it a one-on-one early twice Who's in 2020? That's Volpe to Mingas Jeremy Peña Bobby Witt and like there's a couple other names like Jerry Durand Like Adley's got autos Gonzalez first in Bonilla What's isn't what's his face has autos your don Alvarez has rookie autos Bichette's got rookie autos It's a good. It's a good product overall Sparkle Jacob Gonzalez to the White Sox Langford and shown what our first refractors I know they played it and little George and Kevin sin now. Yeah, who know and Jackson Merrill Nice Colt Emerson first refractor auto 499 going to the Mariners again Mariners doing very well Oh, hems very nice ends Graw and Phillip Abner It is a camfisher to 499 Glass here a white Chris Clark on your first refractors as well blue Hunter Owen to 150 Tray Morgan and Michael Roy first refractor is Brock Raidenberg and Gavin Cross Pullman Chrome coming up after this first base auto Kendall George We go into the Dodgers Congrats lawyers CJ K Fitz first refractor and Roger Arias Sick-looking Joey barkpatch guard. Hell, yeah camfisher To 125 Jack Mahoney for the rocks chase to war ski Nathan Detmer Nathan Detmer that is Brady house First of both me sir and Carlos Jorge Chris M with an absolute spot. Thank you, buddy And everybody you're looking at 19 to go so 24 to go on 10 bucks baseball and drop your link to that 4 to go refractor Luke Schliger to 499. There you go Travis Congrats, buddy Joe Redfield and Sam Zavala. Was it one of those box toppers? Ghost Forrester and Mermer. Oh, yeah, yeah, but he said it's a top stop. Yeah. Yeah, I know what you're talking about yet Colton Ledbetter true green for the raised to 99 Routus introser that is a true green at 299 on the paper Cole Young Jackson Wiggins wife. Oh, well gold Christian Baccaro to 50 Senator Bart see if he's for Melo greater. Oh, yes. Oh, yeah Probably would see why not at this point. Why not? Oh, yeah 19 to go everybody 2020 Bowman Baseball 4 box jumbo break Lucas Gordon lunar glow Chase Davis and Let's get it sold on out to continue on Jacob Wilson to 250 for the ace a Kyfer Lord and Kate Denton a Cade color Regio and Christian Baccaro Christian Baccaro That is a Davison Gutierrez to 199 Aqua wave. That's first Justin Reimer First base auto Hurston Waldrop for the Braves. I don't know cheesy. Tell him to come over here. Drop a link I know honestly Jack early chop our link the hell They got nothing better to do. Come on. Yeah, okay. Look at look at the way this is packed ready Kyle Tell me what's wrong with this every episode. Yep. Been one of those. How is that even possible? I was saying that the other week first if that was Thursday with Sean Makes no sense Must be a lot harder to do that, right Yeah, every other chrome. Yeah, I was like that for a couple boxes I don't know what was going on there first of a Toronto and Garrett Bowman. Nice ones. Yeah Very nice. Very nice Get 19 to go Bowman jumbo 2020 Working out on which seems like a strange subsection. Yeah, that's got to bring some of the Lower levels of society Never mind. We're good. We're good. We're good. We're good Spencer. We're good season favorites That is Mike Bovee for the brew crew to uh, 4.99 congrats on that one trap and Bigger disappointment how boys are real to me to the same spot. I think yeah, but uh What's up thermo thermo Trey the right to 99 I'm gonna get me height man. Let's go thermo ever pulled. Yes I've pulled a lot of one-to-ones in my day a few just a little bit ago like a thousand earlier today Like a thousand at this point at least as a gold Marcello transformative talent to 50 Santana and Davidson Gutierrez Gutierrez It feels like neither team is neither team's fans are happy. So yeah, it's cold here. Yeah, right. I know You're not a cowboy saying like oh, these are not the eagle.  It's he both are potentially firing coaches, which is crazy nuts, dude First base out of Hurston-Waldrup Nice one there for the Braves Lucas Gordon and Alex Clemmie I told Sean. Let me get a Clemmie. It's not mine. It's funny I'm not a fan of it Shady Shady favorite cheers. Yeah, we've done a few. Yeah, but one is only one today so far. Yeah one so far Green Brady house to 99 first of Hunter Haas Oh, yeah, Shady Sean also changed his mind. He's not going for the bills to win anymore. He's going for the bucks Busy. Yeah, let me say that. I don't know. He probably said it He probably said at some point First base subtle max Anderson Grats on that one the first name Friday There's Bryce Eldridge. The horse's name is Friday. Uh, it's just market value more than anything. I would say um It's market value more than anything volt. Thank you there tim Isaiah And David all taking spots tomorrow. Appreciate you guys. Yeah, some of it's the cards that are in there the people that are in there Some of it's the set themselves is like people's favorite for certain rookies or However, it is It's first of Dylan Hut. I've been Tyler It's about to say I'll run. I can't wait for people name for 99. I'm Daniel Bonant I don't think we'll have to give too much away. It is a Brayden Taylor to 125 And Kevin sin. Yeah, it's all a it's all a risk. It's all gamble and first refractor of luchlaga It's probably good Sparkle chase davis to 71 good stuff on that one Nice one there for the um Cardinals I just assume so Drew Hackenberg and Grayson. I don't imagine they would be right in the light in the light I would think he I would think yeah, I can't I can't tell you a hundred percent, but I would imagine at least uh True. No, that's that's valid Tyler drawn Watts Brown and Camden Rashi That Davis is worth it Get that chase davis into a mag cardinals was No numbered oran 199 cold carry Tommy Troy and tanner home Forrester first of emerson and canard Elgato Canard doesn't got dual first base auto tie Floyd the fever Magnetiba Yeah, some of them are expensive too. Uh both because of uh like the amount of hits they have in them Some will have four autos some one one auto. Yeah, so it'll it changes Yeah, I'll fluctuates Uh price eldridge on the baseball factory auto going to the Giants Congrats on that Stuff there again Travis with the Giants First of rodin and ferrata green grass yasser mercedes to 99 first of nevins and base Yeah, we got breaks of all all sorts of different budgets too, which is kind of nice They even pulled me a one-on-one James. Yeah Green to 99 is Joe Whitman Go bills and Sean would say. Yeah I wish they'd be rising It's like a perfect pc card pretty shady. There's base auto Austin tracer one of the Mets First Mike Bovee and Jackson holiday Gonna ready 19 to go 2020 bowman jumbo 19 to go. Yeah, I pulled a couple rugs a year Tyler. I uh I still have them They're around still they are trash First refractor Nolan the plane and Spencer Jones He won 99. It was marcello That is a purple Luke Schlager to 250 Guestimate. Yeah, I appreciate that Timmy first of dillon Campbell and Brady house Terms of speed gun speed depends on the product though. How fast you can go let Dan cook Max Clark lunar glow dan sniffs shoes Weird man, I like to spend his day It's uh, it's odd Very odd sniffing shoes Very odd man very odd You go saying there's something first base auto of Kendall George. I guess man. I guess yeah Just giving you some credit. I'll see you bond and willing to go ahead. Thanks chase rock of fortnight It's a guy from fortnight Gold wave Is noble mire to 50 nice one there marlins noble Maya first gold wave auto to 50 foists Whitman and Patrick Riley about Henry rugs Tyler 199 tj waltin Parents Would check out Green paper coal foster 99 Green papyrus, that is a you on javargas lunar glow nelson rada max anderson and ariel kastro Beed's been offered three games. Yeah, he's going first base auto joe redfield for the angels for a week Honeyman and Alishner's First refractor cambo louis anhela punya be triple double plus 11 50 damn Really? I will Crap will jc young to 250 The pads not bad money.  They scored four and like 30 seconds Literally, I look like five minutes later is four one 50 chase this cheese, man Cheese man leviton and chase and sherry. I'm like bad boy. I bet against them. Yeah, you bet against. Oh god Yeah, you put up like three goals in the period out there, right? Uh, why at length rate to 499 for the rangers and to chase davis Jonah cox and ryan clip odd clip dog Clip dawg Peter m with uh usc and bowman's best at university. Delight. Thank you buddy. A couple of usc spots 24 to go in 10 box baseball Even better. Well, yeah, it's not bad Or sorry, no, well, yeah Still not bad. We'll take Still good solid money there for sure Sparkle ink jake delio to 71 marlins killing it right now This and a gold wave noble minor not bad. Grats on that miami 71 emid olson and alberto lios first refractor michael carrico purple luke schleiger 250 first refractor jake delio as well Is a spencer jones? I'm gonna go a dylan head christian opera and injury walters first base auto is jack hurley for the diamond backs and first refractor mac whorebath glass of skeins how you leave charlie soto charlie soto That is a atlanta morales to 199 the blue jay's first in walder to uh 499 mawayahuna for the gigantes And blake mitchell first refractor Hello, sean. Good morning, sean. Sean. How are we doing? I got the bot 19 to go 20 20 bowman everybody 19 to go for he's terrible and then Sean yeah, now. He's pretty good.  Let's get it filled up Get it filled on up 10 box baseball is gonna be a lot of fun too Chase Chase thumbs down That is a luke keeshal glass of 23 auto for it the twins Grats on that sean thumbs down here they come case dad and tommy lock all out of the woodwork there You know that is a yellow lunar crater 75 blake mitchell Grats on that and first refractor paul wilson green grass to 99 is col young Madness kind of early and brady smith that is no one mclean pj nickles col foster an orange Orange tommy troy To 25 congrats arizona michael s Hell yeah, dude. Take that all day tommy troy to 25 20 to 25 very nice take tommy troy's And brady and taylor first refractor Madden tonight don't run Clark first refractor That is a jonah cox sparkle first for the a's Carlson reed first refractor and connerburns To 1 99 gradient paper homer bush and lunar glow of underneath you valdez first ethanol Kid color to 499 Jackson holiday image variation out of 99. There you go orials. I was not expecting that jeremy b. Congrats, buddy Jackson let's have a cool and jake cunningham ty pete's and first baton t which fractus Jace goes on man on a madden tournament. Oh jace changed his profile picture Notice that quickly that's the uh sketch colvin traver To 150 garret forster is first refractor colton ledbetter That's what i'm saying. True jeremy. Sure. That's a nice one now. That's a nice one, man Is jackson holiday And camp alderman first base auto jackson bowmeester Jace barofin She don't run jace true first jacob cravey and zander mooth Yeah, toad bias with a bowman's best spot Patrick d with the rockies and bowman's best and brad t with a 10 box Tuesday basketball mixer spot Thank you all Appreciate that Come on. Got a bit lazy with the autos in this case. Yeah, the mag stuff was decent, but I know what you mean It's a yellow of louis anhela cunha 275 My running days are limited jason woodward and jack mohoney feel that jace feel that First refractor hollerin and kade morris Let's do a quick recap here To 99 we've got col young chase davis to 150 Cole foster joe whitman yasser mercedes brady house tray morrigan col young to 75 blake mitchell the 50 marcello christin baccaro Base autos. We've got lou kiesel jack hurley jackson bowmeister Joe redfield kendall george austin troesser ty floyd Bryce eldridge max anderson herson wall drip twice Kendall george jake leo pennie mcgroth zack thorton the 499 mike bovey luke schlager Colt emerson kade collard to 99 colton ledbetter Jackson holiday 71 jake the leo to 50 noble mire chase davis to 71 for the Cardinals and then a orange true orange tommy troy That will do it for the break everybody. Thanks again for the filp up next is bowman