DJ, what made Matt Harvey so tough tonight? I saw a sinker for strikes, pounding the zone. He thought his stuff was pretty good today. You've uncharacteristically hidden to quite a few double plays to start the season. Why do you think that is? It's just bad at bats at the wrong time. Nothing in particular. I feel like today I expanded a little bit, trying to make something happen and ended up costing us. And that play in the eighth inning, did you think you were safe at home plate? It wound up obviously not having a bearing on the game, but did you think you crossed? I don't know. I couldn't see behind me. You know, I don't know. From the replay, it looked like it was a bang, bang play scoring and on judges. But now I couldn't see behind me. Go next to Marley Rivera. Hi, DJ, what have you? You're very self-aware at the plate. What do you believe it's happening right now, especially with runnison scoring position, where, you know, which is why you got your nickname of the machine? Why is that not working for you right now? I feel all right with runnison scoring position. I feel competitive. I haven't had a ton of a bats runnison scoring position. But, you know, it's just, you know, when our team needs it right now, the most, you know, hits aren't exactly falling right now for me or really for anyone.  You know, I know we're going to be better than we've been. Can you talk to me a little bit about that?   It's extremely frustrating. You know, that's really all I can say about it. You know, we're all frustrated.


Can you fix the folders? Yep. Ah, don't need to say yep. Right. To assist you with organizing or fixing your folders, I would need more specific information about what you're trying to achieve. Are you looking to restructure your project directory, rename folders, or organize your files in a certain way? Please provide more details about the folder structure you currently have and what changes or fixes you want to make. This will help me give you more targeted advice.  Yep, that will really get confused. Got it. If you have any other questions or need assistance with something else, feel free to ask. I'm here to help. Are there any good fuzzy logic JavaScript libraries?


Everybody forced here ripping two boxes of 2022 Bowman Chrome Baseball choice for Sean D Here we go, man. It's super frack of time ready There's a what black choice is the parallel for one of one year Let's just see something big. How about that? Start off with a speckle first bowman auto Victor Labrada 227 of 299 Labrada speckle base first bowman auto Samuel Munoz. That's a nice hit Base Samuel Munoz and base first bowman auto Jayden Rudd. All right one more purple first bowman auto Alexander of Valleys 40 of 250 base first bowman auto one been Joe Roller line on it there I'll send a boxing wrapper for the roller line And gold shimmer Cameron Colley auto 32 out of 50 On the back. Thank you for the order Sean. We'll get these cards out to you shortly


So, it presents a recent work in algorithmic game theory which is a very active and impactful area of research at the interface of theoretical computer science and game theory. And the story of algorithmic game theory has been a story of impressive bad news. Center solution concepts in game theory like Nash equilibria have been shown to be computationally hard. This essentially means that these key concepts like equilibria for these key concepts there does not exist polynomial and logarithms that compute these equilibria in general. But this is not the end of the story we have to address this bad news. And in the last follow, equilibria are widely used in game theory. So, at high level game study strategic interactions for example games are used to understand strategic interaction like auctions, markets, traffic networks and their games are also used in the study of things like social or biological systems. And equilibria in games model outcomes of these strategic interactions. So, for say for the prediction of outcomes of particular strategic interaction we need algorithmic tools, we need tools that address these hardness results, that address these complexity barriers. So, motivated by these considerations in this talk I present positive results. And by positive results I mean results that identify settings or things that can be done efficiently as opposed to these negative hardness results that identify limits of efficient computation. And particular in this talk I will focus on Nash equilibria which is one of the most central solution concepts in game theory. So, I will formally define Nash equilibria in later slides but let me give you a higher description of this equilibria to begin with. So, these equilibria so in games we have players, players are self-interested entities and players are payoffs. So, players pick some actions to maximize their own payoffs. And these equilibria at a high level you know distributions where players cannot benefit by unilateral deviation. So, they are stable in this in this sense, they are stable in this sense that if everyone and every other player sticks to what it is doing a player cannot benefit by unilaterally doing something else going to some other distribution over its actions. So, as I mentioned there is a lot of application pull for studying algorithmic results for Nash equilibria, but these are computationally hard. In a recent line of deep results it is possible that even in the case of two player groups games where I only have two players it is computationally hard to find an underlying Nash equilibria. But this hardness result has to be added this many applications that require us to find the underlying Nash equilibria. So, the core of algorithmic game theory that questions about computation of Nash equilibria. So, the goal here is to understand given the payoffs of the players under what settings can be efficiently compute Nash equilibria and what settings can be address these underlying hardness results. And this is an important question from an applications perspective, but I have to say that this is an important question from for fundamental reasons as well. And the point is that equilibria model behavior or model outcomes of strategic interaction between human players or organizations run by human agents. So, what is hardness results say is that equilibria requires human players to have solved a computationally hard problem. Now, this is a problem for the this is a question that reminds for the model itself. If the equilibrium requires a computationally hard problem to be solved by a human player we have to rethink the model. So, with this in mind again this understanding when we get efficiently compute equilibria is important for these fundamental reasons as well. And so now, let me move on to the positive results and it is that address these complexity barriers and a very natural approach very sort of classical idea in computer science is to in fact relax the output request. These hardness results say that computing an exact Nash is computationally hard, but intuitively one can think of computer relaxing this output requirement and studying the question of is it possible to compute an approximate Nash equilibria efficient. Again I will formally define what I mean by approximate Nash equilibria, but the goal here is to understand if given the payoffs of the players is there an efficient algorithm that finds approximate Nash equilibria. And this in fact is a central open question in algorithmic game theory today that does there exist a polynomial algorithm for two players even in games for two players finds an approximate Nash equilibria in polynomial. And first part of the talk and present results that will speak to that already address this central open question of computation approximate Nash equilibria. So, as I said this is sort of looking at approximate and so relaxing the output requirement is a typical approach in computer science. Let me ask step back and describe a complementary approach to obtain positive results a complementary approach to address these negative hardness results that in fact focuses on the input side of this picture rather than the output. And this complementary approach is based on a perspective that is quite prevalent in economics in game theory, but it is somewhat unconventional in theoretical computer science. And the idea is that in many studies it is said in economics do not observe these payoffs these payoffs are unobserved theoretical constructs. For example, if you for an economist a consume the utility of consumer is not observed, but what an economist will be observed is what are the choices of the consumer what did the consumer buy and not what the inherent utility is. So, these since these payoffs are observed this is not our input typically what is the input what is given to us is in fact observed behavior it is the choices that are made by the players. So, now in this set up we in fact have some flexibility what I mean is that given a put in given a specific observed behavior a specific observed behavior can in fact potentially be explained by different payoffs. And by that I mean the different payoffs such that the equilibria of these payoffs correspond to the behavior. This is a bit of a subtle point and I will sort of get back to it later in the talk, but just to give you an example say I observed the behavior of players they pick some actions what I can do is explain their behavior by a payoffs which are either in rupees or in dollars. So, to gain an underlying equilibria the strategies and the behavior does not change by the scale in which I specify the payoffs. So, there is some inherent flexibility in these payoffs specification this is a very simple example point is much more subtle than that, but the point is remains that for a specific observed behavior I can have essentially different payoffs that explain this observed behavior. And second part of the talk I present results that is leverage this flexibility to again address the hardness results for equilibria. So, that is the outline of the talk two parts that is clear let me move on to some technical results. So, let me first talk about computation of approximate Nash equilibria in particular show consider the case of two games where I have two self-interested players. Now, this is sort of common set here, but just to make sure everyone has a look on the same page. These two games represent settings where two self-interested entities simultaneously select an action to maximize their own payoffs. So, to kind of get this model right let me just give a very simple example. And typically in these games I list out the payoffs of the players as matrices. So, I have two matrices the first matrix is the payoff of the first player, the second matrix is the payoff of the second player. And these matrices is list out the payoffs of the players in the sense that the actions of the first player index the rows of these matrices and the actions of the second player index the colors of these matrices. So, let us consider the game of rock paper scissors I have two players both players have three actions rock paper and scissors the actions of the first let us say the first player plays rock and the second player plays scissors. So, the first player wins and we lead off the payoff which is 1. So, the first player wins it is a payoff of 1. And when the first player is playing rock the second player is playing scissors, the second player loses and gets a payoff of minus 1. So, for every possible action choice of the two players I can simply lead off their payoffs from these two players. So, that is a two player game that is an example of a two player game. What is noticeable about this example is that here there is no stable deterministic strategy. So, if the first player deterministically decides to always play rock then the second player can deterministically decide to always play paper and will always defeat the first player. So, that is not in the best interest of a rational or smart first player. So, in this setting at this time stable outcomes we have to consider settings where players randomize that is the players pick an action based on some distribution. So, the first player will pick an action i.e. one of the roles based on some distribution and the second player will pick one of its actions i.e. one of the roles one of the columns based on one of these distributions. So, let me generalize this representation throughout the talk I used matrices a and b to represent the payoffs of the two players. So, our two players both players in our personality have ten actions and I have this matrix that lists out the payoffs of the first player and I have this matrix b that lists out the payoffs of the second player. So, first player plays action i second player plays action j, payoffs of the first player is a i j and second player is a i j. As I mentioned before understand stable outcomes and strategies we have to consider probability distribution. So, I will use x and y to denote probability distributions over the rows and the columns respectively. So, here the first player picks an action picks a low based on distribution x and the second player picks an action picks a column based on distribution y. Now, with this notation in hand the definition of Nashi pilgrimage is quite intuitive. It denotes a pair of probability distribution where no player can benefit by unilateral deletion. So, let me pass through these inequalities. So, the left hand side of this first set of probability is the expected payoff of the first player. A is the payoff matrix of the first player and both players are playing according to distribution x and y. I just do this term by term product. What I get is that the expected payoff of the first player is x times y and what this inequality says is the first player cannot benefit by unilaterally going to some other distribution p. So, the second player sticks to distribution y then first player cannot decrease its payoff expectation by calling some other distribution p. And similarly for the second player first player sticks to distribution x the second player cannot benefit by unilaterally going to some other distribution p. So, now from a technical standpoint the problem of computation of Nashi pilgrimage as a following. I am given two matrices a and b and I want to find two distributions x and y that satisfy these inequalities. And as I mentioned this is a computationally hard problem that is unlikely that there exists an efficient polynomial time algorithm it finds these such distributions. So, now let me relax the notion that you consider an approximate Nashi pilgrimage. And the idea is you can look at the following definition which is quite standard in game theory it is very practical in the transfer definition. And the idea is that let us allow for a slack of epsilon that is an approximate Nashi pilgrimage as a pair of property distributions and the players cannot benefit more than epsilon by unilaterally. So, I add this slack of epsilon on the right hand side and get x and y to be an approximate Nashi pilgrimage. It is the same as if there is some inertia to changing the distribution for this sum the players are not optimizing to the last bit and you get an approximate Nashi pilgrimage. And central open version in the algorithmic game theory is to find an efficient algorithm that finds such distributions given matrices a and b. So, the computation of equilibria is a as a long and deep that area of research as I mentioned it is a computationally hard problem it is not surprising that the best known algorithm for finding exact Nashi equilibrium runs in an exponential time. This is a known algorithm by Lenby and Hausen. But there are notable number of classical games where exact Nashi equilibria can be completed efficiently. And in the off node are these zero sum games. So, these are games where the sum of the two payoff matrices a and b is all zero. So, these games model perfect competition. Whatever the first pair wins is exactly equal to what the second pair loses and so on. So, the rock paper scissor example I gave you was a zero sum game some these two matrices 80 by n b I get all zeroes. But the celebrated minimat scissor to one and one and by n b duality when you find exact Nashi equilibria in zero sum games in polynomial time. And here there are other interesting classical games like from the word that was done in 90 Bombay that identifies like classical games where exact Nashi can be completed efficiently. This is what it says that the rank of the sum of the two payoff matrices a plus b is one and again you can find an exact Nashi equilibria in polynomial time. Now moving on to the approximate side of things the best known algorithm for finding an approximate Nashi equilibria runs in time n to the log n an absolute Nashi equilibria. So, sort of famous result by Lipton, Marcaques and Mehta. This gives a quasi polynomial time algorithm for computing approximate Nashi equilibria in two main games. So, you can understand whether this can be reduced to a polynomial this is really important question you have got to be clear. And again there are specific classes of games where approximate Nashi equilibria can be completed efficiently. There is a recent deep result by Lungaila and others that shows that if the rank of the sum of the payoff matrices is small. So, I give you again there is the payoff matrices first there is the payoff matrices second there and the rank of this matrix a plus b is small and again you can find an exact Nashi and approximate Nashi equilibria in polynomial time. Ignoring the poly terms here for piece of presentation. So, the result that I present today will actually contribute to this line of work and will consider a very natural measure of games, the sparsity of the games. And I define this measure formally in the next slide but what I do want to mention and what kind of distinguishes this result from prior work is that it interpolates across the entire spectrum of games. For games there are near zero sum with respect to this sparsity measure will in fact get a polynomial time algorithm for computing approximate Nashi. And for general games there are any time of algorithm that I will mention will in fact match the best known upper bound of the Euclid-Leibniz algorithm. So, in terms of this sparsity we will go from this zero sum poly time case to this general result of Euclid-Leibniz algorithm. And guess the format definition of sparsity is quite natural I look at a game where A and B are the payoff matrix for the two players. And now I look at the matrix A plus B and it defines sparsity to be the number of non-zero entries in any column of this maximum number of non-zero entries in any column of this matrix A plus B. So, I have this matrix I sum it up every column let us say has equals s non-zero entries and this sparsity of the game is s. That is quite the means of one can think of it as a robust notion of zero sum. A zero sum model perfect competition where whatever one player loses is exactly equal to for the second player wins and here the idea is in most let us say in most settings what for the first player loses is exactly equal to the second player wins except for a few exceptions odd cases where both player lose and both player win and those are like the non-zero entries of this matrix A plus B. So, there is definition in mind is it possible that in real life scenarios that both non-zero ones are the only ones we care for? Sure. I mean, yeah, you can have you can have a setting where every entry is just zero both are getting zero and there is one entry where both are getting 10. So, they will just play that. So, yeah. In the light of fact, how would you justify this kind of explanation? So, it's, I mean that's one case but that's not necessarily what will always happen and it might be the case if, yeah, you have, yeah, it's just saying that the money doesn't fall from the sky so it's a general result. So, for those specific cases, you can do something with it. And here you have talked about sparsity I mean rank and sparsity after course related but is there any comment on that? So, rank is sparsely in a but the results are somewhat I mean, they are technically sort of the point being so I did think about it trying to port this result along the result or something of that sort but the change of basis if you going from this to the spectrum it kind of blows the approximation and so I mean I'm not it's still sort of ongoing work whether one can connect the two but in and of itself the two results are incompatible and what do I mean unless you get let's say A plus B is the identity A plus B so the full rank matrix so in that case and this is out of the question it's exponentially large even worse than the it's called a polynomial but for me if A plus B is the identity it's just a sparsity is one and I'm good I can find approximate natural quality time on the flip side I can give you a matrix just A plus B is all once the rank is one you can apply this another end sparsity is completely dense I would like to mention that I'm just working with this definition for ease of presentation the definition is much more robust like you can have small entries you can zero it out the dimensions of these matrices and with that sparsity the result I'm about to mention will hold A and B can be individually spars and again this result will go it's just for one key result and I'm working with this definition so with this definition in my form of theorem state okay let me make this concrete if I look at my rock paper scissors example again then this was a zero sum game of these two matrices is all zero so the sparsity here is zero all columns are zero and in general sparsity can be at most N these are N cross N matrices number now zero entries in any column can be at most N so I mean there's a technical caveat for me sparsity is always max of two and zero but for ease of presentation stick with this definition and here's the formal theorem of the statement what we can show is that in an S pass game we can find an approximate Nash equilibria in time 10 to the log S and the dependence on epsilon is as shown here and as is typical the playoffs are sort of normalized between minus one and one and the useful implications of this result is that when S is fixed the game is near zero sum so per column I have a fixed number of all zero entries log S is fixed, I get a polynomial time algorithm for computing approximate and in general S can be at most N these are N cross N matrices so in that case the running time of the algorithm matches the best known of our bounds of differential algorithm completely interpolates based on this natural sparsity so that's the sort of theorem statement I wanted to mention and I agree for a couple of slides talk about the key technical construction that goes into proving this result and it might seem a bit distant but after describing the result I can connect it back to approximate Nash equilibria and what's coming up is the key technical take away of the top if you want one technical take away from today's top that's the result and the result is an approximate version of the theory so the theory is a classical result geometry over a hundred year old what it says is that if I give you a set of vectors doesn't matter how many if I give you a set of vectors and all these vectors lie in D direction so D direction vectors and any vector w in the convex hull and we express as a convex combination at most D plus 1 of these D i's so pick any w in the convex hull you always write w is a convex combination at most D plus 1 of these D i's doesn't matter how many D i's you have now this is now considered a very natural approximate version of this theorem in the beginning I'm mentioning that this bound of D plus 1 is tight it's not too hard to construct examples particular vectors exactly D plus 1 of these D i's to be expressed exactly but here's an answer in the approximate version let's pick a norm P any arbitrary norm vector norm P and what we can show is that for every vector w in the convex hull the w might require D plus 1 V i's but doesn't matter where w is in the convex hull there always exists a vector w prime that's actually close to W for all w there exists w prime that's actually close such that this D i vector can be expressed as a convex combination of at most 4 P over that's what's going on so there's some scaling going on that I'll mention and like because I'm looking at relative error as a form of theorem statement that I'm giving this set of vectors and all these vectors just for scaling and the result was generally P0 of all these vectors for any norm P where it's equal to Z most 1 so I give you vectors they lie on or inside the P level so what the theorem says is that for every vector w in the convex hull is an excellent close vector w prime it's a convex combination of at most 4 P over 2 square P i's and what's noticeable about this result is that it's dimension free D underline dimension does not make an appearance so you're in a million dimension it doesn't matter if you allow an error of epsilon you can find a vector that's epsilon close that only requires 4 P over epsilon square so the proof I have of this theorem uses ingenious equality one can also instantiate some worries, empirical method from one aspect theory to prove this result in the interest of time I'm not going to be happy to talk about the top line there are many other interesting algorithmic applications that I have and still working on so the point is how is this approximate capital E connected back to Nashiv Kiribriya let's go back to the subject question is it in the convex hull it is and you can so by randomized algorithm it's very easy if you don't know so how does this connect back to Nashiv Kiribriya there's a very higher idea of the connection let's go back to the definition of approximate mass and notice that x and y are probability distributions x is a probability distribution for the rho k and y is the probability distribution of the column there and so Ay is not a vector that lies in the convex hull of the columns A1, A2, A3 these are the columns for this matrix A and y is a probability vector so Ay is a vector in this convex hull that kind of high level connects casually to Nash number of technical details that go into proving using this approximate casually to prove Nash in the interest of time I'll just give this high level picture if you're interested I'll be very happy to tell you the result of this comparison it is here or rather next slide please this one so can I actually choose I believe I can actually choose these the vectors see if I order them one to the other side yeah and I can if the consecutive interval will be twice the is that true so for W or for W no I am just saying that is it is it can I actually pick is it to do it the approximation of the by sparse vector is that there is at least some connection to the sparse this is like there is sparse approximation to W that's exactly what I am saying the point is I mean the point is you can pick different W's for different convex combinations for W's and that is potentially different W's there is at least one such W's there could be many such W's given a W you can find D plus 1 vectors will there exist D plus 1 that can be all the points for each point just look at the thing in this room I can partition it into two triangles so if you are in this triangle I need these and if you are in this triangle yeah talk about this result this combination of approximative Nash in these sparse games considering the problem of competition with the approximate Nash equilibrium in multi-clear games games where we are more than two players and in joint work with Yakov Babichenko and Ron Peretz we present the best known algorithm for finding approximate Nash equilibrium in multi-clear games so specifically we consider games where we have 10 players in action so M is where they are equal to 2 and in the state of the art algorithm the approximate Nash equilibrium for the case when the number of pairs is comparable to the number of actions of each player and the running term of our algorithm is capital N to the log of capital N where capital N is the input size of such a game so I have M players each player has N actions so I have this M-touple with N possible choices for each runway the number of action profiles is n billion and for each action profile I have M possible utility for each of the players the input size of such a game is capital 10 and what we are sure is that when we find an approximate Nash equilibrium for these large games in the capital N to the log of capital N the dependence on epsilon is really important and this research uses an interesting concentration bound for product distributions a bit more qualitatively what we do is to exponentially improve on the running time bounds for my deep layer games by Dr. Muratakis and Mehta and the result of Dr. Muratakis this is sort of a result right here and so that sort of wraps up my first part of the talk I talked about computational Nash equilibrium and the problems I considered I was given these payoffs for two players for my deep layer I also used a computed approximate Nash equilibrium and again let me step back and go to this complementary approach that I had mentioned earlier for positive results and the idea here is that in many settings payoffs are observed what we have is given to us as input is in fact the choices of the players you look at the input is observed behavior we have some flexibility in the payoffs specification in particular for a specific observed behavior you could have multiple payoffs and explain this observed behavior in the sense that you can have multiple payoffs matrices such that the equilibrium of these matrices correspond to the observed behavior let me be a bit more formal about it so let's formalize observed behavior as a collection of probability distribution so these are the equilibrium statutes we have a couple of probability distribution in observation 1 player 1 picks distribution x1 player 1 picks distribution y1 in observation 2 the second observed equilibrium strategy first player picks observation x2 the second player picks distribution y2 we have this collection of probability distribution and now for any such collection for any such observed behavior we have multiple games such that the equilibrium of both these games correspond to this given set of probability distribution it's possible now some of these games can be computationally high and some of these games can be computationally easy what do I need? well we know that games in which the line of one of the payoffs matrices is small we can find all national equilibrium of such games in polynomial time so given a specific observed behavior I can have computationally simple explanation for computationally simple low rank playoff matrices such that the equilibrium of these matrices contain all these observed strategies or I can have a computationally hard game that explains these observed behavior given set of probability distribution both are supposed to be optimal so there is no optimality it's an important point here that this is a testing exercise in learning prediction or these are important problems going ahead all I am saying is that here is this set of observed equilibrium strategies can you give me a statistically simple game which is formalized by land that explains these observed what are those probability distribution things so they are equilibrium they are potentially observed equilibrium equilibrium is optimal right so mine is an equilibrium for A and B so I mean it's a potential equilibrium so you don't know what A A and B is correct but yeah as you mean yeah so as you want to find A and B such that these are equilibrium of the situation it could be A and B that so unknown A and B but X and Y 1 are no equilibrium outside of no equilibrium so here I am giving you equilibrium I am trying to understand can you find a complete simple matrices that for which these are actually equilibrium but then there could be case that I can give you a distribution so that no game can explain them so model of those cases these are observed equilibrium strategies and you want to find a complete simple game thing and the premise is that a simple explanation a complicated simple explanation but yeah but these would be the linear any point in the variables of A and B sure so these are linear so you can always test whether there is some matrix that satisfies but when I have this line constraint then so then you have these linear spaces which you have to check with the line exactly so these are just constructed so in joint work with what we do is sort of identify properties of this observed behavior properties of these observed equilibrium strategies that quantitatively connects to lower angle quantitatively connects to these simple games so I will give you a specific example what we show is that I give you equilibrium strategies that have small support so you have X1, Y1, X2, Y2 so on and so forth and each of these have small support each have small support distribution each of them have support size S and you can always find an edge area then you can always find a game of line 2S plus 1 such that the equilibrium of this game corresponds to or contain the observed shine it does not determine what is this S so X1, X is a let's say I have only 2 distributions X1 is a distribution over N, it's a distribution over the rows Y1 is a distribution over columns I mean a distribution over N and let's say both X1 and Y1 have small support X1 only uses S rows it puts non-zero probability on most S rows and so look at the second observation X2 X2 is a distribution over N, it's a completely arbitrary distribution overlap with X1 does not matter but again in X2 the player is playing only putting non-zero probability on most S rows you have such a generic set of couples of probability distribution the result says that for such a set of observed equilibrium strategies I can find N checks of rank 2 S plus 1 so that these observed satsies are equal to 0 so the support satsies doesn't imply some condition of the rank 3 that's exactly what that's good but you're not missing the depth of the yeah it's not so so it's not uniform distribution so these can be arbitrary distributions so it's not so that's where the first factor yeah it's not even for the if I give you on the diagonal so I just have this set of observations the first observation both players are playing one one no distribution only one action the support size one the second distribution I have second observation both players are playing two parameters no randomization so you plot out these observations the line on the diagonal one explanation is the identity it's full rank so it's not obvious that for such a there's a low rank matrix and this is in this result it uses polynomials and so it's not a it connects to polynomials it's about no trivial observations so you can really look at certain low degree polynomials and make sure that we use those two in these matrices so it's not a there are certain in detail certain and we have similar results for so higher mention one result like that there is a basic assumption that that all these come from a single A and B way so if I give you this some sequence of vectors x and y and x y of support S then it's not clear that they actually you need the additional assumption that they also come they are the observed magically they come from A but from a technical standpoint this makes the result more challenging if I have different matrices I think for each I can come up with a new matrix no no we also if I give you some x k y k all of support K all of support S are you going to just give it this and no other thing will you design the A and B or are you saying that there you know the condition that there exists an A and B for which this I can construct A so if I just give you some x and y and x I can construct completely construct if you allow to feed on the A and multiple A's and B's can be given then your job is easier it is I mean for each so for a single observation I can order the zero sum because there may be you know there may be you see some say for in an x and y way there may be a support which is larger than x 1 including x 1 and larger than y 1 including y 1 which is also larger than x 1 so that maybe of that it might be so these are not the maximum yeah those are you can sort of so these are not generic these are not generic there are some just a testing exercise I mean fundamentally learning prediction all of these are fall over the first step is to understand if there is low rank and this is that you can sort of add additional constraints like these are the only equilibrium so not the only I mean one of the things that given an A, B you know there is this for any people this is like say I, J maximum so you are you are not saying that this I and J would be the I am just saying this in a difficult situation people would play a generic element of I and a generic element of J but I mean I am just for now I am just saying this is the first step this is there are other properties in the section of the sort of chromatic number of the data one such so yeah I mean the high level sort of takeaway message is that to many settings if your observation is structurally simple then you can come up with a structurally simple a comparatively simple explanation a comparatively simple game and hence I guess B is a complexity barrier for natural computing now we also have results of pure natural computing where observations correspond to single action choices for the two players so that is pretty much the end of the technical part of the block I presented two results one on computation and approximate dash the second one that reflects the addressing hardness from a coordinate from a complementary perspective and so going forward and focus was on Nash equilibrium these results had this Nash equilibrium have results of other types of equilibrium and going forward it is natural to ask if such positive results can be obtained for other types of equilibrium and for other settings as well and when you are looking to explore in the immediate future for these games over networks so networks are pervasive today and a lot of networks are now being studied in a big game so the idea is that players and the players are influenced only by its immediate meaning so you have this network and there are some games amongst each other so the network structure over this game the idea is to understand if we can get again approximate Nash equilibrium as hardware the idea is to understand if we can come up with algorithms for positive results for such settings as well and the technical standpoint this is an interesting setting for me because again here a lot of equilibrium competition problems can be cast as mathematical programs and the previous results I mentioned also sort of leverage structures of particular mathematical programs to find the equilibrium and find approximate equilibrium and so going forward I would like to understand the techniques and tools I developed that were approximate mathematics whether those can be extended or applied to address approximate competition in such a way another context where mathematical programs these tools naturally show up on markets equilibrium and price competition in the markets but these are two specific examples of domains that are to look at games over networks in markets my overall thing goal in this domain is to develop a structural understanding of approximation in game theory classes of games that on one hand are computationally tractable and at the same time are expressive they are able to explain large classes of observed behavior so the sparsity result I mentioned is an example of such a result the game is sparse you can find equilibrium efficiently and so on and another direction I am quite interested in working on is optimization problems so I talked to you about a competition of a national equilibrium competition and equilibrium in some context but it is quite relevant in many settings to understand if you can find an equilibrium that is optimal with respect to some object so for example it is natural to ask if you can find an equilibrium with the sum of the payoffs of the players or find an equilibrium it is Pareto efficient and so again this is a very rich area of research and I am quite excited to work in this so far I have focused on my results in game theory but in general I am interested in an approximation in optimization there are some topics and areas that I have worked in I am very happy to talk about these I am interested in conclusion I talked about the following four results today hopefully they gave you a sense of my research interest and I look forward to how these results connect to your chemistry so your which is before the first market game decision can you elaborate because markets are typically given by equilibrium conditions but my point is that in the mathematical program even the sparsity result that I had was using this to solve the philosophical viewpoint markets do you optimize your problem whereas game theory market for example should be given by market players given by very clear complementary conditions exactly, they are not seen as game theory because there will be an inversion of the decision markets so I guess my point here was are you meeting at markets are you trying to enlarge the decision markets or are you the traditional market so right now I am looking at traditional so one complete problem is exchanger it is taboo scar so exchanger typical both are given by market equilibrium conditions of price discovery but again finding approximate and so on remains interesting I am just asking you why have you called it game theory because they are not strategic by game theory are you asking them like encompassing in game theory so microeconomic so I am not encompassing these definitions I am just saying with these tools what can one say no but you are aware market theory conditions are ideal they are not based on rationality or stability they are basically based on ideal the market theory and price discovery and labour is employment and so on and let me finally be aware so these are not really where we allow the players to have rationality and stability I should not say this so that is a different competition I don't have any prediction markets so that is something different what I mean here is classical standard then you said that what is the agenda what is the agenda is it really just computing is it really to support some economic competition some competition that economics economists have posed or does it have an independent agenda so I mean it kind of this is not a single so that is one scheme of work and focus on what I wanted to talk about today what I want to talk about certainly it has so fundamentally there are a lot of game theories coming from engineering design computer sciences it gives design ideas rather than economics it is sort of social science analysis and that fundamentally distinguishes these are there any core agenda by itself which is not coming from economics so designing efficient I think on a competition of equilibrium equilibrium has been defined by a couple of people so is there anything so for example the algorithm word that was not seen by economists before so that is what I am saying designing certain paradigms but even analysis so certain options design descending price options and so on these are game theory constructs rather than just to go back to what I was saying the goal which differentiates is design rather than typically game theory is in the business of understanding like a physicist how do you explain this how do you understand this whereas game theory is also sort of the core separates it from some ideological strong game theory is a design perspective now I know these tools can I design an option can I design a neighbor can I design incentives that lead to competition so it is not like here is the word so in the Indian scenario do you think modeling means definitely certain things that I think are quite useful in some Indian context one of the examples is this thing called being done in I was in Uganda like text options could do with God could do with God and I think that is a very good game scenario platform for pricing something that is very important


Quick check, what's 5 plus 3? Okay, what's 4 and 9? This is difficult, what's 4 and 9? People are weak. So, since we have only 30 minutes, I just checked with them. They said yes, we are going to switch off the projector. So, I shall speed up a bit. But if anybody wants to take something offline, move it happy after this. So, this is going to be an exploratory session on how we can do user research a little differently in our world. That's becoming more mobile, more connected, more and more bite-sized. So, this is going to be an exploratory session from my experiences from something that I've learnt, others are doing as well.  Oh, he's killing me. The sushi is mobile. You can take it whenever you go. It's literally bite-sized. And if you look at how popular sushi bars are becoming all over the world, I thought that was a perfect visual representation of a connected world. So, that's the reason the sushi is there. Okay, so this is the topic. I'm sure everybody knows this was the topic for this year's UX India, which is driving value through customer experience. When I read this, the first thing that came to my mind was is a customer the same as the user. For whoever attended Prachi's session and the Q&A, this would be a repetition, that somebody asked if the customer and the user are the same, right? So, consider this example. I buy a gift for my friend, who's the customer, who's the user. I'm the customer, my friend is the user. I buy diapers for myself. Obviously, he's the user, right? And I'm the customer. So then, if we're talking of driving value, then value for whom? Is it value for the customer or is it value for the user? The reason I'm asking this is, I've seen a lot of people use customer and user interchangeably nowadays, right? And largely for convenience, because they use the two interchangeably. But I've found people at both ends of the spectrum. I've found, in my work experience, I've found people saying, oh, you know what? I've asked my customer. They've told me exactly what they want.  Right? And at the other end of the spectrum, I've found people who do lots of user research, make awesome products, which are, like, perfect for the user. The problem is they totally ignore the customer. So that's why I find a lot of well-designed products remaining unsold, because the customer's not taken care of. You might have designed a game which is awesome for my child. I have one-and-a-half-year-old son, okay? So if I just start giving him to any child examples, please bear with me. So if you make an awesome toy for my child, who knows the user, but the toy takes, like, three hours to set up, I don't want it, right? So that's why long explanation for a short answer. Love their users, but love their customers too, right? Value for both users as well as value for customers. It's a tightrope work and it's easy to stand here and say that, but it's really difficult when we start doing it. But I think as user researchers or as UX professionals, I think it's our job to find that sweet spot, right? Which is what is value for the customer and what is for the user and to find that sweet spot. That's our job. So for today's session, I'm going to talk about only users, but the same thing applies to the customers as well because I'm talking of user research and value for user, but the same thing applies to the customers as well. So if you look at the kind of things that we do in our job every day as UX professionals, as product owners, as entrepreneurs who own startups, you can pretty much put everything into four buckets, right? Either we are discovering what is value for our users, right? Finding out what is that need, what do they want, what do they really care for, what are their aspirations. After we discover that value, it's our job to communicate it back, right? You come back and you tell your team, you know, this is what I found on the field, right? That's the communicating part of it.   So that's a creation part of it. And finally, you assess. You go back and check if what you've actually made is really of value to the user or the customer, right? So this is what I call the value wheel, just some clarity in my mind. So when I talk of... Okay, so if you look at the traditional divisions, right, of evaluated user research, don't think too much about it, but there is generative user research, there's evaluative user research. So this part of it is pretty much generative and this part of it is called as evaluative. Not too much to worry about. But when I talk of how user research can be changed, I'm going to talk of some major shifts which are happening in the world these days and I'm going to talk in terms of how we can bring about some change, how we can leverage those shifts which are happening in these four buckets. Okay, so far? Follow me, am I too fast when I'm speaking? Okay, good. Discovering value. How do we discover value? How do we do user research most of the times? Anybody?  See the user in their own environment. See the user in their own environment which is what pretty much is happening here. Interviews? Talk to them about the daily lives, qualitative, quantitative. But if you look at how we're still doing user research, we're still doing large formats. You say, I have half an hour of your time. I've done that so many times. You have one hour of your time and you say, one hour, I'm not going to give you one hour. And they say, okay, fine, 15 minutes. And I try and hope that I can extend it. But I'm pretty much still doing large format interviews. But look at how people are shifting in terms of how they communicate. This is the major, first major shift that I've found. Anybody remembers this? Archie, Betty, Veronica, everybody's grown up on those. Remember, Betty would come back, Veronica would come back from a vacation, some exotic vacation and sit on the couch and chat for hours with Betty, right? That's how people were talking earlier. Our communication was long, but it was paced out, right? When I was in the hostel, I used to call my parents every three days because they were on the mobiles. There was landline and to go dial it. And that's how communication was. Versus, look at how it is right now. When I go shopping, if I see a nice dress, I try it on and click a picture, send it to my sister and say, what do you think? She says, great, bye. That's it. Done, right? So it's short, but it's always connected. When I go home, I try it again in central pictures and say, what do you think in the evening? Like, how does it look, right? So we are always connected, but we're doing short bursts of communication, right? The other shift that's happening is everyone's moving, everything's moving. I don't need to state this again. Everybody knows the word is on wheels, right? I've done usability tests for like, blogging platforms and all, right? Where you get people into the usability lab, give them a computer, they'll start a blog and say, how do you type a blog, publish a blog? Look at how people are writing right now. You start typing on your computer. You continue typing on your, maybe iPhone when you are in an office shuttle. And when you go home, take your iPad and post it, right? So there are three different devices, three different locations. So everything's moving, everyone's moving. That's the second shift. Now, how can we use this tool to change the way we do user research? To discover value, right? I think the mobile. This is a highly untapped area. Why can't the mobile be used as a tool, right? People take photographs. People take thousands of selfies. Why can't we use mobile phone to actually understand a little better about the environment? This is an example. I found somebody's done this. This was more for market research than user research. But what it does is it says, take a picture, you know, describe your mood, or what are you doing right now. So what they're doing is actually they're blending user research, keeping it short and crisp and blending it what people normally do, right? They're not asking somebody to come to a lab. But they say, you know what? You take tens of selfies. Why don't you just take a photo of what your house looks like and set it to me? And that's one way that somebody solved it. Here's an example. We were doing a diary study when I was working in my earlier company. You know what diary studies are? You give people a diary and we had given them an internet device and they had to every day make a note of what all they have done on that device. What we found is nobody writes anything and typically Indian habit, I'll say not even Indian, global habit of last minute preparations like how we do for exams. Just before we would go visiting them, they would quickly scribble something in the diary. So how we tried to overcome the problem is we would call them every day. Just a five minute call and say, hey, what are you doing? What did you do today? Did you use the device? What did you do? Can you just tell me? That five minute call was very low investment. I can talk to these guys for five minutes, tell them something. And that became a primary source of information. We knew whatever's in the diary is going to be some last minute cooked up story. But we gave them the diary nevertheless, but we relied more on these five minute calls. And you know how often, just before I got here, I was checking, I got a WhatsApp notification. I was checking whose sender message. Even if it's a forward message, we are addicted to it. So imagine if you use WhatsApp for running user studies. I mean, we just need to make that shift. It's possible, might be a little difficult to start with, but I think just moving from that mode of long format to short format. Yo, I don't know if you guys heard about Yo.  You can really send a Yo. But there are people who studied, for example, TV watching patterns. You want to understand how people watch TV in a day. If you tell them, every time you start watching TV, send me a Yo. Every time you stop watching TV, send me a Yo. And just by that simple act of sending a Yo, you can get a timeline of how people are actually watching TV. It's a simple tool. Oops, I'm back. Have you seen this? It's a small camera. It's a small camera called narrative clip. All it does, it takes, I think, two pictures per minute and then uploads them onto a cloud. Right? So you don't have to go clicking pictures. It takes pictures. So if you are doing like a day in a life study, I've done it with doctors and I've found it very difficult because you don't get access. If they are with a patient, you're not supposed to sit there and watch. Imagine if you give them this and say, all it's going to do is take pictures. You get to finally veto if you don't want a certain picture and we don't put that in. But at least it gives you a life log of what that person is experiencing in pictures. Right? So this is another great tool that it's a little expensive, I guess, but if your company can afford it, nothing like that. Okay, so that was using the mobile actively as a tool. The other way of doing this is to use mobile passively. Right? I was just having a conversation with somebody sometime back. They were talking about data mining and they were talking about how when you use a mobile phone, there's so much information, right? Location data, who you have talked to, where you have gone, what you have seen on your, what you have browsed, whether it's acceptable, non-acceptable, you know, whatever. But there's so much of data that's available. And so somebody was telling me this, the other day, saying that, you know, what's within Google, what's the most used feature of Google? So I thought so, but apparently among the Google engineers, it's Google logs, the logs that people have, right, of what site. So apparently they rely, and I don't know because this is an unverified story, but apparently they rely heavily on just looking at logs of what people have done to understand people, to find, we are creatures of habit, right? Everybody will have some or the other pattern.  Can I tag this as home? Can I tag this as work? Essentially they're looking for patterns, right? So the other way to do this, a little advanced way maybe, is to use the mobile phone as a resource, right? The third shift that's happening is this whole 60, six degrees of separation, that's shrinking. Everybody's becoming more connected. I don't know if you can see, I found out I'm connected to Barack Obama through like three degrees of separation. I was like super great, right? I blocked out the guys in between, but yeah, I am apparently connected to him by like three degrees of separation. So that's shrinking, we are becoming a more connected world. There was a time when this whole man with a mission was like, I'm the sole hero, but now that's becoming groups with a purpose. Then meet up groups everywhere. So groups is becoming the unit of thinking. It's not individuals anymore, right? So again, I called it social butterfly effect, but what's happening is people are becoming more and more social and everybody's having these circles of influence which are expanding, colliding. So I meet somebody, I met like five years back at you and linked and through you I find somebody else and so on and so forth. So what's the effect this has? Groups shape identities very strongly, group shape values. So if we are considering, if I'm designing something for him, I can't consider him in isolation. I have to consider who are the people who influence him, who are the people he can influence, right? So that becomes important. You have to consider multiple perspectives. If he's trying out my service today, where his experience ends, maybe he's will begin because he'll come and tell him, then you know what dude, I tried this, this was super awesome, you've got to try it too. So that's why we have to consider groups. The other side of it is because there's so many groups around, we have ready communities of practice that we can tap into. I was doing a study for patient experience recently. All I had to do was go online. I found so many communities, patient opinion, you know, patient matters and healthcare entrepreneurs and there were so many people available just as groups and you don't have to go scouting like, you know, I want participants for my research. If you get into one of these groups, you'll find a group of people that you can tap into, right? Okay, so that pretty much covers this cover. So far everybody is with me. Okay, now I'll come to the fun part which is coming back in communicating. So you've found what is of value. Now I'm going to communicate that to your team. Okay, so what's the classic standards? Do we, how many of us still make reports for user research? Yeah, pretty much reports, no still, right? Okay, so this is one shift which is happening and again, I'm just stating things which are actually common sense but so we are moving from words to visuals. Look at how many things we do in our everyday life which are purely visual. Instagram, medium, nobody writes a blog spot anymore. Everybody's moved to medium, right? There's a nice fancy picture, a nice fancy title like how I taught my grandma marketing or why I stopped eating potatoes and beans and things like that. Five days to, you know, all of that interest, Facebook, YouTube. So we are becoming a more visual culture with literally thinking with our eyes, right? And again, we are wired to be visual beings. That's about fact. So I get it back to our team. We should make it visual. We should make it short, visual and interesting. I'll show some examples of how I've tried doing this at work and I mean, I've had some success. People are at least interested, you know, rather than saying that, oh, if you have a research report, you can read it later. So I've found that it actually works. So this is something that I did. So I did a research on SAP. So I work at SAP. I did a research on SAP's implementation experience. Huge beast. So I came back and I put together this board game. It had, you can't read it. It's too small, but it was like a board game with different parts of the implementation experience. I used construction as an analogy and explained it to them, right? The other thing I used was bit strips. Comics. Anybody is ready to read a three-page comic rather than reading a ten-page usability report, right? So I've used comics. Works well. This is another thing, the study that I was talking about, patient experience. What I've found is the most important thing out there was there were these movements, there were these little stories, right? Ultimately, user research is about stories. I mean, we get lost in this whole data and, you know, 10 out of 17 people sent this, but it's about stories. So I found these stories. I mean, you can go through them, but they were really powerful stories. Saying that, you know, I'm sitting at the doctor, he has a few minutes, I have many silly questions. How do I ask my questions, right? So all of this became the stories that I wanted to take back. You know what I did? I made a desktop calendar. I printed it out. I went to Printo and I printed it like a calendar. You had these quarter-day calendars, I don't know if you've seen. Every day you have a new quote, inspirational quote. I made this into a calendar. Each of them was color-coded, categorized, and it was like a desktop calendar. And I gave out this desktop calendar to my team. And I just leave it on the desk, right? And live it, read those stories, and you'll realize what actually are the opportunity areas. You don't have to do a major session for it. So these are some ways that I have tried making it visual. This is another way that I found somebody else had done, which was to actually put it up. This was a design agency.  And all their research finding, they made posters, they made it like an exhibition that everybody could come and watch. And it's super interesting. And the second thing I think is important here is we come back and we communicate our findings to our team. But we don't go back to those people and communicate it to them. You know, we have a very transactive model of this, right? Because somebody who recruits a user-search participant, do the research, compensate them, give them a little voucher, and say, thank you very much, and that's it, right? But what's happening is it's becoming more and more important to establish communities of practice. So if you find connections, go back to them, tell them, you know, this is what I found from this study. It's just going to keep them connected to you. Somebody was talking in the morning about having that relationship with users, right? So this is what it is. Then you go and communicate it back to them. And these dotted lines of feedback and influence, you're going to strengthen them and finally they'll come back to you, they'll feel more comfortable to you, and you can tap into that resource for getting more inputs, okay? We'll move on. We'll be good on time. What's happening here is where I say create. So what typically was the thing is user-research, many karthiya, it's your time to create, right? User-researcher gives inputs, designer creates, already developer creates. That was how it was thought to be, no longer. Now the model is everybody works together, so it's equally our responsibility, okay, sorry, equally our responsibility as well, to be a part of that messy creation process. So what have seen happening in creation is we're moving from co-located specialists. Earlier there was like a user-research team, a design team which was co-located, sitting together. Now it's moving to multidisciplinary teams. Obviously, everybody knows that. And they are spreading out. Everybody knows 37 signals, yeah? Spread out all over, right? They've written a book on how to actually manage companies which are located in different locations. And collaborative creation. Everybody knows open-ideo? Open-ideo, yeah? Where they post a challenge and then people post solutions, squaki, everybody knows squaki. Squaki is similar, right? You can post an idea, somebody builds up on it and then you post. So we are going towards, spread out, geographically spread out teams and we're going towards multidisciplinary teams. So we are moving from waterfalls to alterations. Again, abandon fact, right? Everybody's talking agile. Everybody's talking alterations. And the waterfall mode is pretty much disappearing. What that means for us is because we are doing lots of alterations, because we are spreading out geographically and because we are multidisciplinary teams who sometimes may seem like Greek-Latin to each other, right? What's happening is that sometimes the value might get lost. Sometimes people, as a team, in the whole thing about features and coordination and meetings and alterations, you might forget why you were doing this in the first place. Many times in the startup project, it takes three months, four months, and people are like, okay, how much? What do you want to do? I hope everybody understands Hindi. Sorry, I sort of go back into it. But sometimes people go like, oh, why did we start doing this in the first place? What were we trying to achieve? What was the need, right? So that's why it becomes more and more important for user researchers or UX professionals to hold on to that value. That's why our presence in that whole creation process is more and more important, right? It's coupled. I don't know. It's pixelated for some reason, but anyway, it doesn't matter. What I've found is doing posters. This is super, super valuable. I used a Wolverine poster here. It's dead simple, right? Okay, 10 minutes to go. I'm pretty much done. The Wolverine poster is, it's dead simple, right? And if you can actually make a poster for your product or your service and just pick it around so that everybody remembers what was a whole need, right? Even if it's a photograph, that kind of sums up what you're trying to do. If it tells you, you know, what your product should be for, who it should be for, that's enough. So I found posters to be super effective. Just at the end of it, before you start creation, simple poster, non-threatening, right? It looks completely non-threatening. Just put it up. I mean, people can remember this each time. The other thing I wish I had time to show this, but I don't. So I'll skip over, but I'm sure most of you have seen this. It's Twitter in plain English. Have you seen this? Anybody? Okay, just go look up on Vimeo or YouTube. It's called Twitter in plain English. It basically explains what Twitter does using just cutouts and plain English, right? So it doesn't talk about the product feature. It's just drawn out with paper and pen. And it's like a stock portion kind of a video which shows what Twitter is, right? So these kind of videos I found sometimes help. That you do a video and leave it. So even if people are sitting, some of the team is in China or team is in India or team is wherever, it's still easier. You can just send a video and say, you know, this is what we are trying to achieve. And that stays with people, right? So what about quirky? We are moving to collaborative creation. I was talking of open idea. So I found a really interesting example of how you can make this fun. Because if you're doing collaborative creation, right? I think it's very important to keep it fun for people. Otherwise, what's the fun? Why do people go to open idea? Because you see such inspiring ideas from everybody. You see that something is being done to your idea. So I found this nice example. Has anybody played this game? It's a game called I-Wire. Okay? It's actually, it looks like a game. But what you are actually doing is you're mapping real neurons. Okay? There are these neurons that they wanted to map. And it's huge, right? So what they decided to do is make a game out of it. When you actually pay it, it's a game for you. But what you're actually doing is mapping neurons, right? So it's collaborative value creation that is made completely fun, right? Try thinking of that. If you have a problem that needs a lot of people to work together, try and see if you can make it fun for them in some ways, right? So that's the create part of it. And the last part of it is assess. This is what assessment looks like, no? Usability testing. Get somebody into a lab. I've done usability testing for maps, a very popular map service on the mobile phone. Do you know how we did it? We called people into the lab, gave them a mobile phone, and made them nicely sit in a place and then try it out. Does it make sense? No. Why? Because when the guy is actually going to use maps, probably one hand is on the driving wheel, one hand is on the phone, and then he's trying to make his way through some crazy banger traffic, right? So it doesn't make sense to do assessment the traditional way, right? A lot of people doing remote user validation where your user can be across the world and you can do remote usability testing, helps because it extends your reach, right? You don't need to travel someplace. You can get wider users. But this is a shift. Almost everything that we do now, we're moving from computers to mobiles, right? I've still not found answers to these is how we see what they're seeing and doing. A large part of user research is not asking for observation. How do we observe what somebody is doing on a phone? It's difficult. It's easier on the computer, right? If I tell you to sit on the computer and observe from here, it's okay. But if you're doing on the phone and I come and see like this, nobody wants it. So I have not really found an answer to this because there's not yet a good, a very good way to do this. Largely, there is software like Dscout which people use, AskMDscout. There's some software that people use for doing mobile user research. But it's still self-reported. You know, you're asking people to report what they feel, what they see. And it's still not the best model, right? So that's... If you find something nice where people are actually doing this, please let me know. I'll be happy. The other shift that's happening is we are moving from usability to user experience. So how do you test your experience? User experience is that simple to test. How do you test your experience? It's spread across time. It's spread across people. It's spread across spaces. And there's so many moving parts that sometimes how do you test the whole experience, right? Another question where I still have not found convincing answers. If you found interesting models of somebody testing experience, please let me know. I just wanted to share this. This is something that I've found. Again, I'm quirky, is how people are doing crowd-sourcing, crowd-sourcing validation and assessment. So there was this thing, if I had an idea, you know, what will people say, you know, I don't want to really expose it or get feedback on it. But actually, a lot of people are now open. I think this is largely because of the entrepreneurship culture that's coming up. I don't know. All it's because of this whole open, connected thing that's happening. But people are actually crowd-sourcing things. So if you have a product or a service that you want to test, why not try and see if you can crowd-source it in something? Try and put feelers into a coffee area. You might get some responses, right? So, I mean, what I mean is people are directly doing this, right? You're hacking it. People are pretotyping things. People are... It's not a typo. It's not meant to be prototype. It's meant to be pretotype. There is this small booklet called Pretotyping Manifesto by this guy called Albert Savoya. Go read it. It's super interesting. It talks of how a pretotype and a prototype is slightly different, right? And how are the ways you can hack things, literally. So, do it the same way. That's a major change I see in companies. Big companies are also starting to think like entrepreneurs. And I'll give an example. So, this is a patient experience research that we were doing. So, we wanted to get stories from people. Remember the stories I showed you? They came after a lot, lot of tries. We tried everything. We tried leaving little things on people's desks in office. We made a Tumblr site. We made a Facebook site. We called people up. We tried our own networks just to get people's stories. So, the point is that we have to be flexible. It's not like earlier that you have a research plan and for six months you have a research budget which you still will need to have. But the point is that I think we need to be more flexible. Try a more mixed media approach because people are nowadays attention span is so less. So, we need to be constantly, you know, trying out new stuff. Iterative research approach. We iterate while we are building. Why not iterate when they are doing research? Right? Not like reaching different stages. But within the same stage also try and iterate your research methods. You might find something different.  Am I going back? Okay. Be creative and be exciting in how you approach this whole thing. And I think if you do that, if you make it fun, you'll discover everybody members. Just checking. Discover. Then, communicate. Then, create. And as I said, yay! Good. That is the thing. So, if you do that, be creative, be exciting and be able to do user research differently and please share it back whenever we meet or stay in touch. I'll be super happy to know. Yeah. Thank you. Yeah, not a question but just to add on to your mobile part it actually a couple of parts launched in last month. Okay. Where it actually records the way it is, the person is having both the screen as well as the camera. You can choose front camera or the... Right. I'll drop a... Sure. So, by the way, that reminded me, IDEO published just four, five days back, I found it. IDEO, just go look up research in digital world, user research in digital world or design research in digital world. It's an article by IDEO and they talk of the different tools that you can actually use like Ethneo and all of these tools, right? So, they talk of the different tools that you can use for doing mobile research or remote user research and all of that. It's an IDEO article. If anybody is interested, look it up.  If you had that, there's a site called remoteresearch.com. Yeah. Type remoteresearch. Hope you'll have all the tools around. Correct. Remoteresearch.com is another site where you can find a whole bunch of tools that you can use for doing remote research and IDEO's article has some bunch of other interesting tools.


think about Beethoven. You know, I love the musical Beethoven. I don't know if you guys like him or not. Maybe the ancient rap Beethoven is not that popular. But I think Beethoven is one of the great artists of all of human history. Beethoven is the first musician of serious music to actually make a living from his music. He actually held concerts and sold tickets. It was capitalism, right? Beautiful capitalism. Mozart, remember, could only make money by doing what? By sucking up to the church and to the aristocrats. Beethoven didn't have to do that anymore. He could count in a middle class, a growing middle class, sell tickets to his concerts and people sat and listened to beautiful music for a long time.


Dennis you're listed as probable with the back. Can we expect to see you go tonight? Yep. How have the last few weeks been for you in this recovery process? They've been the first couple of days was tough because I was in pain couldn't move couldn't you know I was I was hurting I was in a lot of pain but um you know I came in was doing my treatment they left me back on the road trip which was a good idea on their behalf just in terms of traveling sitting on a plane and things like that but I came in every day got my work in and I feel better I'm ready to go now. How tough has the season been for you? You missed some time you came back played well for a couple of games and that went right back out on the line. It's an NBA season man it's like life ups and downs you know I just deal with it as it comes I don't look at it as tough uh shit I've been blessed with enough strength to deal with it I'll be all right. What did you kind of notice from you know watching on TV watching film on that road trip? I feel like we weren't competing at a high level honestly I just feel like we went out there and we was we just going through the game almost like we was trying to get the games over with you know we got to come in every game and feel like we can win and understand that ain't nobody gonna give anything to us everything gotta be taken so we get that mentality back I think we'll get back on the right track. How much do you think that was just the road trip and the grind of those five games just draining everybody a little bit with that mentality thing? Um I don't know man some people say that I ain't really big on the excuse man like I said uh you don't never ask for no easy burden so that's what comes with being an NBA player that's part of the job we gotta go out there and do what we're supposed to do and uh week we I feel like we had to let our fans down and we let our coaching staff down because everybody put a lot into it so we didn't we didn't hold up our end. How good was it just to kind of come back here in this building to have a shoot around coming off of the losses and just what was the message as a group today this morning? Detail we gotta be uh we gotta pay attention to details and have some gameplay and discipline that's what's been the key for us and uh like I said I think if we do those things we come out and compete we'll get right back on the right track. A lot of the guys on the road trips that they miss your athleticism your spirit for what you bring coming off the bench for this group when you look at this Nuggets team what will it take defensively and execution-wise when you have a Nicole Jokic at Jamal Murray on the other side of things? They got guys as All-Stars Jokic MVP um Finals MVP so it's not like oh we're gonna come out here and do this and we're gonna stop them uh we just gotta follow the game plan you know we gotta follow the game plan get them to choose the shots that we wanted to take compete and uh you know we just gotta trust in that we gotta trust in that that's gonna lead us we can't put our heads down that's one thing for sure can't put our heads down and and keep going. A lot of the guys I think are Jaka said that there needs to be more physicality do you see that that was missing on this road trip as well and what you guys mean now? Absolutely absolutely that's just that's a by product of competing you know I'm saying or or whether it was being comfortable I'm not really sure what it was but we got to go out and play for each other you know from top to bottom everybody got to go play for each other uh whether you're getting shots or not you know it's it's all about it's a team sport you know I mean somebody got to sacrifice something if we're trying to win and that just is what it is. During that's when you look at over the last few weeks kind of shots at the rim for the team have gone down has that been something that's been addressed shot selection getting to the rim that sort of stuff? I think people are pretty aware of what's going on um you know universally in the NBA you want to stay away from so many mid ranges and we settle for a lot of them uh you know like I said it's a team game man everybody we got to understand sacrifice you're not going to get to do whatever you want to and that is what it is you know what I'm saying uh we need shots at the rim you got to have room pressure no matter who your team is that's an important part of winning games so we need that. It's one of these last couple of weeks when you were out what were the conversations like and how did you kind of keep in touch with the guys on the road? Well coming in and playing a couple games and then being out for a road trip I kind of felt like I was letting my team down so I had reached out to them before they took off and just let them know like I wish I could be there um and they kind of fuck with me that I couldn't so you know I addressed that and let them know how I felt and then uh you know I would just keep communication with them throughout the games like after the games we all got a group chat and I'll you know I'll send my thoughts or tell them like all right we we got to come out execute we need this one and things like that just trying to trying to have a uh a vocal presence even while I'm not there it's a little tougher when you're not when you're not tangible you know what I mean but you know I try my best so uh let's just try staying communication with the team. And guys are struggling like say Mikael's struggling with the shot like how do you try to as a vocalist try to encourage guys when they're kind of either slumping or going through a rough patch how do you try to you know keep them encouraged on what they need to do? I mean to be honest they ain't really too much vocally you can tell them like hey keep doing this keep doing this but you know people got family you know and the family is going to tell you right or wrong you know hey shoot all the shots or or make all these passes or whatever like that's that's how I go I think it's more so leading by example you know I'm gonna get out there whenever I play it's just uh I'm leaving it all out on the floor so you never see that and that's uh mistake or not make a miss I'm you know I'm going 110 whenever I get a chance too so uh it'll be more so like just kind of leading by example or showing like you know we all out here with you bro we all out here struggling too it is what it is you know I mean it's not just used the whole team. And since you talk about coming out there you giving it 110 percent when it comes to tonight's matchup will there be minutes restriction for you or will that be the same that we've been saying?