This episode was prerecorded as part of a live continuing education webinar. On-demand CEUs are still available for this presentation through all CEUs. Register at allceus.com slash counselor toolbox. So I'd like to welcome everybody to today's presentation on developing momentum in therapeutic relationships. We're going to start out by talking about the ideal momentum in a therapeutic relationship. And you know this is the one we all hope every time a client walks through our door. We'll examine the causes of stuckness and we're not going to cover every cause because that would be like a week long course. But we're going to talk about some of the more common ones. And then we'll review phases of readiness for change and appropriate interventions to increase readiness. Because a lot of times clients seem stuck because they're just not ready to take that next step. And then we'll explore issues in the therapeutic relationship itself that may lead to stuckness and some interventions. So we're going to talk about this stuckness in terms of you know when we think we're doing what we need to do and clients just aren't making progress or clients really never get off the ground. What's causing that. So thinking about it and you can share if you feel like it. What causes your clients stuckness thinking about clients you've had that have either never developed momentum or have gotten in and then just kind of hit a wall and not gotten further and either dropped out or you know just continue to come and don't seem to be making progress. And what do you do in your sessions with all clients not just the stuck ones to keep momentum going because you know the ideas that I have are my ideas and the ones that I've found doing research and stuff but I'm sure you've got other tips tools and tricks that you use to keep momentum going. So one of the things that I do in my sessions with clients is we set goals and we set small goals and we review the treatment plan at the beginning of every session to see kind of where we are on track. I like to set treatment plans up as like a little checklist so we can check things off as each objective gets met. And that way people can see the progress they're making. I also encourage them to keep create a baseline log of how they're feeling so if they're experiencing depression. I encourage them twice a day maybe more depending on the client but at least first thing in the morning and right before bed to assess their level of depression on a scale of one to five and I use an anchored Likert scale for them. And I have them do that for an entire week between the assessment and the first actual session. So that's our baseline and I use that in order to help them look back and see am I improving at all. So am I going from you know a one every single day to a one for out of the seven days. Well that's not where you want to be but that definitely shows improvement.  A lot of times you don't even notice they're happening.  Those are things just like the incremental progress that our clients experience. You don't necessarily notice them unless you like step back and go oh wow you know I have made some changes. So it's important to ask them you know how do you think things are going what improvements have you seen. And like I said I'm a visual learner and visual kinesthetic so I tend to encourage clients to keep blogs keep baselines so they can have something they can look at and go yeah you know things have improved some. They can also do a verbal description or or a written description of how things are right now. And then maybe once a week write an addendum to our things changing how are things with the relationship with their mood with their eating with their whatever they identify as some of their current problems or problem areas that they hope are going to improve. So there are a few things you can do to keep the momentum going but one of the takeaways from this is remembering that people do things that have a reward and they usually choose things that have the greatest reward. So we want to make sure that what we're asking clients to do has the greatest reward and you know sometimes therapy is painful. So not doing it is more rewarding than doing the homework because that takes time and it's hard and you know it brings up stuff. So we need to help people figure out and focus on those longer term rewards the happiness that they're going to achieve. And we're going to talk about that some as we go but remembering that momentum will stay going. If we add reinforcement fuel basically is kind of what I refer to it as because we need to make sure that people keep seeing that this is worth it this is worth it this is worth it. So ideal momentum the client comes in identifies a problem and is highly motivated to work on it they're like I want to make this better. You know tell me what your thoughts are I'll tell you what my thoughts are great session and both of you are so excited for the first actual session. The first session they come in the client actively participates and is open and willing to hear feedback and take suggestions and maybe look at things a little bit differently. Not saying we're telling them what to do, but they're willing to step out of their selves for a moment and look at it with new eyes or consider different interventions. The second session and beyond the client thoroughly completes homework arrives on time participates actively and reports near continuous improvement and I say near continuous because everybody's going to have those hurdles and everybody's going to have those plateaus. But we want to see it you know kind of like we hope the stock market does where it just keeps going up and maybe it levels out for a little bit but then it goes up some more. The reality for many clients unfortunately is the client identifies the problem and is highly motivated to have us fix it in the assessment they come in they're like I'm depressed. I need you to tell me how to make it better. And unfortunately that's more than we can do. I'm, you know, appreciative that they trust me to try to fix it. But in that first session I try to educate them about, you know, how much of a two way street it's going to be and what I'm going to need from them in order to help them figure out what's going to work best for them. The first actual session the client participates but often looks to the therapist to fix it still. So they come in and they're, they'll tell us their litany of problems and how they're depressed and everything and they're just like, and that's the way it was. But they're not offering any solutions they're not suggesting, well maybe if I tweaked this or even asking for well what do you suggest in order to address this issue.  You know, if you're psychoanalytic you really want them to do the free associating and it takes time to get where you're going but a lot of times we are confined, if you will, by insurance panels and how many sessions they'll approve. So a lot of us have adopted a more active and interactive approach to counseling so we need clients to really start taking an active role in figuring out how these pieces go together. The second session and beyond the client rarely or partially completes homework and this is a complaint I hear from a lot of my colleagues and it's one I occasionally have most of the time I figured out how to work around the whole homework issue. They participate passively reporting problems but not looking for solutions and only reporting sporadic improvement it's like yeah I had a good week this week and the next week. The sky is falling. So they're not generalizing their skills. And this is not really where we want clients to be we want them to feel like they're making forward progress and not doing some sort of weird little cha-cha. So the first thing I want to look at is stuckness versus a plateau. We expect clients to experience plateau where gains seem to stall and again I'll refer you back to times where you've lost weight or when your children have been growing up developmentally they will like go through a growth spurt and then they'll stall out for a while. And then they'll go through a growth spurt and you got to go get a whole bunch of new clothes and it goes like that there's it's not a continuous growing thing. It seems like it goes and then it stalls and then it grows and then it stalls and academically a lot of times they do the same thing. So it's important to look at our clients and say okay what's going on with this client right now have they developed a new skill in counseling and they're trying to integrate it and really solidify it so you know they're not making a whole lot of progress on their treatment plan because they're really trying to get this one aspect right or you know maybe they've got something else going on that's sucking their energy. And so they've plateaued and they're holding their gains but they're not making any further gains so then we want to look at what else is going on. A plateau plateau lasting more than about a month in which the client has not reached maximal gains should probably be explored. Now most of my clients come in and they don't have one simple problem, you know they may come in with depression or anxiety, but when we start looking at it there's a lot of contributing factors there. So it's not just one simple treatment plan that they can bang bang boom. So they'll maybe start working on mindfulness and reach a little bit of a plateau with that and then they'll hold it when they've reached maximal gains they've solidified that skill. They're not backsliding on it they're starting to use it every day and they're becoming more aware and content on a daily basis. So maximal gains are good we want people to hold and we'll talk about maintenance when we get there. So we want to look at to determine whether it's stuckness or a plateau is something else going on and the client just doesn't have the energy resources to devote to counseling right now. You know if they are doing gangbusters and counseling and they're just making all kinds of improvements and then they start to slow down or they just it's almost like they hit a brick wall. You know I usually investigate with my clients I'm like what's going on, you know did something change at work, maybe they had a death in the family. You know there could be a bunch of other stuff where counseling's got kind of gotten put on the back burner.  But I don't want clients to get frustrated and feel like they're coming and not making any improvements. So I want them to make sure that they also see that you're holding your own and that's really awesome. But you've also got this other stuff going on right now so you're probably not going to make advancements as quickly as possible. Sometimes the client just loses steam, because it's not rewarding enough, and we've all lost steam before. You know how that goes. So, trying to look at have there been good times, does the client see the benefit of the counseling, does the client see improvements. And, you know if you haven't talked about it, then the client may not have even explored it. So it's important every time the client comes in to really look at the progress they've made and reinforce those gains. And you may also have treatment plan objectives that need to be changed. Life happens. You know somebody comes in and they're dealing with their generalized anxiety, or whatever the case may be and then they get diagnosed with a chronic illness, which spirals them into a depressive episode. Okay, well that wasn't there when we were at the first treatment plan. So we may need to, you know, table the anxiety stuff for right now and just hold your own with those skills and address this other issue that's a more pressing crisis. So sometimes we need to rebalance how we're using our therapeutic energy if it will, if you think of it that way. So, when we talk about stuckness versus plateau another analogy I use is think about a train going uphill. So the trains going uphill and there's a tree down on the tracks, then they're stuck. They can't go anywhere. Versus, did it just lose power, it was going up, and there just wasn't any energy there wasn't any reinforcement power, keeping it going kind of kicking it in the keyster going, I think I can I think I can I think I can. The causes of stuckness I know you've been waiting for 15 minutes to hear about this. The client has competing priorities. We need to talk to them about this when clients come into counseling for their assessment one of the first things we talk about is, you know, aside from what you're presenting issue is the fact that counseling is exhausting. It is, it is a job, and it's going to take some time every day, and it's going to have great benefits in the long term. But it's important to recognize that you're going to have to cut yourself some slack and you're not going to be able to keep up everything you've been doing until now and then just plot counseling down on top of it, most likely. So we talk about that just to make sure clients understand that for them to reap the best benefits they're going to have to dedicate more than one hour a week. Another reason clients make it stuck is they're not motivated to abandon their old behaviors and this is something we'll typically address in the first couple of sessions. What are the benefits to that old behavior what function did it serve for you. Or why are you holding on to that now when we talk about depression you know that's a feeling. So we want to look at the behaviors that may be contributing to the depression the negative cognitions sleeping all the time poor nutrition whatever it is that's contributing to it for that person staying in a toxic environment. So we want to look at the reasons that person is still engaging in the behaviors that are causing the feelings of hopelessness and helplessness and try to figure out what might meet the same needs or help them not feel like it's rewarding to hold on to those anymore. Another cause of stuckness is wrong or incomplete problem identification or cause identification. You know, sometimes you have people who are like I need to lose 40 pounds so my husband won't leave me. So, you know, most of the time and I'm not saying it would never happen, but most of the time there's a whole lot more going on in the relationship than one person gaining weight, which would cause another, which would cause the partner to abandon them. So we want to look at what else is going on in that relationship, because if they lose all that weight and the partner still leaves them, then they're going to feel very defeated and like what happened. I thought if I lost the weight, this person wouldn't leave me. So we need to help them look at the whole, the bigger picture. The goals are too broad, poorly defined or complicated. So if you have clients that come in and you want them to work on their depression or their anger, a lot of times that's too vague. So we need to be more specific because to keep that reward and reward energy going, they need to have small goals that they can complete on a daily or and or weekly basis and see that yeah I did that and okay good for me. That's my reward, whatever it is. The client doesn't feel heard or understood. And a lot of times you can notice this because the client yes, but everything you make a suggestion they're like yeah but you know, and makes an excuse for why they didn't do something, or why they won't do something.  Then that is kind of a clue that they haven't resolved that issue yet. So that's when I back up and I say, you know, I hear this keeps coming up. And so I'm wondering what I'm missing is you're having hard time working to a place where you can accept this happened or come to terms with it or whatever it is. And I ask them, you know, what am I what else do I need to hear, or what else do you want to occur in order for you to feel okay about this situation. The client doesn't understand the importance of homework or the connection to their recovery process. So if we just throw worksheets at them, or put them in group and say okay go home and do the worksheet. They may not get how that's going to help them, or they may only tangentially tangentially get it. It's really important to review the homework to make them apply it to their own life and draw that connection for them sometimes. And a client isn't challenged through Socratic questioning to arrive at their own solutions. Think about when you're taking a test, how much more challenging for most of us, an essay test is versus a multiple choice test. You know, if you can see the answer you can often pick the right answer. When we're in counseling, if the therapist is always saying well why don't you try doing this, instead of asking of the tools you have to deal with that what do you think would have been a more helpful response. If we're always giving the answer, then the client isn't challenged to do the problem solving process in their own head and connect the dots. So, one of the things we want to look at is their readiness for change. And when clients come into your office. A lot of times, if you're doing mental health counseling a lot of times the clients are either in preparation. They're tired of feeling sick and tired, but they're may not be quite ready to, you know, do what it takes, or they're in the action stage and they're just ready. So what we want to do is help clients figure that out, but some clients will come in because their boss told them or their spouse told them if you don't go to counseling I'm going to leave or whatever it is. So pre contemplation, they don't see there's a problem. It's like on a hot day sitting outside and going, it's beautiful. Contemplation, you realize there might be a problem. So that same hot day, you're starting to get a little sweaty and thinking, this is a little uncomfortable and looking around going, I could move to the shade. I might be able to get into the pool. And then you start thinking, No, that pool is pretty cold. It's not that bad right now. Preparation is when the client starts trying to figure out what to do. They're like, Okay, I am really uncomfortable sitting out here in this heat. So I've got to move. And they try to decide what they're going to do. They look around at all those options again, and they start making a choice. And they go over to the pool. So they are ready to feel cooler. And they go over to the pool and they jump in and what happens when you jump in is cold.  And a lot of times with our clients, they come in in the action phase, and they start getting into stuff. And then they find themselves feeling overwhelmed, or they find a hot button issue, and they jump right back out again. And this is a lot of times we refer to this as stuckness or resistance, because they don't seem to want to go back into it again. And it's up to us to help them understand that, Okay, maybe things went a little too fast. Let's talk about how you felt about that last session and identify what you can do to prevent feeling overwhelmed or feeling so much hurt again. At least all at once and help them make it more tolerable, if you will, because remember it has there has to be some sort of reward to it. And certain things if there's painful enough, there's just not enough reward in the world. So we want to help them inch towards being able to work through it. And maintenance, the maintenance phase, they're maintaining their progress on one skill while addressing or issue while addressing another one. So a lot of times we see this when we have people come in who are working on, for example, smoking cessation, a lot of times they use cigarettes in order to help them cope with stress and anxiety. So they come in, they've got underlying stress and anxiety, we help them develop tools to deal with stress and anxiety and distress tolerance and all that happy stuff. Then we want them to hold that and maintain those skills and keep practicing those skills while they start working on smoking cessation, which will naturally increase their anxiety most of the time. So then they're going to have to use these tools that we've been working on that they're maintaining while they're working on this other issue and reducing their smoking. So generally working on one or two problems at a time is better than trying to work on everything all at once, which can end up kind of feeling slip shot. If somebody's using a tool, for example, smoking in order to cope and you pull that away, they have to have something to replace it, otherwise it's too painful. So think about a change that you wanted to make that lost its momentum, you know, maybe it was a New Year's resolution or something. And think about what happened. Why did it lose momentum? Why were the benefits of the new behavior not rewarding enough? And what did you miss about the old behaviors? So I'll give you an example, you know, every once in a while I'll get a wild hair and I'll decide I am going to eat clean. Now I love my refined carbohydrates, they're bad for me, but I love them.  Well, the benefits of the new behavior, you know, I drop some weight, it feels better, I have more energy, that's awesome. But you know what, I also struggle when I'm at mealtime with my family and they're eating all the refined carbohydrates and I miss them.  And I hate paying for something and not consuming it. So those are kind of the drawbacks and, you know, missing just breads and things like that. So when I look at why I'm not usually really successful on eating clean completely is because of the drawbacks that I identified. So if I were going to go back and try to readdress that goal, I would have to look at, okay, so when my family is sitting there eating all kinds of refined carbohydrates in front of me, what can I do so I don't miss them quite as much? Having, you know, things that are made with almond flour, coconut flour, instead of the refined wheat flour, that's one step you might be able to take. I haven't gone that road. I haven't thought about it that much, but that's what we really want to do with clients is look at why was it not rewarding enough? What was not keeping your momentum going? So some clients are just not totally ready for change. And it's important to remember that change causes crisis and crisis causes change. And that's one you can think about when you're on the treadmill or something because it's one of those meta concepts. When you change, you are upsetting the homeostasis of your life.   So the change causes the crisis and then you've got to figure out how to find that homeostasis again. So from a growth mindset, clients see change as everything I can be. I'm here right now, but how much better can things be if I fill in the blank? A fixed mindset is the I am mindset. I am depressed. So I can't see myself as a undepressed person. I can't see the options and the opportunities I can see as myself as a person living with clinical depression, maybe, but that's about as far as I can get. So we want people to be able to envision the possibilities instead of feeling like they're stuck with whatever the situation is. Encourage them to examine the benefits of the old behaviors and we don't do this very well. We often focus on all the drawbacks to the old behaviors and all the reasons why the person would want to change for getting the fact that there's a lot of reasons why they did that behavior. So we want to do this in order to ensure that interventions meet the same need to a similar degree. And that's important.  That just doesn't do it when you're craving chocolate. Now an apple might help because it's also sweet. But you want to make sure that you're kind of trading apples for apples instead of apples for celery. Develop discrepancies between current behaviors and goals. So sometimes what you want to do in the present is going to be a lot more rewarding. But your long term goals, you know, you're doing it for what you're going to get to three, four months from now. So you need to stay focused on, you know, this is kind of unpleasant right now. But once I finish learning these new skills or getting used to not having chocolate, things will be so much better. So encourage clients to increase the rewards for their behaviors and doing their counseling stuff by remembering to focus on the end goal. What is that light at the end of the tunnel that they're working so hard to get to explore and address the drawbacks to the interventions. You know if you suggest to somebody that they need to journal for an hour every day. And they don't do it. Well, let's look at why. Why is that not rewarding to you. Because it's not to some people. So talk about what it is you don't like about journaling each day. Maybe an hour is too long. Maybe you don't know what to talk about. Maybe journaling is just not your thing. So worksheets might be a better option. But it's important to talk to them about what are the drawbacks because they're not going to do them if they don't see a benefit. The thing is they might do them for a month and not feel like it did any good. They're like, what was the point of all that. We also can increase the frequency and or intensity of the rewards for the new behavior. And sometimes that means putting in what I call artificial rewards, you know, letting yourself treating yourself to dinner on a Friday night if you completed X goal every day of the week or, you know, encourage clients to figure out things they can do to reward themselves because sometimes the inherent benefit of the counseling activities just aren't that tangible. So it's important for clients to be able to relax and have fun. If they're trying to be less anxious less depressed and are doing what they need to do in counseling. Then by all means a great reward would be to do something that makes themself happy. Take themself out. It doesn't have to directly relate to a problem they're working on. We want to help them learn how to be happy wrong or incomplete problem identification. Examine the problem from a biopsychosocial perspective when you're talking about something like depression. Yes, you can start talking to them doing talk therapy and addressing those cognitions, and that's great. But there also might be some physical issues like hypothyroid that are contributing to the depression and no matter no amount of talk therapy is going to make that thyroid go up. They also might have hormone imbalances estrogen testosterone that are also causing or contributing to the depression. They might have a toxic environment at home that is contributing to it or at work that is contributing to it. So there are a lot of things we want to look at what's going into this depression because if we're working with someone and all we're doing is treating the cognitions, and there are other contributing factors and vulnerabilities, and we don't address those. How effective do you think we're going to be probably not very effective, unless they realize all those other problems and address them on their own. But most clients don't understand how poor sleep, how pain, how side effects of medication, how stress at work can contribute to their depression. So it's up to us to educate them and help them learn how to draw connections and figure out what can you do to reduce your stress or increase your empowerment in those areas. Examine what the person hopes to get out of the change. So the miracle question really helps us make sure they've identified the right problem to reach their goal. So depression treatment, for example, is ideally going to help them achieve happiness. It's not necessarily going to help them improve their relationship. Now granted their relationship is probably going to improve some because they're happier and easier to be around. But if the main cause of their depression is a dysfunctional relationship, then just working on their depression isn't probably going to be enough. They're probably going to have to work on communication skills and boundaries and other stuff and maybe some stuff from childhood. Anxiety treatment, for example. Again, ideally, and I use happiness as a kind of a general term, clients would want to be happy. They want to be less anxious. Well, that's great. It's not going to help them stop binging. So if they are somebody who is struggling with binge eat disorder or bulimia and they come to you and you can very clearly see that a lot of their binging is a coping response to anxiety and distress. Well, yeah, we've got to address the anxiety, but addressing the anxiety alone is not going to stop the binging. We're going to have to help them develop other coping tools and draw the connections between why they're binging and what they can do differently. If the client doesn't feel hurt or understood, the client will yes, but a lot. So I usually stop and I either say something like I'm not concerned about what you're not going to do. I'm more concerned about what you are going to do. So I'm wondering what you think would work best in this situation. If the client repeats the same issue, again, ask them help me understand what I'm missing or what you want to change that just isn't working for you right now. Another expression of the client not feeling understood is that they just shut down. You know, typically they were active and participatory. I can't talk today. Sorry, y'all. Participatory and treatment and then suddenly there's a change and they just kind of come and they're very passive and reactive and their answers are short. I usually address that and with something like it seems like you might be on the verge of kind of giving up here. Tell me what you see is the most important issue to address right now and how you want it to resolve. So a lot of times clients have gotten caught in this blah of my depression is not improving. So I want to bring them back to something specific. What is one specific thing we could work on right now and how do you want to see it improve and we take active steps at that point to ensure that they start seeing some progress. The client's motivation or readiness seems to wane. If interventions are implemented too quickly or too many at once, the clients may not have a chance to solidify the first thing they learned. So they're learning a bunch of stuff kind of halfway and nothing to the extent that they can actually apply it. You know, think about if you went to a conference and every single day you learned a new therapeutic technique. Would you be competent at the end of that week to go back to your practice and start implementing every single one of those techniques. And I mean like dbt in general, not just the stress tolerance skills. The answers probably know you'd have a pretty good idea, but you wouldn't automatically be able to integrate it into session. I know when I go to conferences I get great ideas. And then, you know, when I get back, I get into session and I go back to doing what I've always done. And it takes a while to start integrating and reminding myself. And I usually pick one skill like when I was learning motivational interviewing. I picked one skill from MI that I would practice using in every session for the entire week. And then it started becoming more second nature. If the interventions target the wrong issue, such as addressing relationship communication issues. When one or both people in the relationship have low self-esteem, abandonment issues, and are projecting the past onto the present.  Well, it may be a whole lot more than that. So if all you're doing is addressing communication, you're not addressing the motivations and other stuff that's causing the dysfunction. And the interventions may be incomplete. So maybe you're targeting, like I said before, you're targeting cognitions, which is something we do very well. But you're failing to also address the interpersonal skills deficits or the physical vulnerabilities that are contributing to what's going on, either, you know, not getting enough sleep or chronic pain or medication side effects or whatever it is. Clients need to understand the importance of homework, baseline and monitoring, I think are important so clients can see gradual improvements. Daily application of new tools to strengthen memory connections and effectiveness is really helpful. So when we learn a skill in group or in session, I encourage them to practice it at least once each day. And if they don't, then I want them to do a worksheet at the end.  They're still doing it, they're just doing it retrospectively. Prioritize doing the work to achieve recovery as they define it. You know, a lot of clients, they're fine coming to session, but they just don't want to make time to do the homework, just like we didn't when we were in high school or college. So we need to make sure that they see the importance. Why is this important? And I emphasize to them if you use it every day, then it'll become more second nature and easier because the habits you're trying to get rid of are already second nature. So we need to develop something that can replace those. But I encourage them to remember why they want to recover. And ask them how does prioritizing ex work, going to the gym, whatever it is over therapy help you achieve your recovery. So if they're constantly going on vacations and going out with their friends, encouraging them, developing that discrepancy between their actions and what they say they want out of therapy. And if they keep forgetting, then what needs to happen so you can remember to do your homework? You know, it's easy to forget if you have to put kids to bed and, you know, feed the dog and do whatever else. So what needs to happen?  There's a lot of interventions that can be used, but we need to ask them what's going to work for you. Another thing I do with clients and I think it's really important. Unfortunately, it didn't make the slides. I don't know why at the end of every session and this includes the assessment session because when they come in for their assessment is generally when they're most motivated. They're like, finally, I might get some relief. So I want to build on that. And I want to develop rapport in that session and I want them to walk out the door with some sort of tip tool or trick that can help them start addressing their issue today. And it can be mindfulness. It can be, you know, I usually choose something very simple that can start helping them develop an awareness of their issue. Or for, for example, if they have a lot of anxiety, develop a couple distress tolerance skills and talk about those in group or in the assessment session and encourage them to practice using those distress tolerance skills between now and our first actual session. They're also going to do their baseline. And we found that when people do baseline charting, they actually notice improvements in their whatever the behavior is because they're paying attention to it. So they're actually they actually improve themselves kind of placebo effect ish. So, but if clients have a tool that they can use after every session, and then when they come in for the next session, you talk about how did that work. And if it didn't work really well, let's talk about whether we should scrap it or adjust it. Then they feel like they're doing something and they're doing something active and it doesn't have to be huge. It can even be telling them about a book you want them to read. So that gives them something active to do and they feel empowered, like they're taking control of their recovery. So do follow up on homework and make sure clients believe that you think homework is important. It's not just busy work that you think you got to give them. Sometimes it's helpful to do homework with them, if you will. So if you are teaching them to use mindfulness, then you practice mindfulness and maybe share one or two things that you figured out about yourself over the past week.  It doesn't have to be anything ultra personal, but help them see that you're not just telling them to do stuff you wouldn't do themselves. Taylor the homework to meet the client's learning needs pros versus lists. I've told you before I'm not a journal or never have been I'm not a pros type of person. I'm a list type of person so I can write lists of, you know, I'll write down the 10 things that made me irritable today or whatever it is. But I'm not going to write this long Dear Diary narrative. Just not how I'm wired. So if people are, you know, resistant, or even before they start doing it, ask them which would you prefer or present it to them. I need this information, put it in whatever form you want. Free form versus worksheets, which is also kind of pros. So you may have them write down how they felt about implementing a particular intervention or how they felt about a certain experience. Not every client is comfortable using all those words and doing that so they may do better with worksheets that prompt them. When I went into the session, I felt blank, and they just have to fill in the blank so they're not having to come up with something that flows and they're not as self conscious. And then some people prefer videos versus books. Some people also prefer old fashioned books versus online. My daughter's dyslexic and she has a hard time reading stuff online, but she does a lot better with actual printed books. But some people prefer videos. I'd rather not. I'd rather read something. So ask your clients what's going to work for you so they don't feel like they're getting homework like they did in high school that they just dreaded the whole time. Keep it short. The total time each day for working on their stuff probably shouldn't exceed about an hour, including the logging they do throughout the day. If it starts to exceed that, it's probably going to start negatively impacting other areas of their life, and they're going to not do not complete it. Help them figure out how to work it into their daily routine, such as journaling or doing their worksheets while they're drinking their morning coffee or just before bed while they're winding down while they're doing something that they would have done anyway. And listening to audio recordings on the way to work or at the gym, if they're auditory learners, some people would prefer to listen to like a 15 or 20 minute primer on the ABCs of cognitive behavioral or or distress tolerance skills or something in the car, because the driving is something they already have to do so they might as well pair the two things together. So help them brainstorm ways that they can actually work their homework in without making it feel like it's drudgery because if it's not rewarding, they ain't gonna do it. So credit questioning by using so credit questioning clients strengthen their memory pathways to the trigger using the skills on their own. So asking them, why do you think you reacted the way you did in that particular situation so they come in, and they tell you that they got into this huge fight with their roommate. Okay, so why do you think you reacted the way the way that you did. And as a therapist as they're trying to explain to me why this happened. I'm listening for them to identify vulnerabilities, you know they hadn't had enough sleep. They had let things build up and hadn't addressed things for a while and then they just had a litany of done me wrongs that exploded. And I'm also listening for them to identify why the old behavior, getting into this blow up argument with their roommate is more beneficial than sitting down and having a talk with him or her about the issues as they come up. Now in that particular example, it could be fear of having this conversation or inadequate assertiveness skills. But what I want to hear is clients being able to start understanding why they do what they do so they can figure out another way to meet that same need that is more helpful. And then I follow up with another question of, Alright, so that happened and you know we've explored some reasons why you think it happened. What other things have we talked about what other skills do you have that you think might have been useful in that situation what could you have done differently. And then how might it have helped you know if they say well, you know I could have addressed this first thing last week when it happened instead of letting things build up. Okay. And how would that have helped if you would have addressed that back then, and then have them go through and talk it out. So I really want them to see the benefit of whatever this new behavior is even if it's not as rewarding in some ways as the old behavior. And then how do you think you can help yourself remember to use that skill in the future. And in this particular scenario, when the person is doing their mindfulness activities. Each day if they notice that something is building up a resentment is starting to build up with somebody. If they note that in their mindfulness stuff, noting that when that happens, they need to address it. I can also give them worksheets in order to help them remember to do that we may start working on other skills to increase assertiveness. But I really want to look at why it happened. What was the benefit. What could you do differently. And if you already have the tools that you could have done something differently why didn't you use them. I want to look at, you know, do you not feel secure do you not feel confident in using them, or do we need to modify the tool a little bit so it feels like it works for you. So three steps to a breakthrough. And this is kind of a summary for clients. They need to define their goal, making each step rewarding and I have that all in caps because it's so important. I know you can't figure that out. Clients need to see the benefit and feel the benefit, even if it's not a direct one to one. If I assertively address this problem, then I will feel happier. A lot of times it is though because they don't feel like that weight of the resentment sitting on their shoulders. To find their goal, making each step rewarding, then develop the change plan learning from prior experiences. So that includes what did you do in the past that worked. What did you do in the past that didn't work. And in the past when you've tried to make this change. Why did it fail. We don't want to fall into that same pit again. Increase and regularly refresh your motivation, cognitively, remind yourself of all the reasons you want to do this, and all the reasons you don't want to do that other behavior anymore. Environmentally, whatever you need to do to remind yourself to the change and what you're going through is worth it. So maybe a collage or pictures or whatever it is that you're hoping to achieve to remind yourself visually in your environment about how much better things are going to be when you achieve your goal. And socially, talk about your goals with people talk about your successes with people. And that you've got a support system that will encourage you and check in periodically and go, hey, how's that weight loss going or hey, how's your counseling, how are your counseling sessions going. That will help people stay moving forward. Another aspect of social motivation, if people are in support groups, then they motivate each other, and they can also help each other get unstuck when somebody hits a really difficult period. They've got social support to go, I've been there.  So change requires a certain amount of momentum. It's important to examine why prior attempts to change have failed and explore why and what can be done to address it. Make sure goals are specific, measurable, accurate, realistic and time limited. So you don't want these broad goals like I want to be happy. Well, what does happy mean? How are you going to know when you're happy? They need to be measurable. You know, again, what does happy mean? Is it, you know, just not blah or is it a four out on a scale of one to five, you want to be a four or better, six out of seven every seven days. That's measurable. Is it accurate? Are you trying to change the right thing? If you want to preserve your marriage and you think losing 40 pounds is going to do it, then is that an accurate goal? Or are you going to go through all the effort of making this change and then feel like you're being let down because you didn't get the hoped for result? And is it realistic? Is it something you can even do? You know, in relationships, there's two people that have to tango. So if it's a one way street and it's a toxic environment, you know, making it into a warden June Cleaver situation may not be realistic. It may be one that you're determined to keep, but we need to look at what's a realistic hope for this relationship. And time limited. We want to make sure that people are getting rewards frequently. Initially every day is good, but that's probably not real realistic. At least once a week, they need to be able to kind of pat themselves on the back and get a reward for doing the hard work. As clients learn new skills and ways of responding, it's vital to practice them on a daily basis and without expecting immediate perfection. Yeah, you're going to use some of their old behaviors and that's okay. You know, just accept the fact that you did it, look back on it and go, okay, what could I have done differently and learn from it instead of beating yourself up because I don't want clients getting stuck going. Well, I had this new skill and I still use the old behavior anyway. I relapsed. I'm a failure. No, you know, you've been using that other behavior for the past 20 years. You've had this one for two weeks. It's going to take a while before you're as comfortable with it. So let's just figure out how to make help you be more comfortable with it. Sometimes motivation and compliance can be increased by social support, environmental reminders and changes, such as the alarms to do your do your homework or, you know, writing on on the mirror in the bathroom that you need to smile more today or whatever the intervention is. Identify and regularly review the benefits of change as well as the drawbacks to the old behavior and log and review progress on a daily basis so you can see and really revel in those incremental changes and go, you know what, this is working. If you enjoy this podcast, please like and subscribe, either in your podcast player or on YouTube. You can attend and participate in our live webinars with Dr. Snipes by subscribing at allCEUs.com slash counselor toolbox. This episode has been brought to you in part by allCEUs.com providing 24-7 multimedia continuing education and pre-certification training to counselors, therapists and nurses since 2006.


It's just one department, biostatistics and medical informatics, but it's okay. Thanks so much for having me visit. It's great to be back in Berkeley. So I'm an applied statistician. I spent a lot of time looking at data, and a good amount of that time is diagnosing and trying to resolve data problems. What I wanted to tell you about is one particularly egregious example of this from my past work. It's not particularly new, but it's hopefully new to you. And yeah, for sure, I'm hoping I don't slip into jargon that's not defined, but if I say something that you don't understand, please ask, and I think you don't need to wait for the microphone to have me fix some jargon that I skip over. I mostly focus on genetics, and mostly the genetics of model organisms, mostly genetics in mouse, trying to identify genes that are contributing to complex diseases or other traits in mice. Mice have a lot of advantages for biomedical research generally, and one of them is that you can create inbred strains of mice. Mice like humans have two copies of every chromosome. These mice are members of a single inbred strain. They're the result of many generations of mating between siblings, so a brother-sister pair for 200 generations. Result of that in breeding, you get a strain of mice where their two copies of every chromosome are exactly the same all the way down. All these mice are genetically identical, but sort of the two genomes, the chromosomes they got from their mom, the chromosomes they got from their dad, are identical. So that is advantageous for biomedical research in that you can completely eliminate genetic contributions to whatever you're studying. An advantage for genetics is that you can then do really big crosses. A given female mouse can only produce 20 or 40 offspring in her lifetime, but because you have genetic, because you have clones of her and her partner, you're not restricted to work with family sizes of 20 or 40, but you can have crosses that are effectively unlimited size, however much space you have for cages of mice. And mice are not unlike humans. We're not particularly interested in curing mouse diseases, but we're hoping that by learning about, say, diabetes in the mouse, that that will tell us something about diabetes in humans. I like to think of mouse as a model for human disease in three ways. It could be that a particular gene for a trait like blood pressure in the mouse is also involved in the human population blood pressure differences. In some cases it could be the same allele, the exact variance that you see in the mouse that are participating in blood pressure variation in humans. If it's not the same gene, it could be the same pathway that, you know, what genes are involved in blood pressure in the mouse could tell you something about what pathways are important in blood pressure in humans. And the third way is that, say, the genetic complexity or the genetic architecture of a trait in the mouse could tell you something about human disease. Like if blood pressure involves 1,000 genes all with complicated interactions in a given mouse cross, you have to expect that in humans it's going to be similarly complex. So maybe it's obvious, but I have to give you a bit of background before you get to the whole, you know, massive data part. This is, yeah, this is the background bit. So the typical experiment I study is an intercross between two strains that differ in some trait of interest. You have a strain of mice that has low blood pressure and a strain of mice that has high blood pressure and you cross them and you cross them again. You do the cross, basically the same sort of cross that Mendel did with peas. Mix up the two genomes so that the mice down here, they, well, they'll get a chromosome from each parent, but because the two F1 hybrids, the hybrid parents have one copy of a pink and one copy of the blue chromosome, here, you'll have genetic variability as a result of this process of recombination along chromosomes. They, you know, these F2 mice may get a pink chromosome or a blue chromosome intact, but they mostly get a chromosome that is a mosaic of the blue and pink, you know, that in, in meiosis that produces sperm and egg cells, these chromosomes find each other and exchange material so that the actual chromosome transmitted could be a mosaic of the two parent chromosomes. So typically what we do is measure blood pressure in each of these mice and then measure genotype along chromosomes, that is, at given sites figure out which mice are homozygous pink, which mice are heterozygous, and which mice are homozygous blue, and then look for sites where what genotype you got is associated with the clinical outcome blood pressure. And this is the sort of data that we would have that you have a bunch of genotypes, say positions along the genome, along each of the chromosomes as, as the columns and mice as the rows, and at each spot you get to see, you know, blue for homozygous blue or pink for homozygous pink or yellow for heterozygous with some smattering of, of missing data in there, these little black specks. And then we have some phenotype. For every mouse we get to see its blood pressure, and I've sorted the mice from lowest phenotype to highest here, but we're going to scan along the genome and look for sites in the genome where, where there's an association between this category genotype and this quantitative phenotype, and so, so, and the, the result of that, you know, at every position in the genome we calculate some test statistic, which for historical reasons we call the test statistic we use, it's called the LOD score, and it's a measure of how closely associated that region's genotype is with the mouse blood pressure. So a region where that thing is, is high, if you split the mice into three groups according to their genotype at that spot, you know, sort of blue, blue, red, red or blue, red, they show some difference in whatever the, the phenotype is, those three groups. But regions that are associated with, where that phenotype is associated with genotype, regions where genotype seems to be affecting the trait, we call those quantitative trait loci or QTL, it's site in the genome that is somehow affecting the phenotype. So I've been working on this problem since I was at Berkeley 20 years ago, and, but I moved to Madison 10 years ago, and that the, the, the first major project I got involved in, we was already, you know, getting, going, ongoing before I got involved, but it was an experiment like this, they were interested in diabetes in the presence of obesity. So they did an intercross with about 500 mice between two strains, B6 and BTBR. The mice were all knocked out for a gene called leptin. As a result, they're all really obese. Leptin is this, leptin is, I mean, the gene produces a hormone that makes you satisfied at the end of a meal, sort of your signal to I'm done, I should, I'll stop eating.  So these mice just eat everything given to them, and they're all really, really big. And one of these strains, BTBR, in this obese state becomes diabetic and the other one does not. They cross those two strains, made an intercross, about 500 animals, genotypes across the genome, lots of clinical phenotypes like insulin levels, but in addition they measure gene expression in six different tissues. So for each mouse, they add a certain age, I think 10 weeks, they kill the mouse, remove all these organs, extract RNA and put that onto a microarray to measure the level of mRNA of every gene in the genome. So for each of these six tissues, for each of these 500 animals, we have 25 genes, we have their measure of expression. We were working on this for two years or so, and we're about ready to write the first paper, and I said, well, I'm willing to sit down and write the paper. But the way that I work, I wasn't, like, I'm going to sit down and write a paper, I'm first, let me go back to the raw data, so I really have a full understanding of everything that happened. And, well, first I have to say that the cross was done in a certain direction. They took a female BTBR mouse and crossed it to a male B6 mouse, and because of that, the offspring should have certain genotypes on the X chromosome. The F1s, the female F1 will have two X chromosomes, one from each parent. The male F1 will have a BTBR X chromosome and a Y chromosome from B6, because his father was B6. That's where he got his Y chromosome from. The cross, those two, the offspring, these offspring that I'm working with, the females will get a blue X chromosome from a BTBR X chromosome from their father, and then a recombinant X chromosome from their mother, and the males will get just the X chromosome from their mother. The males will be hemizygous along in the X chromosome. They'll have either blue or pink, but just one allele, and the females will all be either homozygous blue or heterozygous. But when I looked at that, watching that type data, I saw that there were out of the 500 mice, 26 mice, this was clearly wrong, where either the females were homozygous pink, a whole bunch, or the males had a whole bunch of heterozygous. This is not atypical. It seemed like there were, say, 25 mice like that, not unusual. But I was thinking, we have this expression data. How can I use that to help me figure out what went wrong? Are these mice male or female or what? I could look at a gene on the X chromosome, exist. That is super highly expressed in females and not at all expressed in males. Maybe that would help me to tell. Then I realized, well, we have, well, later that day, I came across, I really figured out that, jumping to the punch line now, we ended up, like there's 20% sample mix ups in this data. Of the 500 mice, it turned out that 100 of them, the genotypes didn't correspond to who they were supposed to be. These are the 96 well plates or 8 by 12 plates where they put DNAs on with little wells in them. Each arrow here is pointing from where the DNA should have been to where it actually ended up. The insight I had, I was thinking, what's going on with these sex-swapped mice? I could look at a gene on the X chromosome. Then I thought, I have all these other genes whose gene expression is really closely associated with genotype. There are lots of genes like that whose, the expression of the gene is super highly associated, especially with the genotype of the spot where it sits. There are other genes that maybe aren't, the expression of the gene is super highly associated with some other spot in the genome, but lots of them anyway where the expression of the gene is super highly associated with the spot where it is. I could use those as tags of mice. I looked at that. Here's that gene, there's a gene on chromosome 1 super highly associated with genotype at its location. If I look at the expression of that gene broken down by the three genotypes, it looked really odd to me, like not how nature is supposed to work. Most of the mice, like if you have the homozygous R genotype, you have really low expression of this gene. If you have the homozygous B genotype, you have high expression of the gene. If you're heterozygous, you're sort of pretty high but not as high, but then there are these other mice where homozygous B, they should have a really high expression of the gene, but they have a really low expression. And these sex swapped mice, if I highlight just those, it turns out that they're sort of enriched to be in the wrong blob, if you will. So there were say 16 mice that I had that looked like their sex of their genotype didn't match the sex of what had been recorded. And nine of them end up being in the wrong blob. So it suggested to me that the sex swaps were sample mix ups and that all these other mice in these other blobs, that they were also sample mix ups. I mean, some other positions in the, well, and if I could, so, if I could, I could use the expression and use this association between expression of an expression and genotype at a site to try to figure out, for a given mouse, I don't believe that it's really heterozygous here because its expression is really low. What do I think it's, what do I think its genotype is at this mark, at this, at this gene? I think it's really homozygous R. You know, all these mice down here, I think really are homozygous R because they have low expression. All the mice that are in the middle, I think are heterozygous and all the mice that are really high, I think are homozygous B. So I could use, say, K nearest neighbor classifier to, from their expression, what do I think their genotype is at this position? For each mouse, I look at some number, say, 30 mice whose expression is similar and I allow them to vote to say which genotype should they really belong to. It allows me to sort of very quickly form these bands of the low expressors are all this genotype, the middle ones are all the middle, the heterozygous and the upper ones are all homozygous B and then maybe some in purple that are sort of halfway between two blobs and I don't know for sure who they are.  In some cases, I can use pairs of genes. We have two genes that are really close together and I can look at their, you know, a scatterplot of the expression of one gene, the expression of the other gene, you know, colored by genotype at that region. It's pretty clear what the three different genotypes are and if I highlight the set of my sex swapped mice, many of them are, you know, some of them are fitting into where they're supposed to be, but more than half are in the wrong blob. So, again, I can use the expression of these two genes and figure out what I think each mouse's genotype at this site should be. That those in this blob are really homozygous B even though the genotype data says they're not and those in the middle blob I think are really heterozygous and those in the upper blob I think are homozygous are. Here's another example of it. I have two genes that are really closely, that, you know, sit really close to each other. Their expression is highly associated with genotype at that location. The actual genotypes for a bunch of the mice don't really fit what I think they should fit. I can use a K-nearest labor classifier just for each individual dot, find the 30 dots nearest to it, and let them vote to say what I think that genotype really is for each mouse. So, as a result, my basic scheme is I have, I focus on a set of genes that I have gene expression that's really tightly associated with genotype at the corresponding location. So, I have this rectangle of the expression values for those genes that have really, really strong QTL and the observed genotype data for all the mice at those locations. And I use that sort of each column or maybe pairs of columns to make a predictor of from, from that what do I predict each mouse's genotype really is at those locations. And then I can ask for given mice, for given mouse, does my observed genotype data actually match the, you know, what I infer it to be? And if it's not, if, if it doesn't match, can I find another row that, say, does match? See?  For each of them, I make a classifier that allows me to take expression and tell me what I think everyone's genotype should be. And then I ask, do they, does that actually match what my genotype data says they should be? And if it, and, and is there another mouse that it matches better? And so, for every DNA sample I have and every mRNA sample I have, I can make a heat map, I can calculate a distance between them. How, how similar is the, the set of genotypes to what I predict the set of genotypes to have been? Calculate for every, for every pair of, for every DNA sample and every RNA sample, what is the proportion of mismatches? And I get something that's kind of hard to see because you, I, we have limited pixels. If we blow up the first hundred, the lower left corner, you get a nice perfect blue diagonal line that the first hundred mice, their RNA sample and the DNA sample, they're really similar. You know, the DNA, this, the set of genotypes I have and what I predicted them to have, they're really similar. And the off diagonal are all not similar. So it looks like these are all good. But if I move up a little bit, you know, the miles 200 through 300, you find that, you know, there are some of them that are nice on the diagonal, but there are all, then a whole bunch that are off by one or off by two. And so I, I know that their, their DNA sample is wrong. If I just take that, that distance matrix and I make a histogram of the diagonal, you know, each DNA sample against its own supposedly mRNA sample, I get, you know, a bunch of mice that are really close, you know, there are RNA sample, their DNA sample look the same.  And if I look at all the off diagonal elements in the distance matrix, look at a histogram of that.  So that, I mean, the idea is that, you know, these guys are all right. These, these are all wrong. And, you know, they're, they're over there. So for each row, I really want to figure out for each row in my distance matrix, if this, the minimum isn't on the diagonal, I want to figure out where is the minimum. And that's similarly for each column. So, so for each DNA sample, I can say, I plot the, the value I get on the diagonal and plot that against the minimum for the row. And I get a bunch of points where they, that, the value on the diagonal, its distance from an mRNA sample to its own DNA sample is, you know, the smallest in the row that look to be correctly identified. I get a bunch in green, green points that it's clearly not, the value on the diagonal is big, but there's another value in the row that's small. So it's clearly wrong, but it's fixable. And then a bunch of others that, a small number of others where the value in the diagonal is, is, is not small, but there's nothing else in that row that's small either. That mouse, I don't have an RNA or a DNA sample for. I have just one or the other. And then, and then I went back to the part of the record, the, the company that did the genotyping, they wrote down where the, where the DNAs were arrayed onto plates, because they, they do them in these, this 96 well format. And if I look at, and that allowed me to make that picture for every sample that I know is wrong, I can find, you know, many of the samples that I know I wrong, I can find where it really ended up. And that's what these, the, the arrows are here. Blow up just one of them. There's some additional annotations. So the, the white dots mean that the correct DNA was found in the right well. The pink circle over there at D2 means that, that sample was duplicated. So it's got a white dot in it. It means it's, the correct DNA was put there, but it was also put in, in B3. That arrow means that it was supposed to be, a sample that was supposed to be in D2 got actually placed in D, in B3. And the thing that was supposed to be in B3 got placed in B4. And the thing that was supposed to be in B4 got placed in E3. And the little green arrow means the thing that was supposed to be in E3 that didn't, I don't know where it went. And so that the green arrow, I mean the orange arrow means that there was something got placed in 7, but I don't know what it was. And the purple arrow that, I mean that, that thing, the thing that was supposed to be there, I don't know where it went. I don't, this one, I, I, it puzzles me. Like how, I don't know how, maybe it's my, my lack. I'm not sure how that this would happen. But the data are clear that this is what happened. That, you know, a sample got duplicated, but put in the right place, but put in another place, and then, you know, three sort of all shifted in some weird way. But, you know, of the 500 or so samples, really 20% were put into the wrong well, and there were mostly these two plates. And let's just look at one of them. So something that was supposed to be on that upper plate got put into A1 there. The thing that was supposed to be in A1 got put below in B, in B1. And that pink circle means that that was in replicate, because it was put in the next two wells. And then everything was off by two for a period of time. But, and then, and then, there was a sample that they, they somehow got back to be off by one. And they were off by one for quite a period of time. Yeah, so, you know, it was clear, it seems clear that whoever was working with these DNA samples was moving them, say from one set of plates to the other through one well at a time. And they may be, I like to think they got a phone call, and they came back and they did things a little differently, or they, and they maybe realized they were making some mistakes and they tried to correct for it. The, the game that I, so, though, I should emphasize, you know, the, the errors that I'm seeing, that really the only way I could, I mean, there are two ways that I can ascribe it to be the genotype data that's wrong and not the expression data, because it could be they mixed up the expression data. One is that I have these six tissues worth of expression data, and the, the six tissues correspond, and it's the genotype that's different. If something is off from one tissue, it's off from all of them. And the second one is this, that the errors are corresponding in a way that is course, you know, has to do with the way the DNAs were arranged in plates, that geometry of being off by one, off by two, whereas the RNA samples, but it, it suggests that it's the DNA that's put in the wrong places. But I can play the same game with the expression data. You know, for, if I look at two different tissues of expression data, I have expression in islets in the pancreas, which is really important for diabetes, and expression in liver, which is also important for diabetes. I could, you know, the rows of these two data sets are supposed to correspond. So I could ask, you know, pick a mouse and let's plot their expression patterns. All the expression of all genes in the islet, expression of all genes in liver, do they actually correspond? What you end up with is just a splotch. Thirty, twenty-five thousand points just blah. Most of the genes aren't expressed. They don't show any association at all. So you don't see anything. So that didn't work. But there exists some genes that are really highly, if I look at a given gene, there are some genes that are really highly associated between the two tissues, you know, bunches of them. Some of them, they're highly associated between the two tissues because there's a common locus that's driving their expression. You know, genotype at a given site is affecting expression of that gene in islets and expression of that gene in liver. So what I'm going to do is just, let's just focus on the genes. You know, look at every gene one at a time and say which ones are highly associated between the two tissues and just focus on those.  Except when it's not, you can find some mice where their expression of islet and liver doesn't look associated at all. But oh, it's, you know, the expression of that mouse in liver and the other mouse in islet, they're highly associated. So, you know, we do this the same thing and the other way around. So, you know, this pair of mice, if I look at a scatter plot of, or a, you know, a matrix of scatter plots of, this is the plot of islet versus liver in mouse 3598 and this is a plot of islet versus liver in mouse 3599. They're not associated at all. But islet and liver in 3598 and 3599 are really highly associated. And the other way around. So, it looks like these two were just swapped completely. And then here's the case, 3280 islet and liver, 3281 islet and liver. You can see that 3280, islet and liver, they're highly associated. Looks like the same mouse. But, oh, this is a good one. I should, this is an example of what it's supposed to look like. But here, here's a bit, so here 3295 islet and liver are highly associated. 3296 islet is not associated with 3296 liver. It's instead super highly associated with 3295 islet. So, that looks like what I would call an unintentional technical replicate. Where mouse 3295 islet was, its islets was done, the mRNA was assayed twice. And 3296 islet was not 3296. The strategy is take two tissues, look for genes that are highly correlated between the two tissues, focus on those, and then for every mouse, you know, focusing on these highly associated genes, make a distance matrix. How similar are they? Or a correlation matrix. And look, you want everything nice and perfect along the diagonal and not nice on the off diagonal. Where it's not, then you figure out there's a problem. And there were quite a lot of problems. I mean, not anything like the genotype data, but more than you, we want to deal with. You know, this hypothalamus, there were nine cases of sample swaps. In the liver, there was a sample swap and then one of these, the arrows pointing from which was supposed to be, who was actually written down to be. And so, you know, that guy was done twice and the other one not at all. In adipose, there was a three way. And in kidney, I can't really tell, but I'm pretty sure that these two samples were maybe mixed and then assayed and duplicated as a mixture because they're really highly associated with each other and not with the rest of their, the panel of tissues I have. But when you, when you, if, one thing, I mean, probably the most surprising thing about it is that 20% of my DNA samples were mislabeled. And we didn't notice it for two years because we were getting lots of cool findings. The findings were strong. If you, if you fix them, the findings get even stronger. So this is, you know, the, the scan of the genome for loci that affect insulin level. In pink is what we originally had with the original data. And in blue is what we got after we corrected the 20% sample mix-ups. Everything increases quite a bit. So that's good. But it's kind of surprising that it was not, I mean, 20% sample mix-ups was not as terrible as it was. I showed you these pictures of the really strong QTL effects of, you know, the expression of a gene is really strongly associated with genotype at the site. So in dashed are the curves that I showed you before. After correcting the sample mix-ups, these come super strong associations. Sort of my test statistic is way increased. But in summary, sample mix-ups happen that my sample mix-ups are worse than your sample mix-ups. With this expression data and genotype data, we have the ability to not only identify them, but correct them. Having these multiple tissues really helps you to figure out the origin of the problem and not just know that there is a problem. I think the general idea has a lot of application in data science generally. If you have matrices whose rows are supposed to correspond, you check that they actually correspond. And if you have some columns that have strong signals, that should give you an ability to really figure out whether your matrices' rows are actually corresponding. A paper describing this that took a long time because people don't want to... It was great that my collaborator was willing to lay it all out because others might not be so willing.  That was at the time that I was working this out. The lessons I take from this are first, don't trust anyone, even yourself.  Don't trust them. Go back and double check everything. Or it could be... Well, if it was you two years ago, it's worthwhile going back and looking at it a second time. It wasn't me, but don't trust anybody. Make lots of plots. One problem that we have is moving from low dimensional data to super high dimensional data, you think that then you're going to be making lots and lots of pictures, but in practice you tend to make fewer pictures. I can't look at 30,000 histograms, so I'm just like forget it. I'm not looking at any histograms, but you need to really figure out a way to look underneath the hood and see the data. You need to make a lot of plots. In this case, we were burned a bit by having made a choice to transform all the outcomes so that the level of expression of genes can have lots of these outliers of points that are just artificially wrong. So we chose to normalize each gene's expression to just sort of cram it into a normal distribution before we did any analysis, just so that we weren't just looking at artifacts all the time. With all of that, so we were mostly looking... When we did look at pictures, we looked at pictures like this, where the vertical scale was crammed into a normal distribution. Everything looks a little weird, but looks okay. Actually, the raw data was this one that I showed you at the beginning that had really distinct clusters with individuals that didn't belong and that were in the wrong cluster. If I had seen this picture, I would have come to the conclusion I would have found this problem earlier. Because we were looking at this picture, we went 10 years in this project before we figured out that this was a problem. It was two years before we figured out this was a problem. Another lesson is follow up all aberrations. 500 animals, 26 of them have swapped sex. You could say, let's just throw those out. We still got 475 animals left. Every little hint that there's some oddity going on, you should follow it up and really figure out what's causing it. Because it could be that there's another 200 of them behind. You might ask, if there are 200 sample mix-ups, why were there only 25 that had swapped sex? Well, it's because mice are kept the same sex in a cage. Two males, two females, two females, two males, etc. They labeled them in that order and they arrayed them on the plates in that order.  Take your time with data cleaning. Ask your collaborators to give you time for the data cleaning, a week, a month, a year. If you have big rectangles whose rows are supposed to correspond, check that they actually correspond. I think the general technique of look for columns that have high correlations and focus on those and see whether those rows correspond is worth doing in all cases. A whole bunch of people involved in this project because it's a big project that others really responsible for the situation that allowed me to spend a bunch of time studying this and write a paper about it. I think they're not on this particular list. These slides you can find on the web. You can find on my website slides for every talk I've given since I was here in Berkeley in 97. Code that I've written and you can find me on Twitter. Thanks very much for having me.