Welcome to liquid margins 43 the craft of instructional annotations a few pieces of housekeeping before we get started. Slide advance whoops too many. You are here for liquid margins. We've got some great episodes of liquid margins coming up. We've got some great. Liquid margins adjacent content webinars coming up. We have a webinar on our on our integration with vital source. Very exciting to talk about our case study there and share with you some data from courses that use the vital source ebook platform and connection with hypothesis in the spring. The data more students don't do the reading very often sometime in some context and social annotation can help them do the reading would probably touch on that subject today. We also have a webinar co-sponsored with J store to talk about that integration again some very interesting data and interactions going on through our integration with J store library platform. And then later this fall we haven't set a date yet we're going to do an episode on regular and substantive interactions which I'm actually going to circle back to in just a second. In our conversation. For more episodes of liquid margins coming up. And let's see here. This liquid margins if you're new to liquid margins is really a pedagogical discussion it's not a how to tutorials so if you're here to sort of chat about strategies and best practices for social annotation across from me in the right place. If you want more basics about how to get started with hypothesis reach out to education at hypothesis and we can set you up with a demo or a more sort of hypothesis one on one webinar to introduce you to the platform. If you would like to turn on close captioning. Oh, sorry. This is about q amp a. We don't have the chat turned on, but we do have a q amp a turned on. So if you have questions for the panelists, feel free to drop a question. There we have a Christy the careless from our customer success team here, helping to answer questions. And we'll would love to bring your question into the conversation. And then if you want close caption, that's also something that you can do yourself in zoom. So go ahead and press that CC icon and you can get close caption. All right. So the craft of instructional annotations. We're here to talk about instructional annotations annotations created by instructors for students and we have really our superstars of instructional annotation some of these folks that are guests today are like, you know, the number one. Instructional annotator from our spring 23 semester. We did look at some data. And then there's also folks that have just been doing this for a long time built out a lot of scaffolding as part of their courses. Long time hypothesis users to share best practice with you. Instructional annotations, not every instructor using hypothesis in their courses annotates themselves. They might just set up an assignment to prompt students and annotation of a text, but keep quiet in the margins. There's nothing wrong with that approach. There are good pedagogical reasons for stepping back from the conversation sometimes. But whenever I look at course data and I see a high number of instructional annotations, I get excited because there's something important happening there. I think. And it relates to something I've been reading about lately the US Department of Education's. Relatively recent guidelines around distance education that they released a couple years ago and they have a big emphasis in their guidelines on online courses involving quote regular and substantive interactions between instructors and students and quote. That's to determine federal financial aid and title for funding for for online institutions offering online. Distance education courses, but of course that's good teaching really in any modality right instructors should be instructors should be interacting with their students. And I think hypothesis is a very powerful way to create space for regular and substantive interactions between students and instructors and instructional annotation is a major type. Instructions have substantive interaction and a major way to establish instructor presence in a course as a guide in the learning journey. Instructional annotations can be signposts that an instructor adds to a text ahead of students reading and annotating themselves. They can be specific prompts that instructors seed in a reading for students to respond to in discussion threads. Instructional annotations can be replies to student annotations, answering questions, offering feedback, or simply engaging in dialogue. And I think as the conversation plays out here you'll see that we have folks that are doing maybe one or the other of those approaches, or sometimes both. So, without further ado, let me introduce our esteemed panelists, an interdisciplinary group of instructors with a high degree of representation in the state of Massachusetts. For some reason today. But we're joined by Lisa delicio biology professor at Salem State University in Massachusetts. Hi Lisa welcome. We're joined by Daisy flame in Professor of English at Springfield Technical Community College in Massachusetts and Marcia. Good. I'm a senior professional lecturer in anthropology at the Paul in Chicago and Daniel Hutchinson associate professor of history. In Belmont, Belmont Abbey College in North Carolina. He also directs the college is digital humanities program. So, this is a relatively informal conversation, but we have some frameworks. There you guys are. There's everybody's. Hi. We'll just make this as much of an open conversation. I kind of just want to start about hearing about where you teach in the context. I think understand a little bit of the context of where an educator is coming from is important. So, maybe you can tell us just a little bit about the school you teach at and the types of courses. And maybe we'll start with you at the top of my screen. Marcia. Hi everyone, it's a real privilege to be here today and I'm so excited to hear from the other panelists as well. I teach at DePaul University, which is the largest private Catholic research institution university. We have, let's see one in three people are first generation that come to us and we have a broad diversity of students. That's something I wanted to put up front because it's so important to hear from a multiple points of view on a piece of reading so that's one of the main reasons I use it and it's one of the main reasons I kind of beg to Paul several times to work there and finally got finally got in in 2008 because I really appreciate that kind of diversity in an urban context. Marcia anthropology. Oh, I'm sorry. Yes, I teach anthropology and I did make a little I teach teach a lot of survey classes like cultural and medical anthropology. And then I teach more classes around women's health and also indigenous studies that are part of my race being raised in Mexico. I teach women's health in the global south anthropology of childbirth, Maya humanities, etc. And this coming year I'll teach a course in the race power and resistance called birth justice. Wow, I want to take these courses. We'll go Daniel and then Daisy and then Lisa. Thanks Jeremy and thanks to the hypothesis team for putting this together and thanks all of you all for being here today to learn more about social annotation. I teach at Belmont Abbey College, a small liberal arts college established by Benedictine monks. It's a Catholic college like DePaul Chicago. We as a small liberal arts college we focus on building a sense of community and trying to really embody mentorship and close work with our students and helping to shape and form them in their educational journey. And I teach at Belmont Abbey College, I teach in the history department and so I primarily use hypothesis for my intro level history courses as well as upper level history courses. We rely a lot upon a lot of close reading of historical texts and documents, and I found hypothesis really helpful for breaking down context and helping students get an informed and collectively examined take on a particular historic text. And I also teach courses in digital humanities where we learn about new methods and technologies for exploring the human condition using some of the cutting edge technologies of today and hypothesis is really helpful in helping students break down critically analyze and use these technologies to empower them in their learning journeys. So that's a little bit about where I teach and the type of teaching that I do. My name is Daisy Flame and I teach at Springfield Technical Community College or STCC. We're not supposed to use stick, but you might hear me use it. So our community college is. It's just a really important place for students who have been historically underserved by education. We have students from marginalized groups, so many first generation college students. So I always think of us as being everyone's first chance, or their last chance, a little bit. In terms of the workload, I teach English, and I do get some upper level it's or creative writing or women's studies but the bread and butter at a community colleges definitely. I have one income to the first two writing courses. The first one is straight up writing and the second one is writing an introduction to literature depending on the professor, I do is introduction to literature with writing. Oh, and I'm happy to be here today. Thanks everybody. I think it's my turn. Yes. Okay, so hi everybody so excited to be here with all of you and share my experience using hypothesis. Hi, my name is Lisa delisio. I'm a professor of biology at Salem State University in Salem, Massachusetts. We're a primarily undergraduate institution with some graduate programs. And we in the state university system in Massachusetts not UMass and not the community colleges but the level in between. We've got the most diverse student body, including lots of first generation students and we're working toward becoming a Hispanic serving institution as well. So I'm a biology professor who looks at larger scale biology so not selling molecular stuff but organism up to ecosystem and biosphere. So I teach ecology and the environment I teach environmental problems I teach conservation biology. And those are the courses that for which I used hypothesis in the spring and it was the first time I used it. But I expect to be using it again in fall with a botany course as well as a honors botany oriented course. So, yeah, I'm excited to be here. I have quite a range of biology from plants to the atmosphere. Yeah, I'm a conservation biologist and a plant ecologist. So it's, it. Yes, there's a lot. Well, I wanted to start off the conversation just hearing about what drew you to social annotation as a technology maybe we'll go in the same order we just did starting with Marcia. Why social annotation. Why are you social annotation and you're teaching at the Paul. One of the things that as a teaching and when you teach in survey courses, you have our classes are kept at 45. But it's still a challenge to learn to know people like in an online setting and to give students a kind of one on one attention and to see them that students that come to the Paul expect. So I started with some of the challenges and that was the challenge but I really, I really got into this. I was using. I used Instagram and I used Facebook back in the day until I felt like it was too poisonous for people to come in through the Facebook and Instagram change dots, change dots algorithms and you know we couldn't be a little community anymore. So, primarily this is about building learning community and also addressing the different ways that students read now I've been teaching for 20 years, and students do not read the way I read in grad school and I don't read the way I read in grad score brain synapses just work very differently. And to be able to engage with a text and to be able to create on the margins sort of that illustration that draws people's eyes down through the meaning of the text. I put a lot of emphasis on them adding color and illustration as well as their ideas, because it just makes reading more engaged in more fun and like one student said they don't feel as lonely when they're reading. You know, when you're sitting there trying to get through 15 pages for a class and you're just, you're all by yourself and some people turn on TV in order to be able to read it or have music in the background, but honestly being part of an engaged learning community. Also, I wanted them to see each other. Too often, the good students are always going to be great students are always going to meet the basics of anything you give them. But other students don't get to see them in. And this is one one student said last year they said I had no idea people in this class knew so much. And I think that, you know, we talk a lot about decentering, not the sage on the stage kind of idea but you know we're making buffets of learning for students were putting lots of material out there that they can use from, from a lot of different perspectives and bring who they are into reading and those are those are those are my primary reasons. There's lots of reasons. Yeah, that's a lot that's a lot of reasons right there feel like you might have stole everybody's thunder that's a beautiful articulation of a lot of the value that I see in hypothesis but Daniel do you want to add anything to that or emphasize anything that Marcia mentioned. Absolutely. I would certainly piggyback on that desire to create a sense of community among students and in this intellectual exploration of a particular topic. In classes, we often will take a historic text and try to break it down. And these texts sometimes can be familiar to students but oftentimes they ask students to approach history and historic language in ways that may not be familiar to them or or they're used to engaging. And so particularly before I guess 2020 we would do that a lot of that in class we do a lot of in class discussion, reviewing of the text and the material, but then then came COVID and suddenly all of my standard playbook just completely no. And I was really desperate to find some way to recreate that sense of dialogue and conversation and community and hypothesis save my bacon. Our institution, connected with hypothesis and I began utilizing it in my courses and while there certainly was a transition to be made there, it really was helpful in fostering dialogue and conversation at a moment when it was sometimes difficult to get students to share to talk to to put their perspective forward and the ease by which hypothesis allowed students to express themselves to communicate with each other to and to really dig into the text made me it really made it a rewarding learning experience, both for me as an instructor I think as well for my students and now it's become a primary toolkit and the way I use to teach historic texts. Yeah. So I came for a very simple reason. I used to I always require when I first started teaching literature I was stunned by how disengaged people seemed. And then I had them annotate what we were doing, because where I, when I went to school every English major ran around with a pen in their hand, you know in a book. And so I had them annotate and saw everything gets so much better. The discussions were so much richer. And then, and I used to grade it because my thing is if you don't grade it they won't do it. So I graded it and so all I was looking for is some way to grade their annotations. I didn't know much for that, but stayed when I realized that I could switch to entirely online texts that were free. So that's awesome for our students who, you know, our poverty is huge at our college and, and then I realized that I could see the texts and Jeremy mentioned seeding the text with definitions prompts, all that kind of stuff. And most happy realization is something that Daniel and Marsha touched on is that we could be a community again in a way that discussion boards just couldn't at all. And I always say that when we're in the classroom and we're discussing a text that we're sharpening our fine minds against each other's. I feel like we can do that again, you know, and of course in English I don't want to make us while I'm making a special plea for English, but we carry very much like this word, these three words, how do they change everything.  That's a great segue to for our token scientists in the room. Right. Everybody else on this.   Like, I think a lot of people familiar with close reading is an important part of the English curriculum and for primary texts, you know, as Daniel mentioned in history, but why is social, why were you attracted to social education for for teaching of biology. Well, we do have texts that they need to read that are denser and denser every year I mean the biology texts have become encyclopedic even within narrower disciplines within biology. And so I try to find readable texts but it's not easy. And I have had students do three to one assignments and canvas canvas discussions. But I would say that AI pushed me over the edge. It became very clear to me right away, like, I'd say over Thanksgiving. That it was going to be way too easy for my students to ask an AI to to create their discussion board submissions. So, and I was really happy to connect with the institutional, sorry, the instructional designers at my institution who were learning me to the existence of hypothesis I hadn't known about it before then. And so this replaced that assignment in my class, and it's much better. So, in all of my classes, they are annotating the text and I'm responding to what they're writing it's close reading is important in biology as well as other disciplines and precision and language when the students right is so important. So, it gives me a chance to give them feedback in a supportive way in a community way that furthers the conversation with their classmates, and I really appreciate that opportunity. I've also used it to have students, especially my non majors, link the text to their major. So they have to include a source and talk about the relationship between what they're reading in environmental problems and criminal justice or nursing. Right. And I've had students who told me that they didn't think at the beginning of the course they thought there would be nothing that connected between there was no connection between their major and environmental problems. And of course there's a connection between every major and environmental problems. And I also students tell me that it was the first time they really thought had to think in a class. When they're forced to make those connections in the annotations. So, I was really pleased with that. That's a neat assignment. Maybe we'll go in reverse order this time and keep going with you Lisa since we since you touch on this a little bit. I'd like to now dive into this idea of instructional annotation and hear how you yourself annotate for or with your students and I haven't gotten the phrasing down but there's I think there's going to be a little bit of a division between the, the suppliers, people who spent a lot of time replying to student annotations and I guess the. I'm going to call the other group, the cedars maybe we talk about seeding annotations. But those are different approaches I think some of you guys do both but starting with Lisa just tell us a little bit about your, you know, how you approach instructional annotation how you're annotating with hypothesis yourself for your students. I'm mainly following their lead and responding to their annotations. If there's some if there's a student who's asked a question that hasn't been that hasn't been addressed yet by another student. I'll ask the other students to chime in. Ask them to come back and take a look and see whose questions I've highlighted. I've also given my own links. I've also given my own connections or ask students to reconsider what they've written in a different with a different framework. And I think that that's very helpful because there's so much misinformation out there, especially around climate change and other environmental problems that I find that that's really helpful. For my upper level students, I'm able to connect them with resources that will deepen their understanding in the areas where they're curious, right I want to follow their curiosity. There's so much to learn I honestly don't care. For the most part, what they learn and what they don't learn as long as they're learning. And if if I can help them on their own individual journeys through the annotations I think that's really helpful. Oh, I think you're muted. So we're applying but really nurturing lines of inquiry helping students down that, you know, independent line of thinking that's that's wonderful.  You're both. Okay, our students need a lot of scaffolding lots. Okay. And so I deal with definitions and context, all the time. The reading typically they don't come in with really strong reading skills, and they typically don't and vocabulary associated with it, you know, or just the educational capital that a lot of us had grown up. And so I really identify with that that cultural capital that's often missing. I can sort of reply and so I think that we're going to talk, I think in a minute about how it works for students but I think that's one thing that really does work for my students is they have I can't do that for them when they go home and read a text or written text, but I can do that as they're reading here so they can read mastery as they go. And then I'm a total text supplier to. So I have them go in twice once with one half their annotations and once with their second half, because otherwise it's drop and run. I made you read everything everyone's annotations that you know what's going on but you drop your annotations and you go. So I've never seen how things progressed, or how I suggested well you know there's a different approach that's possible here or have you seen this evidence. And so when I am annotating I'm also doing it's trying to do it as a model for other students and I do a lot about how to care, how to cordially make suggestions to other people. And I do it as a model to that they can say oh I was wondering since you took that position but I was wondering what support, like, if you saw this piece of evidence. And so then that's the ultimate for me isn't me interacting with them, although I love it. It's that they will do it for each other. That's a whole different, you know, level, which is very exciting, you know, like for all of us. Do you find it does nurture that discussion kind of. Yeah, it really does. I'll be honest and sign this the classroom. But it is, it's, I've seen really really exciting things and I've seen people really really change from like Pat answers to really having dug in connected to it personally connecting to it culturally, asking hard questions. I've really seen so many students transform in the course of a semester. Amazing.  Do I remember. That's right. I do try to do implement scaffolding in terms of my annotations, both within the annotation, both within a single assignment and then over the course of the semester to try to build on specific type of variables, literacy abilities, analytical approaches, the ideas to sort of work week by week to address a particular historical ability based upon a particular type of historic text that that will use. I'll typically seed the assignment first with a set of maybe guiding questions to help stimulate students on what they should be looking for the text or sort of questions or approaches they should take. Like Lisa and Daisy through the text I'll then provide contextual explanations of what's going on in the text definitions of perhaps concepts or terms that may be unfamiliar. And I do try to get back in and offer replies to students as they're making their their annotations that can be challenging for me personally I'll admit but I do that there's tremendous value in that. The best value I've seen in terms of that approach is with something I start every semester off with and that's annotating the syllabus. That's an assignment. I use to both introduce students to hypothesis how it works. It's remarkable when you ask students, what questions do you have when you ask that in class students are often sometimes hesitant, especially on the first day of class to actually ask questions about the nuts and bolts of the course, but through hypothesis students will really good questions, not just sort of practical questions but the nuts and bolts of what to expect of the course but even more sort of dynamic questions about the ideas and the subject and the direction the course will take itself. And, and from the very beginning, through that sort of approach you can begin to create dialogue and maybe even create a sense of community to. But yeah, the goal for, for my efforts and hypothesis is to provide some variation provide an opportunity to build skills and try to foster dialogue and conversation. And Daniel are some of your pre populated annotations like essentially discussion prompts for thread of discussion or are they pointing to other parts of the text versus and take themselves or they're often, especially for sort of the first annotation I set sort of, I set a roadmap for how to approach the document and some ideas and things to keep in mind. And then through the document I will for specific points of if there's a specific aspect of the document I really want them to key in on, then I'll sort of seed that with another question or a prompt for dialogue and engagement. Got it. Marcia. Hi, I do it differently. And that is that's a, I guess that's a good thing I really respect how much you all respond to students. In cases I go over the 5000 and one quarter, because I have all these students working on to in what in medical anthropology because I use it differently in different courses but say in medical anthropology of two chapters a week, and they're annotating those chapters. I seed it ahead of time. There's always the instructions are always at the exact same place. The instructions are always the exact same thing in terms of this is what you must do to get your grade.  You must use either images hyperlinks websites memes in in a few of those.  You should feel very free to have a broad diversity of opinions and not see, you know, not try to cater to what you think a medical anthropologist wants you to say on this. So that is why it's very specific, what you need to do for your grade and the rest is all yours. And the four replies. Actually, I have found students, really kind of amazing at calling each other out on misinformation, which always makes me really happy because I have not been able to do the pass through twice my respects Daisy honestly my respects, because when they go through once they don't want to come back, and that means they aren't necessarily seeing the replies the people that get the religious comments are sort of at the end of that curve. And, but so I had structured scaffolded it so that they would come in twice but then so many people didn't do it and I was doing all this grading two time four times a week and just going out of my mind. So now I just in the page notes. That's where I really see what people came back from the scaffolded part of it is that as we go. I'm saying this you will need for your term project they do an ethnographic interview of somebody with a disease chronic disease. And make sure you're noticing bio socialities, for example, and then when they're doing their interview, they're asked what are the four theoretical approaches that you're taking from the text. So as we're going through there's a lot of discussion on how does this apply to actual people that you might talk to. But yes, I'm less of a respond in the moment. I can use it in another part of the course, but I have found that the grading load is heavy enough the way I have it structured that I can't handle going through twice. And I still think it's, there are a couple times when somebody has said something, or that I think is just odd. I'll write to them in the gradebook when I'm making the comments. I'll put the textbook on the side and I'll come into them. Oh, by the way, you know, when you annotated on x paragraph, you really should know about this, but that's personal to them. I also found that when I jumped in enthusiastically and started replying to people, they kind of backed off and got polite. Because the professor is saying the ultimate thing. And when I'm not in there, there just seems to be a richer dialogue at least I have felt, I have felt that in some, some of those settings. But that's all kind of open ended, I use it very differently in different classes I'm speaking how I use it in a large survey course, not large 45 people is not large. But in a survey course, rather than how I'm using it when I'm asking them to annotate it before coming to class, which is a very different, you know, they're going through and I'll say things like, put up a video childbirth, okay, put up a video of what you learned from TV on how childbirth works and tell us why you chose this piece. So we're going to open it up in the classroom, and we can play some of them and students say well I put that up because of this and it really makes a very rich discussion it's like, it's what you have in front of the classroom for everybody to look at and see, see what each other did and brought to it. And Marcia. So go ahead, Daisy. No, I really want you to. Um, so I saw two things coming up in the chat that were relevant and something Marcia said. So I don't grade twice. I'm not that much of a hero. But when I do grade I checked that they did their first four I can see the dates. And so I checked that they did their first four and their second four on the dates and that they were done on different days. And I did see, is it okay if I mentioned two things that came in the questions. Yeah, great. Okay, so, um, Mama have nor mentioned that it's a considered a discussion killer to jump in on a thread so you got to wait as long as possible. I find that there's a way to do it in hypothesis that really works so if I say student a and student B had a great argument that they were pursuing here. But I think there's a good counter arguments made and it has to do with this character can anybody help us out. I remember when a student has made a point that's good but other students don't seem to have picked up on I could say, can anybody think of an original way to support this point because there's more out there. So, I think it's totally, I understand the point but I think it's also possible to do, and somebody mentioned how do you grade. So as an English professor grading is the bane of my existence, and it's still true here. But I, they were talking about subject in a subjective and an objective so I start with the objective, did you do as many as you're supposed to do. And did you roughly follow the instructions. So let's say based on that you got a seven, right, but you made really original comments, I might bump it up from the seven or figure seven was what you maxed out at right. And then if someone, it's clear that just repeating other students, you know, then they would drop from their seven to whatever it is. So, I'm real clear about the like, here's what you need your bread and butter and then I want to, I want to see your sophisticated most involved self here. I love that I want to dig in a little bit to this to this idea and talk about best practices for annotating because you know we've taught somebody mentioned the sage on the stage model and certainly there are times when when the teacher speaks, you know, in certain context, the conversation stops, right. And you haven't moved away from the stage on the stage model. And so I'm curious about how people would think about this question. You know, Daisy, you sort of started pointing, I guess, one best practices. Not replying with answers but replying with questions or, you know, trying to keep the conversation going to others have thoughts about that and whether you're seeing annotations or replying. Just the proper tone the proper or not proper but the, you know, good approaches to keeping the conversation going because I think all of us want the students to annotate to and for our remarks to be the beginning or the middle of something and not the end of it. Right. Lisa go ahead. So, I find one way to help give the students some confidence in their writing is to draw on their expertise immediately so having them connect to their major and non majors classes. If they're in conservation biology they're each assigned an endangered species so I encourage them to draw connections to their endangered species as we read through the conservation biology text. Or also for non majors have them make connections between the topic and their hometown. So, then, if I chime in it's because I'm learning something from right, or suggesting that that they have made a point that was also made about a neighboring town. So did they realize that their classmates are seeing exactly the same thing this pattern. Can we see a pattern arising in the comments in the. So that's, you know, drawing those connections and recognizing them as experts and I think this idea of expertise is going to be more and more important coming back to my topic of AI.  It's a technical academic term yes. Is is to have the expertise to be able to to discern that. Right. So I think drawing drawing on their expertise and helping them build expertise quickly is is really important for these discussions. Yeah. I love that. Is there anything to add about how you couch your annotations to encourage, you know, the dialogue. One way I try to explain hypothesis to students is akin to a writing medium in which they do have comfort and they do use a lot that social media. I try to tell them look if you can write an Instagram post if you can communicate with your friends in this meeting you do the same thing through with hypothesis. I emphasize to the students that sort of as Lisa mentioned here. I want to hear what you have to say I want your reflections your experience to inform the class about your perspective about this text. For example, in one of my sort of intro world history courses students read the code of Hammurabi this long list of rather intense laws about justice and ancient Mesopotamia, and I ask the students to from their framework from their perspective identify laws that they think are too extreme or in the right place like in how do these issues that people dealt with 5000 years before the birth of Christ how does that show up in our world today and by making those sort of contemporary aspects to allow them to speak to their expertise to their to their worldview. They don't feel like they're going to get an answer wrong. Right, they certainly can share a perspective that we can offer some guidance on or maybe enrich or getting them to talk giving them to share their worldview is maybe the first step in getting them then to explore a different worldview and a different way of thinking which is primarily what the the job of a historian is all about. So I think creating an atmosphere where they feel free to talk in a in a space where they can share their framework for their experiences I think it's a great way to help make the world of social annotation a bit more engaging for students. I love it. Just checking to see if anybody on mute go ahead, Marcia. I agree. And I, especially when we're talking about personalizing and applying the kind of things that we're reading I think that's what's one of the richest things about this they're not like picking out, not defining terms or anything like that unless everybody's like hey hey you really need this term this is an awesome term and I have seated it ahead of time with some of those, but I'm thinking of one where we were, we were just studying how much people travel for medical care. And the people in the class, it was stunning how many people had traveled for surgery for dental care going back to Mexico for this going to different parts of the world go for plastic surgery etc or their parents or you know travel, or people coming to different states, and that one. It's not really that I can jump in, but it's a get they end up with such a rich how the how the text has worked in their own lives, and from so many different perspectives that's what I find really stunning about it. That's great. There's been a couple questions about grading so I want to circle back here Daisy responded a little bit but Marcia and Daniel and Lisa, do you guys grade student annotations and do you have the tips or tricks for the audience. I do grade and I make it easy on myself by having a spreadsheet where I'm counting. The basic is just the counting you did your for your for in your page note. And then the having it open, having the grade book open on the other side because unfortunately still the number will go through but you cannot comment in hypothesis and that's one drawback I can't make little comments like oh you lost, you know so many points or this page really didn't have anything to do with the prompt that was, you know, so those kind of comments I have to have the grade book open on the side. It's not near as I always dread it, because I dislike putting numerical things on people's learning like you wouldn't believe and I think that probably everybody in this room shares that feeling. It's one of those contract things that we're bound to, but being able to comment in the grade book as I go, having it open on the side when it's needed. I do do two of these a week, which means I'm grading 90 of them and that's one class and I have three. So yes it is, I don't like to say it's not a grading burden also. I think of notifications somebody said do you see notifications when I've heard from hypothesis team members are working on that that would be great for people to see. Put enough weight on the grading of these that it matters to them. So, people see right away who are the ones that are just kind of writing along and not adding much material and I figure some of those students know how to get their bees no matter what classes they're in it's like the students in the elevator saying, I didn't read it I still got 80% on the quiz, they can't do that in a hypothesis and everybody sees what that is made up of. So those are just a few comments I make about grading. There's lots of other things to say. I just want to make a quick note before we hear from Daniel and Lisa on grading that I do have a product backlog card for feedback inputs with D2L and blackboard those of you on the call that are using canvas. You already have, you know, you can enter private responses to student annotation sets and canvas speed grader, but we do want to bring that to other LMS is and your snippet there. Marcia will go to go into that ticket on if I can get a recording of it to explain why that's so important. And it's very important for a regular substantive interaction as well because it's that private feedback on the course work. Daniel Lisa thoughts about grading. My approach to grading for for these hypothesis assignments and sort of the scaffold and nature of things. One of the end goals of the semester is to compose a paper utilizing a range of primary sources. So students in writing that paper need to learn some specific writing techniques to effectively put that together. And so my goal for each hypothesis assignments to be for it to be fairly low stakes in terms of their overall grade but I do grade for grammar and composition and I am looking for a particular word count and ideally some evidence of engagement with classmates and that can come in a variety of ways depending on if it's an intro level survey or an upper level course. I'd vary that depending on sort of the, the course difficulty and the course nature but I do set a word counts and I do sometimes set a number of replies I expect from students. There's a challenge with that because then students are just looking to meet that that word count and then hit those replies and then they don't engage anymore. But I do I do seek to grade and provide feedback for this in order to give students that that experience to come to that summative assessment and be able to put that together and hypothesis is a really vital tool for helping me to do that. The grading can be substantial in that regard. I usually do one hypothesis assignment per class per week. It can be between four and six sections of semester so it can be it can add up but I find the time. A good investment. We're working on making it easier including I also have a car. Rochelle Haroldson in the, in the q amp a s about automatic grading we don't have automatic grading right now.  I really want to see what the students are doing. But we do have auto grading and different exploring the idea of auto grading is something that we're exploring from a product standpoint at least in terms of being able to tally. You know, three annotations were required three annotations were you know help with the objective part as Daisy mentioned. Lisa any thoughts about grading. I think it's similar. I'm looking at that count I think it's something that will show the tally would be fantastic. And also, I'm also looking at whether they've included the link. You know they usually have to have a link to show a connection to something else. So, whether it's their major or their in some cases their hometown depending on the assignment. Or their danger species. So, that's really what I'm looking for, but I'm also looking for at least one comment that's thoughtful. And usually it's the comment with the link that's thoughtful. I don't grade for for anything, you know, in terms of their quality of their writing. I think I have time for that and that's not what my course is about. Although it's really important if they say something that's totally unintelligible they don't get points has to be under something that can be understood by me and their classmates. I find that they don't tend to skimp a lot on these assignments, I do them. I think usually every other week. So, and then I'll have a different as alternating assignment on the other weeks. And I can do that if I put enough chapters in one week, you know, it reduces my grading if I have fewer assignments, so it's a lot of grading. I'm going to fold my final framing device here, which is what advice would you give to folks that are starting with instructional annotation, but I did just want to highlight two questions that relate to this.  So I'm going to just do that to reply is maybe you can offer some tips there. And then Eric asks about what are some, you know, reliable and engaging ways to seed annotations so let's just finish with tips for our guests for our for our audience here in terms of best practices for instructional annotation and let's start with Daisy. So I think I, I included a lot of what I wanted to say there's just one more idea that I think is important, and that's providing them with a short sample annotated text so even if it's in one page right. You can you annotate it yourself as though you were the students, and then they can see Oh, this is the model that's what is what is expected.  And so I want them to see that are really sort of more loose sentence can get more of the heart of things than something really long so I don't include word count in mind. And that document helps me show them what kinds of comments will be valued. Great model annotation assignment I love that Lisa and Daniel and then Marcia. I don't seed anything. I'm not sure I have a lot to add because, as I said I'm trying to draw connections for the students between between each other to so I'm really emphasizing, did you see, you know, Tanya wrote this, and that relates to what you wrote. Or if they're responding to each other I really like to encourage that conversation and recognize that they really are having a deep conversation. I think that's, that's really my approach. So you're there, but you're really pointing to, to them to kind of further their conversation I like that. Yeah, yeah. In terms of for my approach, when I think of a particular article or document I want students to read I really want to think as well about what sort of conversation do I want to spark what are the main threads or issues that I think students can get out of this experience. And then, with that in mind I really try to, you know, seed the document with questions and with prompts that will hopefully propel students towards those, those ends that I can get to, but often the most fruitful dynamics in a conversation of course when students take it over themselves and lead it in the direction where they want to take it in and then that's a good moment for me to take a step back and watch the conversation unfold and let the students, then sort of dominate the conversation and that that is a great approach I think to de-centering the instructor from the business of education and getting students to buy into ownership of their own educational journeys. So this is not particularly helpful I guess but I think you really have to start with each particular assignment you want to use, what's the end goal you want to get out of it, what sort of conversations do you want to spark and how do you get students to get there. I think that's great. Backwards design. I also find that in terms of what I'm sorry. In terms of what kind, how we get started with it, and that also addressing the what kind of things do we seed with. I do find seeding the more complicated vocabulary. There's a lot of time and I always call it terms and concepts, and so I tag it with the same thing they could just go through with the tag and see some of the, the vocabulary that they're not used to but they're, I always say, learning a discipline is learning how to speak with the words that we use. So that kind of seeding I will do ahead of time, and also questions that really like Daniel was saying speak to their lives and Lisa, you know all of, how does this person, what does it personally do in your generation let's say the text was written five years ago. What does it mean after the pandemic and how is this is this still true, you know. So those kinds of things but in terms of getting started my biggest advice is expect a lot of work in the first week. I'm always surprised how few people at my own institution are using this I think it's growing, but students come in pretty unfamiliar with it. I have a question how many of you used it before, and then I kind of have a fun document for them to play with some people I've suggested the syllabus, you know, other kinds of play with the class roster that didn't work because the roster changed, but also make a little video of how to actually do the things because I instructional text just isn't working the way it used to they have to see it. And I've kind of been begging hypothesis for that as well and it's just where does my mouse go and how does it go in here it's incredibly simple, and I always tell him, this is going to be second nature to you after the first time you do it. But having that assignment in the first week, and I use the intro to our textbook for it, you know what, what do you look forward to in this text and I also in that one I explain each of the chapters and the main learnings that we're going to do and what are you going to read about. So I do think putting in work, a lot of front work at the very beginning is important. I have other stuff but I think we've covered a lot of them. I appreciate y'all I'm going to finish with some housekeeping and then give you a big thank you. Since we're coming up on the hour here. And your institution is not yet a hypothesis customer you don't have access to hypothesis in your learning management system as our guests to please reach out to education at hypothesis I know the fall is approaching sorry to remind you guys. It's very easy to get up and running as an institution with hypothesis and we have some special deals for for the fall so it's definitely not too late to reach out to education at hypothesis and try to get a starter package going for for for the fall at your institution. If your institution is a hypothesis customer and if you're not a hypothesis customer this is the kind of wonderful services that we provide. We have a workshop program, Chris into careless our CSM manages that she's on the horn here. We've got great workshop series back to school series with some sort of introduction how to use it in your LMS but then annotating your syllabus specific session and unantian your syllabus, and then a session on annotation starter assignments which is, you know, we have some, you know, assignments that you can use in your classes and we also have a bank of assignments that instructors like those on on call today have shared with us so lots of help to get started, and then another plug for another resource available to our customers hypothesis Academy. This is for whether you're new or experienced I think anybody can find something in our hypothesis Academy courses including the hypothesis one on one sharing best practices with educators it's a really amazing community to join. You know, Lisa have one that is focused on the age of AI and annotation in the age of AI and both ways to work against chat GPT type things and also with chat GPT leveraging social annotation. With that, I want to give a big thank you to our guests today. I appreciate some of you are new to hypothesis some you've been around a long time but I appreciate your thoughtful engagement with the tool and with us today and I will just say, we're working on a number of different things to make instructional annotation easier, including annotations from course to course feedback mechanisms and non canvas LMS is so we're going to make your lives easier but it takes hard work to do what you guys do and you're really establishing your presence and you're really there for your students in a powerful way so thanks for being such great teachers as well and thanks for your time today. Thanks Jeremy. All right folks see you next time. Thanks for the opportunity. Great people. Thanks so much. Thanks. Bye.