The meeting is being recorded. So next we have our keynote for this year, which will be given by Yolanda Giebel, who is Senior Director for Strategic AI and Data Science Initiatives at the Information Science Institute, ISI, of the University of Southern California in Los Angeles. She's also a Research Professor in Computer Science and in Spatial Sciences. She's a Director of Data Science Programs and of the USC Center for Knowledge, Powered in the Disciplinary Data Sciences. She's a Fellow of ACM, AAAS, IEEE, and AAAI, a lot of triples and a lot of fellows. And she's in fact the past President of the AAAI, which is a very prominent role in the world of computer science. Yolanda's research is on intelligent interfaces for knowledge capture and discovery, most related to the topics of this weekly workshop. She's a leading researcher on semantic wikis. She's done important research around making the data from semantic wikis readily available for researchers and also around understanding how the community that creates semantic wikis co-organizes and so on. I'm very excited about her talk today, which will be on crowdsourcing to synthesize scientific knowledge. Welcome, Yolanda. The screen is yours. Thanks for being with us. Thank you. Thank you, Bob. Very nice to have me here. I really appreciate any opportunity to interact with the wikimedia community and the larger wiki community. I'm going to talk today about crowdsourcing. We used to call it knowledge collection from volunteers. When we started to work on this from the point of view of AI, knowledge capture and knowledge collection. But I'm going to talk about crowdsourcing to synthesize scientific knowledge. So I'm interested in how crowdsourcing enables us to capture new forms of scientific knowledge. And I'll give you some examples of what we've been able to accomplish so far. Why am I interested in science? Because I think the questions and the problems that science is tackling these days are very close to my heart. So health, understanding, intelligence, anything that has to do with questions about human health and well-being are important to me. As well as the health of our planet, sustainability, the environment, I think that these are big questions that will be key to help humanity. I think that working in these areas makes it a huge different long term for humanity. And if I think about these problems, they are getting more and more complex. And so how can we bring AI into the picture? That's my personal interest. So when I see what's happening in science, I see that collaborations and showing up knowledge from people that have different backgrounds and different expertise is more and more important. This is a figure from Barabasi who studies scientific collaborations. And he's showing co-author networks over the years. So I think you recognize some of the pictures on the left, Watson and Creek starting to collaborate at the beginning of the 20th century. You could see some pairwise and small groups of collaborations. The next picture is a human genome project. So starting to see large networks of co-authors. The one on the very right is from the Atlas collaboration that led to the discovery of the Higgs bosom and the main paper had about 4,000 authors. And I've seen their collaboration networks and they're quite incredible artifacts. And I think it's a testament to these hero findings in science. We want to help make them more ubiquitous. That's something that I would like to see. So one of the ambitions that I have is that we should be able to combine human abilities with other human abilities. But can we insert AI abilities in the middle? So I want to share with you this quote from Gary Kasparov about freestyle chess. It's a new kind of competition that he designed after being defeated by Deep Blue. So Gary Kasparov had never been defeated before in the chess game. And he knew that this was coming. I've heard him say that he was honored to be the human that was first defeated by or the first grandmaster to be defeated by a computer. And so when he created freestyle chess, he said a player can be any number of humans, any number of computers, no matter how many grandmasters or supercomputers you use. And what they found in this competition is that even if the humans were not grandmasters and even if the machines were not supercomputers or the best chess playing programs, what really made a difference is that they had very good roles in the game that they had identified and that each member of that player team had strengths in. So I'm very intrigued by this idea of how do we insert machines and help people collaborate in a way that results in teams that are better than otherwise. And this diversity of talents and skills that is important to bring in. So that's overall my interest and what drives my thinking. And I'll come to this slide because eventually what I would like to see is that AI systems understand enough about science to really gather knowledge and be able to write papers and make contributions to science. And in order to do that, we have to go beyond what we have today, which is a description of a protein, a description of the rainfall and the weather conditions in a location to having much richer interpretations of what scientific knowledge is and how people work together and so on. So I'll be giving you examples of this throughout the talk. I'm going to start talking about crowdsourcing non-scientific knowledge. That's something that we started to do earlier and just sharing some of our lessons learned and our findings there. And I'll move on to talk about crowdsourcing two types of things. One is collaboration tasks and collaboration networks. And then I'll talk about crowdsourcing vocabularies and standardizing vocabularies in science through crowdsourcing. And then I'll reflect a bit on what have been our successes and what remain to be challenges. So let me start with crowdsourcing non-scientific knowledge. Quite some time ago, and as I mentioned, we call this knowledge collection from volunteers. We had a pretty sizable project on collecting common sense knowledge or common knowledge about the world. This has been a bottleneck in AI. We were working on helping people with their to-do lists. So if you jot your to-dos, could an AI system help you organize them, prioritize them, give you a warning if you're running out of time to prepare for something? And in order to do that, the AI system had to really have quite a bit of knowledge about the world, so we went off to collect it. So we were collecting knowledge about objects in the world, like a copy machine, but also about tasks.  And so on. And you can tell this is quite dated because I'm talking about LCD projectors, which is quite something. And a very useful kind of knowledge is if something fails, how do you repair it? So we were collecting all flavors of common knowledge about how the world works in an office environment. And through this, we had many kind of techniques that worked better. So for example, we wanted to be proactive in the prompts that we gave the user so that we would broaden the coverage. So if we say a bicycle has these parts that we know about, then that makes people think, oh, it's definitely missing a certain other part. We were validating a cross volunteer. So very early on, we noticed that if three volunteers, definitely four volunteers coincided in a certain assertion, we could trust it. We were also organizing the collection and the frames that we were presenting to the user by types of knowledge that our system could understand and relate to each other. So we collected quite a large knowledge base. And it was actually one of the largest on the semantic web early on with this kind of common sense knowledge. And the system was called Learner and then Learner too. But this taught us many lessons. Among other things, we learned that volunteers loved to teach this system new things, and they really came back and enjoyed it, especially we were raffling t-shirts for some reason. So lots of lessons learned and we have some papers about this. But we really were, you know, we could experiment online and we could really design what worked better and what kind of designs of these forms gave us better contributions. So this was a very interesting way to come to this. Another project that we did early on is when semantic wikis were beginning to be used more broadly and in fact semantic media wiki had hundreds of sites. We built on something called wiki apiary, which collected data from individual wikis to create what we call the provenance b wiki that really looked at where all of the content of the wikis was coming from and how it was created. And you can see on the front page that the content was going up and up and up over time as these wikis were being more and more used. But we also learned quite a bit of things and just to flash something that you hear, it was very interesting to see that creating classes and categories was not important to these communities. I think we were looking at almost 600 wiki's classes were not very important. Properties were used very, very broadly. The composition of the group was kind of also interesting. There were just a few people that edited properties. Everybody else provided assertions based on those properties and so on. So lots of interesting things about how how communities and crowds use semantics to capture knowledge. So I've always been interested in these topics. Jenny Randicic was in my institute for some time. And I remember discussing this idea that he had that he conceived about having a wiki with assertions about entities that was a little bit more structured than Wikipedia. And I remember having long discussions about what would be a fact versus what might be controversial and the importance of capturing provenance. And then, of course, this all eventually transformed into the wiki data project, which we use very extensively. We have a large project on using wiki data to do integration of data sources for science. And so this is also another example of some a page in wiki data that we visit quite frequently and we're always looking for more information to be captured there. So we've been working on understanding how to crowdsource and capture non scientific knowledge, this common sense knowledge for AI. Let me let me talk about scientific knowledge itself. And I'll talk about collaboration tasks and then I'll talk about crowdsourcing vocabularies. I'm very intrigued by crowdsourcing collaboration tasks because as you saw in those images, collaboration is so pervasive in science. And we don't understand it very well. So I do a lot of cognitive task analysis when I see groups of scientists collaborate, which is every day. And it's very dynamic, very ad hoc, very much of an emerging collaboration. It's seldom the case that they come into a meeting or a collaboration and they know exactly what each of them is going to do and they kind of have a plan. Everything kind of evolves as they understand each other. So it's kind of the blind guiding the blind on collaborations. And it's really a necessity. Any scientist that has data or that is looking at a problem is really reaching out to collaborate with others that have more expertise in certain types of data or understand some other type of model or some other type of physics or chemistry. So this is something that is only going to increase as we have more scientific data available. So we worked on a framework that we built on top of Semantic Media Wiki for collaboration. So we were interested in task oriented collaboration. So looking at how scientists formulate tasks, how they form groups to accomplish those tasks, how the group that works in a task relates to the group that works in a different task, how those tasks are related together. And very importantly, something that you all will resonate with, we were very interested not in the traditional AI frameworks for teamwork, where, you know, you're flying a group of planes. It's clear who is the lead pilot. It's clear who has to run the helicopter in this kind of pattern. So people play certain roles. And if someone leaves a team, someone else comes in and takes on that role. So this kind of prescribed teamwork was not our interest. Our interest was to enable anyone to come in and based on their expertise, their interests and the goals of a collaboration to find their way to tasks that they could contribute to and to creating on their own new tasks that they were interested in. So one of the major difficulties, I could talk about this for a long time, but just looking at a successful collaboration online, we really saw that it's very important to enable newcomers to really absorb what has been done before they came into the collaboration and what was the status of tasks, what were people working on without placing a tremendous overhead on the existing collaborators. Otherwise, it's kind of unmanageable. So so so we we really want to understand how the collaboration process is formed and to make it very evolving. And so we call this organic and organic data science as data science really captures this this interest that scientists have in in analyzing data together. So so what is the what are the tasks that they're working on? Who's working on each tasks? When do they work on them? How are the tasks accomplished? All kinds of informations about tasks. And I've worked for a long time on on planning and task representations and workflows. So I was really interested to see what we could capture from these. So you can recognize here kind of the look and feel of a wiki. And so we were representing on the left, we had a special area where we could see tasks and their subtasks and in the circles, you can see the degree of completion of those tasks. And I think the more reddish that they look, it means that they're a bit late over the schedule. But for each of the tasks, you could see in them, you can see in the middle, we had subtasks, we had an indication of how much of each of them had been accomplished. And then we had some basic metadata about who were the participants, what was the dates when it was supposed to be done, who owned the task and was responsible for it. And then what kind of expertise and focus it had. This is very important to connect people with the right places. And then the rest of the task page was devoted to how exactly it was going to be. And so if the task used a certain software library or model, then that would have its own page on this wiki and the collaborators could actually see a lot of detail about them. So so every task had a unique page, a unique URL and was connected with other tasks through this structure. So what was challenging for us is to build on the wiki architecture. I show this slide not because I can answer detailed questions about how we design the architecture of this platform. But just to show you that we had many functions that we built on the Semantic Media wiki platform at the time wiki based did not exist quite yet. But we were developing these APIs that allowed us to assert new facts, to query about facts, to compare the content of different pages. And then we added this idea of handling different types of categories. So if we were displaying a task, then the page looked a certain way and the metadata looked a certain way. And if we were talking about a piece of software or a data set, then it would look completely different. And a very important piece for us was the tracking the provenance. So who had contributed this? So for example, the software library can have authors who contributed. But then in the wiki environment, who was the editor that had provided a certain fact about a task or something else? And all of that was built as extensions of media wiki at the time. So our universe had these crucial types of scientific objects in the in the framework. And so the users could instantiate these types of pages. So so this really facilitated task coordination. You know, I show you a page that has a lot of green tasks. A lot of pages have a lot of red tasks. A lot of things that scientists endeavor to accomplish are too hard or, you know, they get distracted by doing something else. That seems much more interesting or important. And I think most of all, you can see, you know, this was my own page. But the places where I paid more attention or I contributed more or I had more contributions, we had lots of ways to reflect that on individuals. And I think that you can see here that there's a legacy of, you know, how was this task accomplished? And so we had tasks about organizing the workshop. We had tasks about how to write a paper about something. So in a lot of ways, this is knowledge about the tasks that we as researchers do every day that are really not captured in many places. It's kind of a giant group wide set of to-dos and ongoing to-dos and so on and so forth. So very interesting. So users were truly collaborating. A lot of people were viewing other tasks or, you know, many tasks, most tasks, 72 percent of tasks had several people signed up. But I think what's very interesting is that we started to detect what we were calling social task networks. So what you see here is the users being the nodes and the connections of this network being the number of tasks that two users had in common. So they were both signed up for the same task. And so what you can see is that there's there's already you can see some groups of users that are very connected. They probably have similar specialty or quite complementing complementary specialties. And then you see others sometimes, even if you see a user that's not well connected with thick lines to the rest of the collaboration, they might be really crucial in that they're bringing a very unique kind of expertise. So so very interesting kind of network and and we could see them evolve. So this is an example of a group that started really large and then split into two subgroups that were quite distinct over time. So over a period of several months. So really interesting to see that, you know, there's subgroups and sub collaborations and this can give fodder to those of you, many of you that work on social network analysis, very focused on collaboration tasks. So so it's very intriguing to me to study how humans collaborate and how they accomplish things together and how they connect based on different expertise. And I think that those kinds of social network tasks are greatly understudied and understudied and it would be very valuable to to understand them better. I want to spend time talking about crowdsourcing vocabularies and an approach that we call control crowdsourcing and the success we've had there so far. So so in, you know, I talked early on about environmental research and environmental sciences. And one of the challenges here is that it's very, very different from biology or perhaps chemistry and other sciences where the community seemed more inclined to standardization. So the situation that you see here, and these are pictures from a colleague of mine, Tom Harmon, from the University of California at Merced. His group sets up these little robots that you see floating in the river. They collect data about the river, water quality, et cetera, et cetera. They store it in their own database and they publish papers about the the water quality levels of these rivers that they have instrumented. They go on canoes when the sensors break, you know, in the rain or in the heat. I mean, they really work very hard to collect all this data. And when when other people ask him, you know, well, would you share this data so that we can see the trends in water quality across this region or the whole state? You know, he'll say, yes, sure, it's all here. You know, I'll open up a port and you can download it. But people don't typically want that. They want him to deposit his data in a repository. They want him to annotate it. And he's just not very excited, not just because it's a lot of work. But I hope that you you sympathize and you'll understand that sometimes you go to a shared repository and they start to ask you for, you know, well, what are the synchronicity periods? And like, I don't know what synchronicity period is. They start to ask you for things that you may not exactly fit and you may not know how you fit. And so they have this this very diverse, strongly diverse data collection situations. And so environmental science, if you ask about global trends, it's very hard to to use global data. So so this is our focus. You see on the top right, the famous hockey stick diagram for climate. And so you see that temperatures are raising. This is paleo climate. It's a community that has been studying the climate for the last thousand, the last many thousand years. And every time that they do one of these hockey stick diagrams, it takes them years. And the reason is that the data is collected by individual researchers. And it's really it really takes a long time to pull all this data together. And it's not very easy for them to describe the data in a way that it can be easily aggregated and analyzed. So this has been our target. And to do these hockey stick diagrams, to really understand the climate dynamics, they take samples from all over the world. They take data from samples all over the world. And so that involves many, many, many researchers. So the authors of these papers are consortia, like as you see in other sciences as well. And so we we started a project called Link Earth. And we told them that through wiki crowdsourcing, we were convinced that they could converge on a good way to describe data uniformly without any meetings. And they could just do that from the comfort of their own desk without talking to anybody. And so you could see the the two, three hundred people jumping up and down and saying, OK, let's hear about this. Right. So this was a big, a big, you know, hypothesis that we could do this and it was not clear. And the people in the room, you know, they'll go on on cruise ships. They'll set up drills like you see in the picture to get, you know, samples of coral that have been deposited over, you know, a thousand years and see in every ring of the core that the cylinder core that they extract. Every ring is a different time frame. You know, it's there's no absolute. So they're all proxies for the climate. Right. So if the coral looks a little bit more sad, then it was too hot for that species of coral. What is too hot for that particular species of coral? So these are things that they'll work on while you have another contingent that works on ice cores. So here they're looking at what's trapped in the ice. They're looking at, you know, the size of the bubbles. Other people look at lakes. So we had one single workshop at the beginning of this effort where they actually got in a room, discussed the process, agreed to the process. The main thing was to say, yes, let's do this. Let's try this out. And the champions for this are listed in red. They really, you know, rallied the community and the community trusted them as paleoclimate scientists to to try this new approach because taking years to do each of their global analysis is not very viable. So I'll go quickly over this. There's some papers on this that you can read. But but linked earth really tried to say, you know, as you are using your data set and you want to describe something unique about it, go to the Wiki, create a description for it. And if there's weird properties of a data set that you don't like, don't use them. But if there's a property that you would like to highlight, then add it on. And that way we would crowdsource how they would each like to describe their data. And then we created a very analogous process to what you see in Wikipedia of having editors that will oversee certain areas and and what would be accepted from these crowdsource properties. And the result of all this process, and I'll show you some a little bit of detail about it, it's a new standard for paleoclimate that resulted from just one meeting. So at least we have one piece of evidence for that hypothesis that we could do this fully online and it's being widely used today and increasingly broader adoption for paleoclimate data. So I'm excited to tell you that this kind of wiki crowdsourcing is helping science in new ways and helping them synthesize new kinds of scientific knowledge, which are these agreements to the properties. So this is the paper that they wrote about the standard that actually was selected for a centennial collection by the largest geoscience society. We're very excited about it. And this is how our wiki starts welcoming you as this tradition. And this is a very old picture. But I think I didn't have time to make a fresher one. But the idea is that if you're doing lake sediments and if you're doing coral cores, the properties may be very different and you may want to add your own properties as you are editing this. The data set that you're describing is really your ground truth. Right. You need this property because it's important for describing this data set. So it's very much bottom up. It's no one is just sitting on their desk and thinking, oh, wouldn't it be nice to have blah? It's more, you know, I'm working with this data set and gosh, I cannot find this metadata. And if it's the same as this other data set, so let me add that as I go. So pay as I go kind of semantics. And so and so every data set was downloadable. They had properties. Some of them were provided. Others, you know, the contributors did not provide them. We run into a lot of issues because some properties required separate pages to really provide all of the information. So that made each data set have a network of pages that that made it very awkward. So if we had to redesign this, we would do that very differently. We would allow them to define new what we call crowd property. So so after a few weeks, you would have a bunch of new crowd properties that were not in the core, which is the first gray rectangle that you see. So so as they were adding a new property, we would offer completions, as you can imagine, this would encourage them to adopt what was already there. They could see the descriptions of those properties. And they did not always adopt others. And so a lot of the things that we did was how do we get the community engaged? So we were trying to attract them to community discussions, to looking at maps so that they could see the number of data sets described grow. We would have ways to give authors credit and, you know, the usual kinds of things that you see. But a very crucial thing is that we would ask for their opinion. And so we had all these polls to decide whether or not a certain property was more popular than the group of people that had used it. And so we did a lot of polling to the community. So one of the successes that we had, one of the things that we did well is to start out with a very clean initial ontology. So this took quite a bit of convincing because from their perspective, they had a clean initial core ontology. There had been a researcher called Bob Evans who had created this core set of standard terms and all they lived by this. But of course, when you bring, you know, mathematical logic to the picture and you start to say, well, if this is really not measuring temperature and you're if you're really measuring the health of the coral, which is correlated with temperature, that's not really a measurement. That's not really observable. That's really something else. Oh, OK, well, let's call that a proxy. OK, is it a but it's an observation. OK, let's call it a proxy observation as opposed to a real temperature observation from from. I can't come up with the word a real measurement of temperature thermometer. That's what I was trying to find. So so we completely turned their clean standards over their head because for every term that they were talking about, we could find a community ontology. So we would use folk to talk about researchers and their papers and their collaborators. We would use Prove. We would use Geosparcle. We use schema.org. I think that we were one of the first possibly one of the first science ontologies to build on schema.org for data and really help them think through it. SSN, the Semantic Sensor Network, was extremely helpful. So that guided how we modularized and how we created that core set. And then we just threw them in and then they would, you know, add new terms. And periodically the editors would then merge the new terms into the core terms. And these editors were the ones making these decisions. If the upgrade was simply to, you know, a monotonic change where we would just add a term and that was fine, then we could automatically upgrade all the data set descriptions that they had been working on so far. But sometimes it required semi automatic processes. So so the papers described this pretty sophisticated way to let the contributors, you know, annotate data sets, suggest properties. And then these editors that you see on the left having different functions in updating and upgrading the ontologies all together. It's surprising to me that this kind of approach is not more common and and for us updating an ontology while people are using it. I know that the gene ontology has done that from the beginning. But we're always surprised that that this is not more present in a lot of projects. So so we once the new version was approved, we included it, we annotated where the terms came from. We never removed any term, things that now are increasingly common to do. So the report are these kind of subcategories of annotations. Some of them are particular to tree cores. Others are particular to coral and marine sediments. So so you can see that they they actually form these sub communities and these sub vocabularies. Some of them were applicable to all of the all of the data sets. And then there was something called chronologies that had temporal hypotheses. So this ring belongs to this period and they have these very flexible ways, depending on the uncertainty that they had to describe the dating of certain parts of the core. So that led to its own also common vocabulary. So very interesting kind of work. I mentioned how important it was to take votes. So we thought that if someone doesn't use a property, it doesn't mean that they don't find it useful. They're just not using it for their data sets at the moment. So we decided to do these polls that they could really answer. It was hard to do them often enough through the wiki because not everybody was on the wiki every day, but we tried to keep the community engaged. So we would tweet polls and that was getting a lot more kind of continuous engagements and so on.  But no one got together to discuss whether this term or that term or any anything like that, more about the overall process. So as as they got together for other reasons or other meetings. So so they they were able to engage a hundred and thirty five researchers worldwide, as you see in the map voting on more than six hundred properties. And I think part of what they wanted to get is whether properties should be really, really required if they were really important or if they were optional and nice to have, because that reflects on the lift that they have to do to annotate new data sets or to annotate legacy data sets. And so so this is just to give you an idea. I mean, you know, for for each of the different aspects of the of the standard, this is one of the older data sets. And so the metadata that is really available is lacking in many areas. So for example, you can see the uncertainty aspects that the community considered as required and essential are really not present in the data set. And so they would have to contact the authors of this data set and kind of dig all of that out. So they wanted to be careful about what to require. So so today they are moving along. They're using this standard to decide how to include new data sets in the wiki and in their repositories. Some of the repositories are run by NOAA. They also work very closely with Pangea that I believe is out of Germany. And they also are using the ontology and the standards to guide the creation of notebooks and analysis tools, a lot of their software. So, you know, you just mentioned a property of data sets that you want to analyze. And so all the data sets have that property and they can be pulled together. So their ability to do aggregate analysis is is really amazing these days. So so I'm keeping my fingers crossed that I can tell you soon that the next hockey stick diagram maybe will be done in five minutes. So so that's a that's a big success story and something that's really making a huge change for for this community. We're working now with a neuroscience community. It's a worldwide effort. I think there's about 50 countries involved, hundreds of institutions, thousands of researchers. They self organized into groups. They share they don't share the data. They share the knowledge that they have data about something. So they'll say we have a study on schizophrenia on people from 16 to 34. Oh, we have a different kind of population. We have schizophrenia in older adults. And so they kind of share this metadata, not so much the data sets themselves. So but it's so self organizing. And if you're studying schizophrenia, you might have data about smoking and how that, you know, changes things with aging. So they might have another group looking at smoking and aging and they'll reuse this data. So again, this very organic idea of collaboration. So I apologize, these are very old screenshots. But we describe data sets. We describe the machines, the MRI machines that they use that makes a huge difference when you integrate the data sets. We describe the cohorts that come from each of the data sets. And it's a way to help them also create broad studies with very different kinds of populations. So so the crowdsourcing of vocabularies is kind of an ongoing process. You know, we really have to find the right way to engage. This is a much more fragmented and diverse community, but but very exciting. So I'm going to wrap up and leave time for your questions, which I'm very interested in. And I want to kind of highlight for you what I think have been successes and what are our challenges ahead, which are very, very many. And I hope that you appreciate that we're coming to wikis from a completely different perspective and with certain objectives and really benefiting from all the community's work on making these platforms available and usable for the rest of us. So this has really enabled a lot of research for us. So so I think, you know, as a summary of what I've talked to you about, I think you can take away this message that through crowdsourcing, we are really creating added value and synthesizing new forms of scientific knowledge. So you often see, for example, that, you know, there's a description of proteins and there's a database about proteins and there's many databases about proteins and you see wiki data helping identify that, you know, there's several ideas that refer to the same protein and can we create a single place where we can really describe this particular entity. So the description of scientific entities is super important, describing papers, describing data, all kinds of entities. We are kind of synthesizing new forms of scientific knowledge. So one is these tasks and what are the tasks, sub tasks and what are the workflows that the scientists are following as they do the science. So these are not prescribed workflows that they'll say, oh, yes, this is how hydrologists and agricultural modelers work together. And then they just download that workflow and follow it. But they're really creating a new one from scratch every time. And we're trying to capture that new form of collaboration that they are creating. I talked about the social task networks, right? What, how do people relate to one another through the work that they do and the tasks that they pursue in science? And then I've also showed you that through crowd sourcing, we can actually catalyze consensus vocabularies that did not exist before. And I think there's many, many other possibilities in science to capture new forms of scientific knowledge as we do this work. So in the case of the consensus vocabularies, which is something that I think might be of broader interest to all of you, I think we did a careful seeding of the core initial terms. That's really, really important. That was the same approach that Mike Ashburner took in the gene ontology and something that I would recommend for anyone. And then you can grow that core outward. And that was a key to making everything work for us.  So we need to give them immediate benefit. And in our case, it was this immediate description of their data set and this immediate ability to query the data sets with these new properties. So this property of semantic week is to be able to exercise queries immediately with new properties was very important for them. And then really make it part of their workflow and their ecosystem. So they care about writing software to then access the data sets and analyze them so that integration has to be there. So all of these aspects really contributed. It's still very challenging for us to support this continuous evolution of vocabulary is if you've annotated 500 data sets and now someone convinces the community that it was all wrong to do this that instead it should be done some other way. It's really necessary to make that change, but it's really painful. And there's an entire process to doing that, that we need to really facilitate and make more agile. I think these areas of task-centered collaboration and how we help people work together in ad hoc organic ways is also something that we need to study better. And then finally, we're finding, at least in the neuroscience world, they need to do selective sharing. So in Wikis, we normally open all the content to everyone. Maybe some cannot edit, but everybody can view. And here they're really pushing for selective sharing. So they may not want to expose a lot of details about their data only to the close collaborators until the paper is out. So we say that selective sharing is better than none. And hopefully it's something that is counter to the way Wikis are designed, but maybe something that we can support for them. And we've been working towards that. I showed you this slide at the beginning, but longer term, this is really the vision. How can we capture a lot more scientific knowledge that has to do with processes and synthesizing ideas and expertise and eventually have AI be part of the scientific ecosystem and have humans and people not just play freestyle chess, but really collaborate to solve the hard science problems in the future. And I'll conclude here and take questions, Lila and Bob. Thank you. Thanks so much, Yolanda. Q&A will be managed actually by Christina and Tiziano, who have been collecting questions. Hi, Christina. So we have some questions. How much time do we have? Because we have Ismael with four questions. So maybe we can ask the first and then leave the other if we have time. Maybe Ismael, if you wanted to ask directly the question, otherwise I can read it from the chat. Let me know. OK. Do you hear me? Yes. OK, OK. Hi, Yolanda. I have read something about your works because I'm making my final project for my master thesis regarding data papers. So when looking for ontologies regarding that, I found your link paleo data ontology. But I found a previous one, which to me is really, really similar, the ecological market language. Why do you prefer to start a new one from scratch instead of reducing this other one? So we did reuse some of the community ontologies, as I pointed out. The trickiest part for us was to distinguish the many, many different kinds of observations and indirect observations called proxies. That was really, really tricky. There's lots of ontologies. There's one called ENVO that is used a lot in the life sciences. There's another one called SWEET that was put forward by NASA. Those have maybe a dozen terms that we can really connect to. But our challenge was really representing observations. So in fact, in the ontology paper, we mapped the ontology to ENVO for, you know, it seemed like a good idea to do that, to publish the ontology. But that was our main challenge, that the observations were not properly fleshed out in the other ontologies, and that was what we were focusing on. Thank you for your question. And thank you for clarifying. We also had another question from Adam, I think very relevant. So the question was, how do you handle this potentially sensitive data about patients? So do you store the data or just the metadata? And just in general, when collecting neuroscience data sets, how you ensure the protection of personal information? Yes. So are Wiki only, this is about the Enigma project, the neuroscience project. The Wiki only stores metadata. So we'll say, you know, the hospital in Postam has patients that are 18 to 35 in age range that are non-smokers, they have data from MRIs, from EEGs, from this, from that. So we describe the content of the data set. We never see the data set. In fact, when the neuroscientists analyze diverse data sets from separate hospitals and universities, they actually send out the workflows and the instructions to run the analysis. And the analysis are done locally. And then the results are sent to a central place here at USC. And then they are kind of aggregated. They do what they call a meta analysis. So the data never leaves the organization that collected the data. And what I'm trying to point out is that the data, of course, is sensitive and has its own reasons not to be shared broadly, but the metadata also. So they often are worried about publicizing their data very broadly because then everybody will ask for it or then everybody will tell them, you know, oh, can you take from your cohort this or that or the other? So sometimes they just don't want a lot of publicity about their particular data set. Sometimes, you know, they'll say this data was collected with this instrument. So if you have a certain kind of instrument, then a lot of people will want to talk to you about how you collect your data and your processes and so on. So in some sense, and I think it's pervasive in science, the more that you expose the more attention you get in some cases, and if your data is valuable, that's the case. So they're very private about their metadata even. So that was something that I was very surprised about. I assume that metadata was something freely shared, but they have many reasons not to. OK, there is another question from Bob about the limit of crowdsourcing. Basically, how far can we push crowdsourcing and what are limits that can be achieved by crowd in terms of science? Thank you, Titiano.  That's a crowd. And in the case of Enigma, there's there's thousands. So that's a crowd, but it's a very peculiar kind of crowd. When we were collecting common sense knowledge, it was just, you know, we call them medicines or web volunteers.  Here, these are, you know, modified experts. They really have a lot of expertise in their field. So they're not just anyone off the street or any kind of volunteer. They have a serious job. They want to do science. And so it's a very different kind of crowd and a different kind of crowdsourcing. So it's kind of its own niche crowdsourcing. If you if you if that makes any sense. So what are the limits? I think not everybody has a community outward looking view of science. That's not the case with everybody. So that's one of our challenges, even though the benefits are clear and the needs are very clear, but not everyone has that. The good news is that I believe that there's more people with a community oriented view today than there were 10 years ago in terms of science and science, science researchers. So I think things are shifting in a good vector. And I also think that because the experts are busy and scarce and so on, finding ways to route to them the most controversial or delicate questions while getting the rest of us involved in science in more common ways. So our initial concept for linked earth and that's where the name came from is that it didn't need to be the paleoclimate scientists annotating every climate data set that existed thousands and thousands of data sets that they've collected over decades, but that we could actually get regular citizens to annotate them. And it proved that there's just not enough context and information to have regular citizens do it. So they ended up being just the expert crowd, but just finding pockets of science where citizens can contribute I think is one of the challenges for scientists because there's just too many hard questions and too much data and too many problems that opening science to other collaborators I think it's very important that that's a challenge that we have. You asked a very broad question and I'm sure you know there's many more answers but I'll start there. Yeah, but just to connect to that I think very related if just just in two sentences because we are running out of time. There was a quick question about broadly the barriers that hinder this type of research. What are your thoughts. What are the major challenges. You know this type of research is really a labor of love. So we could be doing what 99% of the community does which is to download data from Reddit or from Wikipedia, or from wherever and study the data that's already available there. I think we're doing a labor of love because it takes a long time to create these environments where we can collect proper new kinds of data and use them and, you know, doing that just takes a very long time so it takes years to forge these collaborations to establish and to explore it so link data was not born in a day and they were not convinced in a day so I took maybe two or three years of convincing and really going into the experiment so I think that's the biggest one. But what a nice way to put it where it's inspiring to end on that note. So I think we are now scheduled for the break. Yeah, I'll go ahead, I think. Yes, so Yolanda thank you so much this was an amazing keynote thank you for for spending some time with us. Yes, let's thank you. Let's clap. It was, it was great. You are welcome to say it up until the end of the work.


First Timothy chapter 4. We're going to be looking today at verses 6 through 10 as we continue our verse by verse study here in the book of First Timothy. So allow me to read to you from verses 6 through 10 and we'll get into our study. And what we're looking at, the topic heading would be the good minister, because that's what Paul is speaking about in these verses, and it's the good minister part one because we'll pick up next time we're together and continue to see other elements that he's outlining for Timothy. So beginning at verse 6, reading to verse 10, First Timothy chapter 4, Paul writes, if you instruct the brethren in these things, you will be a good minister of Jesus Christ, nourished in the words of faith and of the good doctrine which you have carefully followed. But reject profane and old wives' fables and exercise yourself toward godliness. For bodily exercise profits a little, but godliness is profitable for all things, having promise of the life that now is and of that which is to come. This is a faithful saying and worthy of all acceptance. For to this end, we both labor and suffer reproach because we trust in the living God who is the savior of all men, especially of those who believe. Now as we're going through our study in First Timothy, let me remind you that Paul had instructed Timothy concerning doctrine, teaching, that has been taught by false teachers. And he was pointing out that in the last days, error would grow and people would continue to be deceived. What was existing in the early days of the church would continue throughout its history. And Paul was saying that it will grow until it infects even those who once professed agreement with the truth of the gospel.  And that lines up with what Jesus taught concerning the atmosphere of the last days. Now remember Paul in verse one had said, the spirit expressly says that in latter times some will depart from the faith. And that lines up with what Jesus Christ taught concerning the last days and its conditions. In Matthew chapter 13, let me refresh your memory by pointing to Matthew 13 and reminding you that in chapter 13 of the gospel of Matthew, Jesus gave a series of parables. And he spoke concerning various things in those parables, but three of the parables spoke of the invasion of evil into the church. So you see him giving the parable of the tears. You see him giving the parable of the mustard seed and you see him speaking concerning the parable of leaven. When you look at the parable of the mustard seed, the parable of the mustard seed actually gives us insight into what Paul is writing of because in Matthew 13 verses 31 and 32, Jesus said the kingdom of heaven is like a mustard seed, which a man took, sowed in his field, which indeed is the least of all the seeds. But when it is grown, it is greater than the herbs and becomes a tree. Notice so that the birds of the air come and nest in its branches. Now he says the mustard seed was the least of the seeds that would be planted in Israel. So the point Jesus is making is how large the seed grows and what was the result. So he uses the mustard seed as an illustration that helps us to understand that he's speaking of growth. He's speaking of the growth of what would be referred to as the external kingdom. You know, there are those who say this speaks of the church starting out small and growing larger over time. There are commentators who will look at that particular parable and say, see it began in a small way, but eventually it grows into something massive and to some degree that would be true in that from a small group, it grew larger over time. I was reading recently something that was put out by the Pew Research Center in Washington, DC and this particular center gave an estimate of Christians worldwide. And they write as of 2015, Christianity was by far the world's largest religion with an estimated 2.3 billion adherents. Nearly a third, 31% of all 7.3 billion people on earth. And so according to the Pew Research Center, 31% of all people living on the face of the earth are Christians. So from 11 men, there are now 2.3 billion Christians worldwide. The question has to be asked, is that an accurate figure? Are there really 2.3 billion solid born again believers? And is that what Jesus is referring to? That there'll be an incredible growth over time and that the world is going to be incredibly impacted by all of these sincere born again people. When you look more closely at that, you realize of course that many people in the United States to this day when asked what is your religious preference will say they're Christian. But the fact is, is we know that they're just saying that they're Christian. We'll look at that in just a moment, but in reality, aren't born again. When we look at this a little bit closer, when you look at the parable that Jesus gave, you can actually get a different picture because he speaks concerning birds. If you look at, for example, if you look at his first parable that he gives, a parable of the sower and the seed, he speaks concerning the seed that falls on the wayside. If you remember in that parable of the seed that fell on the wayside, Jesus said that the birds came and they took the seed. So it didn't have a chance to actually produce any fruit. And he pointed to the fact that these birds, when later he gave the interpretation, he pointed to the fact that the birds were evil, that they devoured the seed, they snatched away what was sown in hearts. And so in that parable, birds were used to symbolize the wicked one who undermines the word of God. So we have to keep that in mind. The second thing is Jesus spoke of the mustard seed. He said, when it grows, it becomes greater than the herbs and becomes a tree. That would be a picture of inordinate growth, something that was massive but was not normal. So there are those when you interpret this parable who will say that this is a picture of what is called, and I want to develop this with you for a moment, a layer foundation, it's called the visible church. Now over the years, theologians have developed phrases that refer to what we call the church. Normally when somebody's talking about church, they'll say, well, I'm gonna go to church or I was at church. But when you begin to look at the way the word church is used and you begin to look at scriptures that relate to the body of Christ, the church, you'll see that the word church is used to really speak of various things. So theologians have developed through what is called a systematic theology. They've developed a way to look at the church and to define certain aspects of it. So you have what is called the church universal. The church universal speaks of the body of Christ throughout the world. So there's a believer here in the city of Chino and there's another believer that's somewhere in China, there's another believer somewhere, we'll say in England and all. So there are churches, believers throughout the world and it's called the church universal. Then there's the church local. That speaks of individual congregations that meet in certain locations. Again, here in the city of Chino we have church meeting here but there are many churches throughout the area that are gathering together. They're referred to as the local or the church local. The church invisible speaks of true Christians who are recognized by God as genuine. You have the church triumphant and that speaks of believers who have died. They've gone to heaven. It's called the church triumphant. And then you have what is referred to and I wanna develop this a little bit. For a moment you have what is called the church visible and the church visible speaks of those who are the visible community of Christians but there's something important about that phrase because the visible church is visible but it isn't always made up of genuine believers. You see church services always have genuine Christians and others who are present who are unsaved. They're simply there in the services. Obviously unsaved individuals are welcome because we want them to hear the message that we desire to give but though they are present in the service they're not part of the invisible church. The world calls them Christians because they attend church services. And with that said, the church as described in this parable is referred to as being large and it is said to have birds nesting in its branches and that describes evil being at home in the church. So the point is the church is numerically large because the church is infiltrated by unbelievers. Birds nesting in the branches simply signify that unbelievers are at home in the church and the church is infected with evil and that would be what Paul is instructing Timothy about he's saying in the latter times the doctrines of demons will find a home in the church and the doctrines will infect people, they will be deceived, they will embrace demonic error and the fruit of this doctrine is that people begin to depart from faith. So Timothy is to faithfully warn believers concerning the danger of bad doctrine. Why is that? Because to fail to instruct the people properly is to allow them to be deceived. Failure to teach them God's word leads to the church becoming weak and confused and to stand against demonic error filtering in requires them to be strong in scripture. We had a ministry morning yesterday, one of the questions that was asked of me is what do I think is the present danger in churches are not only ours of course but what would be a present danger? What is something I'm concerned about? And I answered very quickly the fact that there is error that creeps into churches and that part of the reason that the error is accepted as doctrine is because the members of the churches are not studying and reading the Bible themselves. And because we're not equipping ourselves through the word of God, simply because somebody opens a Bible and speaks from it, we have a tendency of trusting that they are giving us the truth when in fact we aren't studying the word ourselves in order to be equipped in order that we might have discernment. I'm concerned about that. To stand against demonic error filtering in requires you to be strong in the word of God. So he's saying as a genuine teacher you are to continually nourish and equip them with God's word because God inspired his word, the Bible, from Genesis to Revelation even as Psalm 119, 160 says the entirety of your word is truth. In 2 Timothy 3, 16 Paul said all scripture is given by inspiration of God and is profitable for doctrine, for reproof, for correction, for instruction in righteousness. And so by giving them the word of God he's to safeguard them from spiritual loss through error. And that is the purpose of teaching the whole counsel of God. And Psalm 119 verses 2 and 3, blessed are those who keep his statutes and seek him with all their heart. They do nothing wrong. They walk in his ways. And so Paul is speaking to Timothy as a pastor of the church of Ephesus and he's saying you need to instruct the brethren. And in verse six he says if you instruct the brethren in these things notice you will be a good minister. So the statement that ties the verses together is the phrase you will be a good minister. So learning to be a good minister is one of the cheap purposes of this letter. This is really what we all should desire to be a good minister of Jesus Christ. So his point is Timothy your responsibility is to instruct the church in the things of God and by doing so you'll be a good minister. The word good is a Greek word that can say genuine or noble. The word minister is a Greek word that means servant. You will be a genuine, you will be a noble servant of God. So Paul's giving to us qualities that make up the excellent minister and the verses before us summarize the qualities of this noble minister. So notice what he's to do if he's going to be caring for the church the way that God calls the minister, the pastor to care for the church. Let's look at that together. He says Paul you're to consistently teach and live out the truth. Paul says that to Timothy you're to consistently teach and live out the truth. He said in verse six if you instruct the brethren in these things you'll be a good minister of Jesus. So Timothy instruct the brethren. The word instruct means point things out. That very often includes repetition and very often you will include the repetition of certain basic foundational truths. It's so that people will hear it over and over again and embrace it. Repetition is the way that we normally learn. I learned what my name was because my mom kept calling me by a certain thing. She kept calling me David. She did it for over a year until I realized she's talking to me. That's how you learn your name. She called me other names but I knew that my name David was my given name. So we learn things through repetition and that's how you learn foundational truths. The first time I've heard something doesn't mean that I really understand it. It simply means a layer has been added to my life that gives me a certain dimension of understanding.  In second Peter in chapter one verses 12 through 14 the apostle said I will always remind you of these things even though you know them and are firmly established in the truth you now have. I think it is right to refresh your memory as long as I live in the tent of this body because I know that I will soon put it aside as our Lord Jesus Christ has made clear to me. So he said, I will always remind you. He said, I think it is right to refresh your memory because that's how we learn by being reminded and by being refreshed. So the heart of a genuine minister is to faithfully and thoroughly teach the word of God. That's because we received a sacred trust and that's because we need to value it and that means that we as ministers are to honorably discharge that. It's like what he said to Timothy in second Timothy one verses 13 and 14 when he said, what you heard from me keep as a pattern of sound teaching with faith and love in Christ Jesus guard the good deposit that was entrusted to you guard it with the help of the Holy Spirit who lives in us. He said, I have laid the pattern out for you hold to the outline of healthy teaching. He said, guard the gospel. Timothy you have the responsibility of transmitting healthy teaching and according to Deuteronomy chapter four verse two do not add to what I command you do not subtract from it but keep the commands of the Lord your God that I give you or chapter 12 verse 32 of Deuteronomy see that you do all I command you do not add to it or take away from it. So make sure that that which has been entrusted to you is faithfully given and not changed. And many say it doesn't really matter what you believe about Jesus as long as you're sincere but the Bible says otherwise. What you believe about Jesus is of infinite and eternal importance. It is not to be changed. The word of God is to be embraced as it is. When Paul was writing to the Galatians in chapter one verses eight and nine he said this. He said, even if we or an angel from heaven should preach a gospel other than the one we preach to you let them be eternally condemned as we have already said.  Does truth matter? Absolutely. So our motive is faith. Our motive is love for Christ. We are to love people and we're to tell them the truth. We are to guard the gospel through the Holy Spirit. And that occurs in other words is guardian of the gospel through the guidance of the spirit. We need spiritual strength because of the spiritual attacks that come when we hold fast. Again, remember in verse one Paul had just stated that in latter time someone reject the Christian faith. And because of this Paul is exhorting Timothy to communicate the truth clearly and in doing so he's safeguarding the sheep from deception. You see the neglect of teaching the word of God thoroughly as resulted in an anemic church. Error has crept in and is regarded as true Bible teaching. So he's making sure as he's sharing with him to instruct the brethren in these things, nourish them properly. He says a good minister of Jesus Christ is nourished in the words of faith. He's nourished. That word nourished means to be sustained or fed spiritually. A good minister is fed spiritually by God's word because he knows God. So to be a good minister one must be committed to the personal study of the word of God. There are those who simply memorize messages that are given by people they would like to be like. So instead of studying the Bible and meditating on the word they simply become parrots and they repeat the things that they were memorized in on somebody's CDs.  He sees the need of the word of God. It's like what Job 2312 says, I have not departed from the commandment of his lips. I have treasured the words of his mouth more than my necessary food. It's what Jeremiah was saying in chapter 15 verse 16 when he said, your words were found and I ate them. Your word was to me the joy and rejoicing of my heart. In Psalm 19 verses seven through 11, the law of the Lord is perfect, converting the soul. The testimony of the Lord is sure, making wise the simple. The statutes of the Lord are right, rejoicing the heart. The commandment of the Lord is pure, enlightening the eyes. The fear of the Lord is clean, enduring forever. The judgments of the Lord are true and righteous altogether. More to be desired are they than gold, they are than much fine gold. Sweeter also than honey and the honeycomb. Moreover by them your servant is warned and in keeping them there is great reward. A third thing he says in verse six is he's to carefully follow and apply the word of faith. He's supposed to live out what he gives out. You see a life transforming message is best given by those who have been transformed by it. When God has grabbed hold of your life and transformed you, there's a certain credibility that you have as you communicate it. You're not telling people to do what you say, you're saying do what I say, of course because that comes from the Lord but also do as I do. I'm not telling you to do what I don't do myself. In other words, that's why Paul in first Corinthians 11 verse one could say imitate me just as I also imitate Christ. He could point to himself as a model. In Philippians 3.17 he said, brethren join in following my example and note those who so walk as you have us as a pattern. In Philippians 4 verse nine, the things which you learned and received and heard and saw in me, these do and the God of peace will be with you. So when you're sharing your faith with people it's great if you're living that faith out before you're giving that faith out. It establishes credibility because they'll look at you and they'll say you're living what you're giving and that's what he does. He carefully follows and applies the words of faith. A pastor is not supposed to be up in a pulpit like I am right now saying and acting in a certain way and then an entirely different person when I'm not up here. Your life is to be consistent Timothy, live out what you give out.  He says in verse seven, but reject profane and old wives fables and exercise yourself toward godliness. A good minister rejects profane teaching. The word profane speaks of that which is unholy or ungodly and that's how Paul would describe the Jewish legends of his day. You see the best way to recognize unhealthy teaching is to be acquainted with the truth. Being nourished in God's word and avoiding error produces health and discernment. So in a time when deception is on the increase God's word will protect you.  So he says reject these false things. In verse seven of fifth thing, reject it and exercise yourself toward godliness. Now godliness, godliness conveys the idea of a personal attitude toward god that results in actions that are pleasing. Generally the church reflects the ministry of the pastor. It's been said like shepherd, like sheep. And so the church has a tendency of embracing the things that the pastor presents. And every pastor has a different personality that God uses and that personality is part of the presentation. But there are things that are within that pastor that the members if they remain begin to embrace and to live out. And that pastor has a way of provoking within people certain senses when the teaching of the word goes forth. I was sharing this at a pastors conference 25 years ago or so now. And I was sharing how that if you listen to certain pastors that they actually provoke certain things within you. And I was saying for example when I hear Pastor Chuck Smith, as the pastor Chuck provokes me as I'm listening to him, to wanna walk in faith and to love the word of God. He provokes that in me. When I listen to John Corson. John Corson is like a big brother kind of hanging over a fence, kind of visiting with you. He has that feel to him. That provokes that relationship with him. When I hear Greg Laurie, when Greg Laurie is preaching that there's a sense of need, urgency. You need to get right with God. That's Greg Laurie. When I hear Rawl speak, I want Mexican food. So I, there's certain things. There's certain things that the teacher provokes in you. And there is a saying amongst pastors like shepherd, like sheep. And so Paul is making it very clear that you are to apply the word of the Lord.  Now, that word exercise in the original language is from the word that we derive gymnastics or gymnasium from. Exercise speaks of the rigorous training that every athlete endures if they will compete well. So we are to maintain an unrelenting and persevering effort toward the goal of godliness. Paul says that exercising oneself to godliness is strenuous and ongoing. But the result is godliness, godliness which is virtue is also regarded as a holy and reverent lifestyle. It's revealed by the fear of God, the love of God. It speaks of a lifestyle that is focusing on God and growing in godliness is the result of personal choices as well as disciplines. In Luke 9.23 Jesus said, if anyone desires to come after me, let him deny himself, take up his cross daily and follow me. So living a godly life isn't an option for a select few. It is for all believers. Listen, you don't wake up in the morning just a godly person. It doesn't happen that way. It is something that you exercise yourself towards. It is something that requires the power of the spirit but also a disciplined life. And it's one of the things that we need to understand. You see, on one hand godliness is pursued and is going to be the fruit of spiritual discipline. When I was 20 years old, I went into the military and I was serving in the army. I was going through basic training. As we're going through basic training, a recruiter showed up and invited those of us in our, you know, who are going through training together, invited us to meet with him and he wanted to talk to us about going airborne.  Now, at that time, I wasn't making very much going through basic training. $55 was a lot of money and I thought, I could make $55 just jumping out of a plane sounds good to me. And so I signed up for airborne training and so I'd only been on a plane one time in my life. I had flown from LA to San Jose. That was it and now I'm placed on a plane flying from LA to Fort Benning, Georgia all the way across the United States. I arrive, it's June, I arrive and they put us into airborne training. It's over 100 degrees every day and we do physical training four hours in the morning. They get you up right and early and everywhere you go, you run. We start training with a three mile run and then the rest of the day you run wherever it is you go and you have physical training the different kinds of push ups and sit ups and you name it, all of that's continual.  And so you have to learn how to hit. You need to learn how to roll because when you jump out of a plane it's not just like you see on TV where you just kind of come down nice and soft and you land on a pillow. It's not like that at all.  That's how hard you're hitting. You're coming two thirds the speed of gravity. So it's not like you're just floating, you better know how to land. So you go through all of this training and you finally make it through your second week and now you're in your third week and your third week they put you in a C141 jet. They take you 1200 feet, 1250 up in the air.  I was the third man in what they call a stick. I was the third man from the stick. I was in the stick. I was the third man out the door. So I'm sitting there looking at the jump master, a black cat we called him and he's standing next to a door with a red light and a green light. And at certain point, the plane begins to slow a bit. They open up that door and you have a wind foil and the wind foil is perforated and they say stand up and you're looking at the red light, stand up.  Then they say, check your buddy's equipment. What do we know? We don't know what we're looking for so we pat each other down. I took the guy's wallet. I said, you won't need this. So you check your equipment. You check your buddy's equipment. Then you tighten up. You just, you're right next to each other. There's two rows of guys on each side of the plane and you're watching the black hat and you're looking at a red light and you're looking at a green light. My eye is fixated on the red light but you're listening to the black hat and then the light turns green. That means go and he screams, go.  So you can be going, mommy, but you're going out that door. That's a fact. My very first jump, third man in the stick, I landed on top of the parachute of the guy who jumped out before me. I was sitting on it and I was in shock.  He starts screaming, get off, get off my shoot.  You don't know if you're going to hit to the left. You don't know if you're going to go straight. If you're going to come in backwards, sideways, you can come any way. So you'd better know how to land. And our first jump, 14 guys broke their legs and their ankles. It's not easy. You watch the movies, oh, I can do that. It's your little couch potato with your Game Boy. World at war, I could do that. No, you can't. You think you can and maybe you can, but you can't until you're trained. And you have to have the want to. You have to want to. I want to jump out of this plane. I want to run three miles. I want to do physical training four hours in the morning, three hours in the afternoon. I want to jump out of a 34 foot mock tower. I want to jump out of a 200 foot tower. I want to do that. I want to stand in this line, but I have to discipline myself to do it. See, it's one thing for me to say I want to be a ranger. I want to be a green beret. I want to be a Navy SEAL Delta Force. You can say how you want, that's fine, but you have to train yourself to it. It's not that you can't, maybe you can't. Maybe I have, I'm sure I do. I have vets in here who know exactly what I'm talking about. But when you're just kind of sitting around thinking I can do that, it's one thing to say I can, it's another thing to do. And in order to succeed requires a desire and discipline. And if you want to be a godly person, it's not going to come with you laying down on the couch, thinking that God is going to unscrew your head and drop in all kinds of knowledge and maturity, then you wake up the next morning and they say, super Christian woke up. It doesn't work that way. It doesn't work that way. It comes through exercising yourself to godliness. There needs to be the sense that this is what I want. It's like what Paul said in 1 Corinthians 9, 24 through 27. Do you not know that in a race all the runners run, but only one gets the prize? Run in such a way as to get the prize. Everyone who competes in the games goes into strict training. They do it to get a crown that will not last, but we do it to get a crown that will last forever. Therefore, I do not run like a man running aimlessly. I do not fight like a man beating the air. No, I beat my body and make it my slave so that after I have preached to others, I myself will not be disqualified for the prize. Godliness. There's a discipline. Godliness can be the fruit of the walk that you have with the Lord. So you read God's word, you apply God's word. You learn to worship God in spirit and truth.  You serve Him. You give your gifts to Him. You receive baptism. You receive communion. You learn to share your faith. These are all outward evidences of a commitment to the Lord and it reveals a godly life. Again, it's not the result of self-effort alone. It is made possible by the Spirit of God. In 2 Peter chapter one, verse three, the apostle says his divine power has given us everything we need for life and godliness through our knowledge of Him who called us by His own glory and goodness. And so reject profane and old wives' fables. Exercise yourself toward godliness. Verse eight, for bodily exercise profits a little. The godliness is profitable for all things. Having promise of the life that now is and of that which is to come. This is a faithful saying and worthy of all acceptance. Now remember with me a moment before Paul had referred to the practice of false teachers. They had been promoting remaining unmarried and abstaining from certain foods. Here it seems that they're promoting certain exercises that promise spiritual enlightenment. There are to this day groups of people who will say to you that you shouldn't marry, that you should eat certain foods and that you should exercise and have certain positions and all of exercise in your body and that you can be spiritually enlightened. That's not a brand new thing and Paul was addressing that. And he's pointing out that bodily exercise profits. It does, but for a short time because ultimately we all die. You can work out and you can be powerful and I'm not saying that you shouldn't. I think that working out is good and it's healthy. God gave to you a horse. God gave to you a message. Take care of the horse so you can deliver the message. That makes sense to me. At the same time, we need to understand that no matter how hard we work out, ultimately we all die. That's just the fact. So a good minister pursues the eternal because godliness is eternally profitable. Someone once said a godly life is always the best advertisement for Christianity. So he says, remember that there are certain things you can do but those are temporary. Seek out the thing that lasts. And then finally he says in verse 10, for to this end we both labor and suffer reproach because we trust in the living God who is the savior of all men, especially of those who believe. And so a good minister knows that ministry is labor and is often filled with hardship. In Hebrews 6 verse 10, 10, God is not unjust to forget your work and labor of love which you have shown towards his name in that you have ministered to the saints and do minister. Notice that he speaks of your work and labor. In 1 Thessalonians 5 verse 12, we urge you brethren to recognize those who labor among you and are over you in the Lord and admonish you. So he's saying, listen Timothy, a minister is not lazy. A minister will work for long hours as he ministers the word. It's not what we would today refer to as a nine to five kind of lifestyle. It's what you are, always. I was at a restaurant a while back now. I have a friend of mine I hadn't seen since we're in Bible college. And we're in Bible college and I hadn't seen him and now he's ministering in Pomona. And so we hooked up, he and his wife and Maria and I and we went out for an early dinner. And we were in a restaurant in the city of Claremont. And as we were talking, we're in these booths that are separated so it's like a booth and then next to us is like a, you know, just some, I don't even know what to refer to. It's just like three feet high and separates the booths and then there's booths on the other side, just a separation. And as we're seated there, somebody stands from the other side who's in the table next to us, stands up and looks over to where we're at. And he says, Pastor David, and I look up and I said, yeah, yeah, he says, yeah, I recognize your voice. I said, you did? No, I said, really?   And he says, this and that, whatever it may be and we spend some time together and I pray for him and I leave my bill and I walk out and I said, be generous with your tip. No, I visit with him, I pray with him, that's what I do and I come and I sit down and my friend who's also a minister says to me, does that happen very often? I said, it happens all the time, all the time. Really? I said, oh yeah, and you know what, I love it. I love it. That's why I'm alive. That's my ministry. It doesn't matter if it's standing up here. If I run across people in Costco, I run across people in restaurants, I run across people on planes. I run across people who are waiting on us at airports.  Marie and I have run across people where we've been flying to the East Coast and stopped into Dallas, Texas, Dallas, Texas and somebody walks up, hi pastor, I go to your church, I'm in the military, I'm stationed in such and so place, I'm working with the Pentagon and I'm on my way to the Middle East. Can you pray for me? I'm on my way. I can't tell you what I do, but I need your prayer. It happens all the time and I love it. Listen, a minister is not a nine to five.  It's what you do with your life and he's reminding him. He says it's labor and it is labor but it's the labor with eternal dividends and he says remember that but he also speaks concerning reproach, the reproach. On the one hand you have those who do not share your faith in Christ and they do say things about you and some of the things can be so vicious. All you need to do is read the newspaper and look at how people speak of believers and it's very common and becoming even more so. There's a lot of reproach that relates to it. They don't share your faith in Jesus so you work hard and sometimes you are reproached and not only by the people on the outside but there can be a reproach from those who are even on the inside just the other day. I was sharing with somebody who approached me very openly, spoke honestly to me and said I need to ask you a question two weeks ago. I need to ask you a question. I said sure, of course, what do you need to ask? Do you drive three Mercedes-Benz's? I said not at the same time. No, I said, I said what, what? Do you drive three Mercedes-Benz's? Because I was told that you have three Mercedes-Benz's and you live in a three-story house. I started laughing. I said no, I don't. I have a two-story track home that I've lived in for 22 years and I paid it off. I also have a TLX. I said if you want, you can go into the back. It's right here by my office and you can look at it if you want. You can wash it too if you need to wash. But, no, I didn't say I was playing. But you know, those kinds of garbage comments have gone on for almost as long as I've been in ministry, almost as long as I've been in ministry there have been things like that that have been said. It's just unbelievable how people do that reproach. But you live with it, you know why? Because you love the people even though they say things like that, you know why? Because eternity is more important than momentary feelings that we can have. That's how it works. That's what it's supposed to be like. I'm only, I'm telling you this because I don't know, that comment didn't come from somewhere else. It came from here, my church. So if you're here, I'll give you a ride. There's a lot of reproach. There's a lot of things that people say. There's a lot of things that are untrue and people have a tendency of wanting to believe that which is untrue, they just do. But you want to know something? At the end, the Lord has a way of working and He does the right thing. In Hebrews 10, 32 through 35, it says, Recall the former days in which after you were illuminated you endured a great struggle with sufferings. Partly while you were made a spectacle both by reproaches and tribulations and partly while you became companions of those who were so treated for you had compassion on me and my chains and joyfully accepted the plundering of your goods knowing that you have a better and enduring possession for yourselves in heaven. Therefore, do not cast away your confidence which has great reward. When Paul was writing in 2 Corinthians 11, at verse 23, he spoke of his labors, imprisonments, his beatings and his facing death often. So he says, bottom line, he says, verse 10, to this end we both labor and suffer reproach, why? Well, because we trust in the living God who is the savior of all men especially of those who believe. Serving the Lord, it does include labor, it does include poor treatment, it does include sorrow, and it does include disappointment. But you keep your eye on the prize and our efforts to serve the Lord are always rewarded. In 2 Corinthians 5, 9 and 10, we make it our aim whether present or absent to be well pleasing to him, for we must all appear before the judgment seat of Christ that each one may receive the things done in the body according to what he has done, whether good or bad. We trust in the living God, the savior of all men, especially those who believe. We will stand before him, thus serve him with all of your heart. Two last things. He says he is the savior of all men, especially of those who believe. When it says he is the savior of all men, he is the potential savior for any, potential savior of all men. They're not automatically saved, in other words. He is what is called the potential savior, but it goes on to say especially of those who believe, he is the potential savior for all the actual savior for those who believe. He can save anybody and becomes their actual savior when they say God be merciful to me as sinner, forgive me of my sins, come into my life, forgive me. You're born again. He was your potential in that he could save you. He is now your actual because you've been saved. We take the word of God, we proclaim it, and as he is saying here to be a noble minister, we do so with our eye on the prize. At the end, he will say to us, I pray, well done, my good, my faithful servant, one day we may hear him say it. I look forward to hear him say, enter into the joy of the Lord.  That's the first part. We'll pick this up next time.