So, thanks, Sebastian. Thanks, Cookbot, for organizing this track. My name is Sebastian Siddist, Chris Salzburg, title of this talk is The Elusive Attribute. Thanks for coming to this talk. I was really worried that being the last sort of the last set of talks of the whole conference that everybody would be dead and nobody would come to this talk, but I'm really glad to see so many people here. So this is a track on unpacking rails. And one of the other speakers mentioned that, you know, rails is really, really big. And there's a lot to unpack in rails. There's a lot of parts in there. And there are some of those parts that we know. And there are other parts there that we don't know, but we kind of know that we don't know them. But actually, I think there's a lot more, right? And so this is a quote by a guy you may know named Donald Rumsfeld. It's kind of a famous quote, actually. Probably many of you have heard this. He made this statement in 2003 in a press conference in response to a journalist. He said, as we know, there are known knowns. There are things we know we know. We also know there are known unknowns. That is to say, we know there are some things we do not know. But there are also unknown unknowns. The ones we don't know, we don't know. So I know bicycles. I've been basically on bicycles as far back as I can remember before. I could even walk really. This is a picture of me when I was something like, I think, four years old or so, with my dad on a bicycle. My dad is Dutch. And the Dutch are very fond of their bicycles, you may know. And my dad has no exception. So he raised my friend, brother, and I to have a real kind of respect for the importance of bicycles. And I grew up in Montreal in Canada. I'm Canadian. And you may know that Montreal, well, you may not know Montreal is a real cyclist city, which is kind of surprising, giving you how freaking cold it is there, half the year. People really bike through all kinds of weather you really couldn't imagine. And I was no exception. So I always biked when I was there. And then in my 20s, I moved to Amsterdam in Holland for a couple of years, and Amsterdam is for the world capital biking cyclist. So I was biking the whole time I was there. And then, as Sebastian mentioned, I moved to Tokyo. That's where I've lived for many years now. And I still bike in Tokyo. That is not me. But I have bike through this intersection.  I use them every day. I really couldn't live without my bike. So I know that I know bicycles as a user. I also know that there are things that I don't know about bicycles. So if my bike breaks or something and it's anything beyond sort of getting the chain back on, I'm kind of clueless and I don't really know. So I just bring it to the shop and they fix it for me. But recently, I discovered something that I didn't know about bicycles. Which is that I didn't know how to draw a bicycle.  Try to put the frame, the pedals, and the chain on this bicycle. And you may find that it's a surprisingly difficult thing to do. In fact, it's such an interesting thing that cognitive psychologists have studied it as a demonstration of how our mind can trick us into thinking we know things that we don't actually know. So these are four drawings of bicycles from a study where participants were asked to, as I just said, draw the pedals, chain, and frame onto these bicycles. And as you can see, none of these bicycles would ever actually work. There's all kinds of interesting innovations on these bicycles. But one way you can sort of see, well, okay, so here, sorry. Here's another picture of bicycles. This bicycle may not be so obvious if there is anything wrong with it. And so one way you can kind of see whether a drawing of a bicycle like this would actually work is to go out and actually sort of implement it. And this bicycle, in fact, has been sort of implemented. And this is actually what it might look like if you were actually to build this bicycle. And this is kind of interesting now. This is actually one example, one rendering from an art project, actually, by an artist named Gianluca Gimini. An Italian artist. And he found this phenomenon of drawing bicycles so fascinating that he went out and he started asking everybody he knew to draw on bicycles. And of course, most of the bicycles, like the ones I just showed you, were in some way, you know, they had problems with them. But he found this so fascinating, they were all different. He went out and rendered them all, like this. And it's a really fascinating thing. It's called Velocipedia, so you should look it up. There's some really interesting pictures. This one's interesting because it's not, it's not that obvious. You have to look at it a bit carefully to see what, what would actually be wrong with it. But this bicycle would not work. If you, if you sat on it, it would pretty much collapse. And so, in this, in this process of getting all these people to draw on bicycles, Gimini found something really interesting. This is from an interview where he, he talks about this. And he found that people, even like myself who ride a bicycle every day, they, they generally don't know actually how to draw a bicycle completely accurately. But there, there is one group who is statistically, he found was statistically much better at drawing bicycles. And these are people who actually sort of fix bicycles with their hands. They actually know how the parts work together and they, they know more than just the form, they know the function of that form. So this is a quote from that, from an interview where he says, people who are really into bikes start out from the frame. They represent that small percentage of people who get the drawing perfectly right. Most other people tend to start from what they are most sure of. So for nearly everyone, it's the wheels. I saw many people drawing all the rest in the hope that positioning the handlebars, saddle, and pedals would help them figure out the frame. So what does this have to do with attributes, right? So I think attributes a little like bicycles for myself anyway and people who ride them and people who know them are something that's really familiar to us, right? If you're using rails, you're using attributes and you're using them everywhere, right?  I mean, they're everywhere and they seem kind of simple on the surface, right? But what I'm going to try to argue to you today and explain to you today is that attributes are actually elusive. And they fall in that category of things, like I mentioned, that you think you know, but you actually don't know that you don't know. And one way to bring out these things that you think you know, but you don't actually know that you don't know, is like those drawings and bicycles, is to actually draw how this thing would actually work. And I think if I asked you all to draw how an attribute works, of course the drawings would all be different, like the drawings from that art project I mentioned. But I think there'd be some common elements. And one of those elements I suspect would be that you would draw this. You draw a method, right? This would be like the title on your post model or something. And this is like the wheel on your bicycle, right? It's the thing that you know must be there. And then you probably were talking about active record, you probably draw a column, that's where this thing gets its information from, it's data from in your table in your database.  And this is like the frame of the bicycle, right? And I bet many of you would draw this last. And so what I want to do today is we're going to do this in three parts. So in the first part I'm going to take this picture here, and like that picture of the bicycle, we're going to kind of implement it. And then we're going to sit on it, and we're going to watch it crash. And the second part then we're going to redraw it starting from the frame. And we're going to see how that looks. That's going to be sort of a more accurate representation of how it actually is implemented in practice. And then in the third part we're going to see what's actually wrong with that frame and how we can possibly improve that. Okay, so before to that, Sebastian introduced me. This is my kind of late introduction here. These are kind of my attributes. As I said, my name is Chris Salzburg. My handle is Shiyama, which translates to Salt Mountain, which is approximately my last name. I work for a company called Digica here, and they graciously funded me to be here. So if anybody's interested in working at a great company in Tokyo, let me know. I write up a stuff like the module builder pattern. If you've heard that expression at my blog, digima.com. And also I have a gem called Mobility for Translating Model Data. Okay, enough about me. Let's talk about methods and columns. So to start this, I want to start from a pull request, which I made to Rails actually about a month and a half ago. And the pull request is called Give Generated Attribute Methods Module and Name. It's a very small pull request. It's only a five line change and it's a one line addition. It was merged. So if you go on Rails Master it's there now. And it doesn't really do a whole lot actually. So to see what it does, you can go and you can just take any active record class. I'm taking like a post here. And you just look at the ancestors and this is before the change. So if you look at the ancestors here, you're gonna see a bunch of mixed in modules and classes that we inherit from and this kind of thing. And what I want to highlight here are these two things here. These are actually, they look different from the rest because they don't have a constant attached to them. They are anonymous modules. They're actually instances of a subclass of the module class called Active Record Attribute Methods, Generated Attribute Methods. And this is actually a module builder. I'm not gonna talk in detail about that but I've talked about it before and written about it before. You can look it up. Basically you just need to know this is an anonymous module. And so if you jump to master branch right now or Rails 6 when it comes out, this is what it will look like. And this is what that change, that pull request did. It gave these things a name. So now it's like post generated attribute methods. And that won't really change anything in your application. But it's sort of important because these modules are, in one way, are very important. So to see how they're important, we can just go and create a post here. So create a post, give it the title, the elusive attribute. And then we can go, and one of the other speakers talked about introspection. So we're gonna use introspection here. We can say, post, give me the method object for this method title. This is Ruby now, this is not Rails. We just say, okay, we want this thing. It'll return to you this method object. Everything in Ruby is an object. So you get this method object for the method title. And then we can say, okay, method, where are you defined? So this is actually a useful trick when debugging, by the way. And you can ask this, and if you're in master right now, it will return to you exactly that module, the post-generated attribute methods module. And then we can say, okay, what's this module doing? Let's see what other instance methods it has. And if you call instance methods on this thing, you're gonna see a whole slew of different methods, right? You're gonna see the methods, all the methods that have to do with this title column. So as you see here, title before typecast, title came from user, title question mark. But you'll also see the same set of methods for all the other columns on this post table. So you'll see ID before typecast, all these things like that. And these are called attribute methods. And the term is important because they're actually modules for this active model and also an active record. And this stuff seems kind of actually easy when you look at it on the surface, but actually active record overrides a lot of the methods in active model. And this thing is actually not so simple. And this is not all of attributes. What I'm gonna talk about here, I'm gonna talk about attribute methods. It's not all of attributes, but it's kind of the entry point. If you wanna learn more about how it works, you really have to understand attribute methods. And it's not easy to understand. So that's what I'm gonna try to cover now in the next, however many times I have left. So let's go back to our picture here. We had a method mapping to a column. We haven't really contradicted anything here yet, but we've elaborated a bit. So we can add a context. This method sits in a module, which is an instance of this generated attribute methods thing, and that's in your model. Okay, so now we're gonna sit on the left side of this picture here. See where that goes. So we're gonna assign that module to a mod here, to a variable named mod. And now what I wanna do, this is a bit like a genetic experiment where you kind of knock a gene out and you see what happens with and without the gene. So we have this module. We wanna knock the module out and see what happens if we take out that module because it had all those methods on it. But you can't take modules out in Ruby. You can't just like take a module out of the ancestors. But you can do kind of an equivalent thing which is to remove all the methods on the module. So we can do that. We can just say, okay, module, give me all your instance methods, then iterate over them. And for each one, we're gonna use this method, remove method. It's a private method, so you have to call it with send. So you call remove method, you give the name of the method in. This will remove all the methods. And you can check. You can say now mod, what are your instance methods? And this will now return an empty array. We've removed all the methods from the module. And you can double check. You can say, okay, post, give me your method title. Previously this gave you back the method object. Now it will raise a name error on define method title. So we're sure there is no title method on this thing anymore. So now we can go and say, okay, post, what's your title? And surprisingly, I think, probably to most people, it will still return the elusive attribute. So this module is in one sense, it seems like it's really important, but we've just removed it and it seems to be working fine. So what's going on? So we can go back to our picture here. And what have we done? We've knocked out this method, right? And yet it still seems to be working. And if you know Ruby at any level, you probably know there's only really one way this can be happening, which is that we're hitting method missing and somehow method missing is handling that call and seems to be doing the same thing as before. It's still mapping to the column value. Okay, so now I wanna shift to the other side of this picture. So what we did in the first part here is we kind of removed the methods from that module, we saw what happened. Now we're gonna go to the other side of this picture and we're gonna try to remove the columns, which is gonna be a kind of a similar parallel kind of thing. And what I mean by when I say we're gonna remove the columns, I don't mean we're gonna actually remove the columns from the database. We're gonna make active record think that there are no columns on this table, this post table. And this is gonna be a bit tricky, but we can do this. Grab this thing called the schema cache. And the schema cache is what active record uses to what it sounds like to cache the schema. And the schema is what tells active record, what are all the columns and all the tables in the database. And active record is not gonna let us trick it to do this kind of thing. It's not gonna make this easy for us, but we can do this like this. So we can grab, we can reach inside to this cache and grab this thing called a columns hash, which is an instance variable. You have to get it with instance variable get because there's no public method to get it. And then what we can do is we can set the value for the key posts in this columns hash to an empty hash. And what this will do is that if active record has not yet cached anything, then what this will do is it will tell active record, okay, I've already cached this thing and the value is an empty hash, which means there are no columns on that table. Normally these hashes would be a mapping, mappings from column names to types, but since it's empty active record's gonna interpret that to mean there are no columns on the post table. And we can go and confirm this. We can inspect the post class. And this is kind of funny, right? I mean, we don't normally see active record models whose tables have no columns because it's kind of a useless thing. You would never really do this, but this is what it would look like if you had one. And then you can create an instance and same thing, it looks kind of funny because it doesn't have those attributes. Normally you would have, you know, title, whatever, whatever, but that's not there because active record thinks there are no columns on this table now. And then we can go ahead, we can say, okay, do you have a method, a setter method or any of those other methods? It's gonna raise a name error. It doesn't have any of those methods, which is not really surprising. We can try to set the title anyway. Maybe it's gonna hit method missing. It doesn't, it raises an active model on an unknown attribute error. Then we can just check the attributes and it's a blank hash. So this is not surprising. We've told active record, this table has no columns, so it's pretty much acting the way we'd expect it to act. But now we can do something a little bit different. Instead of creating a post, we can fetch the first post and assume now that our table has one entry in it with a post with the title of the elusive attribute. And it's gonna return to you again this post and again the post doesn't seem to have anything on it, so that's not surprising. But notice that we're making this select statement, right? We're select post star from post, which is always what you do when you grab the first thing of a model. And the database, remember, the database does have those columns on the table. So the database is gonna return all those values, even though active record doesn't think it has those columns. And then you can again go and check those methods like the getter method here. It's again gonna raise a name error, so it doesn't have those methods. But then we can say, okay, post, what's your title? And it's again gonna return the elusive attribute, even though active record doesn't think this thing has any columns on the table. And in fact, you can check the attributes on this thing and it's gonna return to you all of them. So in this case, the ID, the title, the timestamps and everything. And what's happening here is actually that what we're kind of exposing here is that active record treats attributes in kind of two different ways. The one kind of privileged way it treats attributes is sort of the ones that are actually defined in your schema. So your schema says, these are the columns on this table and those are kind of special in a way. But then it treats also attributes in terms of what the database returns and what the database returns you can see if you do this kind of thing here. You can just call, this is what actually happens internally, you can call an exact query on the active record base connection and give it the SQL string that it would normally use and look at the result and it's gonna return to you this active record result object. And you can check the columns on this result object. The word columns here is kind of liberal. I use air quotes for this because these are columns in a very liberal sense. But in this case, it's returning you columns, ID, title, created, updated, and the values for these things. And the key point here is that this result that comes back from the query is completely independent of whatever the schema cache is. And you can check this. You can say, okay, post, what are your column names? And it's still gonna tell you the column names are, there's none, there's no columns on this table. But it's happy to return to you these things and let you access them. So if we go back to this picture now, what's happened? Well, we've now knocked out the column from the schema and this has sort of knocked out the method. In fact, it never defined the method for these things because there was no columns for them. But when we call that title method, what happened was that it went, it seemed to go through method missing and it went directly to the result which came back from the database. And this, I'm gonna refer to this as kind of a backdoor root. So we have the frontdoor which is the one we're kind of accustomed to which we think about which is going from a method to some value which corresponds to a column on the table. But then there's another root actually which you probably don't know about which goes through method missing and gets the value if there's a result corresponding to that name from the result from the database. And this may seem kind of artificial and I've kind of tricked active record into thinking there were no columns on the table and why would this ever even matter? But there's a reason this is here. And as far as I'm the first person to give you the reason for this, I've never found anybody else who's explained this. This is the reason it's here, it's there. If you've ever used this type of pattern, this is not a super common pattern but it's pretty frequently used. I think probably most of you have actually, I've at least seen it. This is the case where you do a query method like select or group and you give in some SQL and then you alias something, right? So you alias like title to foo.  And then you grab the first thing, right? First one. And then you call foo. And now if you think about it, foo can't be a method here, right?  If you actually wanted to be a method, you'd have to define a method every time you get a query like this. It's not practical for ActiveRecord to do that. So the reason why there is that method missing thing there is to handle this case. Where you're aliasing something to something else, the database will now return to you the foo thing and somehow you have to be able to access it. So this falls through the method missing. And that's what it's there for. So we can go back to that picture and to make things a little more concrete now, we've got this title method. Title is the method we're gonna hit when we just do it like the first. So we select post from post. And then that's gonna be our kind of a front door root here. And then method missing, when we do select title as foo, foo is not gonna be a method. So we're gonna hit method missing, but method missing is gonna go through this backdoor root and it's gonna get the value from the result which was returned from the database. Okay, so that was sort of the first section. Now I wanna kind of look at a better, I think a more accurate way of looking how this actually works, which is to start from the frame. And this is gonna be in terms of these things called what are matches and dispatchers. So to think about this, I wanna kind of think about this in terms not of sort of describing how things just are, but how in terms of the kind of constraints that this frame actually satisfies. And there's gonna be kind of two key constraints which are really gonna determine how this works. And the first one is a constraint not of Rails, it's of Ruby. And probably a lot of you are familiar with this. It's very basic thing. In Ruby, methods are fast relatively speaking and method missing is slow. Actually method missing is very slow. So you generally wanna avoid method missing if you can. The second thing is that the schema is static. And this I think is obvious. If the schema wasn't static, it would kind of defeat the purpose of having one in the first place. But attributes are dynamic and this is the one that's not obvious. And that's why I just showed you that select statement example. Attributes can have any name you want them to have. You just alias something and it can be anything. And so ActiveRecord has to deal with that. In fact, ActiveModel has to deal with this. ActiveModel deals with it for ActiveRecord, but this has to be handled. And so the solution that is there in the code has these kind of two different things. One is the front door root where we kind of take the schema, we define the methods for everything that's in the schema and those are fast and those are what we usually are hitting. But when you alias something, we go through this backdoor root. And in that case, we use method missing to handle these dynamic attribute names. And the thing that's gonna sit sort of in the center of this whole thing is gonna be this very innocent thing called this attribute method matcher.  It's an ActiveModel attribute methods towards the bottom of the file. It's a very simple class, it's not complicated. But the role that it plays is not obvious at all and not particularly documented anywhere as far as I can tell. And this thing is just basically it's just a combination of a prefix and a suffix. So in this case, I'm looking at the dirty was matcher, which you may have used. And this thing is gonna, these matchers are gonna kind of run in two different directions. They're gonna be kind of a forward direction for the front door and a kind of a reverse direction I'm gonna refer to it as for the backdoor. And in the forward direction, the term matcher itself is gonna be confusing as you'll see but we'll get to the reason why it's called a matcher in a second. So what do we do? We take all these column from the database and we kind of stick them into this machine here and then it pops out, it puts the prefix in the beginning and the suffix at the end and we get title was. That's not so surprising, that's our method name we're generating here. And then we're also gonna generate this thing called a target, which is where we're gonna take the string attribute and do the same thing. So that's gonna give us attribute was. And then we're gonna create what I'm gonna refer to here as a dispatcher method. Which is just title, defined title was and it's gonna take the arguments and then it's gonna call this method attribute was which is kind of a generic method. Hasn't title which is the name of the attribute and the other arguments. And this is how it's gonna work and this is where the front door code where this actually is used. This is, you can find this in active model lottery methods. This is not terribly complicated but it's going through these matches. So just for reference, there are currently 18 of these matches.  But there's a bunch of other ones as well. Those are those ones that I showed you earlier in the set of methods. So what are we doing? We're taking this attribute name title from the schema. We're calling method name here, which I just showed you. And now we're taking method name which is like title was. We're taking this target, this handler method called attribute was. We're taking the attribute name title and we're sending them to this method defined proxy call, which has really intuitive name. And then you see this generated attribute methods here. That's that module that I mentioned I gave a name to. So it's going to define that method I just showed you on that module. And it's going to do that for every column, for every each of those patterns, right? This is the front door root. This is a little bit easier one to understand. The other way we're going to look at this is in terms of the back door root, the one we go through method missing. And here the way to think about this I think is to kind of take this machine and kind of flip it in reverse. And we're going to convert the prefix and the suffix to a regex. So now we're matching stuff. And this is where the matcher name comes from. The matcher name is really confusing because matchers and matching is not the main thing that it does anymore anyway. It once did. You can do a bit of git archeology on this and you can go back. And originally the main role here was matching stuff but it's not matching anymore. But it still does match. And so when it matches, it's going to be handling something like this. So we get a method like foo was. There's no method foo was. So it falls through to method missing. And method missing is going to pass this to all those matchers. And they're pretty much all going to return nil because they don't match any of the other patterns. But this matcher here is going to match. It's going to capture foo here. So foo is going to be a capture from this thing. And then again, it's going to have this attribute was handler method. And so previously we were defining methods, handler methods, dispatch methods, sorry. And now we're going to be, but now we're going to be actually dispatching from method missing. And this is where the code, this is the code that actually does this. This is like a fantastic method name I think. Attribute method matcher is matching. So this code here you can ignore. I'm going to debate with Rails Core Team to get this removed currently. But basically this is not really important. Of which some members are here right now. This is the important part here. So we're going through all the matches. I said there's 18 of them, right? And in this case for foo was that none of them are going to match except for the was matcher. Well, actually one more, but we'll get to that. But basically the was matcher is going to match here. And it's going to match foo. It's going to say that the capture is foo. And then the handler method is actually was. And it's going to dispatch to that thing. And so we can now try to visualize what this whole thing looks like if we were to actually draw it. So what we're going to do, we're going to put the matcher thing in the middle of this picture now. So this is the matcher. There's going to be 18 of these matchers, but we're just going to look at one for now. This is like the was matcher. And then what? We have for each matcher, we have a dispatch target. This is going to be a method that is actually defined. I'll show you one in a second. So in this case it's going to be an attribute was method. And then we're going to have two different directions. So one method we're going to be defining method. And this is the case, this method I showed you define proxy call. And this arrow here is not a calling arrow. This is like a method definition arrow. So we're defining a method called title was. And then title was calls attribute was with the argument title. That's what that method will do. And then the other direction is that we're coming from method missing. So we're coming this way. And now method missing is calling match with this foo was on all the matches. And again, they're pretty much all going to return nil. So it's going to skip all of them except this one's going to match. And then, and this is important, method missing is going to say, okay, it matches, the capture is foo. But before it actually dispatches, it's going to say, okay, is foo actually in my results? Like, is that foo? Did I actually get something called foo in the results? And if not, it's going to ignore it. But if foo did come back, if we alias titled the foo and there was a foo something in the database results and the query results, then it's going to then do the same kind of thing here. It's going to dispatch to attribute was with the argument foo. And you notice here that both paths are ending up in the same place. And that explains a little bit why earlier on when we took away the module and we called title, it still ended up in the same place because whether you go through the method, which is faster and preferable, or whether you hit method missing, you both end up in the same place ultimately. So the top one here is the front door root. As I mentioned, the bottom one here is the back door root through method missing. And just to see where this actually happens, you can go into active model dirty and there's a class, there are class methods to define these matches.   And as I said, there's a total of 18 of them. Here there are four defined right here. And for each of those things, there's a method. I'm showing you just one of them here, like this, which will actually handle these method calls. So this is attribute was. And it tells you what the previous value of an attribute named adder here was. Okay, so now we come to what is, in some ways, sort of the punchline of this talk. The title of this talk is the elusive attribute. And I've been talking about this was matcher thing. And there are 18 of these matches, right? But what nobody will actually tell you, if you look at the code anywhere, as far as I can tell, is that there is one matcher that is vastly more important than any of the other matches. And you really couldn't have any meaningful Rails application without this matcher. And that's this matcher here. And talk about being elusive, right? I showed you the dirty matches, they're declared, but this one is not declared. It's just the default value in this attribute method matcher's array. You never declare it, it's just there. And not only that, it doesn't take any arguments. We just call new on this thing, to generate this thing. And if you look at the attribute method matcher class and see what happens when you don't pass anything into the initializer, what happens is you get a blank prefix and a blank suffix. So this default matcher is the matcher with a blank prefix and a blank suffix. And what does a matcher with a blank prefix and a blank suffix do? Well, when we go into the front door and we're defining methods, if you pass it a column name like title, it just doesn't do anything to it, it just defines title. And if you go in the other direction and you're coming with foo, it's gonna call match, but match is always gonna return true because the regex is gonna be like always true. And then you're gonna take foo and check is it actually a result that was returned from the database and if so you're gonna call through. So this is the matcher forgetter methods, right? Which is the most important attribute methods we have. So we can go back to this picture that I had here where we had all this was stuff and we can kind of consider what is actually the more important case here, the case where we have a blank prefix and a blank suffix. This seems kind of weird because what's the point of a matcher with a blank prefix and blank suffix? But this is actually the most important one which is one of the things that's really hard to understand about this whole thing. So what happens here is when we have a blank prefix and blank suffix, we define all the getter methods and those getter methods map to a method named attribute because we take the string attribute, we don't append anything, we don't append anything, we just get the method name attribute and that's what's actually called from inside the title method. And the same thing when you fall through to method missing with a call like foo, this will again fall through to the same method attribute. And this method attribute, you can find an active record attribute methods, the read module, and it's there at the bottom, it's very innocently sitting there. It's actually an alias, it's actually a private alias to make it yet more elusive. And this thing just aliases to read attribute, underscore read attribute, which is an internal method to fetch the value from an internal hash of attributes. So that's what it does. So we can go back to this picture now and we're almost there, we're getting there. So the title method when you define it, it calls this method attribute which passed the title and that thing now aliases to read attribute which then gets it from some hash internally. And method missing does basically the same thing, except it's a little more complicated than unfortunately. And this is one of the tricky things about unpacking rails is that everything is quite heavily optimized. And so I mentioned there are 18 matches and 16 of those 18 matches work like this but the two most important ones which are your getter methods and your second by far most important one which is your getter and your setter. Those ones have to be optimized. So actually for reasons that are kind of hard to explain in the scope of this talk, what actually happens is that active model gives a kind of hook to active record to bypass this alias and define it in a custom way. So what actually happens is this. So the title method actually is defined so it basically does the same thing but in a slightly faster way. And now this as far as I'm concerned is an accurate depiction of how attributes are actually defined in active record and also basically in active model but in active record. And I think if we look at this now and I hope at least you've got a sense of how this works but this works, it's reasonably performant but is it easy to unpack? Like I would say no, no way. I hope I haven't lost everybody in the process of unpacking it. So I think we can think about maybe what there is maybe to improve here and that's what I wanna do in the last a little bit that I have left. So in case anybody has been totally ignoring my whole talk because you've been tormented trying to figure out what a bicycle frame actually looks like. This is what a bicycle frame actually looks like. And I think it's actually relevant to look at a bicycle frame because bicycle frames are kind of elegant in a way, right? You can't just make a frame however you want. Like that bicycle that I showed you in introduction there that was based on a drawing would never work because it doesn't reflect the actual loads on a bicycle. A bicycle frames are designed to have the sort of the minimal solution to the problem of carrying somebody on two wheels, right? And I think that's kind of really elegant, right? They don't do anything more than that, it is that. And if you look at bicycles, they tend to have the same type of frames because they've optimized, they've solved that problem in kind of a minimal way. And unfortunately, I don't think that the solution that I just showed you is actually minimal in that way and that makes it harder to understand. So what do I mean by that? So as I said, the reason that we have that whole method missing root that is the hard part to understand and that people don't really know about is to handle this pattern. This is why it's there. By the way, nobody told me this is why it's there. The only reason I know this is why it's there is because I said, why is it there? I yanked it out, I ran the test and certain ones fail and this is the ones that fail. I'm actually not, I mean, there's railcourt team members here but I'm pretty sure this is the reason it's there. Now, it actually was there for other reasons in the past but those other reasons have disappeared and this is now the only remaining reason why it's really there is to support this pattern. And because I was kind of in this questioning mode and trying to figure out what in the world is going on with this thing, I kind of dug deeper and I said, okay, we need to support this pattern, right? But actually, what I showed you in that picture of the frame, it's designed to support more than that actually, right? And if you think about it, because of the way it's designed, it's gonna support this as well. It should. And it should also support this. This is actually the example that I was showing you earlier and some of you may have sort of wondered why are you doing a was on something that's alias that might seem kind of weird but this is supported, this is also supported. If this doesn't seem strange to you, we can substitute this select to this. And now this should start to look a bit weird, right? This actually works. So you can alias the number one to O&E. You can fetch O&E and it gives you back one. You can also change one to two. You can check what one was and it will tell you what it was. And then you can check the change on one and it will tell you a change from one to two. You obviously can't save this because you can't change the number one. But this seems weird, at least weird, if not sort of a bug in a way. But actually it kind of gets weirder. So this is a slightly contrived example but bear with me here. So I work at a company with a payment gateway in Japan they're gonna raise the tax, they say, from 8% to 10% this year. They've said this for many years and they never actually do it so it may not actually happen. But for the purpose of this example, we do a select, we select the amount. We select the amount times 0.08, the 8% to get the tax. We select the amount times 0.02 to get the tax change, that's the 2% increase to 10%. And then we say, okay, get the first payment. We get the amount, it turns out to be 100 yen. We check the tax, it's 8% of 100 yen, so that's 8 yen. And then we check the tax change and that should be two, but it's actually, and this is actually true. This is actually what happens if you do this. And if you were able to follow what I've been saying so far in this talk, if I didn't speak too quickly through it all, you should kind of maybe have an idea why this is happening and this is actually why it's happening. So what's happening is that you get this tax change, you're asking for tax change. There is no method named tax change. So tax change falls through to method missing. Method missing says, okay, do you match this? And most of them don't match, but the change matcher will match on this. It will capture tax. And then it will say, okay, is tax an attribute? And it will say, yes, tax is an attribute because we aliased that thing to tax. And so this will actually work and it will interpret this as the change to the tax attribute, which is kind of meaningless actually. But it will interpret it that way and therefore it will return to you nil because there was no change in tax. Which doesn't make any sense because you can't change tax. You can't change things that you're aliasing in raw SQL. So the interpretation here doesn't make any sense, but this is actually how it will work. And if you actually kind of think about this, this kind of brings up a real point here, which is that I've told you that we have this front door route and this back door route, and they're connected by this matcher thing. And the matcher thing is called a matcher, but the main role is not actually to match, it's to define methods. And its secondary role is to match, why do we want it to match so that we can handle this case where you call foo and foo's been aliased by title. But in the case where you actually match that thing, you only care about the getter methods, right? You don't care about this kind of stuff. And in fact, in the rare case where this actually matches, it's actually more like a bug than what you want. So I think actually this stuff shouldn't even be there. And so why am I telling you all this, right? So one of the reasons is that, well, this is sort of a pitch to allow me to fix this. But I think it's real because I asked a lot of people, do you know how attributes are defined in active record, right? And I didn't really get anybody who could really explain at any level of detail how this works. And actually it took me a long time to figure out how this works. And I think that's not good, right? I think this is bad for Rails as a community. We should know how our attributes are defined. I think this is really important. And the problem is that this code right now is really hard to unpack. It's really hard to unpack because as I said, it's not minimal in the sense of that bicycle frame. It's doing more than it needs to do. And that more is very hard to understand because if you're trying to reason about why it's there, it doesn't really make sense until you look at the history of it and everything that came before. But there's another point that I kind of want to make which is that this is a track on unpacking Rails. So on the theme of unpacking Rails, I think I see a lot of people writing about how they're opening up Rails and oh look it's amazing metaprogramming and it's kind of the spirit of reverence like this is this holy grail or something. And I think this is the wrong attitude, right? I think we need to be questioning this stuff and you need to really be poking it and trying to break it and take it apart and see where it breaks. And because that's the way you're gonna understand, right? That's the way you're gonna understand how it works. And so you don't learn to draw a bicycle by just kind of staring at it. You learn to draw a bicycle or you gain the understanding by actually kind of fixing the bicycle. So I think the message here that I want to leave with is that unpacking is great, don't get me wrong, but don't just unpack stuff. When you're unpacking it, poke it, see if it breaks, maybe 90% of the time or 99% of the time, you'll realize that you were just stupid or whatever, you didn't realize and actually there was a reason for it and then you'll learn something that's great. But that remaining 1% of the time you may learn something that's actually important, right? So don't just unpack Rails, don't just ride the bicycle, fix it. That'll help you to understand better how attributes work, that's how you learn how Rails works and I think that's how we'll help everybody understand how Rails works. Thank you.


So, what we're going to be doing, and it's a shorter one, because this will give you time to work on the assignments, we're going to be looking at databases. I've talked a little bit about some of them. I've introduced you to a couple briefly, but I think it's important to understand a few more of them and be aware of some of them. We'll look at some of the other databases that aren't really just chemical databases, but are really intended to be NMR or MS databases, or pathway databases as well, and then there's another group of databases that we call comprehensive metabolomic databases. So, one of the things that's been going on for a long time is that bioinformatics has been in one silo, and another field called cheminformatics has been in another silo. And they emerged as different fields of information technology at different times. Cheminformatics actually began in the 1970s. It was a very development of chemistry-based software, chemical drawing software, chemical databases, American Chemical Society, CAS databases, and things like that. This was developed before the internet. It was developed before widespread use of personal computers. They were developed before sort of the ubiquity of computing. And they were also developed at a time when software was considered, and should only be considered as a commercial product. So the result is that most cheminformatics tools software, most of the chemical software that was developed was, and largely still is, commercial. Bioinformatics developed in the 80s, really, and it emerged with the growth of the internet, and the model of free, open-source, open-access software. And a challenge has been as we realize that there's some useful chemistry stuff out there, and we need chemistry to do biochemistry and biology, that trying to get those tools and databases from the cheminformatics world and make them open-access, open-source, and free has been difficult. And to some extent, you actually have to build all of our own stuff all over again using the bioinformatics model. So as I said, this is the history, as I say, of the cheminformatics 60s and 70s, user pay, limited access, a lot of it was produced by companies. Bioinformatics through the 80s and 90s was designed from like your biologists, web-based, grew with the internet, and it was funded by public agencies with the expectation of public access. A lot of the software that we use depends on databases. Most of the bioinformatics software now is fundamentally dependent on GenBank, PDB, Uniprode, and Geo, and ArrayExpress. So what databases do is they allow you to consolidate and link information. They allow you to do query or information retrieval. They typically contain a lot of reference values, or reference data, or reference sequences, like protein or DNA sequences, or reference images, chemical structures. A lot of people use databases as well for training, and this is an important area because a lot of what we want to do is prediction in order to be able to predict something you have to practice. So if you have a training set, you can practice on these databases of training set, and then you can use them to test. A key utility as well in databases is similarity searching. So most people in the world of bioinformatics know about sequence similarity searching, but you can also do text similarity searching, structural similarity searching, which is also used in chemistry, spectral similarity searching, and image similarity searching on Google Images. And then the other aspect of databases is to help with aspects of prediction. The primary utility of GenBank is actually to predict gene function, and the annotations that people put into GenBank and the species associations, those are the critical things. Sequences are just tags, hashtags, to allow you to get the function information. So people have used Protein Data Bank to predict structures. They're very good at it now. We can predict properties from structures and from databases, again partly through learning. We've learned about phylogeny, activity, and other relationships. So prediction and similarity searching are some of the main applications for databases in both chem informatics and bioinformatics. Databases evolve, and in the world of metabolomics, it's a young field, and so many cases, many of the databases that are out there are what we call hobby databases. And this is actually how GenBank started. It started as a hobby that a fellow named Russell Doolittle had, and he was collecting sequences in the literature and just typing them in at night. Another person, Margaret Dayhoff, who is considered the mother of bioinformatics, she also started databases as a hobby, and she published books on them in the 1960s, and these are just Protein Sequence databases. Anyways, they were all flat files. Dayhoff databases weren't even flat files, they were books. Doolittle's database was a flat file. But then they actually started doing some things with the databases. As the database grew, Russell Doolittle found that if you started doing a sequence search against viral genes, you could actually identify oncogenes. And the first identification of an oncogene is what allowed sequence searching to sort of take off. As the database grows, and as people want to add more data to it, eventually you start creating what are called curated databases. So this is again an example of SwissProat, which is a predecessor to Uniprot, was a curated database that was handled exclusively by people at the Swiss Bioinformatics Institute. They would choose sequences that they liked. They would curate them and add to them. You couldn't deposit sequences into them. The database grew and grew and grew. And eventually it got actually so large that they had to decide to either go bankrupt or seek permanent government funding, which they eventually got through the EBI and it became Uniprot. As those databases grow and the importance grows, then eventually they can become archival databases. And this is what GenBank and the Protein Databank are. As yet there is no truly archived open deposition database for metabolomics. There's a database called Metabolites, which is disemerging. But it's, I think, only about 30 people have deposited data into it. So it's more a hobby database at this stage in terms of its size. But as these grow, the community grows. The user community grows and the greater demands about it. They don't have to go from the limited coverage that a hobby database may have to very extensive coverage that more archival relational distributed databases have. It also costs a lot more. As the database gets larger and more complex, it becomes very, very expensive. And I can say that a huge amount of MyLabs resources go into maintaining the databases that we offer. It's largely unsustainable. So we're not sure how much longer we'll be able to offer these databases. Likewise, as databases grow, you have to have increasing needs both for standardization. And this is what the Metabolomic Standards Initiative or MSI is about. Better ways for exchanging and sharing information, standard markup language formats, ZML, NMRML, much greater dependence on annotation and automation. You can't do everything by hand, whether it's filling in dates, associating species or descriptions. You can't do it with teams of people after a while. You actually have to depend on computers and text mining. Likewise, as things grow, people want to be able to query in more and varied ways. And so the need for relational databases becomes greater. And web interfaces, graphical visual interfaces. I've seen this slide before, whether it's genomics and proteomics and the sort of missing database and missing killer app that hasn't been around, which allows you to take raw spectra or raw data and interpret it in terms of concentrations and metabolite identifiers. So, up until recently, most metabolomic data was really in textbooks. And it goes back a long, long ways. There's a lot of chemical information at 75, even 100 years old. Arguably the whole field likes behind proteomics and genomics by about 20 years if we look at the database development and when first gene and protein sequence databases first appeared. The other thing is that, in fact, the diversity of users in metabolomics is actually far greater than it is for proteomics and metabolomics. You're looking at not just metabolomics researchers, but analytical chemists are interested in plants and clinical chemists, physicians, drug researchers, people in the field of spectroscopy, people in bioinformatics, people who are involved in standardization and standard formulation. So that's another challenge. And this is one of the things that's maybe why metabolomics has been lagging behind some of the other fields. So there are, I guess it could say, five major classes of databases, the NMR databases, the mass spectral databases, the chemical or compound databases, pathway databases, and then ones that combine essentially all of the above and we'll call those comprehensive metabolomic databases. Usually the comprehensive databases are metabolomic specific, but there are others that are fairly generic. And many of them are aware of at least some of these and probably use some of them every day. So the NMR spectral databases, this is a list of some of them, a spectral database, SBDS, the NMR shift DB is a chemical shift database for NMR, organic molecules mostly, metabolomics, medicine metabolomics, consortium database, and the biomegres bank. The STBS is a database maintained in Japan and it's sort of like the NIST of Japan, it's called AIST, and they collect data and you are able to access and query both NMR, they have IR and UV data, they have thousands of compounds, but you're not able to freely download their data, they have very strict limits on what you can access. So it's been around for a long time, lots of mass spectra, although a lot of that mass spectra data was collected on very old mass spectrometers so it's not quite relevant. Lots of NMR spectra, thousands, lots of FTIR and UV spectra, they have very nice search tools. As with the NIST database and for GCMS, most of the compounds are not metabolites though. As large as it is, compelling as it is, it still isn't something that would relate frequently to the metabolomics community. Biomegres bank, I mentioned this briefly, but this historically was a database for depositing NMR data for large molecules, probably proteins, but about five years ago it shifted into using depositing data for small molecules and they've been steadily adding to a database of more than a thousand molecules where they have NMR spectra and the websites have proved considerably over the last couple years in terms of annotation and in fact the most heavily used part of the biomegres bank ironically is its metabolomic component. So it's now more than 800, it's closer to 1,000 now. They've collected about five or six NMR spectra for each compound. You can search by chemical names, formulas, smiles, strings. A lot of metabolites in this were planned, it was done as a rabidopsis effort, but they've started throwing in mammalian metabolites so it's a pretty broad collection of compounds. A lot of them are now assigned so you have data that not only gives you the spectra but also the assignments. NMR ShiftDB is a database that was developed by Chris Steinbeck who now heads up the metabololites section at the EVI, he also directs Kebby. This database has about 50,000 NMR spectra. It's totally open access, totally downloadable. In that regard it's probably better than the SBDB but again most of the compounds are non-metabolics. These are organic synthetic compounds, most of them dissolve in chloroform or TMSO so not in the typical water or aqueous conditions that we use. You can look at some of the structures there and you can see most of them are not terribly biological. But it is a collection of chemical shifts and structures that are potentially useful. It does have a tool for chemical shift prediction. Given that some of these tools are expensive, it's nice to know that there's at least one that's free. Now it's easy to search by names and structures and chemical shifts. As I say, most of those are in organic solvents so that doesn't make it as relevant to metabolomics. The Metabolome, Madison Metabolomics Consortium Database is really an extension of the Biomagras Bank. So they just basically took all the Biomagras Bank data and put it into another database. But they started adding literature, chemical shifts, literature data so they archived that. So it has an additional set of literature chemical shifts that you wouldn't find. It also includes some mass spec literature derived spectrum. So through that you can do structure name searches, chemical shift searches, and then it includes additional data about the compounds, formulas, names, properties, and images, and then links to other databases. Now mass spectral databases, this is essentially an identical slide to what I showed you guys before when we were talking about mass spec. So I won't dwell on this very much, but the major pure mass spec databases are Metlin, NIST, GOM, and Mass Bank. How many people have used Mass Bank or ever seen Mass Bank? One. Anyways, it's a really extensive collection and in fact the group that runs it really is hoping that it will be used more and more by metabolomics. So it's being developed specifically for metabolomics in some respects. Although the data that it's got is sort of scattered from a wide variety of areas. Has anyone heard of the database called Respect? It's another mass spec database, but it's focused primarily on polyphenols and polyphenol and phytochemical metabolites. So Mass Bank has peak searches. It's actually a very good interface for doing peak searches. It's a fairly intuitive structure. It's very well maintained. It has a wide variety of spectra. It's collected on multiple platforms, triple quads, TOS, FTMS, INTRAPS, QTOS. It covers about 15,000 compounds and collects data from many, many countries. So in that regard, again, it's probably a step up over what you might find even in Metlin. So I talked about Metlin before. We talked about GOM, so I just wanted to highlight Mass Bank. Chemical compound DBs. We've seen these as well. We've talked about Kebby. We've talked about PubChem and ChemSpider. There's also another database called Ligand Expo. So Kebby, as I said, is about 30,000 high quality compounds. They grab them a lot from other databases from KEG, from LipidMaps, from Drug Bank. The focus in Kebby isn't about spectra. They're not about chemicals or biology or their biological interpretation. It's about their ontology, classifying and naming compounds, which seems that it's a little odd in terms of the emphasis because most of us aren't that interested in ontologies of chemicals, but that's what it was established to do. PubChem, again, it's upwards of 30, 35 million compounds, 35 million substances. The requirement is that rather than 1,500 Dalton's, they typically want something that's less than maybe 10,000 Dalton's, so it includes some peptides, large peptides. They get their data from other people. They really don't generate their own. The data is deposited by vendors, and they worked out agreements to have the vendors deposit their data. So Sigma, Aldrich, the Zinc database and others. So there's quite a bit of redundancy, actually. That's why they said 75 million substances. It's about only 32 million unique compounds. PubChem has grown from just being a pure chemical database. They started to add more and more information and linkage to PubChem, to improve their search, to improve the quality of their annotations as well. PubChem-Spider was a duplicated effort somewhat, and it's sort of been a second fiddle to PubChem for a long time. It's maintained in the UK, largely, or by people in the UK. It's a little bit distinct from PubChem because they actually try and include a bit more data. They're very, very focused on the synonym sets, and the synonym sets in PubChem-Spider generally are better than what you get in PubChem. They also are trying to link more and more to pharmacology, to spectral data, so they're trying to add value that you might not find in PubChem. So sometimes there's information in PubChem-Spider that you don't find in PubChem and vice versa. Lincoln Expo is a database, actually. It's of interest because it contains the small molecules that are bound to the proteins in the protein data bank. So as we noted at the beginning, lots of small molecules play an important role as cofactors and activators and signaling molecules for enzymes and proteins. But what it does is it helps link chemicals, metabolites, and drugs to their drug, or metabolite targets. And that's this important linkage between the metabolome and the proteome and to function. And I think people really underutilize this resource. Lincoln Expo gives you three-dimensional structure, not just the two-dimensional structure of a chemical. So that also is important because the three-dimensional structure ultimately is what determines the biological function of a small molecule. So you can search it through names and identifier codes and formulas and smile strings and inchy and other things. There are more specialized compound databases. There's one called 3DMet, so the 3D structure of metabolites. Napsack, which is a plant metabolite database. Zinc, which is a collection of compounds used for drug screening. And then LipidMet, which is a very large collection of lipids that was put together by LipidMet Consortium. So 3DMet has about 8,500 natural compounds in it, so it's mostly KIG. Napsack, about 50,000 plant metabolites, but those are linked to species information. So that's what makes it actually quite valuable. Fortunately, I don't think there's a whole lot of somewhat limited structure data. Zinc is a database of commercially available compounds. This is mostly a drug bank. And then LipidMet has about 30,000 compounds covering most of the major classes of lipids. Originally, LipidMet was supposed to be about lipids from humans. What they ended up doing actually was downloading a huge amount of lipid data that was maintained by another Japanese database, which covered lipids from everything, from plants, microbes. And so what LipidMet has is the same data in this Japanese database, which covers all kinds of non-human lipids. So it's a little spotty in that regard. So those are compound databases, specialized in general. We've looked at some of the mass spec databases. We've talked about some of the other NMR databases. And so there's the pathway databases. And perhaps everyone's favorite database in metabolomics is KIG. There are other pathway databases, Preactome, which is maintained here, I guess, in Toronto. And it's linked to EBI. Bio-psych, Meta-psych, Ecosych. Those are the psych databases that there's several hundred now, I think, maintained by Peter Karp. And then another one that probably none of you have heard of called the Small Molecule Pathway Database. So what the pathway databases do is they start linking, just like LigandExpo links metabolites to proteins. These pathway databases link metabolites to genes, to proteins, to diseases, to signaling events and processes. So that's critical. That's the biological information interpretation that we're all interested in. And we'll talk about these pathway databases and interpretation more tomorrow when we talk about how to go from these lists to biological interpretation. They have a number of tools, interfaces to align to visualize, map genes and metabolites. Many of them cover multiple species. Which is both beneficial but sometimes confusing for some people. If I say keg, it's something that everyone knows or uses and it allows you to visualize pathways. You can see the EC numbers, enzyme classification codes. You can click on things and it's going to point to brief summaries of compounds and their name. About 10 to 15 data fields per compound. So it's enough to get you at least partially familiar with what's there. Right now keg has about 17,000 compounds in it. It also has nine, almost 10,000 drugs. The drugs, they sort of double, well quadruple count. So every drug, it has a salt form. So aspirin with sodium, aspirin with potassium, aspirin with other types of salts. Each of those is considered a different drug. Which as I say is sort of a cheap way of giving lots of compounds. Very extensive glycan database. Very unique, hardly used. There's over 400 pathways in keg. So that's quite extensive. But remember that covers pathways for many different classes of organisms. So for humans, there's only about 80 pathways. And other mammals, about 80 pathways. And you get into microbes, perhaps 150, 200, you get into plants. You get quite a few more. And so when all of those different pathways for the different kingdoms of life are added together, you get about 400 distinct pathways. Small molecule pathways, database, SmithDB. It's a different kind of database that's specifically designed around humans. Or if you want mammalian systems. But really it's about human metabolism and human pathways. Pathways, there's actually more pathways in SmithDB than in keg. It also covers pathways that aren't in keg. Drug pathways, none of those are in keg. A lot of disease pathways, none of those are in keg. And then signaling pathways, again, none of them in keg. And then there's about 10 metabolic pathways that aren't in keg and another 80 that are in keg. So it covers a component of pathways that you don't standardly get in keg. You can search and browse something that's not so easy in keg and it also allows you to do metabolite mapping. The other thing that's important is that metabolism is not about just simply a wiring diagram without context. Metabolism actually takes place in very specific organs and specific organelles and that information is not deliberately not in keg. And that's unfortunate because it's often critical to understanding metabolism. It's important to know the way that metabolism is happening in the mitochondria or the proxiesome outside the membrane, inside the membrane. Whether it's primarily the liver or the muscles in the brain. All of those things are vital to understanding both the function and the role. And so that's what's attempted to be captured in SMIPDB. So this is a pathway diagram in SMIPDB. So in a sense it's a little different than a keg diagram. You can see that proteins are green blobs. The cofactors are marked in pink. You can see various organelles, proxiesome it looks like. You can see the cell membrane. You can see the organ where this particular pathway happens to be. Structures are not just dots. They're actually drawn into the map so you can actually see the chemical structures. The wiring diagrams aren't all straight lines. If there are cycles and events to be seen cycles are drawn. Things are being transported in and out of the cell. Those are drawn in and out of the mitochondria. Those are drawn. And everything is linked. Metabolites are linked to HMDB and proteins are linked to uniprot. And there's also ways for you to highlight on the right side by clicking checkboxes. Certain metabolites so you can visualize. And then each pathway actually has a description of what's involved and what happens. Which again is something you don't get in KIG. Here's a pathway that happens in the mitochondria. You can see the mitochondria sitting inside the cell and which molecules are being used. In this case you can use HMDB. You can type in a list of metabolites that are increased or decreased, reduced. And it will produce pathways or links to the pathways that have been altered. And again you can use checkboxes to highlight things and then take screenshots or data dumps of those pathways. You can map concentrations as well to metabolites in pathways. If you've measured absolute relative concentrations you can enter that and see where things are, where there's a bottleneck, where there's increase or reduction along the pathway. The last database set I guess we'll talk about are the comprehensive metabolite databases. And we've heard about some of them already. We've heard about the human metabolism database. We've talked about KIG. We've heard about MMCD and also drug bank. So these are databases that do or have everything that all the other databases have. So they have or at least some component. So they have either NMR or MS data. They have pathway data. They support searches. They support everything that I've talked about in these other tools. Now some of them have everything. Some of them are missing a few things. Another set of metabolite databases that again most people may not be aware of are the human metabolism databases about human or mammalian metabolism. There's a yeast metabolism database which is about yeast metabolites and the E. coli metabolism database which is about E. coli. Each of these databases has about 2,000 metabolites and has concentration data of intracellular and extracellular for these cells grown on different media. It has very extensive pathways, connections and other pieces of information. These databases correct a lot of errors that are in the psych databases which have many compounds that are not found in yeast or E. coli. In terms of the HMDB, the human metabolism database, we've talked about this before, there's about 40,000 metabolites, both endogenous and exogenous. At 120, we know of being bacterial metabolites. There's probably many more, but it's hard to tell because many bacterial metabolites look just identical to our eyes. Bacteria produce alanine, we produce alanine. Bacteria produce glucose, we consume and produce glucose. So we can't tell. But the 120 unique bacterial metabolites, things like trimethylamine and HPHPA, other compounds are unique and only produced by microflora. Database has lots of concentration data, both on the normal and abnormal, absolute concentrations. Data on hundreds of diseases, thousands of NMR spectra, MS spectra and GCMS spectra. You can search the database using sequences, you can search the database using spectra, you can search the database using text. You can search by pathways, you can search by structure, by drawing structures. And you can download, essentially, all of the data. It grew out of a project that was launched about seven, eight years ago at the University of Alberta. Some people were involved, particularly Jeff at the very beginning, I think, at the very beginning of the project. It was a very small-scale version of the human genome project. Human genome project was financed for $3.4 billion. Human metabolism project was financed for $7.5 million. Human genome project had 15 years to complete itself. We were given three years to complete ours. Anyways, we didn't want to start from scratch. There's a lot of data that was out there. So part of the effort was to start compiling the data from what we could find in the literature. So we developed a lot of text mining tools. We also do our own experiments and confirm and validate that. All of the data that we've done in the human metabolism project is freely available. That includes the human metabolism database, drug bank, food DB, food components, toxin, toxin target database, or T3DB, and then YMDB and ECMDB and a few others. We also were mandated to develop a lot of technology and software tools. Many of you please will be using in the next few days to help with the field of metabolomics. You guys have seen this slide before, but this, as I say, these represent the information that we've compiled through that human metabolism project, both through experimental work and literature mining and collecting data from many, many sources. So these things are housed in these different databases. The most popular database actually is Drug Bank. It gets almost twice as many hits, like seven million hits a year. And the reason why it's so popular is it was the first database to link drugs to the drug targets. And so all of the major pharma industry used Drug Bank because this is how a lot of them are using it to find new drug targets and to repurpose drugs. All the biotech industry was using this. And a lot of pharmacists and medical professionals and others make use of it. It's not a really large database in the sense that it's only covering about 1,500 compounds. But the linkage to drugs and drug targets is what people find most useful. So Drug Bank was the first database we made, and then we moved to applying the same concepts to human metabolism. So one of the features about the human metabolism database is it links all metabolites to their enzymes and to their targets and to their receptors and to their transporters. So it's not just a list of compounds, like Pubcam. It's actually a list of biology, reaction, pathways, mechanisms, linking metabolites to genes, to proteins, to function. Same thing was done with T3DB, which is a toxin target database. So that's to deal with herbicides, the pesticides, and other sides that we all end up eating inadvertently, or that end up in our blood, just from being exposed to industrial pollutants. It's a small database of about 3,000 compounds. There are other databases that are much, much larger in terms of toxins, and all of those are compounds that have never left the lab. So that's, again, an issue. It seems to plague metabolomics. Yes, you can find a goldmine of data of compounds, but they're mostly irrelevant because they have never left a synthetic lab, and therefore they can never be in any organism. The last one is FoodDB, and it's sort of marked in gray because it hasn't been publicly released, but it's a very extensive database on food components, food additives, but also some of their health effects and their functions and their origins in different plant and food species. HMDB contains a lot of information, as I said. It includes pathways, it includes spectral information, it includes details on structures and descriptions of what the compounds are, where they're found, what their concentrations are, what's normal and abnormal, their disease associations and so on. You can browse it. There's lots and lots of data fields, so they're really more like encyclopedic databases. We've gone through some of the spectral searching tools with HMDB. You can try them in the next exercises, perhaps. Pathways, so everything in HMDB that's linked to SMIPDB is in there, so you can view and search by metabolites and pathways. You can look through different biofluids like urine or cerebral spinal fluid or plasma and serum, or about 15 different biofluids. And it's being used quite a bit by clinical chemists because the data is somewhat unique. Drugbank, as I said, is another resource and it was actually developed before HMDB, partly as a trial run. This structure is very similar in terms of offerings and the type of information that it has. Some of it's more specialized to what drug researchers and medicinal chemists want to know as opposed to metabolomics researchers. As a way of browsing, you can search by chemicals the same way you can in HMDB, so you can draw structures, search by sequences the same way you can do in HMDB. You can also extract data through what we call the data extractor, so it's a MySQL query system, but it's user-friendly. You don't have to know MySQL. This is a summary slide just to compare what you might find between these different types of databases. There are spectral databases, there are chemical compound databases, there are pathway databases, and there are comprehensive databases. What I've marked in red are things that you look for like nomenclatures, synonyms, links and references, presence of spectra, presence of pathways, presence of structures, presence of descriptions, chemical properties, and physiological data. Some of the databases have some of this data, some of it have a little bit of it, some more than others, some have partial data, which I've marked in yellow, so it varies. Depending on your applications and your specialization and interest, some of these databases are going to be more useful than others. There are strengths and weaknesses in each of them. I've certainly been plugging the HMDB, but it's not a database that'll cover plants, and it's not a database that covers yeast or E. coli or fruit flies. Keg does cover that. It is a multi-species database, but it doesn't have the mass back, it doesn't have the NMR, it doesn't have descriptions. Without that kind of context, it makes it a little limited for metabolomics. In the future, I think we'd like to see databases that are perhaps as diverse as Keg, but as comprehensive as HMDB. That's going to take a long time, but it's slowly happening. We'll see where it takes us. What we're going to do now is for the next hour or until you peter out until nine o'clock. We have some exercises, and Michelle brought that up. We're going to have you look at you can either continue working with economics, some of you might find that compelling or interesting, or want to refine that. There's an exercise handout that was given to you in the book. There's also a tutorial that we have. And some of you are just listering along. You can try and do all three. There are other options as well, to look at Metabo Analyst. The key thing is that the reason why you have your computers here and why you're connected to the web, all of these tools require that. So I would really encourage you to try and do things. Simply listening to me maybe gives you some context, but all you need to learn is to by doing things. So we've charged you up with drinks and coffee and fruit if you need more to keep awake. Use that. But hopefully you can spend the next hour and part of the evening tonight to see if you can finish out those things. And Jeff and I will be here for a bit to help. I'm pretty much worn out from this long day.


Pay me. What's up, my name is Gixam Saville here. Welcome to another video, guys. So in this video, we are going to talk about PaySafe, which is one of the online payment systems that we cover here on the channel when they announced that they were gonna go public through Spark. So ever since we talked about it, the company had completed the merger and changed the ticket symbol to which is right now PSFE. So that's the new ticket symbol for PaySafe. So they're now trading publicly. So I wanna kind of give you guys some context here and some predictions moving forward. But before we get started, guys, if you're new to the channel, we talk about how to earn money, how to save money, how to invest and build wealth. So there's something that interests you. Go ahead and hit that subscribe button and the notification bell so you don't miss out on new content. So guys, we covered PaySafe here when they made the announcement that they were gonna go public through Spark. So we did a detailed video there where I kind of break down what the company does and where they started and where they are today. And at that point, I said this is gonna be a good long-term buy. But since then, we've seen the market here in 2021 kind of go up and down. And right now the pullback in the market is not only impacting other companies. PaySafe is also a victim of that pullback. So again, with every pullback means there's an opportunity for you to buy long-term. You don't wanna be the guy that's chasing that you're buying at the top and then when the market pulls back, you don't have any money. So if you wanna be the person that invests long-term, the goal is always to find every dip and buy it. You don't have to wait. Just every time there's a dip, you buy. And so right now PaySafe is really sounding the alarm to buy. I personally, I'm holding PaySafe and this is for long-term. And again, the great opportunity to continue to add to my position where PaySafe is currently at. Let's take a look at it real quick. So right now PaySafe is currently trading at under $14. So $13.87, which to me, I think that is a great place to buy. So far I've seen some analyst projections are looking about $25 price point to start with. So if that's the case, at $13.86, that's just almost doubling your investment, just going straight into this particular stack. So before we actually really jumped in, let's go back into PaySafe and just gonna talk a little bit about the company. So PaySafe to me has some qualities that a lot of people don't really cover right here on YouTube. So I want us to really take an in-depth. So if we go here, we'll look at what they offer as far as they do online cash payments, they do digital wallets, they also integrated payments. So you can actually integrate them with your own particular software. You can do online payments. So we're talking about e-comments, like Shopify and all the other platforms that you can actually integrate online payments. But the one area where PaySafe to me is really strong is the fact that they are number one in iGaming eCash network. So they lead that space. Again, we're talking about a lot of people that play games online, Fortnite, Twitch, Spotify, all those different platforms. People use PaySafe for payments to go in and out, which is excellent for them. And then they are number two in global store value digital wallet. So again, positioning themselves with companies that do in the space of online betting like DraftKings and stuff like that. So you will see them come in really strong. And there's no one here called Bet365. So all those other companies are potential places where PaySafe would continue to do business. And as more and more states open up to online gambling or online betting, you would see PaySafe continue to spread into that. And then they are number four in independent matching acquirer in the United States. So we're talking about Itoro, Norwegian Cruise, and then a bunch of other companies that they do partner with. So I mean, just being ranking top five in a lot of these categories already is setting them up to be a great company or a great potential for returns. And then the other thing I wanna mention here is that just from a global volume processing standpoint, they did over $92 billion in payment volume in 2020. That is crazy. So you did, and they're looking for that number to actually continue to grow as they continue to bring in more partners and continue to expand their business. So those are some of the key things that when I look at the company, I see a good future with this company. And again, the company's been around for a long time since our 1996. So again, that just show you some history where there's continuity in people and that trust. So what do I think about PaySafe? I think definitely PaySafe is a good company, not for a quick return, but for a long term. When you have to be patient with this one, buy it and put it in your long-term portfolio. For me, I have it in my kids portfolio and so I know they don't need the money anytime soon. They're only three and four. So by the time they need the money, it would have grown to a substantial amount. So buy and just let it go. But if you bought in, even when the stock was trading at about, let's say $15 or so, let's just go back here a month. So it was over $16. If you bought here, my advice to you is just to average your way into the stock. Take advantage of these dips and average your way into the stock. And really, what we call dollar-cost average and bring your average down to maybe 14 or 15 if that's the case for you. So that would help you down the road so that when this company bounces back and then you're sitting pretty good. Again, that price tag of $25 is out there. A lot of analysts are looking at that $25 price point. Tag at price. And again, you have to hear, I'm not just looking for that quick win. I'm looking long-term. So I'm gonna hold it and just ride it and see where it goes. So let me know again in the comment section what you guys think about this. Did you buy PaySafe before the merger? And again, like I said, the merger just happened. I think that was March 38th. And then it took a symbol, they switched to PSFE. Did you buy it before then? If yes, are you averaging in or did you kind of get out and then they're looking to get in? If you're not in the stock, I'll say this is definitely a good place to start looking. And this one is a long-term buy. It's not something to just look for a quick win. I'm not swing trading this one, I'm holding long-term. I usually would tell you guys when I'm swing trading, in this case, I'm just buying a whole and let it ride. So let me know again in the comment section what you guys think. If you're new to the channel, we talk about how to earn money, how to save money, how to invest and build wealth. So if there's something that interests you, go ahead and hit the subscribe button and the notification bell too. Don't miss out on new content. Also, if you are looking to get started with investing, guys, Weibo has their promotion where if you sign up and deposit $100, you get some free stocks, links in the description. And as always, guys, do your due diligence.