All right, so you've made it to the the last lecture So they're following this style of everything so for the so far They'll be kind of an introductory lecture for the the last module and then there'll be a brief intro to the hands-on tutorial And then we'll go through the last hands-on tutorial, which I think compared to the last one you'll find quite lightweight So it doesn't have a lot of our multiple optional scripts being around and so forth Okay, so this module is isoform discovery and alternative expression and the kind of the main goal is to provide a working example of Sort of a a splicing version of how you would run the cufflinks pipeline So in terms of specific objectives for this mod module really the main one the main technical one is to explore the use of cufflinks in This what they call reference annotation based transcript assembly mode and de novo assembly mode So even though they're calling this de novo assembly both of these modes do require a reference genome So there's kind of a lot of terminology here in this word de novo gets used a lot a lot of people when they hear de novo They think really of de novo de novo assembly and that can be totally reference free And there is a whole area of RNA seek tool development and algorithm development that is really focused on building assemblers that can take raw RNA seek reads and Assemble them into transcripts without any notion of what the reference looks like at all so it's just based on the similarity between reads and inferring that that these reads would be part of a Contag and building these contiguous sequences and then later you would try to map them back to reference sequences Or just annotate them directly as transcripts And that's quite a challenging problem in a lot of ways. It's more difficult than Aligning reads against a high-quality reference. Yes, Francis Right, so yeah, Jared, I guess gave a lecture on the topic of On de novo assembly, yeah, okay so in the in the Table that I referenced for RNA seek. I've tried to give some recommendations for de novo transcriptome assemblers That are kind of specific to RNA seek. Of course, they sort of mean on a lot of the methods And algorithms of the sort of assembly field as a whole But they've kind of been tested on RNA seek data There are some problems with assembling RNA seek data that are kind of particular to RNA seek So there's this issue with a lot of genes having multiple isoforms Which is what we're going to talk about in a bit and that makes assembly quite difficult So you know there's a lot of ambiguity when you have all of these isoforms being expressed from one locus and they are different and they have Functionally significant differences, but in terms of sequence those differences can be fairly subtle So they may share most of their sequence but have you know one exon swapped out with a different exon or a slightly different exon boundary And that complicates the The goal of assembling a transcriptome from scratch So unfortunately we don't have time to cover that That area, but so the type of assembly that we're going to do Is with cuff links and it uses the reference So we're going to start with our reads that have been aligned against a reference genome And then using that information we're going to try to assemble transcript structures from the reads that have been aligned to the genome And there's sort of two modes One is where we're in addition to having the reference genome as a guide We're going to have a reference transcriptome as a as a guide So that's the gtf file that we got from ensemble We could have gotten it from other sources of Transcript annotation, and then the final mode is where we don't even supply that Reference transcriptome We're just going to go straight from the BAM to What cuff links thinks the transcripts look like based on the read data that was aligned to the reference genome So just to remind ourselves where we are kind of Again looking at these Mature mRNA molecules in RNA-seq often And then this module was really concerned with the alternate ways that A transcript could go from a pre mRNA molecule with the introns in place To a mature mRNA molecule where the introns have been sliced out For many, if not most human genes, there are multiple ways that this happens for each gene to produce multiple versions multiple multiple isomers or isoforms for each gene And this is a whole field of RNA-seq and we're just going to pretty much do like an introduction to it There's a huge number of papers and tools and labs out there that are really focused on this problem And what I'm referencing here is actually a specific paper. This was published as In the archive, but I think it might actually be published somewhere else now and they provide A table that's really focused on all of the different tools and methods that have been published so far That relate to alternative expression alternative splicing analysis using RNA-seq data And then this slide is kind of just provided again as a reference. So here's some biostar posts that have really started to answer some of the the big high-level questions about how to detect alternative splicing And again by differential alternative splicing Try to understand the cufflinks and cuffdip output as it relates to alternative splicing and have a visualized alternative splicing and so forth So just to kind of give you a sense of the complexity of alternative splicing that we're talking about here alternative expression is sort of a more general term that includes alternative transcript initiation and polydenylation So if we think about a very simple transcript model So here's a region of the genome with three exons and two introns And a simple splicing path is shown where we're going to basically connect exon one to two and two to three The introns will be removed and we'll get a mature mRNA that's polydenylated with exons one two and three joined together Alternative transcript initiation is the process by which we start basically have different transcript start sites So we can have a transcript initiation happening here at exon one or happening here at exon two And that would give us two alternative isoforms that differ At their five prime ends. This one has an extra exon relative to that one alternative splicing Has several different Well-established forms. So one of them or perhaps the most famous or simple is just exon skipping So where we have two paths through some set of exons where we include exons one two three or perhaps skip exon two And then this gives us Exons that have the same beginning and end but something in the middle is different You can have alternative five prime splice sites being used. So in this case, we have Two different donor sites that are being used One that starts here and one that starts a little bit further to the right And then this can again give you a potentially quite subtle difference At the level of the final transcript where we've got basically a longer version of exon two being used The same thing Can happen at the three prime end where we get an extra piece Of sequence being used as a part of exon three. So these are different alternative acceptor sites being used Again, they give us slightly different isoforms A special form of exon skipping is the use of mutually exclusive exons Where you basically have a exon two a or exon two b that are used So you have completely different exons being used in the middle here The entire intron might be retained in some cases. Usually this is short introns that get retained And often there'll be a non-functional product, but it does happen. So we can have Regular transcript here and then one that effectively has a very long exon two that contains the intron Plus all of what was considered exon two and three in this isoform And then alternative poly denulation is where we basically have alternate three prime ends to a transcript So the transcript either ends here at one exon where it ends At a downstream exon and this gives us differences at the three prime of the transcript And this is you know, these types of patterns have been studied for a long time With various techniques that have gradually become more and more high throughput So sort of the Early technique that was used. It's very low throughput, but it's still considered a gold standard approach Just to basically sequence full length cDNAs To clone those cDNAs into vectors And then to sequence through the vector By some combination of end sequencing and primer walking to give you basically a content that represents the complete cDNA sequence And you do this many times and over time you gather what the alternative isoforms are for a particular locus That's a very slow labor intensive Not very high throughput method Slightly more high throughput method is to do targeted full orc cDNA sequencing Somehow you're not relying on sort of random selection of clones You can really Target the the region that you want to sequence and then you can really discover the the variations that happen In between the areas you targeted, but it doesn't really help you with discovering alternate poly denulation and transcription start sites Uh For a long time we were generating and sequencing large est libraries Where we would just randomly create lots and lots of cDNA clones and then sequence them from their ends And we would learn about how exons were included or skipped near the ends of cDNAs But we didn't generally get a full picture Of what the transcript looked like. So it's kind of analogous to rnac, but the reads would generally be much longer Again, it's not very high throughput And then probably about 10 years ago the really more high throughput methods For studying alternative expression came out and these included things like cage sage and gis that use different combinations of basically sequencing The ends of transcripts in a more high throughput fashion by sequencing little tags that are derived from the 5 prime or 3 prime Or both the 5 prime and the 3 prime end of transcripts And then next end sequencing started and that really revolutionized the field of exploring transcript diversity starting with 454 And selexa where we're able to suddenly basically shotgun sequence a transcriptome with Millions and millions of short reads at a relatively low cost And this has just started to really increase in the last few years where we now Get large numbers of reads that are in the 1 to 200 range at a relatively cheap cost. Yes, frenzies Yeah, like pac bio or nanopore I Yeah, so there's a huge advantage So remember we talked about this yesterday how how much inference goes into trying to understand what a full length isoporn looks like Based on these paired 100 base pair reads. You really there's a lot of guesswork there And it's really suboptimal when you're trying to understand what the full length Transcripts look like and it's just never going to be as accurate as if you could actually sequence each cda from end to end So you can already do this to some degree on the pac bio It's not nearly as high throughput as rnac on the illumina platform, but you can get pretty good quality data So I guess that right now it's kind of like a more efficient way of doing Maybe the full length cda sequencing or the full orc cda sequencing and I have seen a few groups doing that It doesn't seem to have really become wildly popular, but I've seen some really impressive results from that approach um And there's also the you know the read lengths are getting longer on the illumina platform that helps a bit It's a bit hard to imagine right now that they will go from You know at the rate that they're increasing so we're you know doing 125 base pair reads on the x10 platform and You know on the my seek you can do 200 250 maybe So and it hasn't really been Increasing the length has not been on the same kind of trajectory that the throughput has been where we keep getting more and more Through but the length seems like it's more kind of stabilized. It's difficult to Increase the length on the illumina platform so maybe something like nanopore where You're literally feeding molecules through a pore and reading the sequence off as it goes through that pore if they can make that cheap and robust And relatively at least medium to high throughput. It could really again revolutionize um transcriptomics with respect to alternative splice uh detection and abundance estimation And that would be that would be great Yeah Yeah Yeah, so that's a whole other Pandora's box um that a lot of Alternative splicing methods kind of gloss over uh, so a lot of them start with the assumption that You know a locus is can be quite big and the introns can be quite large But they are coming from a single chromosome for the most part And tran analysis that we'll be able to accommodate the concept of transplacing Uh needs to be tweaked to to accommodate that idea and it's similar Uh in a lot of ways analytically to the fusion discovery type analysis, so a lot of people will kind of Conceptually lump some parts of alternative expression analysis in with fusion detection Because you're looking for these novel forms of transcripts in some cases transcripts that are quite bizarre And they can even be chimeric in the sense of incorporating content from different chromosomes either because of A somatic event like a rearrangement where two chromosomes have actually come together Or because of transplacing where the transcriptional machinery is somehow jumping from one template to another By as yet not super established Uh methods Yeah circular RNAs another The transcriptome is a bottomless pit of complexity and weirdness. Uh, yeah circular RNAs Yeah As I think that these areas are like quite Still quite undeveloped a novel in terms of analysis. So people that are working in these areas are really You know doing pretty cutting-edge stuff to think about these weird events There's also a lot of classes of RNA that have been, you know understudied So there's been like a bit of a a gold rush of people looking at different weird sub types of small RNAs at circular RNAs that Transplacing RNA editing all of these kind of interesting aspects of RNA biology that are, you know Generally the sort of forgotten stepchild compared to just mRNA Uh transcripts that make a protein But they're you know, they're all super interesting of course Any other questions? All right. I think this might be the almost the last slide. Um So back to cufflinks so cufflinks tries to Automate some analysis related to alternative expression It that doesn't cover all of the the interesting stuff that we just talked about but covers sort of a targeted slice of it and there's sort of three main assays if you will that are done by cufflinks and Those three are basically looking at Uh transcription start sites So basically what happens is after we run cufflinks and we get some notion of what all the transcripts look like um Cufflinks comes along and it tries to break the transcripts into categories that Kind of line up with certain concepts of alternative expression So if it finds for a particular gene locus all of the transcripts that for example have the same transcript start site and And then contrast them with those that have a different transcript start site. So for this A simple example where we have three transcripts We have two groups of transcripts that differ by their transcript start site We've got the tss one group and the tss two group Uh that either start at this exon or start at that exon So it's going to do a test that basically instead of looking at the level of individual transcripts It's going to group the transcripts and it's going to say how much Expression is using this promoter or this transcript start site versus how much expression Is using this transcript start site. So that that's the first category The second category is I don't know exactly the order, but there's also alternate use of exons and there's Ultimate cds So this the coloring Uh b is like that. So the second one is exon skipping. So it's basically looking at the inclusion or exclusion of this exon So we have one isoform here that doesn't include exon two And then the third category is do they differ in terms of the part of the transcript that actually is translated to protein? so Taking all the transcripts and breaking them down into unique orfs And then summarizing at the level of orf instead of at the level of transcripts with respect to where the exons start and stop So you can see that sort of fat bars here. This is a convention In depicting transcripts as cartoons that you have this thin part That is the five prime utr And then it gets fat here. So that's where the open reading frame starts And then it continues through this exon And then goes to the third exon and then ends and then you have the three prime utr So the the cds portion is this fat part here So we can see we have one cdf one transcript that has this long Coding sequence and then these other two transcripts have this shorter coding sequence So one of the another one of the tests that it's going to do is basically comparing All of the usage of this long cds to all of the usage of The short cds and grouping whatever transcripts need to be grouped together to do that comparison So the conflicts will do the kind of Panelized comparison with all of the categories. Yeah, so it automatically does those three the sort of uh Actually, these are Okay, they've been I guess reordered here relative to up there But it does like a slicing level comparison where it's really focusing on the usage of exons It does the promoter level comparison Where it's looking at the different transcript start size usages And then it does the coding sequence comparison where it's looking at the usage of different Coding sequences and each of those tests get kind of divided into these three arms You get three output files and then the differential expression results in each of those output files corresponds to each of these Three ways of thinking about alternative expression the sort of slicing analysis the promoter analysis and the cds analysis And these are the way the files will be named when they come out of The cufflinks run Yeah, so we're going to talk more about those modes right now It's doing it Potentially without use of a reference transcript dome But all of it still relies on the reference genome So we still have our reads being aligned to the human reference genome sequence in this case And then it's just how we're using or not using the gtf file that contains sort of transcript models So In this module, we're going to learn how to run cufflinks in the The reference only mode is what we've already done. So you guys that's basically what you've done up to this point You supplied a gtf and you basically told cufflinks Tell me about the abundance of these transcripts as they're defined in the gtf by ensemble But now we're going to sort of move on from that into the reference guided and the de novo mode And we're going to learn to use cuff merged combined transcriptomes for multiple cufflinks runs and compare assembled transcripts to known transcripts That's going to help us get a sense of what transcripts are novel versus what transcripts line up with known transcripts And then we're going to perform the differential slicing analysis with cuff diff Um We're not really going to dig too deeply into the output of those we'll sort of do like a brief introduction to the output of those analyses And then we're also going to step back and actually look at the Output from the top hat alignment itself So even without cufflinks, you already get some fairly rich Raw material to do splicing analysis right from top hat. So one of the things top hat does after Uh All of the reads have been aligned to the reference genome Is it finds all of the reads that span across x on x on boundaries? And it summarizes those specifically in a separate file which is called the junctions dot bed file And what that file does is give you one line for every x on x on connection That was observed in your rna seek data and then it gives you the counts So the number of reads that corresponded to each x on x on connection And you can use that raw count information to build quite a rich view of the splicing that's going on in the transcriptome that you sequenced And it's a fairly straightforward file to start with that doesn't involve even running cufflinks So we're going to visualize that junction counts file in igv And we're also going to look at some of the the transcripts that are assembled by igv as well Okay, so the ref guided in de novo mode. So we've already run cufflinks in reference only mode Uh, but now we want to sort of have more potential to identify novel genes and novel isoforms of known genes And this is if you ask the the cufflinks authors, these are kind of the ways that they actually recommend that you run a cufflinks They so they are you know quite keen on this sort of uh hypothesis or let more hypothesis free way of interrogating the data Um, so I think if you look in some of their publications, they'll will just skip right to using the the reference guided or de novo mode So in the reference guided mode, we're going to use a known transcriptome, but it's going to be used just to guide cufflinks Uh, and cufflinks will still try to predict novel isoforms on top of that Um in the de novo mode, we're not even going to give cufflinks the gtf file at all So it's just going to try to assemble the transcripts completely from scratch And the way you control these modes is all based on the use of basically what options you specify when you run cufflinks in the first place Uh, and they all confusingly are named Uh g uh, and they have but they generally have a longer version as well So top hat just to remind ourselves top hat has this dash g option And that isn't is not what we're talking about here But just to try to avoid some of the confusion When we ran top hat we supplied a gtf file and it used that to build a transcriptome reference sequence to align reads against to help The alignment process and then you read the lines to one of those transcripts were then kind of converted back to Reference genome coordinates so that they could be displayed along with the reads that line directly to the reference genome Top hat also has a small g option Which is used to specify the maximum number of multi mappings for a single read and again that is completely unrelated to what we're talking about in terms of alternative splicing It just happens to be a g But cufflinks has this big g option and that's what we used so far So we supplied cufflinks with a gtf file and we told cufflinks These are the transcripts that we expect in this in the human species So these are the things that we want you to try to summarize in terms of their abundance Cufflinks also has this small g option which you also use to supply gtf file But when you use the small g you're telling cufflinks to use this gtf to guide the assembly But to not limit itself to that to those transcripts So it's going to try to predict novel transcripts from the data If you run cufflinks without either the big g or the little g option So you don't give it a gtf file at all. This is what we're calling the denobo mode So there's we're not helping it With any kind of model of the transcript dome that was pre-existing We're just going to rely on the raw data and try to predict transcripts And then cufflinks has Also A way of being supplied a gtf file But for some reason they decided to just make that A file path that you put at the end of the command without actually specifying An option parameter It just it's detected by the order in which you place it in the command So I think this is like kind of like a bad This whole system is kind of a bad design considering these things were all created by the same group They've introduced a lot of confusion potentially Um, so I mentioned this top hat junctions bed file So this comes right out of top hat After the top hat completes its alignment, it summarizes all the reads and supports x on its junctions So for example, you might have a readout that says x on one Being connected to x on two there were five reads that supported that connection So five instances where a part of a read aligned to x on one and then the alignment continued on into x on two And then you might have nine reads that where the alignment started in x on one and continued into x on three So when you see this right away, you have evidence potentially for x on two being skipped that there's A transcript being expressed that is not including x on two Because you can't skip from x on one to three And include two since splicing generally goes in a linear direction So this potentially tells you about alternative splicing that's going on just by looking at the accounts for these junctions It sort of summarizes at the level of unique x on action junctions and all and They observe recounts for each and this is in bed format and that's relatively simple formats You get basically one line for each of these junctions And then in the fifth column is basically the number of reads that support each x on x on connection We talked about it a little bit Yes, this is raw. So An x on skip that's forward I would say that Three prime n bias would limit your ability to see the connection of xons at the five prime end of transcripts x on one Yeah, so determining the where transcription starts Is definitely an application that's quite affected by three prime end bias because like you say If you don't have coverage at the five prime end Because of a problem with your library then you you're missing that information And if you have two libraries and one of them has really bad five prime end bias and the other one doesn't It might look like there's alternate transcript initiation going on just like you're describing But really it's just that you're seeing a difference in the amount of Three prime end bias between the two libraries. So it's something that's you know You really have to watch out for When you're comparing two libraries in the context of alternative splicing you Ideally you want the coverage to not be biased towards the five prime or three prime end and you want the The profile of coverage from the beginning to the ends of transcripts to be quite consistent From one condition to the next condition that you want to compare And there are some tools that will give you the sort of a print out of that what that profile looks like And you'll often see that it has kind of like a horseback shape where the Transcript length is displayed along the x-axis. So the five prime end is at one end and the three prime is at the other end And then there'll be kind of a histogram that shows you sort of where your reads are kind of piling up and you want that to be As close as possible to a uniform distribution where you have nice even coverage from the five prime end to the three prime end of your transcripts not exactly I have to check and so I just actually had this conversation before and it's actually It's not super simple to translate this mentally into the junk edges of the exons that you think about when you're thinking about x-on-xon connections I think that There's like an offset and a length and a Like a reference starting point and between those three sets of coordinates or sizes You can construct what the edges of the exons are but it's a little bit of math to work that out It's that doesn't immediately come out of this these numbers So when you look at it, it doesn't immediately make sense what you're looking at There's there's a little bit of math that you need to do To basically get from this form to the left edge of the first axon Or the work sorry the right edge of the first axon and the left edge of the second axon And that so I I did this I worked this out like several years ago when I wrote A curl module that basically converted this file into something that was More like what you that to me that's the more intuitive way to represent the junction But you know if we sat down We could figure it out again. It's not it's not that crazy But this is something that igv has built into it that it it can take that that file And convert it into these Little arcs that do correspond to the edges of each axon. So This is some rna seek data where we've taken the junction stop bed file and we've loaded it into igv And we're looking at what top hat found in terms of reads that aligned across junctions And you can see that they line up quite nicely with the expected junctions of a gene that happens to be in this region So you've got for example where these expand from here to here, which appears to line up quite nicely with the The edges of these two axons here. And in this case, we've got Apparently nice coverage. So we're seeing representation for junctions that are at the five prime ends The middle and the three prime ends. We've basically reconstructed all of the connections that are known for this transcript and there might be There might be evidence. So there's actually evidence for a novel axon relative to this gene model that's being shown By the observations of these junctions. Can you can you see what it is? Small one on the top how far from the right or from the left In the top row This guy? Yeah, you're right So this junction Are apparently there were some reads that span from here to here Which appears to line up with the edge of this axon But then it seems to be lining up with an edge here and there's no Exxon annotated there And then we have another junction down here that seems to be coming from the edge of this known axon That's also coming over to around the same place Probably the other end of this axon that is apparently novel axon that wasn't known to RefSeq and things where these it's either RefSeq or UCSE that these transcript models are coming from So there you go sort of like a A quick Example of how you can identify Evidence for novel isoforms. So you don't know what the whole isoform looks like But you have some evidence but that there is some isoform That involves known exons from this gene But also seems to be using a novel axon that wasn't previously annotated Yeah, it's this This junctions.bed file that is coming jet automatically produced when you run top hat And I think it's just a little module that is part of the top hat A liner that runs at the end that basically goes through the BAM file and generates this this output from it Yeah, there just counts. So, yeah You don't really have a sense of how believable each of these junctions should be It will limit them to Expected for the most part you get Actual splice sites. So the junctions are usually checked to make sure that they correspond to canonical junctions Or or you can annotate them yourself with respect to canonical Splicings patterns so that you that would increase confidence that it might actually be real Is if it actually uses something that looks like a donor site or an acceptor site Which is fairly well established for human obviously like with other aspects of RNA c for other types of high throughput sequencing more More reads that support the same event gives you more confidence One read is a bit hard to get too excited about one read That's just something but if you have a hundred of them and they all kind of start and stop It's slightly different places that makes it less likely. It's an alignment artifact So there are things you can do to to make yourself more confident But it doesn't it's up this tool doesn't give you like a p-value. It doesn't take those things into account So you would have to add your own sort of notion of that and there are probably A bunch of the tools that I list in the in the resources table that do some of those take some of those concepts into account Yeah, that's a great idea. So we're gonna yeah We're gonna do that in a few minutes and you can see how the alignments of the actual reads Corresponds with the summary of the junction counts that top hat produce Okay, so cuff merge. We're gonna run cuff merge And what cuff merge does is basically combines transcripts that were predicted from multiple runs of cuff links into one view of the transcriptome So this is something that you would do Say you have In our case, we have three Replicates for brain and three replicates for the universal human reference RNA And you're going to run cuff links in this de novo mode in each of them You're going to try to predict what transcripts are present But because of differences in coverage or just random sampling you don't get exactly the same transcripts being predicted in each of those six runs Cuff merge is a way of basically unifying All of your runs back to like a single view of the transcriptome So you can come up with a single Set of transcripts that you're then going to get abundance estimates for It's basically like taking the union of transcripts that were observed across multiple runs of of cuff links And you can also use cuff merge to then compare those transcripts back to a known gtf file So when we're running Cuff links in the de novo mode It's not going to know anything about the structure of human transcripts It's going to try to predict those structures based on the data. But when we get to the end We may want to compare the transcripts that were predicted back to whatever our notion of known transcripts is to get some sense of Which transcripts line up with known transcripts we already knew about which looked like they might be novel So this is sort of a workflow that would be really applicable if you have A species that where the transcriptome isn't very well annotated. You have some preliminary sense of transcripts for that species But you're really you're using cuff links because you want to discover novel transcripts So you would run it in the donut de novo mode and then you would use cuff merge to compare The result back to whatever you whatever your current notion of the transcriptome was and it'll allow you to sort of Bend your transcripts into okay. This one really matches a known transcript and these ones Don't match any known transcripts. So that's going to be your bin of things that you might attempt to validate as potentially new transcripts in your species So we're going to do a comparison of merge gtfs from each cuff links mode and look for some differences in what Transcripts are predicted in each of the modes Which is what's being depicted here And here so we're seeing for example In the reference only mode We're getting this one transcript being summarized In the de novo mode.  So Cuff links is predicting that there's a transcript that starts here instead of over here where the rest of these transcripts start It's also predicting that they're the transcript that is retaining this little intron here Instead of having that they intron that the other transcripts have And then there's probably other little self differences that are outside the zoomed in view right here Okay, so the last thing before we move into the last tutorial Is this question that we've sort of addressed already which is what what happens when you get back to your own lab And none of this stuff works Or there's some problem Or they update the tool or the path URLs are all busted Let me the URLs like break like it's almost like bi-weekly that we have to update that wiki Because things are just constantly being shifted around and Um So inevitably you're going to have you know questions Even if you just try to run the cuff links like the tuxedo suite exactly as we've described it here You know, you're probably going to encounter questions. It's pretty complicated when you start really digging into all of the different options and parameters So, you know feel free to post questions Um on bio stars and we'll probably see them there That's sort of the best way if that if your question isn't already asked on bio stars that it is then then go ahead and ask it there If no one replies, you know, you can try to ping us And your email may possibly not get buried in a massive pile of emails and we might remind us to go take a look Your question but there's also quite a lot of resources that have already been created so the The cuff links authors created this troubleshooting table, which I find it's it's useful It's a little bit comical that they Some I think there's like four or five entries in it as if that was the only thing that could possibly go wrong It makes it seem like there's really just these three things that go wrong So I guess you could check to see whether it was one of these three things And you might get lucky If not, you know, it's uh Usually you're not the first person that's encountered whatever problem is happening So you'll usually if you google you'll be able to find someone who's had the same bizarre unexplainable error message from some log file And if if you are the first person then I guess you'll just have to work your way through it But there is this troubleshooting table and then as I said in the publication that's going to accompany this tutorial In which I've already linked to On the citation page for the the wiki there's quite a much more extensive table of questions and answers That covers some additional Questions and commonly encountered problems and sort of blockers that we hear about So maybe uh, you'll find the answer there as well Okay


Hey, everybody, welcome. As the talk says, this is going to be about kind of how to build a data-driven culture view. It's a lot of material and not a lot of time, so I'm going to go pretty quickly. And I hope to leave enough time at questions so we can dig into the parts that people do care about. Just to give you a little bit of history on where this is coming from, and all of these slides are available right now on datapractices.org. We have a series of open courseware that's up there. And this is one of those things that we're drawing from. A little bit about the history of where this came from. About a year ago, November now, we put a whole bunch of folks that we felt were kind of leaders or visionaries across the data ecosystem, whether that meant the academic world of semantics, data viz, data journalism, open source. We kind of threw them all in a room and shook it just to see what would happen. And we noticed that a lot of the conversations that were happening were very similar to what we saw 10, 15, or more years ago in the software development ecosystem. And we decided that, hey, let's skip that whole waterfall development awfulness that we had in software development, and let's go straight to Agile for data. And so we wanted to come up with some values and principles that we felt best enumerated what was important about modern data teamwork as we saw it. And so that was kind of the first start. And from there, that community has grown and also helped to build all of the open coursework you can see that was relaunched in March on datapractices.org. We have four kind of core values and 12 principles, had almost 40 authors, and we have a ton of signatories already, so definitely some wonderful momentum that we really appreciate. And I'll skip over some of this, just to give you an idea of some of the people involved if you look on the left here, some of the folks that were in the room. There was some really good brain trust, and it was really nice to kind of hear some of their frustrations with the data ecosystem, especially as it's taking off in the realm of the buzzword. But a lot of times it's not keeping up in terms of the actual ability to deliver. There's this really big gap between what people want and what people are able to get from their workforce. And a lot of times it's just miscommunication. So we're hoping that by providing some of this course where and continuing to grow that community that we can kind of elevate the level of data literacy. So here, taking a look at some of the things that we're gonna cover, mostly I'm interested in kind of framing this problem, and then looking at some of the things that are good indicators, or perhaps other things that are good places where you can start. What's the low hanging fruit in terms of I am part of an organization, whether that's a nonprofit, a governmental organization, or whether it's the company that I work for, my day job. How can we make data a first class citizen? Now, I will say that these blue slides, this is designed as an hour and a half workshop for people to actually dive into and I've got like 15 minutes to go through it. So we're gonna skip the exercise portion of this part but I did want to let you know that it is there. All of these slides, as I said, up on datapractices.org and if you look at the first slide, if you hit that S key when you're looking at the first slide, all of the speaker notes and everything that you need to kind of dig into this yourself, if you want to do something like that is all available to you. So, framing the problem. So, I wanted to kind of look at a few different tangible examples of how people are framing the problems in the data ecosystem. I'm gonna look at two books and then just my experience at data.world, I felt there was some really valuable lessons that we've learned over the last few years that I wanted to share that I was intimately involved with. So, first I'll start with the definition of what is data-driven mean. So you can see kind of the dictionary sort of definition but if there's only one thing that you walk away from this talk with, I want it to be that being data-driven is about the people. It's not about the technology. I mean, if you look at the research that's being done out in the ecosystem right now, whether it's Gartner or Forrester or academic researchers or whatever, they'll tell you that the Chief Data Officer, he's the new kid on the block in the C-suite, right? A lot of times they are underutilized, under-resourced and unappreciated. The research is saying that 50 to 60% of those Chief Data Officers are going to fail in their mission in the next three to five years. And the reason for that that we've seen anecdotally is that the ones that are doing well are approaching this as a people problem, an education problem, a culture problem. And the ones that are not getting the results that they're really looking for are treating this as a technology problem. What can I install? What are the tools that I need? If I buy Tableau, that's the answer, the silver bullet. There is no silver bullet. Like let's get that right at the beginning here. So if there's no other takeaway from this, it's a people problem. So there are a couple of books here and the first one here, Play Bigger. This is not about data. So the reason I selected this is there's some really good lessons about building practice. So Play Bigger was all about how do you, it's about category design. It's about creating a new business but creating it in a way that there's a new category. I am creating a category that's called data cataloging and I'm going to tell everybody that's the answer to everything and this is how you grow and explode and dominate a market by creating the market to dominate. So the interesting portions of this book and I won't dive in too deep just because we don't have enough time but I like that it starts asking questions about what are the problems you need to solve? So looking at are we data driven? If we're not, what can we do to solve that problem? Do we have underutilized data? So are we getting poor ROI? Do we have disconnected data? Have we spent the last 10 years aggregating technologies that now have stuff scattered to the four corners of the universe? And then taking those problems and looking at kind of what are the key components? So is my data hard to understand? Is it hard to access? Is it like looking at what are the things that lead to your frustrations? And then at the end of the day kind of teaching you, this book tells you how to kind of define the villain and it's an interesting way to kind of personify the thing that you're trying to defeat, the thing that you are doing battle against, whatever it is, right? Whether it's data silos, whether it's dumb data, whether it's self-serve analytics that no one knows how to self-serve, it's looking at ways to kind of do some of these things yourself. And then the second book I would recommend, highly recommend for anyone in the data space is called Winning with Data. I think this is a fantastic book and especially as it relates to this topic, I mean even the first chapter is titled, Mad Men to Math Men, The Power of the Data Driven Culture. So it's really neat because they kind of, they reframe a lot of the problems that you see in the data ecosystem in a very unique way and a way that's really easy to kind of get your mind around. It's a lot less, I know it's a lot less of that ephemeral well, you know, data silos are this or you know, some of those different things that, yeah, they exist, but they're talking about kind of four key problem areas. Talking about what they call data breadlines, data obscurity, data fragmentation and data brawls. And so going through some of those things, I like the data breadlines. This one is the one that I seem to see most often when I go into an organization. This is the idea that small data service teams exist within the organization but they have to service the data requests for everyone. And this leads to that data breadline. You can imagine the, you know, great depression breadlines where there's a round the corner and everybody's waiting with a soup bowl. It's the same thing with people waiting for their data, right, and a lot of times you may have been in line and you may have been ready, you may have had all your ducks in a row, but some executive comes along and says, I need this right now and they jump the line and now you're waiting even longer. So, you know, it's this idea that we have data experts and they will answer our questions. They are the black magic box and that becomes a problem, right? Because then everyone else is beholden to their time schedule, their workloads, et cetera.  The second one that they were framing was data obscurity. So it's similar but different. It's talking about how there are very few people within an organization that know where all the data lives, where all the bodies are buried, right? You've got maybe a couple of data engineers that know, okay, I know I have to pull these different things from these different pipelines for different people, but if you ask any one of the subject matter experts or stakeholders, they have no idea what exists beyond their own very small niche view of their data universe. And there's data fragmentation. So this is the shadow IT problem, right? This is where people got frustrated with data obscurity or data breadlines and they've decided, well, the heck with that, I'm gonna stand up and access database on a machine that's stashed under my desk and then I'm gonna use that to make millions of dollars worth of business decisions. And no one else knows where that data came from. If it's clean, if it's accurate or better yet, the hit by a bus problem, that person goes away. Now, how are those decisions being made? We have no idea. And then the idea of data brawls, I like this one because if you take some of the problems and silos and things like that, they're built within an organization, you look at, okay, two people have decided that they're going to reference our Q4 sales from last year. Now, they both came up with different numbers. They may be similar, they may be close, but they both came up with different numbers. Now, we spend more time arguing about where did this data come from? How was it generated? What kinds of transforms were done? And so now we're brawling about the data. We're not actually answering the question that we came there to address. And so those four things, I think, really do a good job of describing the problems that you see in an organization as it relates to data. Now, as I said, I wanted to kind of bring in my own data.world POV. So early on, we wrote kind of an internal position paper about the data ecosystem. And we had a bunch of main points that we looked at. We looked at community and governance and kind of the tool fatigue problem and transformation through data and all these other things. But I wanted to read one excerpt about data just to kind of kick it off here. So we believe that data can transform the world causing an unprecedented boom in efficiency and effectiveness. Data can make the world smaller by connecting people to problems and solutions. Data can radically change company cultures too. As William Gibson, speculative science fiction author, said, the future is already here. It's just not very evenly distributed. The future is already here at a handful of these companies that have transformed their industries. Airbnb, Warby Parker, Facebook, Google, Amazon were literally built from the ground up to take advantage of data. At Airbnb, their fifth employee was a data scientist. Today, they have their own data catalog where 45% of their 3,100 employees regularly engage with it. At Warby Parker, they have a similar data catalog but in this case, 80% of their employees are regularly engaged with it. Traditional Fortune 500 and Global 2000 companies are lucky if even 1% of their employees are doing the same or if they have a democratized data catalog in the first place. Most don't even measure this percentage. Ask them. These traditional companies weren't built from the ground up around data. They were built from the ground up around Adams, not Bitz, which is one of my favorite quotes from an investor named Mike Maples Jr. He's very quotable, you can look him up. The new order of the world is that Bitz will control Adams. This creates many exciting opportunities but is very scary for traditional companies. In the 2018 New Vantage Partners big data executive survey, the traditional company's fears of a highly agile data-driven competitor has jumped to 79.4% from, this is a significant drop from like 47% in 2017. So that was kind of our look at the data ecosystem and understanding that there is a large gulf between the haves and the have-nots, between the companies that have incorporated data into their DNA and those that are struggling just to understand how they can even recognize that they have data that's important. So I wanna look at the pillars of a data-driven company. What are kind of the important things that make up a good data-driven company? And so I think the first is data infrastructure. Now this is a tough one, because remember I said that this is not a tool problem, but the problem involves tools. We've spent the last 10 years or more really advancing some of these data technologies, whether they're data lakes, data warehouses, relational databases, all the way down to your spreadsheets and flat files. Like we use all of these tools on a daily basis, we just don't use them together and we don't use them well. So the idea here is looking at these tools and trying to find a way that, or identifying what data infrastructure looks like from a this is disparate, this is siloed, this is not working together well versus how do we put that together? And I think the questions that you have to ask yourselves are around data access, data context, and just making a single source of truth. And so when you look at these things, data access, this is important because historically you have had these, I'll call them the data elite, right? These are the folks that know how to write a SQL query. These are the folks that are your BI analysts and all the way on down the data science chain. And we are doing ourselves a disservice by relying so heavily on them. We need to start bringing in more of our subject matter experts. Yeah, I can be the best BI analyst in the world and I can crunch all the numbers I want, but if the numbers are off, I'm not gonna know it. I'm just gonna know that my math is right, but the subject matter expert can come in and say, yeah, Microsoft, we analyzed our gender makeup in terms of our employees. I know it's bad, but I can tell you that we are not 98% male. So it's that context that's really important. It's the ability for you to bring more people into that conversation. And so I think that's one of the first things that you can identify is, is there this gulf? Do we have the data people in a black, dark room somewhere and it's kind of this magic in and out? Or are more people involved in this conversation? Then I talk about context. This is your data dictionaries. This is understanding. If anyone has ever worked with data from the federal government, you'll know that you get a data dump from them, whether it's a spreadsheet or whatever it is, access to a database. And there are these columns, FS 37 2019 AG. Like, what is this? I have no context, I have no understanding. So a lot of times that data just gets dropped or ignored because no one has an understanding of it. And so bringing the context with you wherever you go and trying to centralize that context in a single place so that the water cooler discussions, the tribal knowledge that's gathered over time, if these things can kind of be centralized and they're just as important as the data itself. So I think that's kind of the other question. And then like I said, bringing this all together into a single source of truth. This is the data catalog movement is something that's big in the industry right now as people are looking at all of these disparate data resources and how do we bring them into a single place? That's what we like to play with, but at the same time, there's a lot of catalog vendors out there. Don't take my word for it. Moving on to data governance.  Like for a dirty term, I guess I should change that. A lot of people think about data governance as the red tape that gets in the way between me and my data. And really, if it's done well, it shouldn't be that. We should have the ability for data governance to maintain things like integrity and security, but also promote things like availability and usability in a way that is coherent across the organization. So it should establish a good framework on how to use your data, not get in the way of actually using it. And then data literacy. I think this is important. Again, this goes back to that getting more people involved in the conversation. I don't have to even be able to spell SQL, but I should at least understand there are ways that I can interact with my data and my data team and other things like that without having to be super technical. And we should provide avenues and resources to those in our organization. And you look at these big orgs like Airbnb or Warby Parker, they're great examples of my receptionist knows my Q4 sale numbers because she can then tell people, hey, this model is selling better than that model as she talks to people and interviews people or our VP of marketing can talk about, she can talk about the sales numbers of in the same way and say, look, now I can go out and tell the market this whereas one person is talking to an individual and the other person is talking to the market, it's important for you to be able to understand the underlying data and how it relates to the business. So data-driven leadership. Don't wanna dive too deep into this one, but a lot of times we will go into an organization and talk and that we're preaching to the choir when it comes to the practitioners. They get it, they're like, yeah, I know, we have data silos, it's a problem, these edicts have been handed down on high and it's mostly a lack of understanding but there's a lot of things that go into it. And so I like to talk about one of the things that you can do when you go in and you want to establish or build your ability to be data-driven is find the places where when you ask questions, someone answers with the phrase, that's the way we've always done it. Because that is the very first place that you should change something. If that's the way you've always done it, it's probably a good indication that that's the way you probably shouldn't do it any longer. The world is changing and you need to evolve with it. Unfortunately, change doesn't happen in a vacuum. It needs to be from ground up and top down. So I think there's a lot of things that go into this but one of the dangers that we always caution our executive friends with is expert syndrome. You've got an executive who goes to a conference and listens to someone like me talk and suddenly decides that they have inferred the answers to all of the universe and that expert syndrome is really tough. So you gotta stay away from that. So we talk about some data decision-making and I won't go too far into this but I think the biggest thing to walk away from is just knowing that there are questions that every person in your org can answer and being able to enable them to do so is important and don't just go build another dashboard. Like go ask specific purpose-built questions and find a way to answer them because most likely you're gonna learn something along the way and you're gonna have a better answer when you come out the other side. Treating data like an asset. Historically, most people have treated data like a cost center. This is a thing that we have to pay to store but they don't think about what this means. At the very best, the most forward thinkers for years just treated data as a way to squeeze a little bit better efficiency out of some process. How to operationalize that asset. Yeah, okay, we've kind of gotten there but sometimes you can turn your data into an additional revenue stream or even better yet and there's a book, it's named Doug Laney and the name of the book is escaping me but he talks about ways that you can use data to barter as well and I think there's some really awesome potential in the market for that going on. I'll skip past some of this stuff. Breaking down silos, I think we all know the restricted data silo. This is the, I can't use that data because it belongs to a different business unit but looking at dark data and unused data depending on whose study you look at up to 80% of data gathered by an organization it goes completely unused. So I think it's just looking for what are we gathering? Do we need to gather it? If we're gathering it we should use it and then dark data, the stuff that lives on people's laptops and underneath their desktop so take a look at those and then just ask questions and I won't go into that, it's pretty self-explanatory. So I guess the last thing I wanna make a point here I'm running low on time is just building your culture through practice. So you want data access? More collaboration, we've talked about that. Data dictionary, better definitions of your data, better understanding, human readable that's the most important part. Data literacy, a workforce of people who use your data and data decisions. So we're actually using data and not our visceral engagement with a problem to answer business decisions. So it's the idea of let data lead you there, be data informed. Sometimes I like to say data driven is not necessarily the answer because you don't want the data to be doing the driving, you want to be informed about it and with it but definitely should be using data and not your gut. So I guess I will stop there and see if there are any questions but that is an hour and a half workshop in 24 minutes. So...


When I've watched CSI, you get the impression that the tests that they run, like DNA tests or fingerprint comparisons, are completely objective measurements made by a machine. So you'll stick the sample into the machine, the machine will flash, match, and maybe even put up the picture of the person who matches this fingerprint or this DNA profile. I think what the shows miss is the extent to which the determination that something matches or is similar depends upon a subjective judgment by an expert. In other words, human beings are involved in it. And that's part of what makes forensic science interesting to me as a psychologist, is how much human and judgment and decision making is involved in the production of forensic science evidence. So the evidence that's presented to the jury is not the product of some machine, only it's also the product of a human being's analysis of what the results of the instrument mean or what the comparison means. And the surprising thing is how often different experts can reach different conclusions when evaluating crime scene evidence. How is it that the two people or a person can look at some piece of forensic evidence and come to the wrong conclusion? What are the cognitive mechanisms that might be going on? Well, it can only happen when there's some ambiguity in the evidence itself. As I said earlier, in some cases the evidence is just clear cut and no one would disagree. But other instances, there is an ambiguity, there could be multiple interpretations and expert judgment is required. When experts approach a task like that, just as any other human beings, they can be influenced by what they expect to see or to some extent by what they desire to see. So people who expect to see something and are highly motivated to see that thing are more likely to see it. They're more likely to interpret an ambiguous stimuli in a manner that's consistent with what they think or want to see. We all do this. Most of the time our use of expectations to help us interpret stimuli is very helpful because most of the time our expectations are correct. But sometimes they aren't correct. The problem for a forensic expert is how to prevent this process of what's sometimes called observer effects, the tendency to see what one expects or desires to see, how to prevent that from coloring one's interpretation of the evidence in ways that undermine the quality of the evidence that's going to be presented to the jury. And I think the best way to do that is to try to minimize the amount of contextual information that the expert receives. So if the expert approaches the comparison not knowing whether it's supposed to match or not supposed to match or what the answer is supposed to be, then it's more likely that the expert's judgment will be determined just by the scientific data and won't be colored by the surrounding contextual information that may create what we would think of as a bias. My sense is that the justice system works best if the scientific experts are basing their conclusions purely on the science and don't allow those conclusions to be influenced by other factors, such as other evidence that might suggest that the person did or did not do it or the police theories of the case or their suspicions about the case and so on. The same argument is made for use of blinding procedures in other areas. So when instructors grade exams, often they do it without knowing the student's name, I think it's a good practice because it prevents the professor from being influenced by other information about the student that may lead the professor to think that this student is likely to perform well or not to perform well. You know, we would like the instructors grading of the examination or the paper to be based on what's in the examination and the paper and not any of the surrounding information. Same thing should go for forensic scientists. This course is about the science of everyday thinking. What advice do you have for people out there who want to think better and do better in their everyday lives? Oh, that's a good question. Boy, I'm not sure I've mastered how to think well myself. I think we all struggle with thinking clearly, with marshaling our thoughts. You know, there are some times when I have been benefited by trying to be very systematic and decompose problems into elements and think about them carefully piece by piece. But the fact is I rarely make actual decisions that way. I think a lot of our decision making happens kind of intuitively through processes that we don't fully understand and really can't analyze. And so the kind of advice that I give people about making better decisions is to be careful about what information you allow yourself to consider. If you're a forensic scientist and you want to avoid being influenced inappropriately by extraneous information, make sure you don't know that information. If you're an instructor and you want to avoid being influenced by how attractive or charming the students are when you grade their exam, grade their exams blindly. So there are circumstances in which less information sometimes leads to better decisions. Knowing what those circumstances are and then blinding yourself to inappropriate information is maybe one of the best ways to improve your decision making. My name is Bill. I think about proof. I think about how proof is generated. I think about how people respond to proof. I think about proof that's put forward that isn't really proof.


Good morning, John. Like an absolute accidental mutant of KitKat, this video comes to you in three parts. Part number one. When you get food poisoning, as I did last night, it's important to recognize that you don't necessarily know where you got the food poisoning. This is because many things that deliver the same symptoms and are food-borne pathogens can have vastly different onset times. E. Cola and Campfula Bacter, they could take like two to five days to get you whereas staff, that might just take 15 minutes. Salmonella can be anywhere from six hours to six days.  So I know exactly what it was, but I just think that's an interesting thing to know. Part two. I have decided to not use Twitter between now and January so that I can sort of have this break, which is supposed to be a holiday anyway, and also be taking a break from whatever it is that Twitter is because it's not like I don't have a lot of stuff I'd like to do during that break. That's gonna be fun and productive. That won't be Twitter. However, that does not mean that I have stopped having the desire to tweet or the ideas to tweet, so I'd like to read to you some of the things I would have tweeted if I had been on Twitter. Is bubble water addictive? This is an interesting question. I like some scientists to tell me. What if we started an instant pho company, like the Vietnamese noodles, and called it unforgettable? That's probably for the best. If we want to save the world, we need ways to make rich bros feel powerful without giving them any actual power. Probably again for the best. That's not a nuanced take because the best tweets are the worst ones, right? We all know this. It's not good to be good at something bad. This was about me. That sounds judgmental, but no, it was specifically about me feeling like I'm good at tweeting, and that doesn't mean that I should do it. Don't go around pronouncing medieval like medieval. You'll never learn how to spell it that way. You gotta say medieval. It's gotta be better, right? Because medieval makes it sound like it's the middle of evil. I don't think that was what it was, or what we intended for it to mean. Well, I've also gotten here, don't ever rag on someone for their music tastes. Music is such a glorious part of the human experience, and literally no good can come from making it worse for someone. I feel like that sounds pretty good. Part number three, John. It's time for the Nerdfighteria census. Yes, I have been sitting in my bed all morning writing questions and modifying questions and adding new social networks that didn't exist a year ago. If you don't know what this is every year since 2013, we've put out a survey to our audience so that we can better understand you, what you're up to, what you want from us, what you'd like to do, what you're struggling with, what you're optimistic about. It very much informs the decisions that we make. It helps us understand this community better. And it's one of my favorite parts of the year. So there's a link in the description. You can click on it and go with all the census. Now, I have intentionally made this video short so that you have some extra time. I'm going to give it back to you so that you can give it back to me when you fill out the census. John, I'm going to go lie back down. I'll see you on Tuesday.


Glucocorticoid preparations, such as adrenocortical steroids, have strong anti-inflammatory and immunosuppressive effects. These medicines are widely used for treating various diseases. Patients who receive steroid therapy prior to surgery may require additional doses of glucocorticoids to cope with the increased cortisol demand caused by surgical stress. Textbooks and research articles suggest that glucocorticoids should be administered in an amount equivalent to about 100 mg of cortisol for major surgeries. More studies are.


This stunning Pictish stone is over 1,000 years old and offers a glimpse into the mysterious Picts of ancient Scotland. The stone features a Celtic cross at the top. Many Picts did adopt Christianity and this stone shows the fusion of Christian and Celtic cultures. Below the cross is two angels that look to be reading books and at the bottom there are animals, one definitely a deer or a stag, with the other side a violent depiction of the animal kingdom. My next video on YouTube coming soon will unravel the secrets of the Pictish standing stones.