And welcome to the homelab show episode 107 the Ansible semaphore modern UI for Ansible I thought this was a neat topic that Jay picked and how you doing Jay? I'm doing really good other than being like really hot So if I start sweating the heat index I think tomorrow alone is gonna be 104 I don't know about today. So that plus a bunch of homelab equipment in my studio equals probably three or four showers today How are you? Yeah? Yeah, yeah It was it was our other friends of the channel Such as our friends radar and some of those guys because they're in Texas and a few of us a few of them over in Oregon They've also had a heat wave. So I guess it's our turn here in Michigan Me and Jay me and Jay are on this side of the world here. We were relatively close to each other in Michigan So yeah, we're gonna severe weather potentially for the next five days They're now know how big but yeah fun times fun times for sure But let's jump into the topic here But I want to mention of something before we jump into it and that's our sponsor for today Linode they're back and they're sponsoring us and things are going. Well, they are a great place to run Maybe you have some things you want to deploy in the cloud maybe you want to use Ansible to deploy that in the cloud and Linode has a cloud offering for you head over to the link down below to sign up for Linode get started with their Process matter of fact, I did a recent video using Linode with my unified controller stuff I really like their new interface. I had actually kind of been slacking Jay told me how good it was and then I went in there Set some stuff up. I'm like, okay This was easy to deploy simple to set up their interface is nice It's a great place to host any of the solutions where you don't want that heat in your Home lab you maybe want it out in their lab a great place to set things up We thank them for sponsoring a show user offer code the home lab show and get a discount get started with Linode Thank you again for being a sponsor And as a side note anybody watching the live the live streamer the video after the fact We have synchronized cameras. Oh We both have a model a camera now because I decided to go to the a6600 So if anyone yeah, if I look any different I doubt it because I said everything the same But if the pixels look better, there's that but today it's not about pixels. It's about semaphore semaphore Ansible semaphore definitely a Interesting topic we've talked about ansible, but of course and I it's funny because in the olden days And I feel like this is less of a problem now We're the older tech curmudgeon's complaining about kids these days only want to use a web UI I think a web UI makes some of these complicated tasks a little bit easier And I believe that's kind of the purpose of this is to give you a another dashboard for it This is a little bit of a discussion me and Jay had Appliance versus custom-built that's another episode we did and one of the advantages you get like I know Can do a lot with ZFS from the command line But true nas makes my life a whole lot easier especially things like when you want to set up replication So I think there's a lot of value in making it easier because unless your core focus is becoming an expert at command line ZFS then maybe you should just run true nas and maybe your focus can then be something else or it is something else So I think the same thing with this ansible, which I'm about to learn about it I you know briefly read through it. I let Jay drive the topic on this one and It definitely looks interesting. So let's hear all about it So the first thing I want to mention is the existence of AWX and ansible tower because you know There's a lineage not really a lineage because it's this isn't really this you know this built the same way But it serves the same goal. So Ansible tower is like your ansible GUI and I think this is something that ansible fans are well aware of but it was Very complicated to set up it the AW AWX is the open source version. So it's the same thing I mean, it's a little bit more complicated than that But essentially AWX tower gives you that GUI layer But it's always just been a pain to set up and I've done it a few times and I ran AWX for a while But I just kind of felt like the there's just it's just too cumbersome to set up and I never really did a tutorial on it But when I was looking at semaphore, I was thinking that it was gonna You know serve the same purpose and and I was right more than I thought because even the sections It's not exactly the same as AWX But you could tell that there's carryover like maybe some of the tabs are titled the same or some of the sections and features Are the same some of the workflow is the same, but it's just a lot easier to use so essentially What this allows you to do is have Ansible playbooks in a repository it can download those and then run them Against your host, but it could do that through the web browser But the cool thing about this is if you're a command line person at your company or you know at home Obviously, we kind of all are we but you know if you're using it, you know You like the command line That's fine But if you have somebody that you're working with that might not be as advanced as you They can also use the same playbooks but through the web browser But if someone doesn't want to do that they could just still use it through the command line Just you know pull down the playbooks and run them manually There's it doesn't stop you from doing that but I do like the fact that for people that just want a gooey or Maybe have you know friends are working with or coworkers at work or something They have you know, whether you're advanced or you're not you have a way of interacting with the playbooks and when it comes to semaphore it's It's just really easy like I was I was I sat down To set this up and I figured I'm going to be here a while You know that just kind of feel that way because I'm just you know setting something up and it takes a while Not that there's a lot not a lot to learn, but it was just easy like beyond easy It was just ridiculous how easy is it's great So I'll talk about the setup process first and then I'll you know talk about more of the features So when you go to their website the first thing you'll see is something like snap install semaphore or something I don't remember the verbiage, but they mentioned a snap package there and On their website you won't see anything else But if you go to the docs tab and then go to the installation page You'll find that there's actually a bunch more ways that you can run this you don't have to use the snap package So for example, there's a Repository package if you want to run it on Debbie and Ubuntu or equivalent or CentOS related distros So you could you know DNF install apt install after adding the repository or you could just download the binary and dot forward slash call it a Day, I mean, you don't even have to install it if you don't want to And they also provide a system D service if you want that you they give you like a you know a template You just change the path to wherever you save the binary if it's it'll probably be user bin Semaphore or something like that if it's installed, but I just did the dot forward slash method just pull down the binary and You could create your custom system D script if you want to run it as a service But no matter what when you pull it down or you install it The first thing to do is run ants or excuse me semaphore setup. It's going to ask you a bunch of questions It's going to use the answers to those questions to develop a Or generate a config file that that'll be in the same directory You could tell it to put it somewhere else if you want It'll just ask you like you want telegram notifications email notifications slack notifications Where do you want your playbooks to be stored when they're pulled down? number of questions that'll help you know differentiate how you want your setup and then to run it you Run semaphore and you point it to the config file that was generated previously and that causes it to run One of the questions that'll ask you is which database server you want to use So it'll give you an option for my sequel postgres or bolt DB, which I've never heard of before this and I still don't know anything about it. So I'm just going to Tell you what I think it is based on what I've seen it's almost like having a sequel light local database file That's not sequel light. So if you don't have a database server That's what I used because I was just testing it out and I didn't want to set up a database server So I went with that and then it Runs on I believe port 3000 just go to your IP or your domain at that port and then it's there and then That point you'll see a bunch of sections and you can start using it. So setup is like really really simple Yeah, and I was looking in the installation You can you got like you said snap package manager docker or binary file and it's well documented as well Like I would just run through the documentation. I'm like, oh, this is easy Like they got the docker compose file all laid out.  I totally forgot about the docker aspect of it So when you have this setup, you have several sections that you could you have to set up I guess this is the only the only thing that was a little bit annoying and it and Considered that what I'm about to complain about is very petty and it doesn't matter So if this is my biggest complaint, then that's pretty darn good So when you go to create a task template, which is essentially you want to run something You want a template that you run against your host It's gonna ask you for the environment the repository the key But it doesn't give you an option to set any of that up right there And it doesn't tell you that you have to have those ahead of time So I found myself going through that the you know filling out the sections and I try to continue Oh, I don't have this So I have to close out of this go back create the thing that it wants and then until I have enough to make it happy But each thing is very super Simple the one thing I had to look up was you have when you create an environment you have to create JSON But I just wanted an empty environment. I didn't really care I just created an environment called production because you can have different environments So if you put just an empty pair of curly braces in the JSON field that takes it and it's fine You don't have to use JSON, but they do make you put those brackets in there at least They won't let you continue without something in there and a space won't work. I tried it Just figured why not But the documentation is very clear. They say just put curly braces in there That's like the the one thing I had to look up other than that I mean it's pretty self-explanatory because you could create a key for example an SSH key Deploy keys for your get repository or what have you you could enter in the vault password if you use encryption You can enter in the sudo password if you want to do that Like if you're setting up a machine for the first time and you know, it's not even really set up That might be a way to do it and you go through all these sections and then once you have those things filled out You create an inventory and you add all your computers to that Our servers or whatever it is and then you can run tasks against The group or you know your environments basically you could create a inventory of web servers Or an inventory of this or that or a group them how you want and then run tasks against those groups even going down to production or Development or something like that if you have a test environment for example So you can really control like which host get which job The end to use it you create a an ansible playbook and then the feature set Becomes pretty much identical to ansible because it uses ansible So you the best way to use it is to have some playbooks in a repository Which is what I did and I had one that just goes through and Installs all updates on all servers. So simple it just basically does the equivalent of apt upgrade on everything and it works pretty well I mean, it's pretty self-explanatory at that point The logging is really good because you could go in and find out. Okay, did this job run? Why did this one fail because it keeps track of everything that you do? So if you want to go back and I didn't test this out But apparently you can replay certain sections of your taskbook So if something isn't working you could kind of try to replay and get back to where you were So they have some really smart features on this that I think are amazing Other features that I wrote down in no particular order You could group playbooks to projects so you could have different projects for different things you want to do You can run playbooks on a particular schedule. That's pretty cool. So you want something to run overnight not a problem I Already mentioned environments inventories access keys and things like that so you can manage all of those and You could not only run your playbooks from inside your browser You could delegate them to be run by another person so you can actually have different users on here So hey, can you run this for me and then give them access to it and then they can just go ahead and do it For you It's just me so I didn't have any I didn't even think to ask you because like I didn't even know about that feature Tell us probably too late get leading up to the the thing but essentially I could delegate access to you I haven't tried it yet, but that'd be kind of fun Kind of another note they have too is you can do LDAP and open ID configuration as well So if you have all their authentication, they do have those other options in here And it asked you about that to the LDAP thing when you set it up for the first time It asked you during the process if that's something that you want, which is pretty cool So you just just go through the questions in your terminal when you when you set it up There is a demo on their site. It doesn't work. It's broken The good news is it's easy to set up as Jay said so you don't have to rely on a demo I want to touch back on one thing though and maybe we'll do an episode on this at some point But there's so many times that me and Jay both been tagged on the socials or comments here about all their Authentication methods. It's cool that things like this Ansible supports LDAP But I'm still not sure if there's enough demand for us to do a video on it because it's usually just your Home lab tools that end up supporting this. It's not usually your other things that you can do the authentication with on here I know that's there's usually a question. It comes up immediately of mentioning like LDAP like hey You should do a video on how you can replace active directory with LDAP or open ID connect And it usually comes down to compatibility of all the different software not having tie-ins for all those things which challenges us more It's not the it's not the tool that doesn't work.  That might make sense when you have an on-prem LDAP system that controls your authentication for your users and things like that At my end and this is one of the reasons why I don't get into it much is I use Ansible for creating all my user account So I never really had a need for LDAP because that's something that Ansible make sure if I want a user account on all of my systems It's very easy to do that and LDAP just seemed like like overkill I think for a lot of people they might be coming from a Standpoint where they're at work and they have to learn it for work or there might be overlap So in those situations even though LDAP is overkill for a homelab It's often for the people that want to learn it that that's the value to them Is that they get a chance to see how this is interacting with other things? But as you said, I mean there's there's often nuances especially With active directory being LDAP plus because you have you know LDAP Compatible directories, but then you also have additional things they add and some of the you know Appliances that you use will be hooking into some of those Microsoftisms for example in the absence of those kind of confuses it and say I don't know what to do because this field isn't there So it's always a messy type of thing, but I think at some point That's something we can look at because there's my forgot fedora's version of this and There's a couple others that have come that are trying to make this easy for people But for me, I just use Ansible for creating user accounts because Ansible is doing everything else anyway So having it do that one more thing and it's all central in that one location for everything I do So There could be value in that if you don't have a learning reason for LDAP in my opinion And just quickly on this topic because it's kind of related because you could be using Ansible to do this Ansible for Managing users. Do you have it managing your passwords upon deployment? Yep, no, but I change my passwords regularly So if I submit the password change it gets applied to everything and the same with SSH keys Like it I could literally just add an SSH key remove the previous one and that'll be done and Make sure it's on the net order Exactly. Well, you know, it doesn't actually matter for mine because Mine doesn't use SSH for the actual configuration which It's using Ansible pull So all the machines are going down get and running it against themselves So even if my password is flat out deleted on everything it would be deleted. It's just going to go and recreate it for me But and I'm glad you brought that up too because that reminds me that is something that So far, I don't think semaphore is going to be a good fit for because if you're if the machines are running it local host via ansible pull Then all the logging and everything is and everything happens on those machines Unless you have like a some sort of a callback plug-in that's going to like forward the logging over to You know semaphore which those types of things do exist and semaphore really isn't made for that kind of thing but What I'm going to use it for is for a utility server that I'm building and I'm going to use semaphore for this so the idea is Maybe the first provision Is something that semaphore can run one-off things like updating packages if there's a A security vulnerability and I want to make sure like all of all of them are updated And I want to do it like and not wait the couple of minutes or something extra for my get thing to pull down I could do it through this as well for a one-off change and security vulnerabilities You could argue wouldn't be a good thing to add to version control because these change constantly so if you're chasing vulnerability fixes and Config management I mean that's pretty much all you're going to do But if it needs to be a one-off run against all of your machines then semaphore might even be A better choice for that But I think if you're running ansible in its traditional way then semaphore is especially going to be a useful utility I would say Um and someone's asking is ansible similar to puppet for configuration or is it more like a terraform deployment? neither terraform Is I mean you could use terraform for some config management But terraform has you know provisioners where you can you know tell it to build a server and then pass off Pass control or actually not pass control but run something from a provisioner which could be ansible or puppet puppet is going to be a more of an agent server kind of relationship Which essentially is what you're creating with ansible pull because You know technically ansible becomes an agent at that point downloading to get repository. It's a lot simpler But puppet if I remember correctly, I don't know if that's that's still the case I thought it would used ruby as it's in tax language. I think so Puppet though it seems to be falling out of popularity like it was years ago. It was very popular Um, we actually have a client we manage now in that sense the merger That's just what we manage for them because that's what they have for their infrastructure But you know, it's it's one of those once you've built it all on this No one's really pushing to say hey, let's spend the time rebuilding this all is ansible But right most modern systems and newer companies have run into or done consulting is always ansible I think when if you were starting, you know, greenfield you're going, what am I going to use today? Uh puppet is not bad, but I think it's better to spend some security problems with puppet because of the client server And it's just older So people have been finding some problems with ansible has such a different structure and such a massive it ramped up fast in terms of community sport And that once you have a project that big and it has a lot of people using it as in Very large enterprise players. You get a lot of security auditing and a lot of You know eyes really on it. So I think that's why it's been I think at one point in time it was the most starred and it might still be one of the most starred github projects out there So it's definitely A magnitude more in popularity over um puppet right now I actually I actually went from puppet to chef to ansible that that was my order A job I was working for a company I was working for They already decided they wanted to run puppet. Um, you know when I started the company had I don't really care I'll I'll learn it and I did my I maintained that for a couple of years and then The next company wanted to use chef. So I learned that and I felt like chef was um Heavier than puppet which is already kind of bloated in my opinion But I didn't know there was a better way But then a friend of mine kept saying you have to try ansible. You have to try ansible.  I told you and I got really tired with Especially with chef explaining to stakeholders why there's cpu spikes regularly because the kiffig tool is checking in and they You know, you got to understand with when it comes to stakeholders at a company It's often the case where they might say the server's 80 percent busy. What should we do? And I'm like, well, that's awesome. You're you're getting your money's worth your server's doing work. Congratulations. What but it's at 80 percent but Do you want like a server that's underused good your your server's used you're getting work out you're doing work for you And then they see further spikes above that when the configuration management tool runs And then you have to start explaining that to them But then with ansible everything's even more lightweight You know, and sure you're going to have a cpu spike no matter what you do but it's Going to be more um, it's just more flexible and it's easier to understand. I think To me, that's why ansible is really taking off because Or has already taken off a long time ago because it's showing people that you don't have to have this super, you know Bloated and just heavy utility to manage things. It's complicated to set up and the syntax could be complicated for somebody who doesn't Really care about scripting, but then ansible shows everyone though It doesn't have to be that complicated you could have all those things but with a smaller footprint And I think it was brilliant that it's such a brilliant way to do it Yeah, it's pretty much and I started with ansible and i'm by far not great at it But I never tried the other tools, but I found it relatively intuitive and said, oh, okay This makes sense. This is your fleet. This is your playbooks and this is the stuff you Oh, just put these ip addresses here throw these commands in here The building blocks from the basic to building it to a more complicated system where I found them relatively easy One of the reasons why I stopped using puppet and this is a long time ago so no hard feelings, you know against them now because I don't know if this is still a case but Um, I forgot which version of debbie and it was the company I was working for they're all in on debbie And so we were that's what we were using and we were ready to move on to the next release because we've already tested it and it worked out just fine but You know that new release of debbie and stable was not supported by puppet and a couple months in I'm like what's going on and then Oh, we don't have it ready yet. And I'm thinking wait a minute. You you're not Like supporting one of the most, you know popular linux distributions. I mean debbie in Unreleased a when like everyone else is supporting it and all these it's like I had to I couldn't I think it took six months or something for them to actually come around and make that work And I'm sitting there like no, I I can't forgive that like if you can't if you can't work it within a reasonable amount of time I mean there's literally debbie and testing to test it against leading up to the release and I just Felt that was negligent and I stopped using it. Um, and I was using puppet at home while I was using chef at work That was not confusing at all So two different hats I was wearing there, but then ansible It's just came to the conclusion. That's what I'm going to use going forward and then you know tools like semaphore. I feel like Give someone that ease of use not that it's hard to use. It's not but if you want a browser Equivalent just for one off all I'm going to click a button and make a thing happen And in a pinch then semaphore is great for that even if you are a command line person either way I think it's it's a great way to get started and it's super easy to set up. It's written and go It's a very fast interface It's supposed to work good on mobile phones. I haven't tried that but you know, it's super fast in the browser. So I don't see why not And something I've noted in the documentation for semaphore is in their key store They don't have support if your ssh key requires a password Um, what's your work around for that? Or do you set up a limited ssh key in semaphore to Manage those machines and that is a way of getting around the password Limited but that's that's more of an I think that's more of an ansible thing because I know you can't I think it's an ansible problem Yeah, I mean there's secret managers and things and ways to you know, do this and I've seen people do it You have to kind of architect that it's not going to facilitate as well built in but Again, people have done it. It's just um, if you think about semaphore is on the top layer Your ansible scripts or playbooks have to work by themselves before you could put them in there So that's also one of those things you have to kind of solve beforehand semaphore is just receiving and already working or known working repository with playbooks So that is one of those things and there's there's going to be nuances with ansible like that because you know Most of the time it's using ssh and if you're using semaphore you're using ssh So I would not use your primary key on there. I would um create a secondary one Oh, yeah that plus consider creating a Dedicated user a system user or something that has permission To the things you want to manage and then have the ssh connection connect as that user so that's limited to what it can do So if it's just a if you're just going to do apt install and that's all it does Then you could just give it at the sudo access to um the apt utility or whatever else If you're configuring all the things that doesn't scale well because a minute you have to do a second and then a third It's already too much. Yeah This is one of those tedious things. So once you move up into the larger security models You you really think about the term is our back rb ac for role-based access controls And this is the way you practice your principles of least privilege These are really important that you assign these users if you just have an update user That way if someone ever gets a hold of your update mechanism that doesn't give them the higher levels of privilege They need to get other things done One work around is to use ansible vault. I mean, it's not going to I mean it encrypts which is great But if there's anything you should already be using this for secrets anyway Nothing should be in clear text if something has to be in the repository You shouldn't have it in the repository, but at least encrypt it and with um, you know the ansible vault You can you can encrypt other things that would surprise you like the inventory file It's you can even encrypt the inventory file So as someone pulls down your repository and they try to figure out what machines you manage They don't know because it's all garbled and It's surprising how much of ansible you can actually um, you can actually I think there's pretty much everything you can encrypt. It's crazy. So you can game that a little bit by First making the repository private and accessible only via deploy key And then on top of that encrypt everything that's potentially secretive and you know that that could certainly help with that You're right. That's one of the things you have to configure an architect and these are growing pains of anyone in ansible Oh, well, how do I do that? I've been using a passphrase for a while What do I do now and then you work through those things and I I think that also teaches you Uh deployment etiquette and how to manage servers because you have to work through real problems You're going to also work through in the real world as well. So I think I like that fact of it Yeah, and it's like I said if you plan on doing this outside your home lab these are all those Scaling considerations because it's obviously easy and as you mentioned, you know Just being able to deploy or change your password everywhere But it gets a little different if you're managing a thousand users You're probably not going to manage a thousand users with ansible but for machine deployments at scale and managing those service accounts on the machine where you want to have it And maybe you have an incident where you go I got a roll passwords on this and you're using ansible pull you can just drop your updated one and pull those hashes all across here So there's there's a lot of different methodologies to think about there But you have to start with you know thinking and architecting it from ground zero your green field and make sure you have an idea That scales up and learning all of that fun stuff if you want to do it in the corporate world Yeah, and I think that this is these puzzles are fun to me. Some people might find this frustrating I think it's fun because you know, you feel good after you figure out a way around these types of things And even like users can scale better depending on how you architect it because if you have a playbook Like one playbook for all your users You can do that because you could create variables for the username and the password hash And then have like one file that's just straight down like you know user Password or whatever it is has all the values in one one file And it just reads those values in when it applies the template But even then like you said a thousand users gets a little at that point you probably do need I mean you've already needed that probably by the time of 100 I mean 100 users is not going to manage here I just want to make sure i'm answering some of the questions that are in the chat kind of about that of Well, does this scale and you know, this is called the homelab show So we're probably we but we encourage people who are wanting to sharpen their homelab scales to get into enterprise That's why we bring this up and talk about it But generally from the homelab show perspective and you getting started with it This is an adequate way to manage your homelab even if you have a few users in it These these methodologies will scale to Most homes don't have that many people.  That's kind of you know, it's funny because it's a question It's got my forums people like I want to set up, you know Some type of central management for all the users of my house. I'm like, how many users do you have? Oh three and I'm like, I mean a fun idea to play with and set that up But just for the three people you have it's probably not too hard to just to keep your life simple So there's less to manage I think um, I don't know if I doubt I know this is not always the case But sometimes I feel like spouse approval drives this because if you're You know managing everything and then your significant other has to Have a different password on his or her laptop versus desktop Then it's like They're constantly asking you that for what you know, why is my password different and how do I do this? How do I do that? Which means that they really kind of start to hate the fact that everything is complicated And I feel like some people they ask for roaming profiles and you know central password because that way The individuals in their house They don't have to constantly have like unsynchronized passwords and files which means the home lab person is probably spending more time Explaining things to them than actually doing their hobbies. So I do kind of feel like there's there's that that that plays into this too but at the same time This is definitely something I want to explore. I just don't know when but I do think there's probably at least some audience for this So we'll see how that goes when I get a chance to do that. Yeah All right, yeah, check out and we'll sum up for it's awesome. It's easy our friend uh christian lempa from um I'm sorry on the name of his channel again. It's really amazing. Um, yeah, my digital life I think it is my digital life for that. I think it's that digital life But it's his channel title also says christian lempa and we've linked his video down in the description He's awesome. He's a really good presenter And he uh does a great video on this particular topic. So we thought we'd link to that So we that way you have someone you can watch for if we got you excited about it And we want and people go but time I want the visual not the podcast or put it in the show notes for you It's not hard if you look up ansel semaphore I don't think you got it. I think it's come when I typed it in it came up with his video on top I believe he's the only one who's done a video on it. So yeah, he he has some great content So so watch the video give it a like give the subscribe block and talk give it a subscribe And show some support because uh, he puts a lot of work into that and I'm not sure if I'm going to cover it Myself at any point. I probably will but right now that's the video to watch so Yep All right. Well, thank you for joining us. Uh, always love hearing from you feedback at the home lab dot show Is how you get a hold of us. We have some feedback episodes coming So we have a couple things. Tom was gonna rant on one of them. I said, no, I'm gonna save this It was a security rant. Um, we'll have we all we have stuff to talk about it's it's some misunderstandings people have And I just want to make sure we clear them up, you know, make sure that misinformation does not rain free We will rain it in we will explain to you how some of these things work and uh until then just make sure everything's up to date That's just the most important part get it all patched All right Thanks and see you next week. Thanks


Hello everybody and welcome to an introduction to genome assembly. My name is Simon Gladman and I'm a bioinformatician from the University of Melbourne in Australia. During this slide presentation, I'll be giving you a bit of an overview of what genome assembly is and why we want to do it. And then hopefully preparing you for the upcoming tutorial. Okay, just a couple of requirements before we start. Hopefully you've all done the introduction to Galaxy analysis and you understand or you've used Galaxy before and you understand how to use it. If not, please go and have a look at that video first. And hopefully you understand a little bit about sequence analysis, mainly around quality control. We're not going to really touch on much quality control here as it's covered in another topic quality control is vitally important in genome assembly. Okay, so the question we're going to try and answer is how do we perform a very basic genome assembly from short read data. Okay, so we're going to talk about DeNovo genome assembly. And I'd like to thank Torsten Seaman, the BULAC, Ira Cook for a lot of the slides in this presentation. Okay, so we'll get started. We'll talk about DeNovo assembly, which is the process of reconstructing the original DNA sequence from the fragment reads alone. And why do we have fragment reads? Well, the simple reason is that none of our equipment can read a full genome at the moment. And so we can only read small pieces at a time. And so what we do is we break the DNA up into small fragments. We send it off to a sequencing lab somewhere. They determine the sequence of some parts of those fragments and then send those results back to us. And what we get is a file full of reads, which are pieces of the DNA that came from the sample that we sent. Now, unfortunately, these are in very, very small pieces compared to the length of the genome usually. And so what we need to do is we need to fit them together like almost like a jigsaw puzzle to put them back together. And so we need to find reads that fit together by moving around and comparing them against one another. There could be some problems though. There could be some missing pieces and some of the pieces could be dirty like there might be some errors in them. Or another view of what we're doing is imagine if you have a stack of newspapers and then you put those newspapers through the shredder.  And they try and glue the draft newspaper back together again. And then so the draft sequence is little bits of stuff stuck together. And then the closed original newspaper again is what we're trying to get. Okay, maybe it might work better if I give you an example. So here we have a string of characters. We'll call it a small genome. Friends, Romans, countrymen, lend me your ears. Okay, so this is one sentence. So we send our sample containing that string off to a sequencing lab and they chop it up into little bits and make lots of copies. And then they read the bits that they can read of each of those fragments and send the result and reads back to us. DS, Romans, count, NS, countrymen, Lee, friends, ROM, send me your ears, semicolon and crime in lend me. Okay, you'll notice that there might have been an error when the sequencing center read our sample. So what we need to do now is we need to find all the overlaps. As you can see here, friends, ROM, kind of overlaps with this DS ROM here, doesn't it? So we can see this DS ROM overlaps with this DS ROM. And so maybe they fit together. This read and this read fit together. And so we'll lay it out like this. And then we'll say NS countrymen, Lee, oh look, there's an NS count here. So maybe they overlap as well and so on and so on. And we find overlaps that make sense between our reads and then we lay it out like this. Okay, so you can sort of see we've laid out all their reads with all their overlaps. And then the next step to do is find what we call in consensus. So that is we look at the evidence of the reads and we try and figure out what the original sequence or sentence might have been. And so for this position, the first position, we have an F and that's the only evidence we have. So maybe our majority consensus is an F. And so on for the R and the I and the N. But here we have D. This is a bit stronger. We have two of our reads that suggest that there's a D in this position in our original sequence. And so we can say with a bit more confidence perhaps that this is probably going to be a D. And this one here is probably going to be an S and this one's going to be a comma and then we'll have a space. And so on as we go to here. And then when we get to here, you see we have three pieces of evidence. We have three reads that kind of overlap here. And two of them are saying there's a T here and one of them is saying a C. In that there's two T's versus one C. So maybe the C was a mistake. And so we'll say that in this position, we're going to assume that our original sequence had a T in this position here. And exactly the same with this position here. Lee landed me and send me your ears. We're going to say, well, perhaps, perhaps this S is an error because we have more evidence to say that it was an L. And so we have a final majority consensus of friends, Romans, countrymen, lend me your ears. And we have reconstructed our original sequence from our reads alone by doing overlap, layout, consensus. So far so good. However, the awful truth is that one does not simply assemble a genome. So Professor Mihai Pop, who did a lot of research into the area of genome assembly, actually stated that genome assembly is impossible under certain circumstances. And we'll talk about what those circumstances are soon. So why is it so hard? Well, unlike our example that we just talked about, we don't have five reads. We have millions of pieces, and they're much, much shorter than the genome. So if we have a human genome has three billion bases in its entirety. Some of the chromosomes are very, very large. And if we've only got short read technologies, the longest ones we can get are about 300 bases. If we've got long read technologies, we can get them out to maybe 70,000 bases, but it's still well short of the length of the original genome. And another problem is that a lot of the sequence is repeated throughout our genomes and throughout the genomes that we're studying. And so they look very, very similar. And there's lots of missing pieces. So some of them just can't be sequenced. They're too high in GC content. It could be lots of reasons. And there could be a lot of errors in some of those reads. So it's kind of like doing a jigsaw puzzle where you don't know what the picture is. Your dog's got to some of the pieces and chewed some of them so they don't quite fit anymore. And one of your children is drawn over the top of some of them with texture. And so it's actually quite a difficult problem. But we have a basic recipe that we can follow to help us. So one of the things we will do is we will find all the overlaps between the reads. We'll see where each read overlaps all the other ones. That sounds like a lot of work, especially for millions and millions of reads. And then we're going to build a graph, which is a picture of the read connections. So we're going to place a read on the table. And then we're going to place another read on the table and then we'll draw a connection between it. And then we'll add another read and draw the connections in and add another read and draw the connections in. And so we'll slowly, slowly build up this giant picture of all the reads and then all the connections to one another. And so we call this thing a graph. And then what we need to do is simplify that graph because it's going to be very, very complicated. We're going to have reads overlap many, many, many other reads. And so maybe we can remove redundancy. We can do a bunch of other things that might help us simplify that graph. Unfortunately though, sequencing errors will show up in this graph and we'll mess it up a lot. And then finally we will traverse the graph. We'll try to trace a sensible path through the graph to reduce the consensus. Now, this is not a trivial problem at all. In fact, it can be quite difficult to do this, but this is what our General Assembly recipe is. And in pictorial form, you can see here we have our reads. We find out all the overlaps and then we lay them down and draw all the connections between them. And then we start to look for paths that visit each of these things once. So we can go from this read to this read to this read to this read and we'll pull out this green to purple part here. So green, blue, purple, and we'll pull out this consensus sequence here. But then what do we do with these parts? And what do we do with the fact that this one has lots of connections going into it? And this one has lots of connections coming out of it. It's a lot more complicated than we think. And this is what a realistic graph looks like. It's very complicated. And tracing a path around this, this part. I'm sure we can go from here down. And then do I go this way or that way? I don't know. I can trace the thing. I'm actually it's a lot more complicated and number these things. It's a nightmare. It is genuinely no fun. So what ruins this graph? What makes it so complicated? Why can't we just jigsaw it back together and make it really simple? Well, read errors. They introduce false edges and nodes in our graph or they introduce false connections in our graph, non haploid organisms. So polyploid organisms like humans who are diploid have this thing called heterosciogosity where one version of a chromosome might be slightly different to the other version of the chromosome. And so these can cause detours in our assembly graph where we can have two possible paths through our graph. This makes it very, very difficult. How do we how do we choose which one to go down? And how do we know that one isn't just an error or it's real? There are so many things we need to think about. And then repeats. If the repeats are longer than the read length, it's actually impossible for us to assemble. But if the read length is longer than the biggest repeat, then maybe we've got a chance of assembling it. And repeats causes nodes to be shared and we get locality confusion. And so we tend to read, but then we have no way of knowing which path to take out of one of those reads. Okay, so we're going to talk about repeats because they're really important in our genome assembly. What is a repeat? Well, a repeats a segment of DNA, which occurs more than once in the genome sequence. And they're very, very common. Things like transposons and satellites and gene duplications occur regularly in pretty much all genomes, except for viruses because they're usually very compact. But pretty much everything else, bacteria, fungi, high eukaryotes, plants, animals all have huge numbers of repeats. Okay, how does this affect our assembly? Well, if you imagine that these black lines here are our reads and we've found overlaps and we've laid out this nice consensus here. We've laid them out and then we found this nice consensus in this black region. But in this red region here, we know that that red region there is identical to this red region here. Right, but we have different sets of reads from it because when we broke the DNA out and set it off to the sequencing center, they read this part and this part and this part and this part and this part. But to us or to the computer, they look identical. So this red read here looks identical to a combination of these yellow reads here. And so when we do overlap detection, these red reads get piled up on top of the yellow reads. And so even though they came from completely different parts of the genome originally, they get lumped together. And we can't split them because, well, this part of this genome here, we know that maybe it's connected to this red section. But then when we're coming out of it here, we don't know whether to go into the yellow section or the red section. So we can come into it. No problems. We don't know which way to go out of it. But this part here assembled up really nicely with no repeats in it. And so, yeah, sure, we can assemble that part. But this repeat, we're going to have this collapse of the consensus. And we're going to just produce one repeat instead of two. All right. So the law of repeats is it is impossible to resolve repeats of length s unless you have reads longer than s. And I've written it here twice because you need to keep it in the back of your mind. All right. However, we can do some tricks. I'm going to talk about scaffolding and what that means. So how do we go beyond contigs? So contigs are contiguous pieces of DNA that we can put back together. So this section here, this nice long section here where there's no repeats. There doesn't seem to be any problems. And so when we extract that out of our assembly, we get this piece here called a contig. Whereas the repeats get collapsed into another contig. Okay. So we want to go beyond contigs. But their sizes are limited by the length of the repeats in our genome. We can't change that and the length or the span of the reads. And we can use long read technology to hopefully overcome these repeats. But we can also use tricks with other technology. We can change the type of reads that we produce. So say we have this example fragment. This is a fragment of our genome. So we've sent our sample off to the lab. They've broken it down into small little pieces. So it'll fit into the machine. And then normally, well, originally what they would do is they would read one end until the machine stopped working. And then they would call out the read. And so they would only sequence one end of the fragment. However, they got a bit cleverer. And then they thought, well, why don't we sequence both ends of the fragments? And so they sequence the first bit. And then they sequence the last bit. Now, we know that this particular read is related to this particular read, right? Because they're on the same fragment. And we can also measure that roughly the length of this fragment. And so we kind of know how far apart these reads need to be when we're doing our assembly. So when we do our overlap detection, et cetera, et cetera, what we do is we record where the pair of these two reads lie in relation to one another in our consensus. And we can say, are they roughly the right distance apart? Are they too far apart? Are they too close together? And so we can exploit that kind of information in our assembly algorithms. Okay, so when we do scaffolding, we're doing exactly that. We know that the sequences are related to each other because they came from a single fragment. And we roughly know how long the fragments were. And most of the time the pairs will occur in the same contig. But occasionally the pairs will be on different contigs. And this is evidence that these contigs are linked together. And this is what we're talking about here. So sometimes we have one of the paired reads on this contig, another one of the paired reads on this contig. So that's evidence that these two contigs are linked. And we can also get a direction for these contigs. So we kind of know which way they need to face in relation to the other ones. And then we can say, you know, this paired-in read matches one here. So we can connect those together. And there's another one that came from a bigger fragment. And, you know, they sort of join up together there. So when we lay the contigs out and try and assemble those using the evidence that we have, we can sort of see that there's a contig here, a contig here, and a contig here. And we kind of don't really know what goes in there. But it's probably going to be some kind of repeated element, probably. Or simply an area of the genome that we didn't sequence or couldn't sequence. But this will give us what we call a scaffold. And then we can go back and target these kind of areas and try and fill them in. Okay. So how do we assess how good our assemblies are? Well, we desire our total length of all of our contigs to be similar to the genome size. We want to assemble as much as our original genome as possible. So if we've got a five megabase genome, say any coal wire or something, and we've got three megabases in contigs, then we've done a pretty poor job of assembling. But if we have, you know, close to five, 4.9, 5.1 megabases of contigs, then perhaps we're getting close to having an assembly of most of our genome. We want fewer larger contigs. So we want our contigs to be big. We don't want to have lots and lots and lots of little ones because that's not going to tell us anything. We want to have a few large contigs that make up most of our assembly. And we want them to be correct. And we need the ways of checking to see if they're correct. Because remember, we don't really know anything about this genome before we do any of this work. So we don't know what it looks like. We don't know what the contigs are meant to look like. We kind of know the length, but that's about it. So how do we know they're correct? Well, there's some ways we can sort of figure that out. And to do this, we have some things called metrics. There's no real generally useful measure because you don't really have any prior information. So we don't have a true set. We can't say, yes, this is 99% true because we don't really know. But what we can do is we can sort of measure the number of long contigs, or we can sort of look at the total number of paces in contigs and compare it with our genome length. We can calculate this thing called the N50, which is a statistic that gets used a lot in assemblies. And it's basically a measure of how together my assembly is. So the N50 is the length of that contig from which 50% of the bases are in it and shorter contigs. So imagine we have seven contigs at our assembly with length one, length one, length three, length five, length eight, length 12, and length 20. Well, the way you calculate the N50 is you lay them out in order like that. And then you sum them all the way up. So when you add all these numbers together, you get 50. So our total number of bases in contigs is 50. And half of that is 25. So now what we want to do is start at the smallest and add them together until we get to 25 or above. And then the length of the contig, that is the last one that we add to our sum, that is the N50. And so we go one plus one, two, five, ten, 18. It's not 25 yet, so we need to add the 12 on. We get 30, which is greater than 25. And so the last one that we added into this sum was the 12. And so our N50 of our assembly is 12. And so basically we're saying that 50% of our bases are contained in contigs bigger than 12. Okay. So there are two levels of assembly. There's a draft assembly and a closed or finished assembly. The closed or finished assembly is usually our goal have a finished reference sequence for our organism of choice. However, sometimes we get to the draft assembly stage where we get the end of the scaffolding step and we've got a number of non-linked scapholds. We've got gaps and we've got unknown sequences in bits of them. But we've got probably 80% of the genome sort of put together and laid out in a plan. And this is fairly easy to get to. And sometimes that's enough. But the closing or finishing assembly, we want one sequence for each chromosome instead of a bunch of scaffold. It takes a lot more work because we need to look at each gap individually. We need to figure out what goes in each of these gaps. Small genomes are becoming much easier to do. So we can do a whole bacteria now. And we can almost guarantee we're going to get a closed genome out of it at the end with long read technologies. Using Oxford Nanopore or PacBio or one of the other long read technologies. Large genomes like human genome are much more difficult even with long read technologies. It's still not easy and it's still very expensive and still the province of consortiums. And for example, the human genome consortium. So how do you actually go about doing an assembly? So we have an example. We culture our bacteria. We extract their genomic DNA. We send it off to a sequencing centre for say a luminous sequencing. And what we get back is 250 base pair per end. We get back to text files from a little while that we sent off to our sequencing centre. Now what do you do? Well, we use a tool. We can use some assembly tools. In some of the tutorials, some of the early tutorials that we have on the GTN, we use tools like Velvet and the Velvet Optimiser and Spades. I have to point out please that Velvet and the Velvet Optimiser are training tools. They are, however, very good for teaching people how to run assembly tools and what's going on behind the scenes. So if you want to learn about assembly, then Velvet's OK. But if you actually want to do an assembly, then use something else, something like Spades or a base. New blood doesn't really exist anymore. SGA, all paths, soap. There's hundreds of them, canoe. The list is endless. We can also not just assemble genomes. We can assemble other things like metagenomes. And we can assemble transcriptomes using things like Trinity transubis. And there are many, many, many others. In fact, if you look up genome assembly and Wikipedia and then go to the list of tools, it's almost you have to scroll through like five or six pages of tools. You'll be doing an exercise but not using Velvet a bit later on. Hopefully if you do the assembly tutorial. So thank you for listening. So this concludes the introductory slides for genome assembly. If you would like to know more or learn a little bit more about genome assembly or would like to have a look at some of the algorithms that genome assemblers use. If you click up here on the top left of the slide deck, it will turn you to the topic page on assembly. And you can see here there are a lot of different slide decks in this section. The ones we did were these ones. But there's ones that go into the details about the brand graph. This one here, a deeper look into genome assembly algorithms is actually really good and explains to you exactly what's going on inside the tools if you're interested. All right. Thank you very much and goodbye.


Today, we're going to talk about the shop right supermarket of Flemington, New Jersey and what we like and what we don't Hey there njroute22.com here with a little chat about the Shop right supermarket in Flemington, New Jersey, especially since this crisis Of the past four months. This is what we're how we handle the crisis anyway, you can see here this Shop right is in a strip mall Over in Flemington, New Jersey just south of the Flemington circle. They call it It's on the northbound side of Route 31 for anybody watching this in, New Jersey that's never been there It's kind of an interesting supermarket from what we understand. It's the highest grossing sales Shop right in existence out of all the shop rights that are around as you can see here There's a bed bath and beyond there's a marshals. There's a bunch of stuff in this little supermarket strip mall here We used to go to this supermarket, you know sporadically and we had our we liked because it was a large Supermarket they one of the best deli sections in out of all the shop rights But we didn't particularly like the the parking lot. It was always very busy It was a kind of a frenetic type of supermarket and it just Something about it. It was a good supermarket, but we didn't like going there, but we do like their shop from home It's located in the back unlike most other shop rights. You just park in the front and What we like about their shop from home is that they have more slots available for pick up Because people in our family don't want to go into the supermarket and get infected by the green slime but You can get a lot you can pick up your food. They're almost on demand Same day almost It's located in the back. So they have a nice little separate section for you to pick up your Food and you could see from the video here, which we took with our drone It's it's in the back of the parking lot here and but one thing I want to point out about the this Flemington shop right and one of the reasons I like going there is because over in the back there is the Dutch country, whatever it's called the farmers market the Amish market and That's a really nice place to go. I mean, it's it's really a special place to shop in New Jersey They have some of the best prepared food and cheeses and creams and fried chicken And all sorts of stuff and great beef and bacon and and so on I really love the Amish markets One of my favorite places to go in New Jersey But the shop right we have to say that if you're gonna do pick up and you don't feel like going to the supermarket Or I don't want to offend your family members with having to wear a mask or go into one of these supermarkets This this shop right in Flemington is probably the best when it comes to the shop from home aspect We had trouble in the beginning and I don't know how they were in the beginning We didn't start start shopping there until like a couple months ago They do a really good job Overall, and it's it they've never missed an item and or been out of stock So we were like if you like it hit the subscribe And the bell and all that other stuff and we will talk to you soon


�우� pcf Love controller says Y instructing thing is right leaflet raising a concern about the NHS in Wales by the Welsh government putting things right raising a concern about the NHS in Wales The NHS in Wales aims to provide the very best care and treatment but sometimes things may not go as well as expected When that happens you should raise your concerns with the staff involved with your care or treatment so that they can look at what may have gone wrong and try and make it better. In NHS Wales, this is done through the process known as putting things right. Who should I talk to? The best place to start is by talking to the staff involved with your care or treatment as soon as possible. They will try to resolve your concerns immediately. If this does not help or you do not want to speak to the staff, you can contact the health board or trust's concerns team. If you have a concern about services that you have received from your general practitioner, GP, dentist, pharmacist or optician, you should normally ask the practice to look into it for you, but if you prefer, you can ask your health board to do so. Each health board or trust has their own concerns team. To find their details, go to the Health in Wales website, www.wales.nhs.uk, forward slash our services, forward slash contact us, forward slash NHS complaints, or phone NHS direct Wales on 0845 4647. What will the concerns team or practice do? They will listen to your concerns to try to resolve them as quickly as possible, look into your concerns and speak to the staff involved in your care or treatment, put you in contact with the right person to help you, let you know what they have found and what they are going to do about it. Please tell the concerns team or practice if you need information or communication in a different format such as large print, braille or audio. How soon should I tell someone about my concern? It is best to talk to someone as soon as possible, but you can take up to 12 months to do so. If a longer time has passed and there are good reasons for the delay, the concerns team or practice may still be able to deal with your concern. Who can raise a concern? You can raise it yourself. If you prefer, a carer, friend or relative may represent you, but you will need to give them written permission to do this. Can I get support to raise my concern? Yes. The Community Health Council's independent advocacy service provides free and confidential support. Find your local community health council by contacting the board of community health councils in Wales on 02920 235558 or www.communityhealthcansals.org.uk inquiries at waleschchc.org.uk What happens once you have raised your concern? The concerns team or practice will contact you and may offer a meeting to discuss your concerns. Look into your concerns and talk to the staff involved in your care or treatment. Aim to respond to you within 30 working days of receiving your concern. If they cannot reply to you in that time, they will explain why and let you know when to expect a response. Some concerns may take longer to look into. What if you're still not happy? If you are not happy with the health board's response, you can contact the public service ombudsman for Wales. Public service ombudsman for Wales, telephone number 0300 790 2203 www.ombudsman-wales.org.uk 1. Ffordd i'r hein gau pancoed CF-35 5LJ


Mangroves are breeding grounds for myriad marine species and an essential resource for coastal human communities. They filter water, sink carbon, trap sediments, stabilize soil, improve water quality and protect coastlines from erosion and storms, providing ecosystem services to millions of people around the world. Healthy mangroves can provide huge environmental, social and economic benefits. Local community participation is key to mangrove protection and sustainable management. The Coastal Fisheries Initiative works with local and ancestral communities to conserve, monitor and reforest mangroves along the coasts of Côte d'Ivoire, Ecuador, Indonesia, Peru and Senegal. The people are living here. They're living in the forest. They are living in the forest. When we were here, we were all living in the forest. The mangrove was not safe for the people. It was a woman's life. People were living in the forest. When the mangrove was in the forest, they were all living in the forest. When you're on a trip, you need to be careful. Cravit was neglected. They say this is a place where the people are living. And there is the same place where they live. And they have no reason to be breaks. But Cravit was neglected. I have a lot of parents. I want to live here. Cravit lives here. In Ecuador, thousands of small-scale fishes work in the mangrove forests of the Gulf of Gayaquil, which cover 126,000 hectares. Lauren is a harvester and fisher from Puerto El Morro in the Gulf of Gayaquil. She says that over the past five years, shellfish have become scarce due to mangrove deforestation and she is fighting for regulations to protect Patas de Mula's clams. Empowering women in fisheries value chains plus sustainable mangrove management plus an ecosystem approach that lets organisms replenish themselves and nature do its job equals thriving coastal communities.