Yeah, that's I. Hey, people tell me if you can hear us. This is amateur hour. I've only done 343 of these. So. So welcome to blog Thursday number 343. Tom and Jason debate about virtualization and tech talk. If you're looking for some merger talk, Lauren stop video slash biz me and Jason been rocking out business videos over on business technicalities. I've made that URL short because I can't spell technicalities consistently. Yeah, I spelling spelling is hard. Yeah, we need AI to spell better for us. If you see me looking over here, this is where the control is we're using our my main camera. That's why we're not sure we're assuming that we're 44 seconds in and people can hear us. So type of message. Hello, a hello. Actually we'll go all the way up to the top here. Good morning. Pushing a little early for you because I know you're in the West Coast area of the world. It's afternoon for people in the UK. Sure, the people in the UK like that is earlier. Yeah. You know, I got to say something. No, I have just a ton of French followers now. My Twitter engagement is a lot of French now and just kind of like, well, that's kind of novel. I, you know, by the way, the, the something we're going to talk to is XP and G. They're also a French company. So is CrowdSec. And there's one other big company that we started working with that's French. That's open source. Seems like a lot of open source stuff coming out of France, which I think is cool. It's, it's funny that we're both in hunter shirts. Oh, yeah, yeah. That's completely coincidence. We didn't plan. Yeah.  So this started as an accident was one, I was since I injured my arm and I can be out of the sling because I can support it. I can't reach though to put a ponytail. So I let my hair down and then everyone's asking me questions. You'll get PTI presumed or improve the range of motion. Yes. Yeah, I can do this now and it doesn't hurt as bad. I've been doing that. Actually, if you look at this little PT sheet of do this with your hand, do this. So yeah, I got all kinds of PT stuff I'm doing. For sure. All right. First couple of questions for people that actually me throw this banner up on there because I always encourage people. I've been, I've even been replying to some of the emails because I set this up as a separate email, which is the blog Thursday at LawrenceSystems.com because sometimes people just want to point something out or send something to me. And I know you're not one of the usual spammers because purposely this is not embedded anywhere. So the scraper's okay. They got to scrape it off a screen, which it seems to be slightly more challenging. But if it's not, if it doesn't show up in any of the thumbnails, it has to go through the whole video. Yeah, they're too lazy for that. This any stuff. But so this is a question someone just sent in. Hi, Tom, hope you're doing well. Shoulders doing better? Yes, it is. So they're comparing the tail scale performance tail scale versus zero tier and a number of videos you have been using tail scales, wondering what brought it down to that side of the fence and updated thoughts and any similar systems like Nebula or NetMaker. I'd also be interested to hear cool use cases for mesh VPN overly networks that you can talk about where you're employing them with clients. I might have a couple client videos that I can talk about with that. I will admit I bet Jason hasn't heard of this one. This one is really cool. If you've not, they're branding in, so you tap on string error, throw that in here. NetMaker has really got a pretty website. Yeah. But what I've heard from people is it's kind of a little hit and miss. And I haven't tried it myself. It's a automated system. So you just point to nodes and it automatically builds all the tunnels and builds a wire guard mesh network, which because it's n times n minus one divided by two tunnels you end up with. Yeah. So it's kind of cool. It's open source and you have to build your own. Well, you get to host your own. So good and bad. It's a little bit more work to set up than the other ones like zero to your tail scale, which have their own coordination server. You get to build your coordination server in a cloud so you maintain control of the networks. I think that's a cool feature, but it's also, I don't know. I got to admit for an open source project, that's a website usually don't see one that nice for something that's 100% open source. I just haven't spent a lot of time with it. A few people have told me it's kind of buggy, but I will admit the reason I went for tail scale, a lot of it just has to do with the fact that it's built into PF Sense. And there's kind of a, I don't know if I'm going to talk about this, I don't know if it's worthy of a whole video, but I did switch to a MacBook Air. And yeah, and you like Mac. And yeah, he likes Mac. We're both, in matter of fact, we both have the same Android phone, but we both, I fought Mac. I would have told you I hated it for a long time, but I tried it and I said, Oh God, I like this. Yeah. So for me, I ran a Buntu like I've never really been, I've used Windows systems and I've had to, but I switched to a Buntu and then I honestly got tired of fighting with graphics drivers. So because of that and because of my needs are not that great these days, but I do need Word. I do need those apps that don't run well on Linux. And as a business person, you know, it became, I got tired of messing with it. I need things to just work. And so the problem is with the Buntu and that is I tinker too much and then eventually it doesn't work and then I lose productivity while I'm trying to make it work again.  It turns out, pretty sure the graphics driver problems don't go away in Mac. It's a hardware problem. It's not actually a driver problem. So I still have GPU crashes and stuff like that that happened. Less so on this M2 one, but it's just a good experience. I'm a FreeBSD guy. The user base, user space binaries for OS X or Mac OS are all FreeBSD forked. Julian, whatever his name is, was a big part of that when it first happened. FreeBSD core contributor got hired by Apple. Yeah. And to kind of bring it back to the VPN side and be having a MacBook Air, which also has tail scale support. So I have tail scale on my MacBook Air. I have tail scale on my PF Sense. And I also have like, you know, if I wanted to watch a movie when I travel, I've been using it. I have tail scale on my phone. You put all this together. And I think tail scale, I'm going to do like a long term review. I wanted to use it for everything for a while and go, damn. And I have zero problems with it. I should do it. I still use, I got one of the humble bundle VPN deals back in the day for like lifetime access for 200 bucks or something. So I've been using that when I need it. But I am definitely curious and will probably end up setting up tail scale at home. But I mean, I'm Maraki here at home. So I'll just set it up behind that. You set it up behind it. But it's one of those ease of use. And we've consulted with a handful of clients on it as far as like commercial solutions. The one that, there's a grocery store chain that we set them up with it. And it was really impressive. And because they had a problem where the install company put the same network at range, so they couldn't VPN. So every store was cookie cutter to same right down to the ranges. So this IT guy that in here, he goes, crap, how do I solve this? Because I need the database backed up between all these two one server turns out an overlay network is a perfect solution because it adds an extra adapter. So it doesn't care. And then they can use the tail scale IP addresses to specifically bind and do the coordination between all the inventory everything. And it built an incredible network. And we've had a few other random consulting calls from just we've talked to the team internally of IT. We didn't do the implementation. We just kind of explained how it worked. They go, Oh, that's it. One of them. I kind of like they watched my video and now didn't need consulting. They go, they almost like couldn't believe it worked that good. They're like, Yeah, it's kind of neat. So I mean, to me, this is just the next evolution of DM VPN, right? So I'm an old Cisco guy by trade. And I we were talking about DM VPN. It's been a CCI topic for eight years now, right? So it's this isn't new. It's just made its way down to the open source world more recently. Yeah, it's pretty neat. Um, hold on, someone had one more question. I want to lose all joking aside, no small business was learning if there is an add on to PF sense that you recommend that I have. As far as like the I have my small business video set up for PF sense, I think it's from 2021. Those still apply load PF blocker. Be careful with Saracada false positives. We literally got burned yesterday. We burned ourselves. All right. There's, um, Snort actually has a little bit better of a rule set now because if you snort, they have a pull down in PF sense that just automates it to like it says real aggressive as in spend all day tracing false positives or not so aggressive. But PF blockers not bad. I specifically my PF blocker, I call out like, Hey, block all the tour sites. Um, and I can't remember CIS army has their list. There's a few lists and I have a video called blocking threats of PF blocker. I did in late 2022 or early 2023, but the list haven't changed. Well, the list have been updated with the names of the list the same. So those are the two things that recommend as first packages for PF sense for small business. What are your thoughts on paid like emerging threats or some of the other paid. Yeah, those are, I think those are good. Um, but for small business limited protection most of the time is business email compromise and silliness. So they're cool, but I think they're for like a small business. I think they're like 80 bucks a year or something. They're not expensive. Yeah. And it does right. Emerging threats obviously it's a group. I think it's Talos, right? Isn't that a lot of it comes out of Talos and they are on top of and yeah, a lot of it's probably not aimed at small businesses, but it will definitely catch it. If you're getting hit by an ODA that has been caught. Yeah. And hopefully if you're small business, you don't have any ports open. So it's just less of a concern in general. That's usually not true. But yeah. In a perfect world, you won't have any ports open. The second question came in and I this is one of those wording things of what people said versus what they want. And the way they worded it's a little funny, but this may trigger some things in people's heads to understand the question better. Is it possible to route multiple private IPs on the same port on the public IP of PF Sense? And if the PF Sense has supported what option would you suggest? Well, you're asking for probably more like HA proxy where you will set up a proxy that redirects on the back end to get the front end where it needs to be. So 443 is all I have open, but depending on the matching URL, and I do have three tutorials on HA proxy, even including one for doing wildcards, and the third one is kind of not a tutorial. It's all the things you probably did wrong. And of course that starts with fixing DNS, because most of the consulting we do on HA proxy is your DNS was wrong, your matching was wrong, you misspelled the URL, that's a common one, because it has to be that URL. Host headers. Yeah, the host headers, the SNI has to match. If you didn't do the matching, you paste it in wrong, or you're connecting to something that has a self-signed certificate, you didn't check that box of don't validate certificate, because HA proxy by default wants to validate certificates on the inside, and of course they're probably not valid search, that's why you're using HA proxy. Yeah, we use, I mean I've got good amount of HA proxy experience, but I tend to use Nginx for that, but they both work fine here. Yeah, if what you're trying to do is web, that's what you want to do. I mean you can do, so the NAT rules, the back and underlying NAT system supports a combination of source and destination IP, so you could have this source IP on this port gets this, this source IP on this port gets that. I'm not sure that PSense exposes that in a way that you can actually do it in the UI, and the troubleshooting on that's impossible. Yeah, it can be tricky. Don't do it. Yeah. Tried Linux on a video creative, music didn't, it's, I've been, since it was painful 10 years ago, I switched to Linux on the desktop. That was a lot of work back then. Here in 2023, I barely ever fire up my Windows virtual machine, it just rarely happens. DaVinci Resolve is my go-to, and DaVinci Resolve is actually, if you were to talk to them, they'll push you towards either a Mac or a Linux version over a Windows one. They really like and they work so well on the, the new M2s and things like that, like it works awesome. I'm in the process of migrating. Proxbox appeared DB and 12 LXD, now that it's in Bookworm Prospery. Maybe it's still learning, but looks promising. Is it LXD or LXD containers you can do in Proxbox, do you know? LXC. LXC. Yeah, I think Jay from Learn Linux TV has a video on LXD. I'm not using it, so I don't have, I like containerization, but I'm not, I'm not up on the latest terms of this question here with any comments on it. So, yeah. IPv6, man. It's the year of IPv6 for the last 20 years. Yeah. I remember when it came out. I know, like my hair wasn't gray when IPv6 came out. I mean, I still, so I mean, I think the biggest adoption problem there is I still come across plenty of hardware that just doesn't support it, or if it does, it doesn't hardware switch it, right? So it's like the performance of throughput on, we'd have to look at like the NETK firewalls. I'm curious to see how much of performance hit it is to go to IPv6, because some of the network cards don't do TCP offload. They don't do a lot of the features that you get with IPv4. Yeah. So you end up giving out a lot of performance because a lot of it goes to the CPU then. Yeah. It just becomes a problem. Home Assistant, Zigbee MQTT XCPNG. Well, the problem with XCPNG, and it's not a problem, it's just extra work, is if you're passing through one of these Zigbee MQTT, which happened to happen in my hand, you end up with a problem of USB pass through, mapping it. It's not, there's documentation how to do it. I have not, I don't have a documented video, but there is documentation in their forums about it. I do like it on a Raspberry Pi. That's my preference. A Raspberry Pi worked pretty well for this because it's always on, and you can use things like Wake On Land. I have a lot of my systems. I just power all off, and then from my phone, I have it connected to Home Assistant, and it sends a Wake On Land packet and wakes up all the systems. That's how I turn everything on that's here. But I really like doing it on the Zigbee and Z-Wave. And this little device, has Zigbee and Z-Wave on one USB device. It's like 30 bucks. And I've put, I've already been replacing light switches and everything in my house on this. I have an upcoming video on Home Assistant, but if you go to kit.co.slashlaurancesystems, and it's linked in the description, I have an entire parts list for all my Home Assistant stuff. So if you want to copy my build, even though I haven't done the video, I did do the, I did do the part where all the things I know that work with Home Assistant, because I think that's the hardest part.  The next 25 hours of labor is all, does it work with this switch? Does it work with that switch? How do I get this configured? It's not the loader receiver. And it works, but it's not a great experience. Yeah. So, yeah. Figuring out which one's the right one. All right. This is on my to-list. Check MK. I'm going to look at that sometime in the future. I've used it extensively. Oh, really? Yeah. Yeah. We've deployed it in production a number of times. It's cool. It does, if you're going to run Nagios, run Check MK. Yeah. It comes with a bunch of tests out of the box. It's pretty easy to configure. The UI is kind of gross, but I mean, that's part of the course for all of those monitoring solutions. Yeah. I'm a fan. Cool. So, he has more experience than me. So, check MK. Thumbs up from Jason over here. Well, let's see. One of your recent videos, XTP and G, we're talking about a new utility to migrate from VMware. Mention you made to a video on the process, any chance that's on the roadmap? Yes. Absolutely. Me and Jason, now that we've got the merger done, like we have to do business things so we can afford our lifestyles and buy servers that we can load XTP and G on and load VMware on and do migrations. That's absolutely. Matter of fact, Jason's, me and him, we're going to coordinate on a video for how to do upgrades. Jason has a whole, in his email, I said it looked like a Windows activation key. He's got all the VMware certs. And so do several people at CNWR. So, there's a lot of VMware knowledge that I don't have, that the merger has now brought and we will bring to this channel. So, yeah, VMware videos are in the future. And of course, that also means VMware migration videos are in the future. But the tool itself has had several updates. They now can support migrations, warm migrations, incremental migrations. They added a whole, like they've, this has been what they said is, it removes the pain points. I talked to Oliver and his team and it just removes the pain point for people go, I really want XTP and G, but I'm all in this environment here and it's always supported importing like the VMDK. Well, you know, I think you had to use QCOW. You had to use. There was conversions. Yeah, the QEM. They automated all the conversion. Yeah, they converted.  And they've got some really, they've really enhanced the tooling on that. So, yeah, video will be in the future for that. Newer to virtualization, how hard or straightforward is it? Or even possible to copy move a 2012 VM server to a new server? There's no cluster. Like you want to go P2V, physical or virtual? Virtualized, yeah. Yeah. I have a video on how to do P2V. When you go to P2V it back kind of the same hardware. Remember, you need to shrink the hard drive first because otherwise it won't fit. Bryce, that's a call out to you if you're here watching because you had to save that day. Yeah. OpenSense has NGINX package with configurable security rules. Interesting. Oh, so a WAF. Okay. Yeah. I mean, I run, we tend to, I tend to run mod security for that kind of stuff with NGINX. Yeah, Xavix, I don't use it as much anymore. As a matter of fact, if I'll share this tab, there's some little point out that, hey, Tom, your dashboard's old. You're running an old version of Xavix. And it's up to date in terms of security. But it's because we used to use it to monitor all of our PF senses. Now I just use it to monitor mine. It was like we tried it, we set stuff up, and it was novel and now we don't. Yeah. I prefer check-in-pay over it, honestly. Yeah. And that's why I want to look at it. I guess to go some of the updates since Xavix is kind of breaking things and I need to rebuild it to the new version to stop that, but I don't care enough. Because it's just monitoring, it's monitoring my PF senses and that's it. Let's see here. How do you set up a clinic, set up a client to get a PM with audio? I think PMC supports it. I would use... Don't use VNC. Yeah. Don't use, don't use VNC. What's that other tool? X to go. Yeah. X to go is way better. Okay. For your RDP will work. I think some variants of VNC, real VNC or tired, one of the VNC variants, I think supports audio, but from a security standpoint, don't use VNC. Yeah. And when we get around to it, once we get the debrief done, we want to talk about VNC and the security incident with a client. That's going to be a debrief video we do at some point. So VMware versus the XSI, we're going to do a more concise, not just a live stream on this, we'll do a more concise video because this obviously is cool and I talked about this before, but it doesn't really tell the whole story. So me and Jason, we'll talk about some of the nuanced differences between this. I think a lot of it might be because I'm bringing XCP and GE to the CNWR team. So Jason's not as familiar with it as I am, of course, and I'm learning the VMware from him and together we want to make some videos to kind of talk about the comparative differences. We'll do that in a more recorded, organized format, but what's your initial take, though, is everything's in a weird place. Everything's in a weird place. So I told you this when we talked about doing this today that some of my experience is poisoned by a Zen experience on an Oracle data appliance. They've since moved to KVM for the virtualization in more modern versions of it, but the ODA on top of it is disgusting. It's a giant conglomeration of really crappy Perl scripts that configure all the Zen stuff and it's running an old enough version of Zen that a lot of the features just aren't there. So that poisoned me on it a little bit. So I've been running, I've got XCP and GE running at home for a bit now. I have, so I am now running Proxmox and XCP and GE at the house. I have not migrated my Proxmox VMs over to XCP and GE, although I'm going to, right? That's, that's my plan is to build a two-note cluster there. It works. Some, there's, I, there's some oddities about it. And as you said, some of them are just a lack of, oh, that's just done differently there. And some of them have to do with how pooling and live migrations work. It's just, I'm used to basically being able to like, pull the VM and just drag it to the new host. And it pops up with the migration screen, click, click next and you're done. Yeah. And it's all done in XCP and GE like this. I pulled something just so you can see there's, I ran H top, just so you can see the VMs not restarting or anything like that, but we can live migrated over here to the other one, select a migration network. I'm going to leave that alone because I'll let it do that. But you can, if you have a specific way you want to do it, but we can do that and we'll go here. Takes, this is a 25 gig interconnect. So it migrated within, what was that about 10 seconds? Yeah, how much land does that host have? Not much. Yeah. That's, I mean, that's going to be the limiting factor is you got to basically snapshot memory and copied over four gigs. Yeah. Hold on. Let me probably not four gigs and use. Let me try something here. Let's actually do this. I just, I got a hike. It's going to show my email address. There's a way I want to change the filter view on this, change this, and we want to change it to defaulting to the lab. Actually YouTube. There we go. Then we'll go, that was just YouTube. There we go. Then we'll add the YouTube tag. There. Now when I click home, I don't have everything on there. Security by obscurity. Yeah. Security. I just don't feel like, I don't want to click on the wrong one. This is a production system. There's like people working that will stop working if Tom screws up. But let's do this here. We'll stop the VM. Well, I could do it without stopping it, I think. Well, no, I got to see what the lamp ram is set to. Let's give this 32 gigs of RAM. Now actually make it use 32 gigs of RAM. So run like a sort of benchmark or something on it. Yeah. Actually, you know what? Do I have a pharaonic system? No, that's on another server. What is that tool? Stress will do that, right? I think so. Yeah. So 32 gigs. Actually, how much RAM do we have on this host? Let's use more. How much free do we have, I should say? That's the number. Yeah, you're fine. 32 is fine because we're going to be, so you're going to have to transfer 32 gigs of RAM. So I mean, that should happen in a best case scenario. You should be able to transfer that in about 12 seconds if you map that out. Yeah, but we'll give it 40 gigs. Yeah. 50 would be the number because it's two times you're in the bandwidth, but it's fine. 40 is fine. 40 is plenty. Yeah, and it does have the ability to do dynamic on here. So that is actually an option. I've got a whole video about how the dynamics works. It's committed, right? So in VMware, that's a reservation is what we call it in VMware learning. Yep. So you should pull that comment down. Oh yeah, probably. So when you do the comparison, you should include Jays, Proxmox sites. You get all three. That would be a good thing. I'm sure I could grab one of our guys in the office so we could even throw Hyper-V into the mix. So for what it's worth, I load Hyper-V. Me too. We've used it. We've got several client systems that are on it. Every time I use it, I swear a lot. And I'm like, they took everything from VMware and made it dumber. Yeah, I don't see it as a long term. Like I feel like at some point, Microsoft is just going to drop it. Well, so it depends on how you install it, right? So if you install it with just core, I don't consider it a type one hypervisor, right? Because it's a full windows install underneath it. But there is a way to install it where that's not the case. Try it. I thought VM state, VM heap. Sure, if there's a, yeah. You want it. You want the M, yeah. Yeah, dash M. So how many CPUs that add to? Give it like eight if there's two CPUs out. Yeah, but M is for memory. So we get a C. Well, you got to give VM bytes. So you want to give it dash FVM bytes and give it like 32 or gig or... YG. Type in bytes wrong. 32 capital G. 32 G. Well, I should keep that box busy. Did I do it wrong? So you want it like that? No. Or do I not need the M? I just put VM bytes. Oh. VM dash bytes. Yeah, it's dash bytes. Capital G. It's just down. It's just capital G. Yeah. Just work with it. No, it doesn't work with lowercase G. So much trouble. All right, we're back. Oh, geez. I clicked the wrong button. All right. The good news is this stayed in the show. Is this stressNG? Yeah. Okay. Does it need a time? Give it like dash C2. Yeah, it does. It needs the number of CPUs I'd give it a dash. No, we don't want it. I don't care about it. Yeah, it works. IO would be fun too. Yeah. So give it like dash IA10. Yep. Let me change this to a G. It needed a time after it. Yeah. Is that all we're going to run for 10 seconds then? Yeah, I just want to make sure it runs. Okay. And actually it uses the memory. We'll do it live. We'll do it live. Figure out how this thing works. You got to put our back on our comments view there so we can see people. Are you there? Uh, we are back. We hope. We hope. All right. It did a thing. It did a thing. Did it use memory? No. It did use CPU, but it didn't actually consume the memory. So let's figure out why it didn't. The end bytes. Do we have to go to dash M? Also? Maybe. Maybe we need to dash M. Yeah, go to dash M1. All right. Because we didn't have, we didn't give any workers doing memory. So we'll set it for a longer time and actually do the migration. Yeah, that's a more fair test. Although I still don't see a lot of memory usage. Yeah, we don't see the RAM usage. Run each top in another tab over there. Well, run it for like a minute so we can see it. Oh, 60. So M, I don't know, let's put 8. Does that help? Hogs to CPU 8 VMs fail. Out of memory. Okay. So we can't do 8 processes each consuming 32 gigs of RAM. Yeah. Oh, I saw a spike in something, disk. Yeah. Okay. So it's definitely spiking the disk. Yeah. A lot. So we're hammering on the drive, but we aren't seeing the memory usage. So is this NFS? Or so one of the questions I had was local storage, right? There didn't seem to be a good way to do a migration that required moving the disk and the host at the same time while it was live. Yeah. So if you want to move this to at the same time, you can. You can migrate this even if we chose a completely different server that's outside the pool, for example. Can you choose a local disk on that or does it all have to be shared storage? Nope. There's a local disk option right here. Okay. So it's on rise in one so I can choose local storage and that'll migrate it to that local storage of that device. But of course, that'll break the ability to move it between there. Okay. So in VMware, I can do both, right? I can tell it to move storage and host at the same time. Yeah. And it will do that. Yeah. Technically, well, let's do that. Let's migrate this over to local storage because it's on rise in one so we can move to rise in one storage without an error. Yep. Oh, no, migrate. VM, we want to migrate storage only. So we do a storage only. So rise in one, hit okay, and it should migrate. Should be reasonably fast. This is not a very big VM. Yeah. Just a base install of Debian. Well, now for that thing still consuming CPU there, or disk there, then it would be, but I think I'd probably stop by now. Well, we can let's do it while it's migrating this here. I'll set it for like 10 minutes. Yeah. All right. So now it's got a high load, which should slow this process down, maybe. It should. It's suddenly got a synchronized drive. Yeah, well, now it's been a long two seconds. Yeah, we messed up the time estimation. We arose it because the stats here say somewhere it's doing something. We got a lot of CPU load. Yep. We just jumped up to four minutes. Yeah. But it's still going. It's still chugging along. It's chugging along, yeah. Yeah, so it'll do that. Can you set the interconnect time between them looks like? Can you monitor that in there? Not in here. It doesn't tell me the interconnect. Okay. Yeah, so you can't monitor like the network usage of the host. Actually, maybe. Of the host. There's a CPU usage of the host. There's a network throughput right down there. Yeah. So you can't actually do it. Oh, internal error. We broke it. We broke it. Oh, internal error. We broke it. We broke it. We broke it already. Yep. It failed to migrate. It's under too high a load. So it probably said there's something going on from what we predicted to what happened. So if we stop it, it'll probably finish migration. No, I bet you canceled it. I bet you the task is gone. Yeah, the task is gone. Yeah. But it did that because it probably it's got some error catching internet. If it predicted something and something changed, it said, all right. So now we did break it live. But moving storage, you can move storage. And once it's on a system, if we were to migrate it over to one of these, when you do the migrate again, it'll automatically move the storage. OK. Because this is on a shared storage and it knows this storage right here is shared. Yeah, it'll just move the CPU. Yeah, it just moves the CPU. So there is a methodology. If I have two standalone hosts with local storage to migrate between them while it's live. OK. Yeah. You also have the ability to do warm migrations. I forgot. I did a video. I haven't done a video on it. But it's a new feature they added. It's a way to coordinate all the migrations in there. And they also have inside here. Yeah, because that's equivalent of going, if you have DRS and going to maintenance vote on VM where it will automatically evacuate everything off the host. Yeah. Yeah. I also have these here where you can build jobs. And when you build the jobs and the scheduling, this will actually let you do things like if you wanted to migrate based on time of day, move things. They've got a whole scheduling engine for this in here. This is something that I think you can do in VMware, but I think they made it kind of slick here because you just create a bunch of time things. You can do some of that stuff in VMware, but only if you have Enterprise Plus licensing. So a lot of the really cool, fun stuff is gated behind Enterprise Plus. It's called DRS there. DRS actually can do cool things like minimize power. So it's like it will migrate everything to various hosts and then shut unused hosts down and then turn them back on when the load. Okay. Yeah. So you can actually bring hosts up and down depending on load. That's kind of cool. Oh, yeah. We're hitting the memory now. Yep. Well, this is what's interesting. When you do the memory management, it compresses it. The ballooning. Yeah. It does the ballooning and sends it back and forth to the minimum use. That's why we're trying to make it consume more memory for doing that. Yeah. I presume it uses a kernel driver to do the ballooning and not just a user space process. Okay. Yep. It's got a kernel driver for it, which is also built into Linux. And the ones from... They're signed if you... Because you can load the Citrix ones. So the weird relationship they still have is Citrix has the signing key and VATES does not. So Citrix drivers work perfectly fine in Windows. Yeah. VATES isn't bothered because basically they don't need to. Like Citrix does it. And Citrix is well supported by us because VATES is the one developing all the TPM stuff that's going into Citrix. So they kind of have an uneasy relationship because essentially this project was spawning off of Citrix trying to turn everything into a license. Yeah. And so they pull a red hat and then it'll be contentious. Yeah. Well, VATES is pretty clear on everything going to be open source. They've a long time open source advocate for things. But yeah, that's... I like the backups being integrated. Yeah. That's neat. It's novel. One, I wish for Veeam support. We can make it work for Veeam support. It's just a little grosser because we are looking. So full disclosure, we're starting to deploy at least for our own internal like BDR devices we put out there. We're... And you know this, we're putting XCPNG on them and managing them that way. We want to back them up. So we're basically running the Veeam agent on the host. Right.  So it's just a two step restore then. I can't directly restore it to hardware that way, which is in the case of these is actually perfectly fine. Yeah. Yeah, maybe and maybe they'll have Veeam support sometime in the future. I don't know if that's on the roadmap. That's a Veeam problem. It's not a... Yes, Veeam has to rate the integration. The good news is Zen's been around long enough to be documented that Veeam support's probably not a big deal. The big advantage is shared storage over live migration with local storage with shared storage and cluster. You can automatically restore Veeam if a host dies, can't do that local. Yeah, I mean... Yeah, HA. Yeah. Yeah, you can have it automatically restart on there. Yeah, I mean, by all means, shared storage is the way to go, right? Like, and they've got like... Oh, the exosan stuff is half baked, I will call it. Yes, that is because exosan 2 is supposed to come out late this year. It might be a third baked. It might not even be half baked. They admit and if you read, they even have a whole write up on how bad the spaghetti code is. Yeah. Their own admission, like this is spaghetti code. Like this doesn't work the way I thought it would and I don't think I would use it the way it actually works. Yeah, but when they come out with exosan 2 later this year is kind of the release for that. They're going to have a new version of Zen. They're going to dark mode. There's the most exciting feature because that white is just blinding. But the exosan V2 is going to be substantially better. Yeah. Like they're also rewriting the exo API. So the underlying storage management is being rewritten too. Yeah, you should have that next question because we can talk about that too, right? Because this is not unique to either one of them. So generally, I do not. I use a dedicated network, at least a VLAN. It's not necessarily dedicated physical infrastructure but all of my iSCSI traffic lives on its own VLAN. Typically on the host has its own network adapters associated with it. If it's 10 gig, that may not be the case. But I generally try to keep storage traffic on its own because you're 100% right that you end up with all the storage traffic leaving the host. You build a storage network. That's what storage networks are for. And it's fine. It works fine. Yeah. Or a new fiber channel. You could definitely do fiber channel.  And you can pick up cheap fiber channel stuff because no one wants to do it anymore. So you could definitely just use fiber channel too and then it's not hitting the network. I mean, still kind of is. But yeah. And these, if you notice the network here is <IP_ADDRESS>. We have a dedicated storage network at the office as well. It's its own dedicated separate thing because not mixing those. This question has come up a few times because someone asked why I don't define them in PF sense. I'm like, why would I? I do not want my any chance of my storage network routing to the outside world. I do not route the storage network at all. That's just not. It has no default gateway. Yeah. It has no routing device on it. And there's no DHCP on it because I statically set everything. A lot of my networks don't have DHCP, right? Like management networks tend to not have it. If I don't want random hosts appearing on a network, I'm not going to run DHCP. Yeah. It just solves a lot of the problems because we go to the production VMs. They're also split here because if you look at the networks, this is why the XO itself, one leg, this is where it talks all the VMs and moves them around. The 20 network is where it bounces them around for the actual storage. This is how the backups are so fast. It's completely separate storage. It just makes life a lot better. Less risky. Yes. To see everyone features like Serene, Incident, yeah. Yeah. We do sure backup most places, instant virtual machines. I think we have that some places. We do all sorts of application or stuff. That stuff's not that hard to do. It's a little harder. We're doing Oracle backups there too, right? And so it's a little harder because you have to integrate with R-Man to get Oracle backups and the Kriask and all the other stuff that goes along with it. But yeah, if you're using Veeam, you should be using all those features. Yeah. That's part of the beauty of Veeam. We're still using that we brought over the MSP360 because people asked what we're settling on. And honestly, the goals eventually get to one backup, but it's not hard. The MSP360 works. My choice for backups says we hated that one the least for the three woman. It wasn't that I loved it. I tested a Cronus. Did it work? Yes. Is the dashboard horrible? Yes, it's really, really bad. And it's like they turned into an upsell machine. Yeah, no, they did. Yeah, I hate that. You want to use us for cybersecurity? No, like if you're the best in backup, then go do something else, right? But like when you're middle of the pack and backup, like, what are you doing? Right, and Comet backup, I don't know if they were fixed this, but we looked at it and one of the problems, Comet's really cool. I was really close to pulling the trigger, except they have a way, but not way, they don't have a way of getting the new clients on boarded without everyone sharing a key. And I'm like, this is a problem. Like this is a security issue. And because of that, it kind of ruled them out. And it's like, yeah, we've had people mention that before. And I'm like, well, you should probably fix it. Yeah, it probably has to do with signatures and stuff like that. It's probably not easy to do. They have to use subordinate keys and whatever. But what you have to think about is, if someone were to pop one of your clients and everyone had to share a key, they're like, I didn't get a key to this client. I got a key to your clients. So this was actually a problem back when I had my CVE in ConnectWise is that there was a lab tech deployment key that was common and you could get it off any of the hosts and it basically let you register your hosts with lab tech. So they changed it so that is no longer given out willy-nilly. I mean, we'll probably move your clients to Veeam eventually on the grand scheme of former clients, on the grand scheme of processes and tools that we're going to integrate. It's pretty low on the list because what you have is working. It's mostly doesn't require a lot of work to care and feed. There's very few tickets because it really is. I mean, we monitor it, but it's kind of a set it and forget it because the agents work fine. They just work. The setup's not difficult. It's easy to navigate. The dashboard is easy to look at. And yeah, sometimes those are like, it works. We've done restores. We've restored things for clients and it works. We do have one co-managed running Ninja backups, I think, still, but Ninja backups have. So since that, what's the names in charge of it? It's gone through some major refactoring. They feel bad. They're going to watch this video and they're going to see that. I went, oh, when you said it. Oh, I did too. I'm the one that look, I was just honest with them. I said, I can't use this product. I'm on their advisory and go to their counsel stuff and things like that. I told them and they took not just my feedback, but others that had experiences and they really put some work into it. So it's worth revisiting. But yeah, that's a different topic all together. Also, if you want us to, so if I want to do a tools video because there's so many more tools to talk about now. Yeah, I think we have 36 different things users can log into. There's just a lot. And it's, I think people, we want to give you some insight into the back end of running the MSP. And there's just so much, like I think all of you for the kind words of saying Tom had a mature operation, but there wasn't everything being done that could have been done. And so expanding further has been really interesting and insightful from like, hey, look at all the things we do now. I finally, I joked with Sean. I was like, I'm finally a connect-wise customer. Not finally a ninja customer. And he's finally a ninja customer because we like ninja for the RMM, but we manage works. They've done a lot with the managed the way you put nilier on top of it. I think your guys actually are starting to like manage. They do. At least, yeah, the employees like it. I don't hate it. I don't love it, but I don't hate it. You don't have to do a lot of it because I've exempted you from time sheets and all the other things that are, yeah. Yeah. How easy is it to pass you a PCI device next to PNG, next GPU, HPA? It's not hard. The write-up is pretty clear where you LSPCI from DOM0, you find it, you pop it into a list and exclude it so DOM0 has no access to it and you figure out which VM you want to. They do not have, because this request comes up from home users, it is not a priority for them right now to do tooling in XCPNG to allow device pass through. But if you like it better, Unray does a better job of this for home users, and so does TrueNAS, it's actually getting pretty good at it because a lot of people want their TrueNAS scale system and the Kubernetes thing they built, Kubernetes Docker coordination. They actually have pull-downs now, so your cards can be passed right through. I think Proxmox does it too. I think they have a, yeah, I think Proxmox 8 has a UI for it. That's one of those little questions. Yeah, it's one of those things that I think if you're, when you're choosing it just as a home user, you don't care about the scalability and cool things I talk about in XCPNG. It may be that Proxmox is a better fit for you because you're like, I don't want to do that command line thing, Tom. I want to pull down. Yeah, and now as we go through this, some of that stuff is that there's a lot of things that you still have to go back to the command line in. Once you get out of the common use path, you're back to CLI land for a lot of the XCPNG stuff.  So my suggestion, I did a review of the Zima board. The Zima board is way faster than the Raspberry Pi. It's really cool. Orange Pi, I think is another one, right? Or orange something. So the problem with those, Jeff Gehrling did a good video on this. The problem is support sucks on some of them. Yeah, it does. It does. You just have to accept that. Yeah. If you buy it off of Alibaba, the support's probably not going to be great. But I just did a review. There's a couple of Turing. Turing's one that's stalking me right now, but I think those are only compute modules. I don't think they're full. Yeah, Jay's got a giant Turing Pi setup. Yeah, yeah. That's probably why it's stalking me, because I watch the videos. Yep. So the Zima board is pretty cool. I did a video on this. It's 179 with 8 gigs of RAM, and it's x86. Okay. There was a Rock, was another one. Yeah. The Rock x86. There's a couple of companies that make these. Matter of fact, in the box over there, I just did a review. Those, I can't pick it up to Carrie, so that's, you're going to take that back to the office. Like this? Nope. The other little ones. Yeah, the little ones. Those are those little mini PCs. Those are really awesome. I did a review of both of those. So that's how we're deploying now for BDRs. Yeah. Are those little 1.1 liter PCs? Yep. We're not 1 liter. They're the bigger ones. The one with like 4, 6, 2, and 8 gigs in them. But either way, the Zima board is a good choice for running it. It's got USB 3. It's got a PCIe connector on there. If you don't need 8 gigs, they started like 119 for the less than 8 gig model. So Zima boards are kind of a good alternative to the Pi. Libra is another one that I've seen. Yeah. There's a handful of those mini PCs. Odroid. That's the one I was saying. Odroid. Odroid. Odroid has so many models. Yes. And like basically every configuration you could want. And Patrick from Serve the Home, he's got so many good reviews of some of those boxes. Patrick just dives deep into it. He's awesome at it. We'll answer a few more. Eventually we got to do other work. Proxmox needs better UI for SRIOV. Yeah, probably. I don't know how many home users need it, but... We could talk about that someday. The unnecessary complexity I see in some of these home labs. Yeah. Some of them really insist they need a SEF cluster to run their Plex system. Yeah. And then when it goes down, they put in a consulting request to get it fixed. I'm like, why were you doing that? Yeah. Do we use Teleport to manage NMSC? Not right now. Teleport's pretty cool. I don't know what that is, so I haven't looked at it. You've actually... What is that called? Jumper? I saw it. There you go. Oh, that. Okay. Yeah. I see you've seen this before. Teleport's pretty neat. Christian Lempa. He's my friend from Germany. He's done several good videos on this. What they did was they copied SSH. It's based on SSH wrapped around their Teleport tool. It's all open source. It's really slick how they built it. It's probably just SSH for the wrappers. Yeah, and they're very open about how they do it. It's essentially SSH with wrappers, but from a control standpoint, the thing I don't like about it is an agent, and I don't want to have to run more agents. Yeah. That's what stops me from using this, and there's a couple competing tools for it. I just don't want to load any extra that I don't have to. Yeah. We've been talking about that, even with regards to the XAPNG stuff, because what makes the most sense for us to do is to run a centralized Zen Orchestra box to manage all of these, but I am very afraid of that attack surface. So we have not done that to date. We might we? Yeah, we might. We'll end up tunneling it in, only allowing it from the IPs, access to it from the IPs externally that need access to using some sort of SSH tunneling or a tail scale or some sort of other methodology to get the traffic back, and then very much limiting access to that from internal networks, from trusted hosts and that. So you'll have to go through a jump box to get through it. It's kind of a complicated thing. It's limiting the threat surface as the really big piece that you want to make sure, and of course, back to being agent-based, I don't want to load extra agents, especially on my XAPNG Dom Zero, because now I may have a conflict that causes and breaks something else, so I'm troubleshooting a problem I created. And that's kind of our... There are other ways to solve that. You could just do an SSH with port forwarding. You could do... There's a bunch of ways to do it. Yeah. SSH VPNs are still my favorite combination. Just VPN in, SSH in. Keep it restricted to certain IPs. All right, we've got a few more questions and we'll wind it down here for coming up on the hour. In the hour, yeah. Home users, and I need it, really, to post some Rack and Lab info in the forums or something. Yeah. I mean, so over the years, my Home Lab has gone from a ton of hardware down to... I think I've got three 2U servers running, maybe two 2U and one 1U that run in the Rack, and I could get it all down to one. At some point, I tinker, but I need stuff to just work to, right? So it's like I try to avoid... I have lab environments that become complex, but I'm not running production stuff on them. No. The team, and this is actually cool, I'll pull up their YouTube real quick, because I didn't watch the video yet, but it's 45 drives. That's what I want to do. I want to get that back to go at some point. So the good news is, these are the... Let me find it. Well, I'll just leave the title up. But you can find this video. They had a challenge posted to them in the CEP forums, and it says, can we achieve 10,000 IOPS using CEP and SATA SSDs? But they built this out, and walk you through all the steps they do. This is a long video. How many boxes is that? Just one box? Probably. But what I like is they walk you through how they step through the process, how you build it. They're really good on CEP. So what kills you there in the end is the native command queuing depth of SATA is only two. So you don't get any right combining, and that makes it... I guess for IOPS, that's fine. Were they able to get 10K IOPS to the disks, or was it hitting the memory buffer? You know, I didn't watch the video, so it's not my to-do list. They just posted this. I'm gonna watch this too now. Yeah, and by the way, I need to invite you next time we do a round table with them. I'll bring you on there. They love hearing ideas and stuff throughout them. I won't say, but when Wendell was on there, Wendell has a clever idea that they're gonna build at the lab. It's gonna be part of the Creator Summit that I can't make it to because I canceled my travel plans there, but there's some cool videos that are gonna come out of there. I'm gonna fly out separate to go hang out with the people who are at 45 Drives and go to their building. And maybe when I go, you wanna go too. They were gonna give a whole tour of the facility where they build these things. So... With a rack of 3D printers, because a bunch of the parts seem to be 3D printed, which is kind of cool. Yeah, they're just neat people. They're really, really strong engineers there. But they build some of these ideas out, but they also talk about the complexities of them. And one of the other problems is if you want a good SEP cluster, it's not just SEP, you usually go, hey, I'd like to have several systems clustered together. The interconnection between those becomes the limitation of your speed. Synology actually has a really cool HA system where you can take two systems and create... And I have a analogy at the office that I'm gonna be doing this on, again, because they sent it to me, but it's got dual controllers in there. But the interconnected controllers becomes the limitation for your IAPS when you put it in that mode. That's why things like Infiniband exist, right? Because they're very low latency, very high speed, basically just PCIe interconnects between two systems. Yeah, so that's where people think they want it. But then when you start realizing all the components to make the cluster work, you'll know why we deploy way more ZFS than clusters in SEP. Eventually, our beam target storage will probably move to SEP, honestly, because we are... Yeah, I have so many. I'm up to like three or four relatively largest true-nest core boxes now, right? At some point or another, it makes sense to combine them all into one SEP cluster and just expose a giant NFS volume. The jury's still out. After the more time I spend with the 45 Drive Team, they used to do cluster. And they said, look, I said, cluster was cool, but we had like almost lost clients over this and because it couldn't scale to the performance. And I think they told me like they have over like 20 petabytes of data with a client. It's more like object storage to me, right? It's like if you're storing... It doesn't... It seems to fall apart because we looked at using it for a bigger enterprise-y client of ours that where we have a job batching system that lives into... There's four copies of it running, right? So basically we do blue-green deployment. So we've got the active one, the passive dev one, and then there's the secondary sites. There's four copies of it. We ended up just doing a centralized NFS server because Gluster fell apart with the 200,000 files that live in that one directory, right? It's just it could not handle a high number of inodes in one spot. And that's exactly what their experience was. SEP handles this differently because they have storage spaces that are dedicated to managing that. So you can manage it to say, all right, this is the extra caching. They said they would have with the client that they switched off a Gluster. I think they said they would just like... Just to open up some of the directories was a one to two minute pause. Yeah, yeah. It took... So that's what killed it. We set up this Gluster stuff and then I did LS. And it took literally two and a half minutes to get an LS response. I'm like, yeah, this isn't going to work. Yeah. It's just it's not practical. Yeah. I don't know where you're at, Jason. I think you're in Canada though. So morning. Morning. You said we should chat. You connected with me on LinkedIn. So say hi sometime. If you want to talk about videos and stuff, he's actually done... I think he's got a few proxmox videos that he did. Another YouTuber. Let's see. Two Synology Doctor Hose. I still have H.A. Firewalls, multiple switches, MTSP, for G backup though. Interesting. MSTP versus PBE, RTSP, right? Like I mostly have moved away from MST because of the... Unless it's multi... Unless MSTP isn't what I'm referring to. Yeah. To me it's multi-spanning tree protocol. Is it multi-spanning tree protocol? We're guessing... We have a lot of acronym collision that we're going to do it. Yeah. Because I think that's just called MST in most vendors' worlds. Yes. You can run up to 16 spanning tree instances and then you just map VLANs to them. Yeah. So you're saying the thought of cluster storage has lost its luster? Yes. Yeah. I mean, I think for certain workloads if you have a lot of large files for a video editing workflow, it might actually work fine because you're dealing with few large files. It seems to be a problem with metadata essentially and how long it takes to do the metadata lookups and collisions. And I think Ceph is a little bit different because it's more like block storage than it is individual. It's just different the way it's architected so it's less of a problem. Yeah. And the way TrueNAS is doing it, people aren't going to like... I've talked about this because the number of people ask me about it but then they realize how they implemented it. So TrueNAS scale and we'll pull up their site because I'll show you how the wording is. I like TrueNAS scale. It's not a dig at scale, but it's an implementation dig. So, you know, they're building this whole scale out structure which is cool. And they talk about it, but, but, but, Docker, VM, blah, blah, blah. The way they do this is the only way to make this work is through their... You have to license their cloud controller. There's no interface for Gluster in TrueNAS scale. So this is where you're... We're going to split hairs about open source. TrueNAS scale, the open source server still is built on open source and Gluster's open source. They're just not going to give you a management interface for it. You have to buy their close source. Yeah. Okay. So that's how they're getting around the fact that it's open source. The board is going to cloud manage this and then it's not bound by any of these licenses. Interesting. Yeah.  They'll host it for you. You can host it yourself. You can host it on a TrueNAS scale system. It's just a VM that spins up that connects through APIs that will talk to this. And it's kind of cool. It gives you more than just... What is it called? True command. They have a whole core enterprise. True command. So the True command system is what manages all this and it connects all of them. It gives you a cool dashboard. You can connect all your TrueNAS systems to it. It's got a lot of features. Okay. So we have a request from somebody that we're currently trying to fulfill right now that wants to not run multiple... Will this talk to core or only scale? No, both. Okay. That might be a solution to the thing we were just talking about. Oh, yeah. And this is what... Well, if he wants to use it, this is what he has to run. As opposed to using Gluster. Yes. Well, no, this coordinates the Gluster. Okay. So it's still Gluster on the back end, but this is the Web UI for it. Okay. Will it do just... If you don't want to do Gluster, it will do centralized management of just the TrueNAS... I mean, that may be all easy. After, honestly. Yeah. So it's cool. I actually demoed it when it was in the beta and I didn't like it enough at the time. And I was very honest with them and there's a reason there's not a video. I'm like, sorry guys, I'm not going to do a video because you won't like it as I told them. And I'm good friends with the people that I existence were a reseller. I said, when it gets more polished, it's actually got a lot more polished. Last time I looked at it, it looks good. They did a nice job. The way you... Like, it's pretty now. It's not like this... It felt... It looks like TrueNAS. Yeah, it looks like TrueNAS. They found a common theme. They've put a lot of things there. They're very open about the pricing on it. The subscription pricing. And it all is subscription. But this is what I know the homeland people are going, but Tom, I thought we were going to get Gluster with TrueNAS scale. Like, you are. As long as you buy the license for this. It's not expensive. You already could have Gluster with TrueNAS scale. Like even... You just don't get a UI around it. Yeah. You just have to run it off of the command line. So that's the one thing is a little bit... And the only place that even has any UI for Cep is the team at 45 Drives. They built some that are pretty cool. They're CepDeploy. They have a video on... I think they call it like CepDeploy, which is... They built it on top of Cockpit. It's pretty cool. I like where they're going with it. I think there are other Cep UI front-ends too that do exist. Okay. I've not looked... Besides them, I haven't seen many people doing it. It's not... Almost anyone who does Cep, like it's used all the time. It's a really popular system. It came from the open stack world. So because of that, there's open stack stuff that supports it. It's going to be your DevOps engineers, fingers on keyboard, punching away at it. It's... And those people don't want a web UI that would slow them down and remove the automation. So... Yeah. Most of those things are not intended to be... They're intended to be configured with automation, not with humans. Right. So that's why there's no UIs for them. Exactly. All right. We'll spend an hour. Thank you all for joining us. This was fun. As I said in the beginning, Lawrence.video.biz, if you want... We've been doing a lot of talk about the merger, all the behind the scenes on that, for those who are curious. We will be covering some of the VMware stuff, though, more in-depth. I'm excited to argue with Jason about VMware. I'm probably wrong about a few things. I've used VMware. I'm not completely blind to it, but he knows it way better than me. But we want to do more XCPNG as well. So... Yeah. Fun stuff. And we want to do that VMware upgrade video. I think that makes sense. I put that in our list of ideas. That one we should start getting done quick. So I'll get the groundwork laid for it. I'll build a couple of VMs. We're going to do it in VMs first, and then if they want to tinker on hardware later, they can. But that's like, I want to do it on hardware. I'm like, no, you don't, because it's like an hour to reset it between every time they play it. Yeah. Play, getting with it. And that's when I build my scripts for my videos. Like, I always tell people, if people watch me work, you would know I'm not as good as I look on video. I'm editing out all the bad. Seriously. There's like 10 times Tom went through this, Tom typoed it. Do you think I leave the typos in? I do. Yeah. When I do them. Yeah. Because I hate editing. Yeah. Editing. Editing is hard. Matter of fact, editing, though, is how you learn how to shoot better. And because once you've made a lot of mistakes, you try really hard not to make them. I'll just rerecord things rather than edit out the mistakes. I recorded the video that's uploaded right now that I'll get published tonight. I recorded it twice, because I realized how many mistakes I made. And I'm like, I ain't editing that. It's actually easier for me to read. It's only a five minute video. So it's like, it's easier for me to say this again in five minutes and spend 20 minutes editing out the things I said wrong. All right, man. Thanks, everyone. See you.


Dana and I'm a member of the Open Education Conference team with the OpenEd 2023 conference being just around the corner on November 7th. I wanted to take a quick moment to show you how to log in to Oxford Abstracts. This is the platform we'll be using this year to host the conference. It is where all registered attendees will be able to view and join all program sessions. Let's jump in. To sign in, you'll need to visit oxfordabstracts.com and select sign in. If you already set up your Oxford Abstracts account, simply log in or reset your password. Otherwise, create an account with the same email address you used to register for the conference. Once you're logged in, the first page you will see is the dashboard. From here, click view to be directed to the conference home page. The first time you log in, you'll be instructed to set your time zone and create your name badge. Setting your time zone is important as it will allow the platform to populate the program schedule according to your time zone, making it much easier for you to schedule your attendance accordingly. Next up is your name badge. Oxford Abstracts offers a range of features for attendees to network with one another through comments and live discussions using a name badge. The default setting is on, which means any attendee can send you a message or video call during the conference, but Oxford Abstracts cannot connect to your video or audio unless you actively grant permission through your browser. You can turn the chat feature on or off at any time. If you'd like to use the chat feature, you can enter your real name or use an alias. Entering your role and affiliation makes networking with other attendees much easier. You can also provide links to your socials to connect with attendees outside of the conference platform. So you've set your time zone, created your name badge. Now you'll be directed to the conference home page. Here is the conference home page. This menu on the left hand side offers various ways to view program sessions. Click program schedule for a complete calendar view of all program sessions and activities over the course of the three day conference. Each colored box represents a session in the program. You will notice sessions are color coded based on session type, including live sessions, prerecorded sessions, socials, plenaries, discussions, and poster sessions. To join a live session during its scheduled time, click view session, then join zoom. You will be redirected to the live presentation in zoom where all live presentations will be hosted. To view a prerecorded session, click view session and a window will appear with the presentation that you are welcome to view at any time during the conference, starting on the first day. If you have any questions or trouble logging in to Oxford Abstracts, please send an email to contact at openeducationconference.org. Stay tuned for more information about chat features and networking opportunities in Oxford Abstracts and other platforms we will be using during the conference. See you on November 7th!


Prime Minister Hon. Philip J. Pierre on August 8th tabled a resolution in the parliament to transfer $5.4 million from the consolidated fund to the contingencies fund to finance refurbishment works and repairs to school plants around the island. Because, you know, the passage of Hurricane Brett caused damage to some schools and educational establishments in the country, I considered necessary because of the urgency of the need to repair these buildings, these school buildings, that the cabinet and the ministry of education thought it necessary to get money from the contingency fund to transfer money from the consolidated fund into the contingency fund to refurbish repairs, Mr Speaker. Mr Speaker, that comes in line with the educational support that we have given to schools this year and every member of parliament got an allowance to help in the educational education, to buy education supplies for students going to the constituency and I stress every member of parliament, Mr Speaker, every member of parliament. The completed repairs to local school buildings will strengthen resilience against natural hazards and enhance the classroom environment for a more conducive learning experience for St. Lucia's students. Prime Minister Pierre is appealing to St. Lucia's private sector for cooperation and compliance with his government's recently implemented value-added tax exemption policy that applies to imported goods such as select building materials, personal care and female sanitary products. VAT is paid at the customs and every 21 days there is a reconciliation of VAT paid and VAT collected. At the end of 21 days, if you have collected more VAT than you paid, you are refunded your money and if the government, you collected less VAT than you paid the government, you are giving back your money. So they got nothing like old stock, new stock because the VAT you paid on the old stock on the stock that you have at the point of reconciliation is going to balance it off. So I want to say to the members of the private sector, work with the government. The government has shown a level of generosity never seen before in the history of St. Lucia. Please, please, this is a plea for the items where the VAT has been removed allow the people to benefit from that concession, please. St. Lucia consumers can expect to benefit from a 12.5% reduction in retail costs as a direct result of Prime Minister Pierre's VAT amnesty. Conversely, the government is offering companies and business owners a 100% waiver on penalties and interest on VAT arrears up to financial year 2021 if they settle their taxes in full by May 1st, 2024. More than $200 million in outstanding VAT arrears can potentially be redirected back into the economy. St. Lucia's farmers and manufacturers will be exempt from importation charges related to the health and security levy to ensure that the manufacturing sector and the agriculture sector do not pay health and security levy on the imported raw materials to ensure that the manufacturing sector does not pay the health and security levy on the raw materials and the packaging material similar to the agriculture sector, Mr. Speaker. Out of an abundance of clarity, the manufacturing sector will not pay health and security levy on the raw materials or on the packaging materials once it's certified by the Mission of Commerce. The cost of packaging products, raw materials and agricultural inputs will not attract the levy to keep production and manufacturing costs down for suppliers and retail prices lowered for consumers. Office of the Prime Minister


The World's Honored Watch is Lawn Jeanne. Lawn Jeanne watches have won ten World's Fair Grand Prizes, twenty-eight gold medals and more honors for accuracy than any other timepiece. Lawn Jeanne, the world's most honored watch, is made and guaranteed by the Lawn Jeanne Whittenall Watch Company. It's time for the Lawn Jeanne Chronoscope, a television journal of the important issues of the hour, brought to you every Monday, Wednesday, and Friday. A presentation of the Lawn Jeanne Whittenall Watch Company, maker of Lawn Jeanne, the world's most honored watch, and Whittenall, distinguished companion to the world-honored Lawn Jeanne. Good evening. This is Frank Knight. May I introduce our co-editors for this edition of the Lawn Jeanne Chronoscope? Mr. Donald I. Rogers, an editor of the New York Herald Tribune, and Mr. William Bradford Huey, editor of the American Mercury. Our distinguished guest for this evening is Mr. George A. Sloan, chairman of the United States Council of the International Chamber of Commerce. The opinions discussed are necessarily those of the speakers. Mr. Sloan, I'm sure that our audience would like you to tell us first what is the International Chamber of Commerce? Mr. Huey, the International Chamber of Commerce is a group of national committees of businessmen in some 30 nations of the world. In this country, we have about 400 business firms in the United States. Now, it's an organization of world businessmen designed to promote world trade. That's correct, sir. And designed primarily to promote United States trade? No. An international group designed to promote international trade, movement of trade, free movement of trade across the barriers of different nations. Now, are we today, are we Americans selling a great deal of goods abroad? Not today, as much as we normally would in peace times. I see. Are we shipping a great deal abroad under the aid program? Oh, yes, yes, indeed. And what American industries are most concerned with foreign trades? What American products are going overseas now in the largest quantities? Well, in the export field, I would say normally, your electrical manufacturers, automobiles, the rubber industry, and some textiles, business machines, and so on. Is their resistance abroad to American products coming into those countries? Not resistance on the part of the consumers, I think, is a great desire for many American products abroad, but there's difficulty in buying those products. They like dollars. Exactly. Well, in order for those countries, European countries, France, England, to buy American goods, they must first get dollars, and in order to get dollars, they must first sell their goods in the United States. Isn't that true? That is true. When they sell their goods in the United States, do we not bring about some harm to local industries, to domestic industries? Generally speaking, no, if our entire economy benefits, why it's best for America. I have in mind the wool of Great Britain, or the woolen goods of Great Britain. They constitute the largest single export of Britain to the United States last year. Do not our local woolen and worsted industries object to large imports of British woolen and worsteds? Yes, I think they do. I think you have hit upon an exception to the general rule. You might have picked the cheese industry, and you would have had the same problem. But I think that our reciprocal trade agreement arrangements out of the State Department offers a means of straightening out those problems where they exist. Our people, sir, I believe you will agree, I think a great deal about world trade, and after the two wars why we've had a great deal of experience with it. Now, in your vast experience, sir, is there more enthusiasm in America today for world trade? Are our people interested in trading with the rest of the world more today than they've been at any other time? I should say definitely they are, Mr. Huey. I think if there's any one reason that has brought the need of that world trade home to them more than anything else, there's the heavy tax bill that's necessary for the sending of billions of dollars in relief and grants and economic aid and different kinds. So they're beginning to realize that the only thing that can take the place of that relief is more trade and the selling of more goods to America, will our industries allow that, sir? Will our industries allow that? Yes. Well, where the shoe pinches, as I said before, where it's a matter of serious unemployment in an industry from that kind of competition, why they have recourse through the reciprocal trade arrangement and through our tariff commissions and so on. Now, you are interested and engaged in promoting the total volume of world trade, aren't you? I'm interested in promoting two-way trade, exactly between the free nations in order as a means of preserving the peace. And you regard trade itself as hopeful as possibly contributing to world peace? I can't think of anything that, in a practical way, short of building up armament for military defense, I can't think of anything that's more conducive to world peace. Now, our people, Mr. Sloan, you recall that at the end of the First War, we had something like the smooth-hauled tariff act, and there was terrific resistance in America to world trade. Now, do you think that that resistance has largely disappeared? I don't know that it's largely disappeared. I think that that resistance is greatly lessened, yes. And you think that the principle of reciprocal trade, meaning the principle of accepting that we must trade with the rest of the world, we must let them sell here as well as sell goods to them. Do you think that that's firmly accepted in the United States now? I think it's firmly accepted in all major industries by the leaders in those industries. How about the American consumers? Do you think that they are convinced? Let's call them American taxpayers. Are they convinced that we must have two-way trade between nations? Otherwise, we are going to have to support Europe through such devices as ECA or the Marshall Plan? Well, I think you have given the answer in the latter part of your statement. I think that it's that this heavy aid program we have had through Marshall Aid and so on, which was very essential, has convinced them that there must be some other answer. Is it essential now? Marshall Aid? Yes. Nothing like the extent to which it was in the beginning. I look upon Marshall Aid as very essential pump priming. Well, is it essential to the extent of seven and a half billion dollars as we have appropriated in 1951? I don't think so. I think that it was correct in 1951. But when we hear that the administration is thinking of a similar amount for 1952, I think you're getting away from the principle of pump priming, and I think you are running grave danger there, causing the people abroad to become too accustomed to this help and to get away from self-help. So you, on that point, sir, you are in favor of cutting Marshall Aid approximately in half for 1952. I think the mutual aid program should be cut approximately in half. Now, our American people have always liked to think that we are a self-satisfied country that we don't have to import raw materials.  I'm awfully glad you raised that question, because it's one of the strongest proofs of the need of international trade, Mr. Huey. We are absolutely dependent upon imports of tin ore, some 97% of it comes from abroad. We import a very substantial amount of our boxite needs for the aluminum industry, some 47% from abroad, and so on. And we are even importing oil, I believe, today. We are. I can't give you the percentages, but we are importing oil. Is it true that our steel industry couldn't exist without its imports of various minor metals that go into steel? Manganese, about 100% from foreign countries. Without it, we couldn't make it to the end. Not only that, Mr. Rogers, we are coming to the end someday in the future and the not-too-distant future of our iron ore reserves in this country. That's why you see the steel industry going to Labrador today, exploring the possibilities there, and you'll see the U.S. Steel Corporation going to Venezuela. Then this isn't pure altruism on our part. You're certainly not. We're looking out for the United States. We're looking out for Apple Sam and the people of this country. I'm sure that our audience would like your predictions, sir. You don't think that we're going to have anything like a high American tariff at this period deliberately designed to curtail the sales of foreign goods in the United States? No, I don't think so. I don't think we'll ever return to the smooth hauling tariff. You think that we, as a people, we've also reached a period in our development as a nation when many of our resources are being exhausted and we must have raw materials beyond our own geographical boundaries. So you think that it's intelligent selfishness on our part to develop world trade, don't you? Enlightened self-interest, exactly. Turning quickly to the ECA, do you feel that the ECA is being administered adequately, fairly, with competence in Europe? Mr. Rogers, when you say ECA, you mean mutual aid. ECA has such terminated quite recently. I think they've done a remarkably good job. I think, though, that in all of our efforts abroad in the nature of economic aid, we have so many men that have become more or less professionals in this matter of giving away American money. And not American patriotism. Well, and they're losing their knowledge of economics, I'm afraid, and they're not concentrating as much thought on helping those countries to help themselves as they are in just learning how to give away money. Mr. Sloan, I'm sure that our audience has appreciated your view. It was a great deal tonight, and thank you for being with us, sir. The editorial board for this edition of the Lawn Jean Chronoscope was Mr. Donald I. Rogers and Mr. William Bradford Huey, our distinguished guest was Mr. George A. Sloan, chairman of the United States Council of the International Chamber of Commerce. And tonight, I'd like to tell you how to take care of your Lawn Jean Wittner gift watch. First, the winding. Wind your Lawn Jean Wittner watch once each day in the morning before putting the watch on your wrist. Never, never wind it on your wrist. There just isn't room for the fingers, and this uneven pressure can lead to expensive repair bills because of damage due to excessive wear. Now, about setting your Lawn Jean Wittner watch. When you set it, pull the stem out gently, and then turn the hands forward or backward, whichever way takes the least turning. I don't have to tell you not to drop your watch. And please, keep it dry. And I have a special word of caution for the ladies. Face powder and watches just don't mix. Please don't keep your Lawn Jean Wittner watch in your handbag unless the watch is wrapped in tissue paper, and I don't mean cleansing tissue. And the same caution goes for the bureau drawer where you keep your maker. Give it simple care. Take it to your Lawn Jean Wittner jeweler for periodic cleaning, and your Lawn Jean Wittner watch will give you years of faithful and dependable service. It'll become an honored friend that you can trust. To all new owners of Lawn Jean Wittner gift watches, congratulations. And to those who were perhaps disappointed and not receiving one, good luck for the future. Lawn Jean and Wittner watches are made and guaranteed by the Lawn Jean Wittner watch company since 1866, maker of watches of the highest character. This is Frank Knight again inviting you to join us every Monday, Wednesday and Friday evening at this same time for the Lawn Jean Chronoscope, a television journal of the important issues of the hour, broadcast on behalf of Lawn Jean, the world's most honored watch, and Wittner distinguished companion to the world honored Lawn Jean, sold and serviced from coast to coast by more than 4,000 leading jewelers who proudly display this emblem agency for Lawn Jean Wittner watches. This is the CBS television network.


So, our next presenter is Reese Feist. He will be talking about Diplopia in the Child. So that was a lot of corny, and I'm sure everybody's kind of bored now, so we'll go to neuro-ophthalmology and everybody will come back up. So this was a case that I saw back when I was on Pete's service. So it's called the ER to evaluate a seven-year-old boy with a complainant double vision. So about 10 days prior to his presentation, he developed mild fever, bilateral leg pain. They didn't really do anything at that point. On the 17th, so like four days later, his mom noticed that his right eye was kind of wandering inwards, mainly at bedtime, and then around that time he started developing symptomatic double vision when he was watching TV. On the 23rd, he started having difficulty kind of walking around, going into walls, and that prompted a visit to the ED. He's got a past medical history of gross and fine motor developmental delay, speech delay, essential tremor, memory issues, ADHD, and then obstructive sleep apnea, which has apparently resolved on his last sleep study. He's had tonsillectomy, adenoinectomy, no past ocular history. He had two siblings with amblyopia and then an ant with some type of congenital blindness, and then he takes Intuna for his ADHD. So when I saw him down in the ED that day, his vision was 20, 20 in year in both eyes. He wasn't able to get a pressure on him, but his visual field were fully confrontation. He had minus one in ABduction in the right eye, and I felt like also, I didn't put this in here, but maybe minus a half in ABduction in the left eye as well. He didn't have any apherapupillary defect, no anisocorrhea. He had full color plates and red desaturation. For his balance, I didn't take my prisms down to the ER, but it felt like he was about roughly 30 diopters of rightisotropia. The remainder of his eye exam was totally normal. He had normal anterior segment, normal posterior segment, so he kind of pertinent things with the ABduction deficits. No evidence of disc swelling. His margins were totally crisp. Given his neurologic history that he was already a patient of neurology, they also came down to evaluate the kid. They called him a cranial nerve six palsy, felt like the remainder of his cranial nerves were intact. He was noted to have full strength throughout, and then good reflexes, normal sensation of attack, a normal tone, no tremor or rest. So while he was still in the ED, they got a CBC, which was essentially normal, just a little bit of lymphocytosis, slightly high use centiphalos, but his CMP was normal and his TSH was normal. I just like to get ACE more than even the uveitis people, and that was also normal. Really the only thing, he had kind of a slightly elevated ANA titer. They weren't able to get the lumbar puncture initially, but the next day, they had a normal opening pressure under one white cell, normal protein. It was a slightly traumatic attack, so a few red cells. He wasn't able to tolerate sedation on the initial day of the emergency visit, so all they could do was a quick CT brain, which was right as normal. The next day, they were able to get an MRI, but it was slightly motion degraded, and so the neuro radiology hedge was solving the radius, maybe under appreciated, but no real evidence of anything on the MRI, so it was essentially normal as well. So he got admitted, and then somehow I didn't see him again during the course of the hospitalization, but at some point he was also given a diagnosis of a cranial nerve for palsy, which I'm not sure where that came from. I didn't see any evidence of that on my initial exam, but they started him on an oral prednisome taper for kind of a presumed, like, post-viral cranial neuropathy. He was discharged home two days later and then had followed with the neuro-ophthalmology on the 26th. So this was, you know, slightly reassuring the top part, at least. His service was examined, which was kind of roughly what I saw down in the ER. So about 30 diopters of righties of tropium primary gaze. Dr. Warner felt like his exams were consistent with bilateral cranial nerve six palsies, but really the only difference here, his vision was still excellent, normal color vision, all that stuff, but when she checked his reflexes, he was found to be hypoereflexic, kind of in his lower extremities, which was a change from his actual neurology admission. So anybody kind of have an idea for the next step of what you'd like to do? Who's on is? You can pick on people. Nico, you're on neuro-ophthalmology now, so.  You got a slightly elevated ANA. Anything you're going to do? I do. Besides some of the Dr. Warner, so. I'm certain for the hypoereflexia. So maybe EMD studies at this point is maybe one thing to look at. Yeah, I think that's a great idea. So what she actually ended up doing was kind of along that same vein of thinking. So you're thinking of something kind of in that Guillain-Barré spectrum. So she actually ordered any ganglia cytina bodies, and then these came back positive. And I'm not sure who this was scanned into this chart, but I can imagine her writing this. But so Guillain-Barré syndrome is an inflammatory polyneuropathy characterized by acute onset, rapid progression, symmetric muscle weakness. So the overall majority of the cases occur one to two weeks after some type of respiratory or GI infection. There's a ton of different subsets of this syndrome. So most cases in North American Europe are acute inflammatory demyelinating polyridiculine neuropathy, AIDP. And then there's more axonal forms. These are kind of distinguished on paper, but can be really difficult to distinguish clinically. In terms of the symptoms that patients have, the electrodiagnostic testing can be less than helpful sometimes. But on pathology, they say the AIDP is characterized by inflammation of the destruction of the myelin sheath over the nerves. And then AMAN is more actual direct damage to the neuron's cell membrane. So this was a table from one of the papers just kind of talking about some of these other subsets. So in addition to these three main ones, there's some other even more rare forms. So Miller-Fishers is one that we're aware of. Acute panisababnomias kind of affects the sympathetic and parasympathetic nerves. Acute sensory attacks of neuropathy and then pharyngeal cervical brachial weakness. And then there's just a whole spectrum of you can even get kind of a brain semencephalopathy that often coexists with kind of encephalopathy features often coexists with some of these other ones. So the clinical features, I'm not going to go through all of these, but we're all familiar with it. So it's kind of typically thought of as an ascending paralysis. The characteristic thing on lumbar puncture is a cytobium and a logical dissociation. So you get a normal cell count, but you have high protein on the CSF. Nerve conduction studies are typically become abnormal after two weeks. You can do MRI, which can show some enhancement of the nerve roots. And then about 50% of patients with just classic Guillain-Barré syndrome can have positive mania ganglisside antibodies. So Miller-Fisher variants, the one that we kind of encounter more here, is characterized by the tritophthalmoparesis, a taxi and a reflexia. You can have just pupillary abnormalities. So anisechloria, slight reactivity, light-mere dissociation, which can indicate an internal ophthalmoparesis. This is classically associated with ADGQ-1B antibody, and it's reported to be present about 80% to 90% of patients with Miller-Fisher variant over a period of weeks. In the first week of study, after presenting with symptoms, about 48% of the patients had positive antibodies, but no CSF. And so this still has a super high sensitivity for the antibody test, but it does become positive towards the end of the disease. So by about three weeks, most of the patients had positive GQ-1B antibodies. There is, like we saw in that sheet a while back that this is the answer on, there's a lot of different ganglisside antibodies that they test for and kind of classically associated with different subtypes of Guillain-Barré. And so this is just a table showing those. So just to finish up quickly, the patient course, so one month he was seen back, he had no change in his trabismus exam. He was still about 30 diapas of esotropia. He was alternately patching and then followed up in the pediatric neurology clinic, and they were weeding him off his steroids. He's coming back, I think, towards the end of this month, and we'll follow up with Dr. Warner again. And then, you know, important to note that for most of these patients, they typically experience a full recovery over a course of months. So hopefully he'll get back to his baseline, but he was pretty esotropic when I saw him. So that was kind of short, but Dr. Warner always skips my neuro-ophthalmology grain rounds, but this is my second neuro-ophthalmology grain rounds. Because she knows you're so good. So any questions on that? Actually, one thing I forgot to mention too that I think I just skipped over, I found this pretty interesting was on the, and actually I don't know where that went. In one study of about 100 patients with just isolated cranial nerve six palsy, so excluding tumors, excluding a high ICP, excluding diabetes, they found 25% of the patients had any GQ1B positivity when they tested them. So 25% of patients might kind of fall into this category, because any ganglia site antibodies aren't particularly associated with just normal people. It's kind of a pathologic association. So it's kind of something that made me wonder, you know, how many of these things have I missed just with a lot of just the, we call it just a normal six nerve palsy, but, Dr. Olson. So a fascinating case and a couple of things are important is that a lot of us have just thought of classical Guillain-Bret, these super sick people and ventilators, and then we've got all these variants, and some of them can be pretty subtle. And so all of us need to be aware, just like you said, I mean the thought that a lot of these odds are sick, so we get better on it, so we get better, a lot of these could indeed be a variant. The second one is, is that due to Zika virus, you know, we're going to see a lot more Guillain-Bret. Guillain-Bret is getting to be quite common down in Brazil in areas where there's a heavy endemic disease. I mean, this is one of the responses that adults have that have a much higher frequency. So we've got all of you a bit more sensitive to the fact that this is what we're dealing with, and that it is immensely more common. You know, the biggest mistake neurology did is they should have taken this to neuropomology, right from the start, so I think our team probably would have picked it up. I'm glad to hear that even the muscle weakness tends to get better. I mean with Guillain-Bret, you just didn't follow it. Do they ever need to have muscle surgery? I mean, essentially at this point, the feeling is, is it almost 100% given time we're going to recover? Some, like anything else, some don't. I think that you have to follow them long enough, and that will vary depending on whether they are in the amyloid jet range or not. You got a young child, we don't wait a year, an adult we wait a long time, and Leonardo is talking, you can also temper us with Fresnel prisms on glasses or something to try to reestablish binocularity, decrease risk of amylobia, loss of binocularity in a child. This child at age seven is right on that customer. You may or may not have issues, so that I wouldn't hesitate if it is just staying the same and not getting better over, you know, three or four months to go ahead and do some eye muscle surgery, get the eye straight. But you're right, most of them get better, and that is a great case. I think the consult king, I think he said records for a number of consults done, certainly times spent doing consults, was just agonized over consults. It was a wonderful job, and this was another one of those cases. It was a lot of hard work and much appreciated. Thank you. Great case, thank you.