Hi, I'm going to do a presentation about Videoland and VLC. I'm from the Videoland project. I'm the president and I've been working on VLC for a long time, a bit too long probably. So this presentation is going to be full of cones and other nice stories. Please stop me if you don't understand what I'm saying or if I'm going too slow or too fast. If you stop me and ask questions, it means that you're still alive and that's great and I prefer that. So this is a story that takes place in 1996, 5-6. Basically here you can see is the campus of the École Centrale Paris which is an engineering school in the south of Paris. Basically it's one of the best engineering schools. In the 1960s the state said, well it's named Centrale and we don't have any place to put it in Paris. So we're going to put it in the middle of France. The alumnae said, oh no way you can do that, we want you to stay. What the alumnae did was they bought this piece of land and then told the French government, hey we bought this land, we're going to build a campus and at the top of the campus you can build the school. Don't worry I'm still speaking about VLC. And because of that the school does not own the campus. And the campus when it was installed at the network that was managed by students was installed in the 1980s. It was amazing with a great technology called tokenaring. I don't know if anyone here still remembers those things. It's a great thing and it was amazing however it's very bad if you want to play video games and especially when you're going to play Doom. So in 1992 and 1993 the students started to ask the school, hey can we have a new network? And the school said, ah we're really sorry this campus is not owned by us so we can't do anything for you. So they went and see a lot of partners and then there was someone from TF1 which is one of the main broadcasting networks in France said, okay if you're able to destroy your tokenaring network by sending video on it, I pay you a new network. And we are in 1994 so there is Pancham but no other mix. Decoding Mbeck 2 in software was almost impossible and we are like 10 years before YouTube. But they say okay let's do that. So they start the project and after two years they managed to have some streaming of the DVB satellite and it was really cool. It was able to play 35 seconds of video and then crash because no memory left. But that's fine because the demo was 30 seconds and it was working. It was even a multi-platform, I mean Linux and BOS. And after that they say well okay that's great. Let's start again and actually do that for other people. So this rebooted the project in 1998 with a few ideas. The first one was open source. The second one was modularity which is one of the base of VLC and this is one of the reason why VLC became popular. And it was still network oriented because it was just a client for a streaming solution online. In 2001 after two, three years of battling with the school they managed to re-license all the projects under GPL because the school said well sorry but you are our students, the IP is for us. But the students won. Proprietary. The idea of the school is that they wanted to sell an Mbeck 2 decoder video and audio because some guy of MIT made that for the Mbeck 1 decoder. Yes, sure. But VLC is just one of the projects of the video land project. VLC was at the beginning just a video land client and of course there was a server. There were some completely crazy projects that are dead like VLCS which was a way to do basically multicast of a network that is not multicast by abusing VLANs and changing directly people from VLAN when you wanted to play a video. But more, lots of libraries for developers including the famous libdvcss to work around the DVD encryption and decryption. But also x2so, probably the most used encoder in the world now. And other stuff. We have a software libraries for decoding Dolby, DTS and also all the Blu-ray stack. So at the beginning this was basically the video land streaming solution. So on your left you got the inputs which could be anything basically whether it would be DVD, Blu-ray, acquisition card, DVD or even satellites. And then the restreaming on the network after transcoding or not. A lot was focused on the multicast network that we had at the École Centrale Paris. And then you have a few clients whether they are VLC or not VLC. So that's more or less the story of the project. The problem is that we have to work with multimedia. So I guess everyone is familiar a lot with multimedia around here? Yes? It's okay, fine. So what is multimedia?   This is the thing we have to deal every day. Great ideas about having some wave audio put in MP4 formats remixed in MKV or WebM and then stream over adaptive streaming with a different codec for audio of course that is going to change every time you change the bitrate. Stuff like that. And also the problem also is that in multimedia everyone thinks he understands everything but actually no one does. The more I do multimedia the more I realize that I don't know anything. The problem is that many people including from big companies including people from YouTube and so on believe they know everything and then try to do something and then they realize that it is complex. So the result of that is that in multimedia technically everything is broken. There are 42 ways to do something. Container is basically the box of format to put the different video and audio tracks.  The worst of course are AVI but at least it has the excuse of being from the 1980s. FLV which is probably the one the most used on the web those days great. Or AUG. Also too many codecs have bad designs. Most of the codecs are badly encoded because someone found a script online and I did a few stuff and many just put something inside something else. The reason is that for example you have an encoder that is very expensive but everything else in your post processing channel is only understanding these formats and you should buy a new compatible encoder but it's going to cost you money. So it's easier to work around and hack it in order to put it in it. So for example we have amazing stuff like H265 put in FLVs or AVI using the direct show format of Windows 95 and people complain on the mailing list all the time to support that. Yes. The rest is that it's not about only codecs and containers but metadata also is very broken. How do you put an image inside FLV or AUG? Well it's great because AUG has metadata that is only comments, text comments. So you're going to take your image basically base 64 it and put it in a text field inside a binary format. Amazing. There are two ways of having basically the metadata of H264 and some encoders even on the same platform some decoders want it in one format or on the other one. For example on Android depending on the chip that is going to do the hardware decoding you need to put one format or the other. And so on. I don't even speak about DRM because DRM is always bad but the thing is open source is absolutely everywhere. It's used by almost everyone, all the TVs most of the smartphone have some part of open source codecs that were developed by the FFMPEG and the LibEvi project. And it's used in large kind of domains from TV station to hospitals to education. However, there is a big problem. The first one is the USA. The multimedia community is underrepresented in the USA because of patents mostly and everyone is afraid of that. So a lot of multimedia engineers in the USA who work on FFMPEG are basically working on wrappers around codecs in order to never touch the codecs. A lot of the projects are abandoned. We have a lot of forks. And we can't say that most of the open source software media are easy to use and VLC, I'm also to blame on that. Also the professional in the broadcast is still fighting against open source software on their field. But that's not really important. Multimedia is not that difficult technically. Of course because I'm working on that but it's mostly about knowing a lot of those quirks. Knowing a lot of those formats and have those subtitles. It's easier to code on VLC or even on some codec than working on a kernel, for example. Anyway, VLC. So VLC is a media player. It used to be the video line client. Usually when I go and introduce myself and say, well, you know, I work on VLC. No one knows about that. Yeah, yeah, you know the code that play videos? Oh yes, that works every time. It's insanely popular. The reason the code was selected is a very old story of the call Central Paris that I can't really explain on a microphone. But yes, there was a bit too much alcohol in this story.  Everyone knows, I mean, no other media player has a cone. Everyone has just a play, basically a play button somewhere. So that was a great marketing idea by people who had no clue what they were doing. VLC is known because it's everything. Mostly it was, so in the beginning, of course, it was only on the right platforms, Linux and BOS as I was saying. But VLC integrated all the codecs on Windows and so you didn't have to download something else. And even if you were downloading something else, it wouldn't change VLC. Everything is self-contained inside VLC and that made it popular. So we play most of the formats and we're working on that. So VLC tries to play everything and also tries to play it everywhere. Everywhere including OS 2 because the last version of VLC has finally been fixed on OS 2. The three developers are very happy. They don't have any users yet. But also we are now on Android and iOS. We're working on Windows 4 and everywhere. And also, of course, on Linux, which was an original board. So how big is VLC? VLC is around one million downloads per day on our website and that's not counting, of course, any Linux distribution. It's not counting every download.com and all those other places. When we started counting in 2006, since then we've had some two billion downloads, which is kind of nice knowing that we are a very small team. So VLC can play audio CD, DVDs, Blu-rays, lately, mostly non-encrypted, but if you're smart you can have them decrypted also, network stream, external hardware for the DVB satellites, the TV inputs, also the cameras. So basically everything that you could play we tried to make it playable. So I'm going to show you a few historical viewers of VLC. So here you can see the first DVDs that was played with video land client as you can see. It's, of course, a GoldenEye DVD and that's the reason why all of the code names of VLC until 1.0 were from GoldenEye characters. It's a GNOME, I don't know, one something maybe, or something very, very old. And this one is, I think, the GTK UI we had in early GNOME 2. That is, of course, amazing design as usual. But what's interesting is that it's basically a playback of a multimedia TSTream Direct Live. This is the first UI we had on Mac and you still see it said our video land client. This is the version that is based on Wits that was the most known and was the one that most people use when VLC became popular. It was great and stayed a long time until we switched to Qt in 2008-2009. And, well, VLC runs also on the GNOME 3 and it's the best thing because you know, ponies. So it's important to say that we are mostly volunteers and very, very, very large part of us are volunteers back on our free time. So we are really on the bazaar side of the scope even if we have one of the most used software for normal users. Since 2009, we have now a nonprofit organization that is behind who gets a bit of donation in order to send people to LCA. And that's the main goal, of course. We organize video land dev days, which is a big conference about multimedia where most of the people from the multimedia community, most of the people from FMPEG, LibAvy, X264, come by. Everything we do, basically all the collaboration is done, of course, online since very early. We've moved to Git in 2007, so a bit before everyone else except the Linux channel, of course. And everything we do is on IRC, and mainly if you're not on IRC, basically you're not in the project. And we have a few simple rules. The core team of VLC is between five and 10, depending on how you count, or maybe three if you count differently. But we have a lot of these contributors arriving, 150 per year, they come, they give us a patch, and they go away, which makes it a bit weird on how we accept code. The way we accept code is not based on is this feature really useful, but it's mostly, is this feature going to be maintainable? Because of course everyone comes and say, well, I'm going to maintain this patch. Yes, sure. I'm going to stick, yeah, technically no one does. So we have absolutely no marketing team, almost no legal team, but what's important is that the technical code is correct and standard, and people who code more have the more power in the community. Most of the time we arrive through consensus, and sometimes we offer some people to fork, but so far there is no important VLC fork, except BIOS. We use Git in a SVN way, which is we always rebase before committing, so we don't have a lot of merge commits, and we have basically a master that we break and commit to and a stable branch where we cherry pick commits. I think that's pretty standard. So one of the main question is why is VLC, why did it become so popular? And there are weird reasons, but there are also technical reasons, and the main ones are because it was modular, because it was network oriented, and because it was written in the same language. So why is the networking oriented important? Mostly because, well, when you were in the 2003 and 2004, and you were using Emule or Idonki, of course no one of you were doing that, I'm sure, and you started to download something, it would take, I don't know, half a day or a day or more, and if you try to play it with any other player, it will just complain that the file is not complete. But as VLC was developed as a network engine, a playback engine, it would just like ignore and try to go directly. So for example, if you were downloading a Disney movie, like only 50 megabytes or maybe even less, you were able to see if it was actually a Disney movie and not an adult one, or if you were looking for an adult one, then it was not a Disney movie. So that's quite important, and that helped a lot in having VLC used. The other reason, it was mostly on macOS, that it was the only way to play DVDs for a long time. So, VLC does not exist, and I'm sorry to say that, VLC is a 200 lines of code wrapper around LibVLC, which is our API to basically create media players. LibVLC is a very stable API above the core, and everything else is done in modules. This module design is one of the reasons why VLC also was popular, is that when you want to do a cross-platform video engine, for example, you're going to take, I don't know, one idea is to say, well, I'm going to pick OpenGL, because OpenGL works everywhere. Well, if you ever use OpenGL on Windows or even on some Linux card, you know that's not the case. So having different modules is that you can take the best technology on every platform. This is why we're able to maintain an OS2 port without destroying the code base, because basically it's modules that no one cares about, except the OS2 maintainers. This helped VLC to spread on numerous platforms, numerous platforms, and especially on non-open source ones. It's also good because it helps the code maintainability, because then most of the units are 1,000 lines of code, more or less. So the core, LibVLC core, yes. As you've seen, the project is very bad with names. So it was video land client that became VLC, and then everything is LibVLC, LibVLC core. So the core is quite light. It's basically doing the operating system abstraction. It's able to do memory allocations, mostly networking, thread handlings, and loading modules depending on the platform. It's also able to do clock synchronization, which is basically synchronizing audio and video together. Everything else is outside. So the core of VLC doesn't support any codec, doesn't understand what a codec is almost, and is quite simple. Above that, we have a stable API. So LibVLC core is not stable. We break the API and the API quite often. But above that, we have the simple multimedia framework, LibVLC, that is done to actually write applications. And then we have bindings for all the major languages, especially because we have a binding in SWIG where we can generate them for other people. To give you an idea, in an usual VLC installation on Linux, you have 300 modules that goes from codec to IO, video output. And if you want a new feature or format of, I don't know, Commodore 64 mod chip files like people did, you just add a new module and submit it to the play. VLC is around 820,000 lines of code. So the core is 100, 120,000, and everything else is inside the modules. What's also quite interesting is that the core depends on almost nothing except libc and iconv. But the modules depend on a lot of external libraries, including LibVl codec, LibVl format, and so many others. When you compile the whole VLC for Windows or for a number of Linux, usually it's 10 to 12 million lines of code that you build. The VLC on Android version is a bit less because we don't have streaming, so it's 6 million and a half. Most of it is C, like 50%, and then the rest is C++. And assembly. So VLC's code is in this amazing language called C-minus, or call it as you want, some C-object, which is basically abusing C in order to have a kind of object abstraction and having pointer-to-function objects that are pointing to other opaque structures. And that's how we do different modules, but it seems to work quite fine. Most of the modules have their own private data on each session. And this is mostly how it works. A lot of media players are written in C++, for example. And when VLC was starting, the C++ compilers were more limited. And when we see, of course, when we compile C++ modules in VLC, they are very important. So having that helped also on importability because ports were done without C++ at all. So how does technically VLC does video output? It's a network-oriented graph that is built at runtime when you want to play something. This is quite similar to what G-streamer, QuickTime, Directional Media Phone is doing, is based on what is probing and from the stream going to ask for different modules. So the first part is usually the protocol, HTTP but also file or DVB. It's basically a way to get from a URL, get a stream, a row stream. Then you've got the format, which is basically the thing that is going to split the video and audio tracks and subtitle tracks. And then based on that, it's going to have a video decoder or audio decoder, subtitle decoder, that are going then to video filters and then to the output or for audio filters, the same audio filters and the output. For subtitles that are text-based, we usually have something, a text renderer like free type and then we blend it on the video output in order to display it. But also with VLC, because it's full media framework, you can re-encode directly and re-stream, remux and re-stream if you want to use VLC to do streaming. As you can see, interfaces are also modules in VLC, which is something that people usually don't expect. And interfaces are a module of LibVLC Core and it's not the interface that is built on LibVLC. There is a small difference from other media players and other multimedia frameworks is that if you look closely on the left, you're going to see that the first two arrows are basically not in the right direction. Basically, it's the format, the demuxer, that is going to ask for more data from the protocol layer, saying, hey, you know what, I need data and is going to ask it data. So for example, when you go on YouTube, it's basically doing a lot of stuff on the server side that you get most of the data soon and then going to throttle your connection. If you're going to launch a YouTube video and you have lots of connections, what's going to happen is that you're going to download almost everything without playing back and so if you close, you lose a lot of resources. On VLC, it's not done like that. Basically, you have for every protocol a set caching that is done in milliseconds. So for example, for a file, we're going to need 300 milliseconds or for HTTP, two seconds. As soon as we miss data, we're going to ask the protocol layer to get this fixed caching. That's interesting because it helps saving resources and network resources. I'm not sure people actually care about that anymore, but that's something. So how do we do module probing? So every module has what we call a capability. Basically, it's the category of the module. It's going to be an access, so protocol, DMAX, format opener, codecs, or filters, mainly. And it has a capability and a score. So what the core of VLC is going is that it's going to ask for all the modules of the right capability and in decreasing score order and then it's going to run the probe function and check if it's yes or no. If it's yes, then fine. If it's no, it'll go to the next one. And that's pretty simple because you can add new categories if you want to have some special modules. But the problem is that as you can see, it can be quite slow, which is why we have a module cache in VLC so that when you're going to open the codecs, you're not going to open every module but just the right ones. So VLC is almost a full Multimedia framework. So why do I say almost is that in theory you could do everything that all the Multimedia framework in practice, the API so far is mostly done for playback. So basically, you want to play a video inside an application or you want to do a different media player based on VLC. So it's mostly playback, filters, all the control you want with LibVLC. But, well, there is very limited streaming. We've done DVD reapers and CD reapers based on LibVLC. We've done subnailers also for LibVLC, so in Nautilus or in GNOME. And it's also directly used in the port of VLC on iOS, on Android, and Windows Phone, and the others. It's also used by Phonon. We have a Phonon backend that is probably not the best one. And it's also used by XML projects that we don't know about because LibVLC is LGPL, so people can use it as they wish. So just to give you an idea, LibVLC is extremely simple to use for the main use case, which is basically you create a VLC instance, which is LibVLC new. Once again, amazing names, as you can see. And then we create one or more media players because in an application you might want to play more than one video.  We set it and we play. We have an example of LibVLC to have an almost full media playback for GTK, based on a text 150 lines of code that play, pause, stop, and even synchronization of audio. So it is done for that. It is very simple to do for that. And it's even simpler if you reflect it with Python and so. Questions? Still alive? Okay, we'll see. So now I'm going to show you a few stuff that you don't know about VLC that are quite funny. For certain definitions of funny, of course. VLC is extendable in Lua, so all the internal APIs are mostly exposed, so you can do and sharing on Twitter, and also to do parsing of websites. So if you put a YouTube URL inside VLC, VLC is going to act like a web browser that just is able to play video, and it's going inside and then with a Lua and some regex is basically going to give you the URL and play it without destroying your machine because you're still using Flash. VLC also has something called service discovery, so basically you can find playlists, or it's really used for UPNP, Samba, but also SAP. And also you could do playlists directly, and it can give you some stuff like that where you have a repository or video of podcasts and you integrate directly with VLC in crypto. VLC can do screencasting, which is completely useless because VLC screencasting is very bad, but it gives you the unique opportunity of doing this joke that is used, but everyone loves, which is an infinite VLC. So every, I don't know, every six months there is some guy on Reddit or Twitter doing that and then retweeted everywhere. It's useless, therefore it's great. We can do Mosaic in VLC, and that's important because now you see that VLC is not just a media player. Because to do a Mosaic, basically you need to have not one input, but here 20 inputs that are decoded in 20 different decoders and then merge at the display time and then you can even, instead of displaying, re-encode that. So people do Mosaics directly in VLC instead of buying some, and that's by default in VLC and you don't need to code anything to get that. We can do, of course, picture-in-pictures or world filters, so that's really cool if you want to not have big screens and not spend too much money and you have just a few HDMI TVs. We play karaoke because why not? We, of course, play media because it's the most important format on Earth. We have, of course, a console and curses interface because, well, we live in common line, right? We need a UI. Actually, I don't, but it's still using X11 or OpenGL to output, so if you're over SSH, well, you're going to use Lipkaka directly, which allows you to be able to watch a movie directly on the server through SSH. Amazing, very useful. I want to end this as key art. We have, of course, a web interface where either you can control and have it playing locally or re-stream and re-encode in video HTML5 or VLC-compatible format. And so many other weird features that we have that are useless but great to use. Especially in the next version of VLC, we have a VHS filter. The idea of the VHS filter is basically you take your good video that full HD and then you add artifacts and random problems on it to remind you of the VHS times. We also have a puzzle interface because, well, basically you're watching a movie and it's boring, but you want to watch it so you can actually click and move the... I can show you after if we have time, I'm not sure. A bit more serious, we've spent a bit too much time lately on VLC and Android. So VLC and Android is completely open source. We work and support from 2.1 to 5.0. It has all the format that VLC does except it doesn't do streaming yet. And the biggest difference is it has an actual media library for audio so you actually can use it for audio while on the desktop, not much. Porting to Android was difficult, mostly because the Lipsy of Android is very bad and because instead of taking one that was already used, they decided to rewrite one and they had no clue what they were doing. So there is no good way to output audio on Android yet. It's very difficult to output YUV in a standard way on Android. But after almost two years, we are finally able to... So we released something in July 2012 as a beta and then it quickly evolved to be less ugly. That was the version that was quite popular and now we almost finished the... Google of two years ago and they changed it again. The good news is that we already are on the next design. Yes, so... Yes, we have a lot of... I think we have three users on Android, which is quite nice. We also have a port on iOS, which is bad but quite interesting because it works faster than Android even without hardware decoding. And lately we've been porting to VLC on Windows RT, which is the Metro UI. We don't really care about the Metro UI, but what we care about is mostly Windows 1. Future projects? Well, one of them is the port on Windows RT, Windows Phone, I was saying. We are not going to work on Firefox OS because porting VLC to JavaScript with no thread is not... VLC for Ubuntu Phone, or maybe it's called Touch Now, we don't know if there is anything actually to do. BlackBerry probably not because they can now run Android application directly, and for Chrome OS we hope to do exactly the same. So, almost my last slide. Should contribute to VLC and to libvlc. The first thing is to use it and promote it and say it's also... I didn't put it... to your friends, report bugs because, you know, there are lots of... send samples. And also code on VLC or use libvlc in your next project. Thank you. I think we have five minutes for questions. You want? No one wants. This is the way. Is there any traction on... So, we have a Chromecast streaming module in VLC. So, the problem is that the... the goal of Chromecast is basically to play something from the cloud, right? So, it's a full... So, it's basically... the Chromecast is going to pull and it's really not done for pushing to it, right? So, for VLC and Android it's very simple. We know where the file is, we could just send it, right? But then it's only the compatible part. So, what we did is a full streaming module that is merged in VLC 3.0 and that is able to push and re-encode all the time to the right format. However, because of that, when you seek or when you pause, basically on the Chromecast you're posing on the Chromecast, right? And so, the streaming is still going on. So, when you resume, there is a lot of issue because of the timestamp that should increase. And, of course, and also the Chromecast is very broken in the type format they support. And so, it is coming and you can try it already. It works, but I'm not sure it's actually working for like normal end users. Okay, thank you. Other questions? No questions? Yes, sure. By C minus minus, is there a separate place where that's documented or is it just a set of conventions for the way that you guys do object stuff? No, it's mostly an internal joke. It's not really standardized. I know a lot of projects are doing it. A lot of the older VLC developers were really against C++ but the way it's done is mostly a funny joke and it works fine for our use case, which is to have like a minimal abstraction for modules where you don't know what's inside without having a big inheritance and so on. And in terms of hardware acceleration, is there much in the way of the module support for 2D and 3D acceleration? You mentioned GL, but it's not everywhere. So what do you mean by acceleration? Well, I just mean... I've been at a talk recently about OpenCL and stuff, so I'm just always interested in why it's offloading, CPU load. So when we speak about... So for the Militia Media Bear project, when they speak about hardware acceleration, what they want is decoding acceleration because for a long time we already have... we already have hardware acceleration for the display, which is direct 3D or OpenGL or X-video, which are basically...  But the biggest discussion lately, especially on Linux, where we have two or three different APIs, is hardware acceleration, which is a decoding part. And the problem is that most of the codecs are very difficult to do in OpenCL, for example. So usually it's kind of DSP parts directly inside the GPU. VLC supports hardware acceleration or all platforms, but on some platform it does it in a bad way, which is it sends the data to the GPU, ask it for decode, and then gets the data back from the GPU, do the filtering, and then displays it. But most of the time this get data back is very slow because it's not what you use to do for GPUs.  And also it's doing mem copies, and as we say in multimedia, mem copies murder because it's basically murdering your CPU. So what we do is what we call the full GPU acceleration where you send the data, you made it decode by the GPU and then you're going to control the GPU in a different way to when it's going to display it. And we have that but not in all platforms and not all the APIs. It's going to get fixed for 3.0 for all of them. Yes, maybe I can show you the puzzle if I find it. Again, I'm still taking questions if you want. Yes. No, no questions. I hope it's going to work, and no it's not going to work. So of course I broke my tree because it's not funny. And yes, okay, so I don't have a working VLC, which is not that surprising. I don't watch video anymore, I think. I have never time, and every time I got it, I break it. Oh yes, I do. Yes, so then I can do that. And this is probably going to be around here, right? And yeah, it's supposed to be an easy video and there will be something written on it. But oh yes, I got a corner. So yeah, wow, I got one corner, right? And then what? Yes, we're getting there, right? So anyway, completely useless, but therefore completely necessary. If there is no questions, I will stop on that. Yes, there is one question. Yes. How do you go about getting add-ons like that? This is a video filter.  And then there is some kind of bezier thing to draw it. But the code is open, so you can check it out and I can help you to improve it. Thank you. And on behalf of LCA, we just have a small token. Oh, thank you. How do I disconnect that?


We should just put two pieces of salami right up there for SRK's nipples. You think so? Yeah, why not? And then right underneath it, big beast tasting nuts. Why don't I just print out pictures of SRK, there's ample pictures of his nipples. That's true. Do you think there's another actor who's had their nipples exposed more often on film? Kim Kardashian. That's true.  Kim Kardashian, greatest actor of all time. Right, folks? It's true. You can follow us on Instagram, Twitter, Twitter, Twitter. Thanks on Patreon, follow us on Twitter and subscribe if you haven't and hit that notification bell. Speaking of Shahrukh Khan, we got a new ad from Shahrukh Khan. What? Yes. Another ad? Well, no, this is not, but this is Dubai. We've actually seen him long time ago. He didn't, because he lives in Dubai a lot of times. Yeah, he did a come to Dubai. Yeah, yeah. This is another one, but it just released, like just yesterday. I'm going to guess not just because I see the beginning of the thing there. I have a feeling he's going to spread his arms wide. He's going to spread his legs. Wow, and thumbs up will come out. What should be our next Shahrukh Khan film? That's not highly contested. Not at all. Here we go. That's cute. Dubai presents Shahrukh Khan into Dubai. There he is, arms spread. Cuts. Great job, Shahrukh. Was that good? Man, thank you, Arish. All right, guys. Pack it up. Hi, baby. Yeah, thank you, hon. No, no, no. I'm not at home. I'm at the shoot. Oh. Yeah, and I have a long day. How boring, Dad. You're in Dubai. Walk around, have fun. That sounds like a good idea. Dad, I love you. Love you too. Just walking by Shahrukh Khan. There we go. Shahrukh Khan was cute. Shahrukh Khan sent it. Yeah. That game, great. Yeah. Amazing architecture. Nobody does it like Dubai. Like Vegas on points. Yeah, pretty much. I'm back home now. How was your day? Did you have some fun? Thanks to you. I had the best day of my life. It's hard to have a better spokesperson. If I go to Dubai and I don't see SRK dancing with me, I'm suing for false advertising. That's true. I would like to dance with SRK. That was a long ad. Well, it depends on how much you're paying Shahrukh Khan. You need to get your money's worth. That's true. Get your money's worth. And we know we've done a lot of ad stuff before. A lot of the advertising things we've seen are almost like short films sometimes. And that was, if the job for that was to highlight the beauty and the excitement and fun of Dubai, they did a great job. And they have a pretty good spokesperson. I don't know what other cities have a better spokesperson. I mean, I guess we've seen some that have a big B a while back that he was the spokesperson for, but it's hard to have a better spokesperson. True. Well, it's interesting because he is a fantastic spokesperson, but in America, people would be, a lot of them would like go, why is he the spokesperson? It's like, well, because he's an international phenomenon and you live in a cave. Just me, I know. I live there once. It was a nice cave. It's a nice cave. It's a little bit damp, a little dark. You pretty much can't get a better spokesperson. I love his hair. I could tell. I love his hair. And I could tell. Like, especially in, where is it? I know you like on the beach. On the beach. On the beach. Oh, yeah. Look at that. And obviously that's for Pathan. Is that what the film was called? The new action one. Yes. His new one that's coming out. Also, 2029. Did you guys know that 2022 is the year of our Lord and Saviour Dabigabadakum? But she has like five films coming out this year. She has that one. The one we saw. But then she's got, oh, and I know this one's coming out next January, but then she's got one with Rithik coming out. Hey, Fafa, get working. Slacking. Come on. Film a month. No excuses. One a month. It seems like, maybe it's, maybe I'm just reading into it. It seems like Shahrukh Khan's coming back into the limelight a little bit. Sure seems like it. After his hiatus. Yeah. Yeah, it sure seems like it. Yeah, we've seen like a lot of, it's been working a lot right now. Promotion. Yep. A lot of promotion stuff. Maybe he'll start doing interviews. We know you come here sometimes, Shahrukh. We live in Los Angeles. We'll come to your place. No problem. I mean, it's probably better than probably. Maybe. A little bit better. We'll see. But, yeah, we'll judge. And if you'd like to come and see a little later, please do. Anyways. Yeah, let us know what should be our next Shahrukh Khan film because nobody ever asked us to do Shahrukh Khan films. Ever. Very rare. And I know a lot of people on us do like watch alongs, and I'd love to do a watch along at some point. Unfortunately, a lot of his films are about three and a half hours long. And I have a toddler. I can't leave my wife alone with a toddler for that long, guys. So, like, maybe if it's one of his shorter ones. Or, also, my wife loves Shahrukh Khan. Like, whenever we watch a Shahrukh Khan film, she's like, I love Shahrukh Khan films. I think they're so, like, it's like we talk about, they're just enjoyable. Yeah. They're almost like rom-coms. Yeah. They're enjoyable to watch. Yep. Maybe we'd have to, like, come over after Lee then goes to sleep. Yeah. And all. Because people ask us for watch alongs all the time. All the time. But also, there's, there's, if it's a three hour film, right, I can probably only use five minutes. Absolutely. Have fun editing that. Those don't sound fun. Anyways. Yeah. But yeah, let us know what the next Shahrukh Khan film should be.


Statistics and Excel, binomial distribution, multiple x drive to work in traffic example. Got data? Let's get stuck into it with statistics and excel. You're not first a word from our sponsor. Yeah, actually, we're sponsoring ourselves on this one because apparently the merchandisers, they don't want to be seen with us.  Like our, trust me, I'm an accountant product line. Yeah, it's paramount that you let people know that you're an accountant because apparently we're among the only ones equipped with the number crunching skills to answer society's current deep complex and nuanced questions. If you would like a commercial free experience, consider subscribing to our website at accountinginstruction.com or accountinginstruction.thinkific.com. Required to you, but if you have access to one note or in the icon left hand side, one note presentation, 1570 binomial distribution, multiple x drive to work in traffic example tab. We're also uploading transcripts to one note so that you can go into the view tab, immersive reader tool, change the language if you so choose, be able to then either read or listen to the transcript in multiple different languages, tying into the video presentations with the timestamps. One note desktop version here in prior presentations, we've been thinking about how we can represent different data sets both numerically with numbers such as the average or mean median quartiles and pictorially using tools such as the box and whiskers and histogram, noting that the histogram is generally the tool that we visualize when thinking about the spread of data and then we can describe the spread of the data on a histogram using terminology such as it's skewed to the left or it's skewed to the right. We're now looking for those mathematical equations representing curves that sometimes can approximate actual data sets in real life depending on the scenario and if they can we would like to be able to do so because those curves can give us more predictive power about whatever it is the data set is working with. This time we're going to be looking with a binomial type of distribution. We looked at the conditions for a binomial distribution, one of them being that we would have to be able to organize our information into a success or fail situation such as a coin flip, having a success or a fail defined as success possibly being heads fail being tails. We talked about a sales call situation where for every call the success would be if you got a sale, the fail would be if you didn't we're not talking about gradients of success in this case right we can't really we can't say well we got a sale of a 200 item versus a hundred dollar item we have to define it as either success or fail. So there's a lot of other scenarios where this could take place which might not be obvious as first so here's another example n equals the number of fixed trials which are going to be drives to work so we're driving to work we're going to say five times that's going to be our fixed trials the probability of success meaning success in this case we're defining as no traffic so when we're driving to work if there's traffic it's not a success if there's no traffic success now again there's no gradients here we're not saying well there's a medium amount of traffic versus a high red zone traffic versus a yellow zone traffic no we're saying either binomial traffic no traffic no traffic success traffic not a success and unfortunately there's only a 12% likelihood that we're going to have a no traffic situation so then x is going to be the number of times no traffic out of five so in other words we're trying to see what's the likelihood that we get three drives to work with no traffic out of the five drives to work which is going to be fairly low you would expect given the fact that 12% likelihood per drive that we're going to have a traffic free experience okay so if we were to do our binomial distributions x equals the number of times no traffic out of five so here is our equation now remember there's actually two of them with the binomial which is a little bit different than some of the other distributions like the poisson distribution the latest one as we talked about before is this range one which which has more flexibility but it also could be a little bit confusing sometimes so it might be easier to use since the other one is closer to the poisson the other one so we'll look at both of them so this equals binome.dist.range and then we're picking up the trials which are going to be five the probability per individual trial the 12 probability of success and then the number we're looking for is three out of the five that's what these numbers represent and that gives us our 1.34 percent the likelihood that we get three days out of five that have no traffic given the fact that each individual day we only have a 12 success rate meaning no traffic is only the 1.35 we need a new job for crying out loud or maybe move closer or something it's just ridiculous i mean i'm wasting half my day in the car i try to listen to tapes but it's just so it's hard to focus anyways the number of times so we can also do it using this formula equals the binome.dist so this is similar to the to the poisson one where we have the numbers is a similar argument up top the trials that we have similar argument but then we've got the cumulative versus non cumulative so we got the numbers we got the trials we've got the probability and then we've got this last bit over here which is cumulative or non cumulative now that allows you to say ask a question such as what's the likelihood not simply that i get three but from anywhere from like zero to three or something like that well it'll take a cumulative impact but it's more difficult to pick something in the middle right whereas the binome.dist.range allows you to have the multiple arguments on this last argument so it can allow you to do a calculation of something like in the middle like two like if i was to say what's the likelihood that i have like two to three days so i will give a little bit more of of an explanation of that here so let's imagine that we're saying that p is equal to two so i could use the binome.dist.range which has the arguments the trials the probability and the numbers the number now being two out of the five instead of the three out of the five we could use an argument of the binome.dist argument which now it has the two up front i'm just typing in the two because the order of the of the arguments are different orders but it's the same basic arguments and then the trials and then the probability and then we have whether it's cumulative or not in this case we're going to say it's not cumulative because we're looking for exactly the two so those two both of them are fairly equal in terms of the in terms of the ease of the argument and we can also if we were to plot out our data this way we can this is often a useful tool to plot out the data meaning we can say what if the chances of x are being from one up to five and then we can plot out our data and for this we're using the binome.dist.range you could use the other binome.dist as well but this range function gives you an array function which makes it a little bit easier to enter something like this because to it enter this into the system we're going to say that the number of trials that we have is going to be once again the five i made it absolute so we can copy it down to the range the probability of success is going to still be the twelve and then we have the numbers which is going to be this array that's what this stands for with the hashtag and this will just be a spill array which will give us the results so the probability that we have zero days is quite high so because remember that we had a 12 percent likelihood that we don't get traffic so it's a high likelihood that we do get traffic so that means that to have zero days of traffic free a traffic free week is not is still not it's likely that we don't get any traffic free time and so and then what if we get one day of traffic free day thirty five point nine eight percent two days of traffic freeness nine point eight one percent three days one point three four and so on if we plot this it looks something like this and if i was to compare that to to calculating just for the for p equaling two i can then say i can then find it right here right p equals two which is this nine point eight one all right if we do another one i'm going to say that this time p is going to be less than or equal to two okay less than or equal to two so we do that a couple different ways i can say all right i can look up here and go less than or equal to two meaning i'm going to include the two so i could say well it should be five fifty two point seven seven plus thirty five point nine eight plus nine point eight one ninety eight uh so we get to that that ninety eight fifty six that's the one way we can get there that's the result or we can say binom.dist.range and we enter the trials the probabilities and then uh the numbers and now i'm picking up two numbers and this is going to be the more flexibility of the.range i'm not saying it's cumulative we're we're picking up those two numbers and we could list those numbers this way if we were to list the upper and lower limits the lower limit now is zero because it's that's as far down as we can go and the upper limit is up to and including two and those and that's what's being included in this argument we could also use the binom.dist which still works pretty good because i got the numbers the trials uh the probability and then we have to have the added argument to say do you want this one number or do you want the cumulative in this case we're going to put a one which represents the cumulative which basically does the same way thing we did up top which which sums it up that way so what if we had an argument of x is less than or equal to one well if it's less than or equal to one i could use my summing format i can go up here and say all right well less than or equal to one means it would be up here of 52.77 plus 35.98 likelihood which comes out to 88.75 so we can use that method and or i can use the binom.dist.range trials probability and then the numbers and now we've got the the two numbers that we're putting in place and if i look at it in terms of my ranges here it's going to be zero goes down to zero on the lower limit and goes up to and including one and then the cumulative still works quite well here because now i can just say it's the numbers the trials the probability and then i'm going to just say it's cumulative instead of zero which is which is non cumulative so both of those seem pretty comparable but but then so now this one if i go x is greater than or equal to two well so now i've got to say okay if it's greater or equal to two now i'm going from from two down so that would be you would think 9.81 plus 1.34 plus 0.09 and then this one is too small to count so 11.2425 about and so i can do that i can do that with that summing way i can do it with a binom.dist.range the argument being the trials the probability and then i'm looking at this range which sometimes is nice to put in a table the lower bit of the range being two and then we're going from two up to five now here's where the the older one runs into is a little bit more difficult right because now if i say well what if i do that with the same format that we had with the poisson which has this cumulative argument instead of being able to put these two numbers for x right what how am i going to do that because i want to go from two up to five and the cumulative only goes from zero up so what i would have to do then is do the cumulative thing up to five and then subtract out the cumulative up to two or one whichever one i'm trying to include here so in other words what am i including it's it's uh greater than or equal to two so greater than or equal to two so that means the lower limit uh is two and the upper limit is going to be five two is going to be included so you have to do something like this if you wanted to just calculate it with that old binom.dist you'd have to say okay binom.dist numbers trials probability and then cumulative minus binom.dist to subtract out that first half of the curve uh and you got and so this is where it gets a little bit more messy and this is where that range function or one area that range function works well let's do one more here and then we'll take a look at another format uh hold on i deleted the wrong thing let's see let's delete this all right so now it needs to be x is just greater than two not greater than or equal to just to notice that subtlety here because now we're saying it's not the lower limit is at three because we're not including the two so from three up to five so if i did that up here i can say okay so it's greater than two but not equaling two so that means it's going to be the 1.34 plus 0.09 or 1.43 about and i can do that by adding it up i can do that by the binom.dist.range where the last argument are going to be the last two arguments are the three and the five or if i use the old format again it's a little bit long because i'd have to say binom.dist up to the five minus the binom.dist to to take out the first bit up to up to and including the one but not the two right so it gets a little bit messy so we do that in excel if you want to practice doing that in excel note also the the binom.dist.range also has that spill that spill capacity so if you were to set up your lower and upper ranges for a set of questions like this then it becomes quite easy to use the spill feature in a binom.dist.range as opposed to the to the cumulative because you can then set up your lowers and your uppers you can go binom.dist.range put in your trials your probabilities and then and then the numbers are going to be these two ranges so now i just for the numbers instead of having two and two and zero and two i select this range and then this range and that will then spill down the results so that's another reason why this binom.dist.range is a little bit more flexible but oftentimes to me i like to build something like this and and and you can do this fairly easily with either of the the functions so so that's another example and just to note that this example might not be as intuitive right but but because traffic example versus a coin flip probably comes to mind when you're talking about a binomial kind of thing at least it does for me but there's a lot of things that you might be able to put you know that you might actually be able to put into a category of success or non-success and have a similar type of scenarios


This story really demonstrates everything wrong with the Democratic Party, so Joe Biden, the president of the United States, the most powerful man on the planet, for those of you unaware, just signaled to folks that he can't do anything about gun control. Come on, man! As Alexander Bolton of the Hill explains, President Biden said Monday that it's up to Congress to outlaw assault weapons and strengthen background checks for gun sales, I can't dictate this stuff. Quote, I can do the things I've done and any executive action I can take. I'll continue to take, but I can't outlaw a weapon. I can't change a background check. I can't do that, Biden said, after stepping off Marine One on the south lawn of the White House. Asked if he is optimistic for getting a deal to address gun violence after Senate Minority Leader Mitch McConnell said last week that he had tapped Senator John Cornyn to negotiate with Democrats. Biden replied, I don't know. Quote, McConnell is a rational Republican. Cornyn is as well, he said. I think there's a recognition on their part that they, we can't continue like this. We can't do this. Now, this is the best part. Biden and McConnell successfully negotiated several deals when Biden was vice president, most notably a deal to keep many of the George W. Bush air attacks in place and avoid the so-called fiscal cliff at the start of 2013. But the president has made clear in recent days that Senate Democrats led by Senator Chris Murphy will handle the negotiations on Capitol Hill over gun control. Now, yes, it is true that the president cannot introduce legislation. I think that most reasonable people acknowledge that he has to sign the bills that Congress passes into law. But as the president, you have power and influence, right? Where you lack power, some might argue you have the influence to get things accomplished, to get the ball rolling in a particular direction. So you can make this an issue. You can raise the salience of this issue. You can say this bill that Democrats have passed in the House is not being voted on in the Senate. So protests call up your member of Congress. You can call on Congress as president to put a particular bill on your desk. But Biden doesn't think he has any power as president. Like I'm not sure why he chose to run for president if he's not going to exercise the power that he has again. As president, your power isn't unlimited, but your influence is virtually limitless. You can hold a press conference every single day until something gets passed, calling on Republicans, naming mansion and cinema in your party who are refusing to lift a filibuster in order to get gun reform accomplished. But he won't do that. And I found one of the best tweets I've ever seen with regard to Democratic Party leadership. Jonathan Larson tweets, I don't think congressional Democrats understand that voters see them as the cops standing around outside the school. And that might be the most accurate tweet I've ever seen.  But sometimes they do worse. Like we're talking about Biden currently. But Democratic Party leadership in the House, Nancy Pelosi, Stanley Hoyer, Jim Clyburn, they actually were campaigning for someone who got an A rating from the NRA. I'm of course talking about Henry Cuellar. This is an anti-choice Democrat in Texas. And he also is a gun nut. He advocates for gun anarchy. He's fiscally conservative, socially conservative. There's no reason any Democrat would want to support him. The only reason why you'd support him if you're Nancy Pelosi is so that way you can become the speaker of the House. If there's more Democrats, then you get the gavel, right? But functionally speaking, Henry Cuellar is indistinguishable from a Republican. But yet House leadership chose to endorse and campaign for Henry Cuellar over a progressive woman who would want to enact gun reform legislation. So this is Democratic Party leadership. They are incompetent at best. But at worst they help out the opposition. They pretend as if Republicans are good faith actors one time. And again, they've spit in the faces of Democratic Party leaders. And you know, Democrats turn right back around and say, more please. I guess that they're masochists at this point. And I love that Joe Biden says that Mitch McConnell is reasonable. This is an individual who stole two Supreme Court seats from the Democratic Party. When you were Vice President, he wouldn't let Obama fulfill his constitutional duty as president and appoint a new Supreme Court justice. And a week before an election, they rushed through Amy Coney Barrett's confirmation after Ruth Bader Ginsburg died.  I mean, to say that Mitch McConnell is reasonable isn't just incorrect, but it legitimizes this individual who has done everything in his power to turn the Democratic Party into this toothless organization that isn't capable of governing. And it's just it's shocking that they continue to do this to themselves because you think that they'd have some like self worth. And they think, OK, we kind of look foolish at this point, right? But no, so Biden is going to leave it all up to Senate Democrats.  No, because it's not going to accomplish so long as you try to pass it with the 60 vote majority. You have to do it either using budget reconciliation or by getting rid of the filibuster and passing it with all 50 Democrats. But the problem is that Manchin refuses to call out Manchin in cinema. Biden refuses to name them and say, we could get this done, but these two members of my party, they're blocking it. Call the office protest.  And honestly, I don't know what he's even doing in the White House. Like, what are you doing, Joe? Just resign if you don't want to be the president. You ran for this job, you wanted the job and you have it now. So if you're not going to act, then I just I don't even know what to say. Just just go home. Do you enjoy watching independent news shows like the Humanist Report, the Rational National and the Majority Report? But oftentimes, YouTube doesn't deliver our videos to your subscription box. Well, I've got a solution for you. It's called the Optout app available right now in the iOS app store coming soon to Android. Optout is an app made by and for progressives where they take all of the most popular independent news shows and they put them in one convenient location. You'll find all your favorites on there like the Humanist Report, the Rational National, the Majority Report, and the app is updated multiple times per day. So your news feed is constantly up to date. If you enjoy watching independent media, this is the app to get downloaded today. You're getting nervous, man.


a nice quick break, coffee or something else. So next up we've got a session that where Oling will take us through some of the parameters, the QM parameters, in CPG input files. So actually looking into the CPG input files that are generated by the interface and going through some of the steps that would be useful to know about. Hello everyone and so yeah this section is going to focus on CPG UK parameters. So we've already had a go at using the interface and seeing how when you run the interface a CPG UK input file is generated automatically and we've taken this and then just this has gone in and run CPG UK in the background doing lots of steps. Let's run each FD step. So what we're going to look at now is how would we want to go about changing anything within this file and why we might want to change it. So we've already had a lecture this morning discussing different basis sets and potentials and exchange correlation functionals within CPG UK and the different types that are available. So now kind of going to put these two things together and look at using these within CPG UK and how you might go about changing these and investigating which ones you might want to use and verifying that what you've done is a sensible change. So what we're going to look at is sorry so yeah what we're going to look at in particular is changing the basis set so changing from using the standard basis mollups that we've seen used in in the runs we've been doing so far. I'm looking at using the B3 lip functional so in the CPG inputs we've been using so far they automatically generate with PVE. PVA is great functional but maybe if you want to look at using a hybrid functional with more potentially more accuracy and a greater higher level theory how would we do this within CPG UK and look at adding in a distortion correction so using the grimmer DFT TD3 correction and another point to cover is how we check the convergence of the grid so this involves tuning the cutoff parameter within the CPG UK input file and look at how we check for convergence of the energy and showing that you've chosen a good value. So I'm going to start off by covering just a bit about the CPG UK best practice guide and so this is something we've put together to cover QMM simulations within CPG UK so this is kind of was put together for see using standalone CPG UK with QMM on its own of a lots of it is relevant to using it with in combination with Gromax so here are some kind of the information that you can find within the CPG UK best practice guide and I've put links here stuff that's particularly relevant when you're using it with Gromax so for example if you want to look at understanding the QMM part there's a section that covers the QMM so the importance section within CPG UK it gives descriptions about all the different parameters you might have and why you might choose these and some information there it gives overview of basis sets and the types available and the different descriptions of all the different types so covering the single zeta double zeta and the polarization ones that we've discussed earlier overview of exchange functionals available the different types how you use these within the CPG UK input so the different sections that you would actually add into the file how you might do these overview of more more advanced ones so hybrid methods PB0 and B3 lip also covers some information about pseudo potentials, dispersion corrections and then an overview of important QM parameters so what these can be set to why you choose these values and how important they might be there's a bit about troubleshooting at the end so yeah that covers most of the stuff there there's also overview of how to run CPG UK so setting up what the output actually means they might have looked at this yesterday it prints quite a lot of output what is actually going on here it's useful to understand so yeah it's just something to be aware of and during this exercise we will refer to it and point to the relevant sections so part one of this exercise is going to look at changing the basis set so we're starting off with the input file this will be a CPG UK input file on its own and you'll have this input file and the PDB file which contain the coordinate information so this is kind if you've gone away and run the interface this is typical input file that it might generate and we're going to look at tailoring options within this input file and changing these so the first step is we're going to look at changing the basis set from this basis mall ops basis to the hfx basis so we're doing this here in preparation for the next part of the exercise where we're going to be looking at using B3 lip when you're changing to different functional and look particularly hybrid functions you might want to look at changing the basis set in combination with this so particularly using basis mall opt with hybrid methods it's quite a large basis set and if you try and do this your calculation will be really slow especially if you've got a larger number of qm atoms so yeah this is why you might want to change it to the hfx basis so you can find the download the input files from these locations here and yeah get set up on archer to log in touch to again and we'll start running some of these calculations so key points to change um you'll have to change the basis set file name so this is the file name for for the basis acts and i'll show you where these files have come from and what's in them in a second and then look at changing the basis set option um within these as well so this is given for each element so your key line here is the basis set file name which is here is basis mall opt i want to change this to hfx basis and uh further on down in the input file you'll find a different all these sections for each of the different elements and here your key line is basis set um so here which we want to change to tz vp this tz v to p gth so this is a triple valence potential with uh two polarization functions so here are the instructions um i'll just switch to sharing my terminal now because i think that's gonna be more helpful okay can everyone see my terminal window yep great um so here i have all the files we're going to need for the start of this exercise so if you need any of these you can get them from the commands shouldn't exercise so just first go and grab those um and then the job script i've also placed in the practical instructions so you can just copy and paste this into a text file so yeah this let's just take a look at our input file so at the moment this is the typical one generated but when you run the interface so you can see you have the basis small opt your potential name there um pbe used as the exchange function exchange correlation functional and you put your cell your qm atoms listed already which has been generated nicely and your link atoms and all the information there so yeah the key line you want to change is the basis set file name which is found towards the top and then these within these kind sections you want to change the basis set itself so yeah i was going to explain what are these basis basis set files so they are actual files um if you've built cpgk correctly they're included from the data directory so if you know where cpgk is installed on your computer so in this case it's installed in this directory here um workshared work y07 shared cpgk cpg8.1 there's also a data directory within this and if you ls this directory you can see all the available basis set file names so you have here your basis small opt and your hfx basis and yeah not only basis sets but also the potentials are stored here um some different parameter files so there's also the parameters needed for dftd3 for the dispersion correction so these are all stored here um and if you ever want to see what available basis sets there are for a particular element within a certain file you can do like a a grep search for that element so for example for hydrogen you want to check for hydrogen within these files um or a particular file shared cpg8.1 data and then your basis small opt uh this gives you all the different basis sets found for your hydrogen um and you can see here the dzvp basis small opt that we're currently using and it's the same for usually the same for most of your common elements they'll be the same available ones so for carbon for example and then yeah for our change that we're making in the hfx basis oh it didn't work as i thought um okay maybe i've just searched for the wrong thing but yeah you can have a look at all these files and they're pretty useful to see what basis sets are available yep can you see the questions or can you see the chat i mean oh sorry yeah i'm looking at the wrong place okay uh okay uh yeah so for a copper to use these even people copper do i need to use these uh more like to say the short range one basis set um so for copper let's just have a look at the available ones quickly i've forgotten my elements copper means to see you here yeah so your option there so a lot of the metals only have the short range um marked basis sets available so yeah this is probably your best option so yeah that's our chance for short range um cool so yeah i suggest if you're if there are any other questions um people go on to looking at that exercise so just it shouldn't take you too long just make those changes within your input file and then go ahead and run it so i'm gonna do this here um you should be modifying the basis set file name just changing the basis set at the moment okay so that should be running so yeah just raise your hand once you've submitted the calculation because we'll go through looking at the output together and just see what you care like or cut spit out warning messages when things have gone wrong and conversely a job completed successfully message um so yeah it does spit out warning messages um you have to be quite careful sometimes it spits out warning messages and they're buried within a lot of text and you need to find them so sometimes it will do a lot do a full crash and it'll be very obvious that things have gone wrong um other times yeah there are warning messages buried within quite a lot of text so you can usually do a search for the keyword warning um um when it's completed so we'll look at this at the second but it prints timing report at the bottom um so you can always tell if it's run to full completion but yeah you don't have to be quite careful sometimes if particularly things like scf not converging um can kind of get buried particularly if you're running like quite a long md calculation okay cool yeah so some people have done this um and yeah there's a very small difference in the energy so basically this step is a preparation step towards the next exercise so if you've run this calculation mine's not done what i wanted um oh i've made a typo that's great um what did i miss oh i didn't change the basis to mallot from mallot right did i change the line i did great so this was just potential let's try that again yeah thanks for noticing that's quite easy to mistake mistakes like that which explains why people could work confused about faces set by an arm to potential i know actually what's wondering about that yeah uh okay hopefully okay brilliant this looks like it's worked um so yes we've got our cpga output file that's been produced and we've also got these gromax restarts.wfn files which have also been produced so these are wave function files so these basically can be used to provide uh like a starting guess for an scf run if you're then to run this calculation again um so these files are actually like quite important for what we're going to be doing for the next step when we use to use uh move to use in b3 lip we're going to take these files and use them to provide initial guess and this will be really helpful because they can change the exchange correlation function and these will help really speed up the calculation so reading these files is always a good thing to do um yeah regardless of what you're doing it really helps with the first scf step um so yeah have a quick look at our cpgk output file so here is output um so print out a lot of stuff at the start mainly reading in all your parameters that you've set in the input file you see here we're doing a qm calculation which is good um and yeah so it starts printing stuff about the scf so the convergence of the self consistent functional um you can see here the energy is printed out and this conversion factor slowly getting smaller um so it left the inner loop after reaching 20 steps um this 20 steps is a parameter set in the file so you can modify this for it to do more steps before it um goes around and then does a another outer scf loop um so all this is explained in the best practice guide if i'm going a bit quick here but eventually you'll see this number gets smaller and then it'll print scf run converged in 15 steps so you know it's been successful um and then it'll print out the final energy here so this is your total energy the qm energy this is a printed atomic units so heart trees um which isn't always the most useful unit but this is the default unit used within cpgk so if you want to convert this into a more useful unit um just go and look up the multiplication factor so i've written somewhere in the notes the multiplication factor to convert this into kilojoules primal um then a lot of statistics and the very end here references um it prints a time in report to this always be printed at the very end and you can see the run time for the whole calculation here um so this cpgk is the whole thing it's just the sub routine it's the whole code um and then this will be your run time for the whole the whole calculation and then also prints time reports for all the different sub routines called which is sometimes pretty useful okay so yeah this is what you record if you're looking at the run time and then further up the energy um which i showed so yeah within your notes you can see i put some commands for extracting the energy and the run time from the output so yeah the next step is to just make a note of these um and then also we want to keep this wave function file for the next part of the exercise so to ensure that it's not overwritten we're just going to rename it because what would happen if you ran this calculation again it would overwrite the restart file wave function files the same thing again so we just want to kind of checkpoint this so um we're just going to rename egfp restart wfx and then that is saved um yeah so that's the end of the first part um so yes um raise hands if you're ready to move on to the second part um i'll answer some questions although the qm atoms are defined in the cp2k input file the pdb file has the res student res name column filled with mm and qm file is there a way to instruct through file that info from the pdb file or needs to be redefined in the input file um so it has to be defined in the input file that is the part where cp2k knows that they are qm atoms and the thing is here that everything we're using has been generated by the interface so the pdb file and the input file are kind of generated at the same time based on what the qm atoms are so it's yeah it's kind of all done at the same time so it's when you're producing this pdb file with the qm atoms you're also producing the input file in the interface so yeah it was it's not really a point um but yeah the cp2k needs very much needs um the qm atoms in this format in the input file that you see but yeah hopefully the internet if the interface does this all for you okay yeah good point Arne um yeah gramax restart when function has nothing to do with gramax so yeah this is just a name um this is defined in your input file it's the project name so it's yeah it's just a name you could put anything there and it would work just as well but this is what is output when we run the interface into this file okay so i'm going to move on to the second part um let's just clear that um so yeah second part of this exercise we'll look at adding in a b3 lip and dispersion corrections so we've already looked at this the input we have this uses pbe however we can quite easily change this within the cp2k input file so we're going to look at changing it to b3 lip um so this is a hybrid functional and half of this will be half of it well not half but a portion of it will come from how to focus change and thus thus it's more accurate but you'll find it's more computation expensive than gga methods such as pbe so your area of interest in the input file if you scroll down you'll see this xc section um so this is what it looks like at the moment we just have um some parameters specified here the density gradient cutoff and tau cutoff and we have the xc functional section here which just lists one thing that we're using pbe so what we're going to do is be as we'll change this to use dftd3 and so yeah quite a lot of changes that go into this it looks like quite a lot of text but i'll just quickly explain it um so you can see here um you have the different um contributions which make up b3 lip so yeah these numbers here tell you the amount of this contribution and list it so using 81% of the lip correlation 72% of the back 88 exchange and 19% of lda correlation um so if you look back on the lecture this is actually in the formula for b3 lip so all we're doing is just directly specifying how much of these different parameters we use um so this part comes in the under the xc functional so this is kind of your gga portion um of the hybrid method and then we have this hf method so the hearty hearty fox section um and here we specify 20% of the hearty fuck exchange to be used holy yeah uh just wondering at the moment i'm seeing your terminal is that right oh i forgot to switch okay yeah just just checking yeah i think as soon as you start explaining the the different percentage contributions i couldn't see to reflect it in the in the terminal so okay yeah i'll switch to sharing the different um sorry everyone okay so yes and these are the different parameters i've just been describing um so this is the section that you'll be modified in your file um so yeah we have the hearty fox section here um we're using 20% of the hearty fuck exchange um so yeah all these parameters just basically make up the description of the through lip um so we have this screen in section so this is to help stabilize the um hearty fuck exchange um some parameters here so epf schwarz um this is a parameter you can kind of play with if you find that your calculations are a bit unstable do what you might see is like weird energy jumps when you're running the scf so something look at um and then you have this truncated operator here which um basically prevents self exchange interactions across the neighboring grid neighboring grids so um yeah it's got a cut off which should be less than half the cell to prevent this interaction across neighboring grids uh memory you don't really need to worry about um this is just set to set the maximum memory that you want to consume um it'll depend on your on the machine you're running on and how much memory there is per process and then uh this well the van der Waals potential section so this is for your dftd3 your grimo dispersion interactions um basically you're telling it you're doing a pair potential you want to use the parameters for dftd3 which are in this particular file um this is again in the same directory as the basis sets and potentials and the type and then your reference functional which is speech lip and the cut off um so yeah i think that covers that part um so some other subtle changes you will also need to make um so in the last section we made these uh these wave function restart files um we just need to add a line here after where you specify the potential tell it to read these particular this particular restart file name so it knows the name of it and we're also going to change the potential here um to one that's optimized for belip rather than pbe um so yeah we noticed before um this potential was dth pbe um just changing this to dth belip because it makes more sense to use this in combination with b3 lip than using pbe um so yeah it's just a small change but yeah 50k has a lot of these potentials and they're particularly optimized for using with particular functionals so it's just something to be aware of um so yeah i'll let people do that um and yeah let me know if you have any questions so just yeah make these changes and run again um yeah make sure you've recorded the energy from the last one or you've moved that file to a different place because it will overwrite your cptk.out file uh yes you can also use dispersion correction with pbe so yeah you don't have to go all the way and add in um hybrid functional and pbe um any reason why the cutoff is 16 angstroms um so it's kind of yeah cutoff for to be within the realm of how you want your how far you want your dispersion correction interactions to be um your van der Waals interactions so it's kind of a typical number it's probably something you could play around with but yeah 10 sort 20 angstroms seems to be the norm oh yeah so yeah why one might want to use a restart wave function so yeah it's usually when you run a first your first scf step with without a restart wave function you'll see um it does a way more like scf steps to try and converge um and this will like obviously take more time if you're supplying the restart wave function it's already got like a nice initial guess um to start with and it will just basically speed up the calculation and reduce the number of scf steps um and you'll you'll see this if you when you're running the second part compared to the first part you should see that first there will be a lot less scf steps in the first step it'll still take longer because we're using a more expensive hybrid functional so yeah don't get confused about that but um so we felt long can we calculate this with different functional basis uh yes um well here we've got the same basis set um I think usually using the same basis set is key um but it can be with a different functional um so obviously it won't be a perfect guess but it'll it'll help the count help with speeding up the calculation so obviously your system is still the same like your atoms and your coordinates so yeah has any people done and looked at the energies um so you see there's a slight change in the energy okay so yeah the smart energy here is using b3 lip and then the one below was the first one we did with uh pb0 um so you can see it's roughly 0.04 difference in the energy here this is in heart trees so what's that in kilojoules from all okay yeah so quite a big difference in kilojoules from all so yeah this is just a static energy calculation so you can see there's been a difference however if you were like doing this for real you'd probably want to go away and calculate your actual property of interest um so if you're running like an empty simulation you might want to go away and compare the two or if you're calculating say an energy barrier or something like that um oh yeah the time so how long did this one take 25 seconds how long did my other one take which is upper directory I don't know about the same okay that's interesting um so yeah this time yeah so I noticed this too um I'm guessing it's because we've supplied the scf wave function for this one so in the first one um the first part of the calculation we'd have done we did what so we did 20 scf steps and then another 15 so we did 35 total so we did more at the start and this one it converged quite quickly I believe yeah so only 13 steps so you can see each one of those steps is taking 0.4 and in the other one just hopefully we put it 0.3 so yeah each scf scf step was shorter in the first one um so this is the time per step um shorter and with pb zero but we ended up running more scf steps because we didn't start with the preconverged wave function mine's been stuck for only for 12 minutes oh no um I mean check what's going on inside the output file if it looks like it's converging or not converging it might be that it didn't somehow didn't read the initial wave function file um and that it's it's kind of stuck trying to converge the scf or it might just be something's gone wrong on the compute node I don't know it's a bit worrying yeah double check um double check you've got that that file there the the wave function restart file in the right place and double check the you've supplied that not uh that line which says the name of that file and that the spelling is correct okay yeah so I'll move on to the third exercise now um the third part of the exercise we're going to look at converging the cutoff so usually you have this well you have this integration grid using per map in the electronic density um in cpdk and you want to check that this grid uses fine enough um so you have this cutoff keyword in the input file um that just defines the cutoff in ryberg which kind of relates the finest level of the multigrid so basically the higher you're playing with cutoff the finer the grid um so yeah this is kind of an important step when running a cpdk calculation you always want to check that your cutoff is large enough um otherwise your energies could be wrong um so in the interface the default cutoff written into the cpdk input file is 450 rybergs um so you'll see that uh if you currently open the file that you have at the moment um yeah so this is quite a large fairly large value for the cutoff so this should automatically so it's set to kind of you know be be sure that the cutoff will be large enough regardless of what you're doing because you don't want incorrect results um so yeah it's a reasonable value um so here's for example it's the cutoff total energy versus cutoff for your initial input so the pbe example um you can see here um it's quite a drastic change in energy if you use anything below 150 um and then you can see it stabilizes quite nicely um above that value um there will be some steal some like subtle differences in this value that you can't see in this graph but yeah so 450 uh quite a large value um yeah so cutoff is kind of a funny value um if you yeah you need to use a large enough value if you go way too large it can quite slow down the simulation quite a lot so you have to balance this um but yeah you might want to look into tuning this you definitely don't want to use a value too small maybe sometimes using a really large value could slow down your calculation quite a lot um so this exercise we're going to look at just converging the cutoff um for our new system so we've made changes to the input file we've changed the basis sets from functional the potential with added and dispersion corrections um whenever you make major changes if you were running cbtk on it so and you'd want to go and double check that your cutoff is still reasonable and we converge it um so it's basically what we're going to do here um kind of run in multiple calculations and changing the cutoff each time and recording the total energy yes on to pulling these calculations with the hybrid so taking your input that you've just modified and added in um your b3 lip and dispersion correction um yeah your value is this cutoff here should be quite near the top so just change this example um write it and then run it and record the energy um you're meant to want to make sure again that you're using the restart way function file otherwise you could run into trouble with taking a long time okay i have a question how do we decide the scale c scale x and values in the xe part so these values come from the definition of v3 lip itself um so i don't know if i can reshare the best practice guide quickly okay so for example here with pb zero the definition of pb zero it's comprised of 75 percent of pb and 25 percent of party foc exchange um so this kind of defines how these values are set so you've got 25 percent here for the heart to fuck exchange and then 75 percent of the gg exchange and then 100 percent of the gg correlation you can see the equation for it here b3 lip is more complicated than pb zero so it's basically comprised of an lda part which is of the exchange function um some different parts the heart to fuck exchange another part of the gg exchange and some correlation from lda and that's going off the screen i can't okay make it bigger um yeah some other parts um so this is covered in Emiliano's lecture um so basically these are the different parameters and this is how they are defined in the cpdk input so you can see um we've got this contribution of the lip coloration here which is not going any one and this is set here um in your input file um so yeah it's somewhat a little bit confusing to work out where they all come from some of the mobile straightforward so you can see that's 20 percent of the heart to fuck exchange so 0.2 there um yeah some of them have been come around we have to subtract values from others so you can see yeah your coloration is 90 lda and 81 lip sums to a hole and then if you sum these the exchange parts up as well that sums to 100 percent as well so that's where they come from okay so we have some values already so we have okay one for 50 one for 150 one for 250 for 500 um yeah so we're missing 100 200 300 350 yeah great um yeah if you've done one and you filled it in uh feel free to pick a value in the scratch pad that hasn't been done yet and you can add in another line if you want okay yeah so a big difference for 50 okay um I see that we've got values and most of the cutoffs now which is good I'll let people continue to add those while I just answer some of the questions um so I'm wondering how much the influence will be on the court when I apply a setup when I'm doing um yeah I mean it's difficult to say um so for certainly if you use a very low value of the cutoff your results could be wrong um we actually did a kind of a look at this as well um modifying the cutoff and you see if obviously you can see there's subtle differences in the energy here between different values and then if you go ahead and run like a long MD simulation kind of the effect of that and affecting the energies and the forces which are calculated will sort of be magnified over the course of the run so yeah it can kind of have more of an effect if you're running a long MD simulation um will we see some variation in energy for two calculations of the same cutoff yeah there's a bit of variation um you can run um the same CPGK calculation with exact same parameters twice and the energies will not be exactly the same um so you see the I mean you'd expect them to be the similar have values here for example where two people have done the same thing and yeah that's kind of what I expect so if you look at 250 here um beyond say one to six decimal places um is where you typically start to see variations just yeah this is very small at this point in terms of your actual energy um I was going to quickly share so I've just made a graph I basically took everyone's results and I've just made a little graph here um so you can see this is kind of what the energy is doing with the cutoff so before we kind of said 150 plus it looked like it sort of converged that was difficult to tell here even 200 might not be enough so I'd say like you could possibly say 300 plus 350 should be a suitable value but yeah you can still see some variation going from like 200 to 250 um so yeah that was a quick summarise of that um I will now just cover a bit more of the performance so this is the very last part this is an exercise this will just be me talking for a bit the very last part is just looking at the performance so we've kind of seen how performance can be affected when you change um to use in different functionals so these are some results I got earlier and I did see a difference um from switching from PBE to B3 lip and you could see the runtime had gone up um I don't know what I've done differently this time um so yeah um you expect change in season like a more complex functional such as B3 lip to increase the runtime overall and we only did like one step here so when you're running an MD calculation you're running multiple of these steps you're calling it once every MD step so you do expect the effect to be larger overall um and our first step's a bit weird because we're dealing with sort of doing the first like setup and the SCF calculating the wave function so yeah we had strange results because we're reading in the wave function the second example with B3 lip and not in the first one so but yeah you can see that the time PS SCF step was longer for B3 lip so yeah here's some similar results looking at different functionals so I want to PBE and PBE 0 so this is a the time per MD step this is a lot bigger system so you'll notice time per step is huge here um compared to what we've been looking at before where we only had around 20 qm atoms this has 68 but you can see it's almost a sign of a double in the time taken per step just running on one node of Cirrus here um but yeah you can see it takes much longer but also the the scaling the speed up here is a bit improved so you can kind of justify running on multiple nodes um yeah also some stuff about qm region size here um so I've looked at changing the qm region size so you have 19 and 253 qm atoms but for the same system and you can see the effect of runtime there on runtime there and then some information about using multiple threads so the moment we've just been running on a single thread for these calculations but sometimes it's advantageous to use multiple threads per process so MPI plus open MP so there's some different results here just showing the effect so example in this system using six threads is a lot faster than using one thread so these are just things to be aware of when you're running your own system if you want to look at the performance give you a rough idea of what to expect and kind of what changes might make to help improve the performance um how many cores on nodes you want to run on um so this will all be added to the best practice guide shortly but important thing is just experiment with your own systems cool um I'll have a quick look at the questions but that's all I want to say um could you explain conceptually how these calculations are parallelized um it's yes it's very complicated um it mainly is parallelized over the grid function so when you do multi grid yeah yeah it parallelized over that so you have a grid points and you have a grid grid which consists of a large number of points and they can be split that points could be split along the nodes and each node take some small part of the grid and to wait everything in that grid uh in that part of the grid and then they uh uh sum up all contributions for moments in a way this is a pretty similar algorithm as like PME works in gromax is classical md it's also decompose the system into smaller parts and uh and calculate them separately yes uh fft is also utilized for the grids to pass it to the reciprocal spacing back uh density from it and uh but of course this is a very well known parallel algorithms which are already implemented in let's say fftv library and so on yeah it's also employed actually very funny thing might not many know but in previous p2k at least it's a 7.1 I don't know how it's working 8.1 didn't test it but a sp2k very well liked when the number of mpi uh tasks which are running should be uh some uh square of some number for example 84 a square of 8 why because because of that grid decomposition which it sees so uh the best performance usually which I observed was exactly when your number of mpi tasks not open in peter's but mpi tasks is square of some number uh yes uh don't choose yeah not prime numbers but for course it's better to choose a square of some number yes okay yes I guess it provides but yeah it's very funny it's actually connected to the parallelization scheme over the grid switch cp2k implies it comes to how it splits the grid over the nodes so I think we uh we realize that this is one of the areas where we probably the best practice guide of holly link to the plan to expand with some more guidance on this in future also what I saw from the holly stock that uh if you noticed uh the default cutoff which dromex with the interface set up is 450 readbacks and this is a quite high number in reality why it's done so it done so because it should be suitable for most of the system you can imagine so probably in real uh calculations it always worth to reduce it because it it speed ups the your simulations a lot in reality so yeah but 450 is kind of safe uh cutoff which suitable for 99 percent of the system biological system uh is there are some unparameterized molecules have to be treated in the mem part it's possible to use cp2k because the it seems quite promising because of the f t it charges hash and optimization derivatives and so on so yeah um for parametrization of the molecules usually you should use uh software which not connected to the qm implementation but to the mm so for example finder port field usually one should use uh um pic chamber uh and in that um pic chamber the parametrization procedure is explained i guess and in anti chamber for example you need to generate him uh several uh so you need to generate uh cube files with electron density uh from the several geometries of that molecule which it generates you and usually it should be done with gaussian what they suggest is gaussian uh with some like this relief i guess six three one g star basis but yeah i mean you also can use cp2k in that case yeah so the rest charges is not enough for usually for the force field for force field you need to do a rest charge in several points and then in several geometries because yeah the force should cover more geometries and on the left is that it ever ages through somehow uh yeah that's one of the procedures another if you don't have i mean if you want to use a qm for parametrization it's better to go to the particular force field which you want to use and check how it should be parametrized usually yeah it's indeed parametrized qm like both charm and ember are parametrized qm calculations and basically check uh the procedure it's not a good idea to use another method then the standard user to the force field because then you have inconsistency in your force field because uh your protein for example you parametrized use one method and your ligand will be parametrized in other term which is not the best idea in general yeah but if you want to do a qm mm then it's fine it's completely okay to use whatever mm charges you think will be suitable because anyway you will then switch to qm qm uh uh a description of your ligand for example and qm of course is not dependent on that okay all right well um thanks very much thanks Dimitri uh and thanks Holly uh for useful session so uh now we have a break um lunch or whatever your local time appropriate uh meal is or or not and we'll see everybody back again at uh one o'clock uh central european time um for the next practical