Hi everyone. So as you said I'm Joe Conway. Is this too loud? No. I've been in and around the community for almost as long as it's been around. I started using Postgres in 98 and started becoming an active community member around 2000 and I'm one of the committers. I'm on the PG infrastructure team. I'm also on the board of PGUS as Debbie is. So I'm going to give you kind of an overview of what it takes to get Postgres up and running. I'm not going to be able in 50 minutes to cover that and also sort of a primer for starting SQL. So this is going to be mostly kind of administrative, although I will go into a little bit at the end in terms of data types that are available and some hints about PCQL that maybe you'll find helpful. So first of all, how do you do the installation of Postgres? Most of you are probably going to install Postgres using some kind of a package that exists for your OS. So if you're running Windows, you'll go to the Postgres site, you'll download the Windows installer. If you're using something like Red Hat, you'll probably get the RPM from them or maybe you'll get that also from the Postgres site. There's Debian packages on the Postgres site as well as directly from Debian and Ubuntu and you could also potentially install from source. And if you're doing kind of development work, I would definitely encourage that and it's not nearly as frightening as it might, difficult as it might sound to people. Just to give you a very quick, I know this is an introductory talk so I'm not going to go into it in much detail, but I just, right before we started, compiled Postgres on my laptop and it takes just under a minute to compile. So I mean there are some dependencies you need to install, you need to get the source from somewhere. There are source tar balls available including the kind of a daily snapshot of the development if you want to track that. You can download those or use get and then you just basically do a configure, a make and a make install and you've got Postgres and it does not take all that long to do. So once you've got Postgres installed, the first thing you're going to need to do is actually initialize the cluster because after you've installed Postgres, there's basically still no database on your system. Depending on which OS you're on, there are different ways to initialize the cluster and this talk is basically focused on Linux variants. I'm not going to really cover much in the way of Windows here, but there are Windows tools. If you download the installer for Windows and you get PgAdmin with that and then PgAdmin actually has kind of menu options for doing things like initializing the database that make it fairly easy. If you do install Postgres yourself, you're probably going to just use the native tool that Postgres provides, which is called initDb. And so in the simplest form to initialize Postgres, you just do an initDb dash capital D and then PgData here is just an environment variable that represents the directory where you want Postgres installed. The important thing to note there is that directory needs to be owned by readable and writable by the user that Postgres is going to run as, which is normally Postgres. It doesn't have to be. You can actually install Postgres under your own user account on a Linux system and if you do an initDb logged in as yourself, then Postgres is going to run as you. Again, that's not normally going to be the case if you're installing from a package for your OS, but if you're installing it from source, that might be. If you install from Red Hat or sent to OS 7, it comes with an actual script that's called setup. So Postgresql99.5, that's the 9.5 version, dash setup and you tell it initDb and that will create a cluster in kind of a well-known location for a Red Hat system. Similarly on Debian, there's a PgCreateCluster script. So you tell it create cluster 9.5, main and it's going to put, the one difference to note here is that in a Red Hat variant, basically everything goes into a single directory, including the config files. But on a Debian variant, the data goes into your data directory, what people will typically talk about as PgData, but the config files actually go in the slashetc directory, which they consider to be more standard across other types of servers. So there's a lot to be said for that actually. But in any case, you should note that if you're working with two different types of systems, you're going to find these things in different places. And notwithstanding what Debbie said, I will probably stop and ask for questions as I go to some extent. So if you do, you know, there's a lot of material I'm not sure I'm going to be able to get through the whole deck. We have a lot of questions in between.  So there's anything about that part before I move on? The slides will be available. Every conference kind of does it differently, but there's a Wiki page on, if you go to wiki.postgresql.org. On the Wiki, there's a page that's called, I don't know, I always search, I always Google for postgresql, presentations, conferences, and Wiki. And usually you find there's one page that sort of aggregates all of the conferences. And then for each conference, there'll be a link into the Wiki that has a page just for that conference. And then all of the slides are normally linked up there. And I will definitely put mine on there. And in fact, I often for the conferences end up creating that page. So I will definitely do that. Okay, great. So once Postgres is actually initialized, now you can finally start it up. And again, there's a variety of ways to do this. Part of the reason for that is historical. I mean, Postgres has been around as other people have alluded to. Postgres actually started 30 years ago in Berkeley and has been open source for 20 years. So some of this is a consequence of evolution of capabilities over the years. But one note here, again, if you're probably as a beginner not gonna be doing this, but there's something called single user mode, which can sometimes be needed to recover or do certain operations. And if you needed to do that, you probably are already engaging with some Postgres consultant to help you do it. But in any case, this is the command, you can use the Postgres binary dash dash single and point it to the data directory and give it a database name. And it basically will launch Postgres into the single user mode, which is sort of limited in capability, but it turns out to be important sometimes. If you want to do a manual start of Postgres, and again, this is typically going to be the case where you've installed from source, not using the OS packages. There's a binary that's provided with Postgres called PG underscore control. Again, use the dash capital D, give it the data directory that you want to run. This is kind of an optional, you give it a path to a log file and you just tell it to start. And again, these, there's lots and lots of other options for these kind of command line things. I would encourage you if you haven't already, let's see if I've already got it up. If you go to the Postgres website and write in the middle their documentation and then pick your version. I mean, it can be pretty daunting to find stuff in here, but all over the nitty gritty details for those binaries are going to be available under either client applications or server applications. So that PG control is a server application. And if you went into there, you would see all the different options, which I'm not going to try and go through today. So now if you're, if you installed from a package and you're running this more or less on a production system or on a production system, you're probably going to use the service facility of the OS. So on CentOS 6, that's just service PostgresQL start. Red Hat 7 started using SystemD. Actually, some of the Debian versions also use SystemD now. So if you're using a SystemD system, it's going to be system control start and then PostgresQL-9.5 would be for version 9.5, 9.6 for version 9.6, and so on. On a Debian based system, again, they provide a script called PG control cluster, 9.5 main start. So those are all different ways to accomplish the same thing. That's going to get the service up and running. Now, how do you stop it? Again, if you, if you're in single user mode, basically, you just control D, which will exit out of the program that's running. If you've done a manual start, you can use this PG control again. You do dash D, tell it which data directory. And as an aside, one of the reasons you're, you're doing this with this specifying the data directory all the time is because it's perfectly possible to run multiple Postgres clusters on one machine if you want. Basically, each one has to be bound to a different port, but you could run multiple copies of the same version or you could even run multiple copies of Postgres at different version levels on the same machine. If you're doing, like for instance, I do Postgres development, if I'm fixing a bug and it applies back through Postgres 9.2, I'll often run Postgres 9.2 all the way through the current development tip all at the same time so I can apply and test the patch. So here we're doing, we're specifying the data directory, we're saying that we're gonna stop and then we're giving this dash M fast, which is telling it how we want it to stop. And I'm gonna go into that in a little bit more detail in the next slide. Suffice it to say at this point that probably what you want is this fast option and that's what if you're using the operating system control, that's what it's gonna use. I'm 99% sure. So again, Red Hat sent OS 6, service Postgres QL stop, sent OS 7, system control stop, Postgres QL 9.5. And by the way, this service file might not be called Postgres QL, it might be called Postgres QL-9.5. It kind of depends on where you got the package from. If you got the package from Red Hat itself or if you got the package, the RPM package from the Postgres development group repository, it might be slightly different. You could figure out what that is if you look in the ETC init.d directory and see what the script actually is named. And then finally on a Debian based system, you would use PG control cluster, 9.5 main stop. So in terms of these shutdown modes, it is kind of important to understand this to a degree. There are three different modes that are supported, smart, fast, and immediate. Smart basically says I want Postgres to shut down, but I want all of the existing sessions that might be running to finish first.  And that sounds like you probably often would want that, except the problem is some of those sessions might last for a very long time. And if you're really trying to get Postgres to shut down now, that can be kind of annoying. This fast option will terminate all the existing sessions, but it does it in a graceful way. And what that means is it's doing a proper shutdown of all of those back ends. It's not just sort of cutting them off at the feet, right? This immediate option just summarily kills everything. That's essentially equivalent to pulling the plug on your server. When Postgres comes back up after an immediate shutdown, it's going to go into recovery mode, as if it had crashed. So as if Postgres had crashed or if the OS had crashed or the server had lost power, that's what's going to happen if you do an immediate shutdown. And the problem with that is, number one, Postgres is going to take more time when it comes up because it's got to roll forward through all of the files, the transaction log that had been accrued since the last checkpoint. And also you're going to lose some important information. There's some bits of information that are not maintained if Postgres is not shut down in a controlled way. In particular, statistics about when auto vacuum has happened or needed will be lost. And if you've got, I think it's unlogged tables, if you don't have a graceful shutdown, the data in those unlogged tables will be lost. Any questions about all that before I move on? So now let's talk about some other administrative things. What if I want to terminate just a particular session? Well, there's a couple, again, as with many, many things in Postgres, there's more than one way to do this. They all, in the end, boil down to the same kind of thing. But if you wanted to do this from the command line, there's a lot of ways to get this information. This is kind of my favorite command. If you do ps-fu and then the name of the user that Postgres is running under, that gives you just the processes that are running under that name. And on a typical system, the only processes running under Postgres are the main Postgres server processes. I mean, if you've got someone logged in as Postgres, they may have some other processes running. But typically, this is going to give you just the processes that are involved in Postgres. And if you grep for, say, the name of the database of the session that is logged in, you can get the PID.  It's important. Sigterm gives a very specific signal that causes that back end to terminate in a graceful way, again.  If you wanted to do that from logged into Postgres, you can use this. Again, there's different ways to get this, but this query is one way. You can say select PID, state, and then clock timestamp minus state change from PgStat activity where database name is equal to some database of interest. And you can see here that, and actually query would give you this text of the SQL. So in this case, I'm seeing a PID, I'm seeing a state, and here you can see this one is idle in transaction, which, as an aside, you generally don't want to see. That means your application is probably misbehaving. You should never really see that for more than a transient period of time. You can see how long it's been in that state because of this calculation here. So this has been idle in transaction for 26 seconds. And the last query that ran was a begin. So it started the transaction. So that's the one I want to terminate. Well, now that I've got the PID, there's this function PgTerminateBackend and that will do, that essentially does exactly what this did up here. So now instead of cancelling the back end, which will basically, again, pull the rug from under whoever is logged in, you could simply cancel a long running query, cancel the query that's being currently executed. So in a similar way, I'm going to use PS to find the PID of interest, and then I'm going to send a different signal, SIGINT, which is basically like hitting CTRL C. In fact, when you hit CTRL C, that's the signal that gets sent to the process. So if you do kill SIGINT to that, it will cancel the existing query that's running. Same thing, you can run that query off the PgStatActivity table, find the PID that way, and then say PgCancelBackend, give it the PID number. Okay, any questions about that before I move on to some configuration stuff? So configuration of Postgres is, by and large, kept in two files called Postgresql.conf and Postgresql.auto.conf. And as I said earlier, those files on a kind of red-hattish system are going to be in the data directory, more than likely. And on a debian-ish system, they're probably going to be under ETC Postgresql. You can use a hash to denote a comment in those files. And the format is typically, you know, name equals value. Pretty straightforward stuff. Peter, somehow I don't think you need an intro to Postgres. Feel free to sit in, I don't care.   If you change one of those files, and this one you're not going to change directly yourself, I'll cover that in a minute. But if you change this Postgresql.conf, and you want to see the change actually take effect, you have to tell Postgres that you made a change. So that's what we're going to do right here. This is one way. As you can use a query, select pgconf reload. And you can see I'm actually going to send that through the psql, which is the command line client as a command line parameter. And I want to do that as the Postgres user. So there's other ways to do that, which I'll show you later on. But you can specify the user on the command line for psql, if that's not who you're logged in as. But this will cause Postgres to reload its configuration. You can also use that pgcontrol program that I was talking about earlier. Again, you have to tell it which data directory, so it knows which cluster of Postgres to act on. And you tell it reload. The service control supports it as well. So service Postgresql reload, system control reload, Postgresql. And then finally, just like the other options, you can send a specific signal to the Postmaster PID. It's called SIGHUP. So when you send a SIGHUP to the Postmaster PID, Postgres will reload. And basically, all these other methods are just doing this under the covers. One note that I haven't made so far, but you'll, when you do look at these slides later on in a number of places, I've put a link directly into the documentation for you. That will show you the relevant information that's on the slide if you want to go into more detail. So when you're modifying configuration, you can make persistent changes. And you can also make changes that are procession. If you want them to be persistent, you're going to either do, as I was just talking about, modify Postgresql.com is the most likely way that you would do that. You can also actually modify the startup script for Postgres and pass in configuration options right on the command line. That was something that was done more often years ago. It's not very often done now, but it certainly still works. And then in the last few releases, I think since 9.4, there was a new command called alter system. And so basically alter system is a way to make changes to Postgresql.com and technically not directly Postgresql.com, but that Postgresql.auto.com is what gets written by this alter system set command. So if you from within a SQL session, do that, it'll write out to that auto.com file. And then in the auto.com files in turn, included in the Postgresql.com at the end. So changes that you make through alter system get persisted in that way. So you would say the other other ways that you can make persistent changes to configuration are through alter database and alter role commands. And these are a little bit different than the other in that if I do an alter database, database name and set some configuration variable to some value, it's going to change that value just for that database. So now whenever someone's logged into that database, it's going to have that that variable is going to have that value. Similarly, you can do that for a role. So in Postgres, users and groups exist still because they've existed from the beginning. But at one point, they were all re implemented as a generic role. So actually, a role is a role that can log in is basically a user and a role that cannot log in is considered to be a group. But in any case, if I say that a role that's going to log in typically is what you would do this for, I set some variable equals to some value, let's say I want, I'm going to cover this later if you're not familiar with it, but there's a setting called work mem, which has to do with how much memory you have available for certain operations when you're running queries. If you have one power user and you want that power user to have a lot of work mem, you can alter that user and set work mem to some big value. Now only that power user gets that large value and everyone else gets the standard value. You can also change some of these things per session. So there's a set command or reset and a show. And you can also find out those values from a view a system view called PG underscore settings. You can directly update that view in order to change one of these values. You can select the setting using a function. And you can also change the setting using a function. So these are again, multiple ways to do the same things. Now one thing to note here is I've been talking about this as if, you know, any of the variables can be set in this using this method or one of these methods. The fact is that some variables require a postmaster restart in order to take effect. And there's some variables that require a reload in order to take effect. And there are others that are session setable. So these things here, and these things here are really only going to work with session setable variables. You can find out which ones those are with this PG settings view. It has a column called context. So if you did select name comma setting comma context from PG underscore settings, it would show you all of the configuration variables on the system, how they're set right now. And that context field would tell you, and again, you'd have to go into the documentation probably to figure out which word means which thing. But basically that context will tell you which things are setable per session. Any questions about that before I move on to specific settings? Yes. So like I said earlier, the question is, is do some of the settings require Postgres itself to be restarted versus just reloaded? And that is definitely the case. There are certain settings. Perfect example is shared memory. Another one is the max connections. If you want to change something, some of those settings will require you to actually bounce Postgres. Whereas when I was talking about reloads earlier, a reload doesn't actually restart Postgres, it just rereads the configuration files without stopping and starting Postgres. So there's no interruption of service if you can get away with the reload. But there are a handful of things that you can only change by restarting Postgres. In the documentation on the .org site, but as I said, is this large enough to see in the back? So you can see some of these are Postmaster context. This is what I was talking about just a minute ago. Those will require a restart. The ones that say SIGHUP, you can probably figure out from what I was talking about a minute ago, SIGHUP means a reload. If you see user, that's something that can be changed by a normal user in a per session basis. But again, keep in mind that if you do that in the session, it's not persistent, it's only for that session. There are also some, I don't know if there's, see if I can see an example of it. Yes, super user. So there are some things that are per session setable but only by the super user. And then you'll see some of these are called internal and those are actually kind of for information only, but they can't be changed. So this block size is actually compiled in parameter. Anything else before we're going? Okay, so now I'm just going to cover like a handful of the more common things that you might need to think about. There's, I don't know exactly how many configuration variables that can be changed. There's probably a couple hundred or so. Regina, do you know? There are a lot. You also don't need to be here. So I'm just going to cover some of the more common ones, basically. This listen address controls what interface Postgres is listening on. So in other words, and again, the default values kind of depend on the distribution. Some different distributions sort of default things differently. But if this is set, for instance, to local host, that means no one will be able to connect to Postgres from another machine because it's not listening to anything but local host. So you could change that to an asterisk and it just means listen on all interfaces or you can change it to a specific IP address for one of your interfaces and it'll only listen on that IP address. And actually from a security best practices standpoint, you probably want to specify an IP address and not just put a star on there because you may want to restrict connections to the database from one specific subnet. As I mentioned earlier, Postgres listener runs on a particular port. The default port is 543.2 but it can run, you know, essentially on any port that's not already occupied by some other service. And if you want to run multiple versions of Postgres or multiple Postgres clusters on a single machine, you would have to change this port so that they differed. Now, whatever you change that to, you also got to make sure that the client knows that and most of the clients have options to specify. They all default to 543.2 typically but you can tell the client I want to connect on 54432 or something. The max connections, this is the maximum number of connections to the database is going to allow. One thing I'll say here is if you find yourself wanting to set this to like a thousand or more or really even over 500, you probably need a connection pooler. You don't want to set Postgres up to use thousands of connections. You really want to use connection pools. So, you know, when I go in somewhere and I see them with a very large value here, I immediately start asking why. And there are performance impacts if you start having a huge number of connections, even if they're not all being used. For one thing, each postgres back end consumes, you know, fairly small but still significant, let's say 10 megs. And let's say I set a connection pool for 5,000 connections, just do the math, right? Superuser reserve connections, this is kind of your emergency backstop. If you need to be able to log in as the superuser to fix some problem, you don't want to be locked out because all of your connections are used, right? So, this allows you to specify how many of those connections should be reserved only for super users. And then SSL, it defaults typically to false, SSL has to do with encryption between the client and the server. There is a performance penalty for that. However, you should be aware that unencrypted postgres connections, if someone's sniffing those packets, they could replay them. And potentially other bad things, right? So, if you're not using SSL between your client and your server, you better be on a trusted network. Or maybe your client and your server are on the same machine. Shared buffers, this is the buffer cache. So, when postgres, when you do a select and you pull data out of a table, postgres basically reads it all up into the shared buffers. And then this is kind of a fixed-size pool of memory where that stuff will reside until it needs to get pushed out because of other things that are being used more often. So, the whole purpose of that is performance. If your data is all read into shared buffers, it's all in memory, and now the next person comes along and queries the same data, instead of going all the way to disk for it, it's just going to pull it out of shared buffers. If you change that data, it gets changed in shared buffers and then gets marked as dirty and eventually gets pushed down to disk in a very secure way. The general guidance here is that shared buffers, you typically hear people say, use between 25 and 50 percent of RAM if it's a dedicated database server. In older versions of postgres, postgres didn't do so well if you went more than about 8 gigs of RAM, but then again in older versions, that wasn't so typical. And now these days you can find, you get many database servers that have 256 gigs of RAM or a terabyte of RAM or something like that. On older versions, you probably still don't want to go above about 8 gigabytes, but on newer versions of postgres it's more efficient and you can you can set that fairly large. The thing to understand though is if you're working set, the data that's constantly being read by the clients can fit in that, then your performance is going to be much better than if it just barely doesn't fit. Because if it just barely doesn't fit, you're going to have a lot of turnover. And that's actually going to, it's actually will be slower to have a large shared buffers if your working set doesn't quite fit. Well I would start with a quarter of a RAM, but the point is you really need to try it with, you need to experiment with your own queries and your own tables and your own working set and optimize that value because there's no way for anyone to give you you know the perfect answer. In some workloads it'll turn out that setting shared buffers to 2 gigs even though your database is 20 terabytes is going to be the best performing and it'll perform better than something that's larger. In some cases setting shared buffers to 32 gigs might make a huge difference in performance. So there's really no way to say ahead of time what's going to be better. There you know there is this general thumb rule 25 percent of RAM but I would just point out you need to be careful about just accepting that you should really test a few different scenarios. Okay I'm going to get into that right now. Yeah so huge pages if you've got very large shared buffers you'd want to use huge pages. This is an OS thing that is I think it was supported after Postgres 9.4. Basically if you're using really big chunks of shared memory you want those pages to be allocated in big chunks in order for it to be more efficient. WorkMem is used by query nodes for certain operations that kind of the two best examples are if you're doing aggregates there's a type of aggregate called a hash aggregate. Build a hash table in memory for each of the groups and then it just sort of updates it. That's a very efficient way to do an aggregate if you can. You also sorts will use WorkMem so if you need it if there's a sort node in your query if it can do that without spilling the disk that's more efficient and basically WorkMem is what Postgres will try not to exceed so if it does the calculation and it says in order to do this sort I need two gigabytes and WorkMem is set to four megabytes then it's not going to try and do that in memory it's going to spill the disk. Well this value is settable per session but you should understand that it may it's per node a query is broken into a bunch of nodes so when you're querying you know five tables that are joined and you're you're doing order by and you're doing group by you might have multiple nodes as the data is kind of gathered up to be returned each of which use WorkMem so the important thing here is shared buffers is once for the whole cluster so you can set that to half a RAM and you know and be reasonably certain that you're not going to drive yourself to swap but again this goes into a number of client connections if I've got a thousand client connections and they're all running queries and this is set to 128 megs again do the math you might drive yourself in the swap pretty quickly so you have to be careful about setting this value too high but setting it higher in some cases will really significantly improve a query so using tricks like having a power user that's got a higher value or just setting it in a particular session prior to a query to a higher value just for that query can be really useful maintenance work mem is used for in older versions of Postgres it's used for vacuuming and it's also used for index building in the newer versions of Postgres is actually a separate setting which I didn't list here called auto vacuum work mem but at default to the value of work mem so you can actually split the two out the other thing to know about older versions of Postgres you could not usefully set this value more than about one gigabyte more than about one gigabyte I think up until Postgres 9.4 after Postgres 9.4 you can actually set that to a very large value and it'll get used so this is a case and again this is per session settable because it's involved in potentially auto vacuum you do have to be a little careful auto vacuum kicks in and there's a setting for auto vacuum this is how many workers do I want to have so and that defaults to like three but let's say you set auto vacuum workers to something like 10 and then you set this value to something like five gigabytes and all your auto vacuum workers kick in on large tables now all of a sudden you're using 50 gigs of memory so you got to be a little careful about this one not quite as careful as you do with work mem but for building a specific index if you can build it all in RAM I've done testing and it'll build at least in tests I've done sometimes as much as three times as fast shared preload libraries is something that's used for extensions to Postgres it's not in many cases it's not required to load do this but what happens is if you've got a library listed here it will get loaded when Postgres first starts which means it's already there when the client connects the way Postgres works is when a client connects to the postmaster a process is forked so when it's forked it's just a copy right anything that was there when it was connected when it when the client connects goes with it when it forks so if you preloaded libraries they're already there as soon as the client starts up and don't have to it doesn't have to be reloaded so that's kind of a performance tweak that you can do if if you want and you're using some extension like PostGIS say you could load it here it'll be loaded already whenever your client connects instead of having to load per session in some cases because of the way they work some extensions actually require you to load them at Postgres and at time and those will typically tell you that and I'm probably gonna run short on time here what time do I have to? okay all right I guess I'm not doing too bad I'm on slide 12 or 43 that's a fairly complex question so the question was is there are there statistics available that would or that would tell you that these things are set appropriately for your for your workload and there are statistics on the system they're not really going to tell you whether work mem is set correctly there's an extension actually that you can you can use that will give you more information about what's in shared buffers I've only got five minutes yeah that's what I thought yeah 10 minutes okay I'm gonna try and speed up a little bit and get through the more of this material okay well let me just get through the configuration parameters anyway so there's parameters on on the right of head log which is also known as the transaction log you always want fsync on while level is set based on whether you want to do replication or not synchronous commit is is typically on but that's one that you can consider turning off the implication is that if there was a crash in the middle of a transaction you might lose that transaction but you won't actually end up with a corrupt database checkpoint timeout and prior to nine five there was a checkpoint segments and after nine five there's a maximum wall size basically this has to do with how many pre allocated transaction logs there are if you've got a very busy right heavy database you would want that to be larger you would want this checkpoint timeout to be larger and this checkpoint completion target to be larger these are all kind of the defaults there's some planner parameters you should know about effective cash size you typically want to set this to something about half of available memory sequential page cost maybe three quarters sequential page cost and random page cost this is the the relative cost of doing a sequential read on the disks versus doing a random read on the disks this was set years and years and years ago based on some testing often based on real workloads and real hardware today I find that random page cost is too high and should be lowered to something like two maybe even one point something and if you're basically if your whole workload fits in shared buffers you would actually want to start approaching one because if everything's getting read out of memory the two are pretty much the same but again that's something that you might want to do some testing on your actual data and your actual workloads and your actual hardware there's all kinds of logging parameters I'm not going to try and go through all of these but you should know that the what you can log where you can log to is very flexible in Postgres including using facilities like syslog and the event log on windows you can control many different things that can be logged log min duration statement is set to off by default but this is something where if you want to look for long running queries you can set this to say a thousand which means one second so it's in milliseconds and now any statement that takes more than one second is going to get put into your log if you set this to zero it's going to log literally everything if you've got a busy system that's probably going to cause performance impact in and of itself but it can be useful to do that every once in a while where you just do it for a specific operation you set it to zero you can do that on a per session basis you run some operation in your app you turn it back and now you've got a snapshot of every query that your app ran during that operation sometimes that can be really useful especially I've seen applications where they run hundreds of thousands of individual short queries so every one runs very quickly but there's so many of them the whole operation is slow and you can often rewrite those kinds of things where they do a set operation instead of a bunch of individual rows so in miscellaneous there's a search path in Postgres Postgres supports schemas tables go in schemas basically the search path controls which object is going to be seen by your client so if you just similar to like an executable program on your operating system if you had two copies of the same program in different locations the one that's that comes first is found first in the path is the one that gets used same idea here you can set a statement time out so you can it'll cause an error statement takes more than that time to run you can set a date style and a time zone so I guess that's that's all the time we have so is there some free and easy to use analytical tools that you can recommend about analyzing your system if your account is set right and what is the bottleneck like what prom can review something like that well there's a there's an online there's a website um called Depez Peter do you know the name of the website explain.depez which will do an which will allow you to put an explain plan in and it'll and it'll tell you basically give you a more easy to read version of an explain plan there's also a there's a project out there called PGTune that some people use which will help recommend settings PGSTAT statements is built in I mean looking at that is very useful anyone else have a quick question okay Joe will you be available later for people to ask questions and hackers out or maybe around the corner and the yeah I'm actually room hosting quite a bit but I'll be around the conference until Friday and if you can corner me someplace maybe tonight during the reception or something I'd be happy to spend more time with you well thank you Joe very much for your presentation let's give him a hand for him thank you so much


Hello, everyone. Today we will be having a talk on Building a Debian Derivative by Alex Doyle. So please enjoy. Thank you. Hi. So let me just say straight up that I had a lot of 20 minutes for this talk, and then 10 minutes for questions, and I've got 20 totals, so I will be blowing through a few things. So some of the slides, they will be posted up wherever we get the slides posted. I'll figure that out later. But let me introduce myself. I'm Alex Doyle. I'm a Build and Release Engineer at Cumulus Networks. I helped Debianize our build system about three years ago. It went from a monolithic image build to a build that actually built packages and could be upgraded a little more dynamically than it was. Because it's Debian, it's a much more sustainable development model with being able to churn out bug fixes and stuff. So I've got some familiarity with how Cumulus Linux builds. In today's talk, I'll be talking about what makes Debian derivative different from Debian. The components of a derivative build system using Cumulus Linux as an example. And I won't be doing a comparison with existing build systems. I think with derivatives, the requirements of what they're supposed to achieve differ enough that you can't really get a good one-to-one corresponding there. And I'll conclude the talk with problems to expect if you're dealing with a derivative and lessons to be learned from that. This talk is an introductory level talk. I wanted to try and broaden the audience, so I won't be delving too deeply into build system details until the very end. There's going to be a lot of introduction at the beginning. So if you're familiar with how build systems work, the first five minutes of this may be a little boring. So the example I'll be using is Cumulus Linux. It's an operating system for white box network switches. The logical question is what's a white box network switch? And it's a switch where the owner can install their choice of operating system. They're usually less expensive than switches with proprietary operating systems. You can think of it as an embedded PC where the network controller has a dedicated switching ASIC and a whole lot of network ports on it. So as a Debian derivative, Cumulus Linux pretty much looks like Debian, plus switch ASIC code from vendors like Broadcom or Melanox, adding Cumulus-specific software to help interface to the chip and make everything manageable, and then you've got Cumulus Linux. So why would anybody create a derivative? Well, basically a derivative is Debian plus other stuff, and the reasons for other stuff may include software that can't be added to Debian. Like I said, the ASIC code is proprietary. There may be strict package interdependencies. Cumulus uses its own version of the kernel, and the ASIC code is tied very hard into that. I'll talk a little bit about that later. You might need a custom installer for your target hardware, in which case we're installing on switches, a little different than PCs. And you may need to deliver bug fixes faster than upstreaming allows, which is not to say that Cumulus doesn't upstream fixes. We do. It just takes a little longer than them getting out into the field with a regular release. But the Linux kernel networking and free-range routing projects are just two examples where we've contributed a lot of code. So common requirements for derivative. Generally to produce it, you have to build custom packages. You have to include Debian packages. You'll need an installer to deploy it sometimes, and then you need to make repositories and the installer available. So components you're really probably going to need. Build hardware, obviously. At Cumulus, we use shared build systems. We find it's easier to support and maintain a couple of identically configured build systems rather than everybody's laptop. You'll need something for source control. It's Git, because it's well supported. It works, and most people kind of at least a little bit understand it. If you have multiple users using the same build system, you'll need some way of isolating them because as they do build, dependencies will come in that will change the configuration of the system, and hilarity will ensue. So we use trutes. I don't know if anybody's got the pronunciation on that. Okay. Good. I didn't lose a bet there. And it creates a virtual file system to isolate builds. So when the dependencies install, they don't interrupt anybody else. Then once you've had that environment, something to help build with it, we use sBuild, which is also used by the Debian build systems, which sets up the true build environment, installs dependencies, build packages, runs sanity checks on the packaging, and cleans up after itself. Once you've got a package, you need to use ReBrow. So we use Depload to do that. And then once it's in the repository, you need something to actually be the repository software. We use ReBrow, which has nice advantages of being able to sync some packages from upstream and then handle adding our own. Things I found that were nice to have when we started developing things. Server configuration build scripts. You can use Bash, Ansible, whatever works. Basically, the idea is to get a reproducible build server setup. You don't want magic machines where your code only builds on this one particular system because reasons, it's super irritating to deal with. And if you have the scripts, you know what the standard is, and you can apply them to virtual machines and run experiments. So that when you're trying to introduce new features, it's easy enough to check, hide your mistakes from everybody else and then publish stuff that works. You may need build and test automation, so we've got Jenkins to run builds when code gets checked in and to run tests on new builds. Because we're a distribution, we'll be shipping a collection of packages and we need to track that somewhere. So we have a release manifest file. It's basically a collection of packages and versions. It gets used by our installer code to use when installing on a switch. So it basically bundles up all the libraries and takes them with it. And it's also handy for creating previous releases for bug tracking purposes. There's our installer program, which I just mentioned, which puts our code on the target to enable any hardware quirks of the target. You know, hardware tends to have a number of differences, especially when you're dealing with switches, so being able to understand this is slightly different than that or this is a different model, super helpful. And then in the development of this, I found I really needed one tool that would build any package. I'd originally thought I could do this with Jenkins, but I ended up using a writing a cumulus master build tool. Other distributions may have different tools for this. But I needed something that would put all the package build quirks in one place, and it's a good interface for the automation to connect to it, like Jenkins. And then I also found out that with doing a derivative, it was really helpful to have a build tool wrapper, which would help enforce conventions when doing builds and could also simplify the user learning curve by setting defaults. I'll talk a little bit about this in a minute.  Once the code is committed, Jenkins takes over because it triggers on a code commit, checks out the code from Git, builds the package using the master build tool wrapper, it uploads the build package into the local repository, repro updates its index of repository packages to make it available, and then the distribution manifest file is updated because the new package is now available. But these are general workflows that theoretically work in theory, obviously, but in the real world there's a couple of obnoxious details to take a look at, and I've picked out three of them. One, there's a management problem with understanding where packages come from and what you're based off of. And in Cumulus, we have three different types of packages sort of defined by where they're sourced. First, there's unmodified Debian upstream packages where we pull them from upstream repo and just put them into our local repository mirror because we don't modify them at all. And I'd just like to take a minute here to say a big thank you to the Debian security team for those security patches because every time one comes down and I don't have to deal with it apart from putting it in my repository, I am super happy. Then there's patch packages. So we take upstream source, put Cumulus patches on it, build it and then upload it to our local mirror. The kernel and free range routing, as I've mentioned previously are two examples of that. Then there's Cumulus-owned packages where we are effectively the upstream where we just build and upload. Examples of that would be the proprietary ASIC code from vendors that we have to manage and interface code. The second thing to consider is software release tracks. At Cumulus, we've got two software tracks. One is DEV, which is the initial developer commit, should work, code there needs test and released where the code has been tested in DEV and is promoted by the package maintainer and it's good to ship. So because we have two release tracks we need to double everything else. So we need DEV and release versions of obviously Git has two, you promote from DEV to release. We need two package repositories, one to hold code for the development code and one to hold code for the release. And then we need built-shoot environments that will reference either one of them depending on what we're building. So it's an additional complication. The third thing was reasonable version numbering. So one of the things I had to struggle with coming in was thinking, well, we're building our packages two, three times a day. Aren't we going to have ridiculous version numbers by the time we release? Unfortunately, Debian versioning is thought of this, that if you have a Tilde in the version number, it creates a pre-release package where its version is lower than the package without the Tilde. So for example, package foo1.2 is a higher version number than foo1.2 Tilde7 and foo1.2 Tilde8 is a higher version than foo1.2 Tilde7. When we want to release, we build without the Tilde in the name in the version string, and it will install over the dev builds. So this has a couple of advantages, but since we're doing, you know, messing around with versions anyway, if we put a time stamp in there, then that guarantees new builds have higher priority than old ones and we have an idea of when the package was built. And while we're messing around there anyway, we can put a git hash in so QA people can immediately correlate with a developer to say, hey, this version of code looks like it's causing the problem, take a look at it. And so what this looks like in practice, this is a dev build of the free range routing package. And just to break this down really quickly, the FRR40, that's the upstream version that's based off of the plus CL3U1 is the cumulus patch version that's been applied to that. Then after the Tilde, we have the time stamp starting with 15 and the git hash starting with the .9F and then AMD64 dev means it's AMD64 binary. When you release all that stuff with a time stamp and git hash and Tilde is gone. So lessons learned or I didn't see that coming. So four things that turned out to be more complicated than I initially expected. The first thing was the master build tool. So getting back to this, there were a surprising number of build quirks that I hadn't expected. I thought I could just have Jenkins run sbuild and that would be it. But there are things like the kernel where it has to run make to be able to generate its dev in control file so that sbuild has something to work off of. There's things like putting the pre-release package version in with the time stamp and git hash. The release track that you're trying to build for. What git branch do you want to use? Being able to set that to head or roll it back one version if there's something in there that's breaking the build for the time being is really handy. Determining what repository you're uploading to. Is it dev? Is it release? Updating the installer manifest which has to happen after the build. And then it's also a good point for Jenkins automation so our Jenkins master doesn't have to be an expert on package building and I don't have to know a whole lot about Jenkins. And it also provides the option for me to debug a build without using Jenkins which makes it a little more portable if you happen to be building on a laptop. Then there was a surprising amount of social engineering involved in rolling out build tools and getting people to use them. So things to keep in mind about some developers and I certainly followed to this category so I'm kind of comfortable propagating the stereotype. But they want to build fast which builds clean. Nobody's rebuilding from scratch every time. You want to iterate as fast as possible and get reasonable results every time. Because you're there to write code and run code and build time is downtime. And with this mindset the build tools can sometimes be seen as restraints rather than supports. The corollary to that is any build system that sucks will be worked around. And this will cause to quote Thomas the tank engine confusion and delay as things are slightly different for reasons that aren't immediately obvious. And I think there's maybe a bit more of this mindset with a derivative where the end product focus isn't deviant. Not all developers want to have to understand things like packaging because they're here for the other stuff. That's why they were hired. That's what they're working on. It's not their area of expertise. So sort of given this, how do we work with it? First thing I found is the developers are customers. You incorporate their feedback wherever you can. Don't give them a reason to work around the build system. If there are features that will make their lives easier and you can implement them do. When you can have the developer and the build environments match when developers start debugging something they can use a copy of the build that the build system uses. So they're starting from a common base of a common build environment and that helps keep the build from breaking. I think maybe once every three four months somebody checks in something that breaks a build that built in their environment that didn't on the build tools. So that's been really helpful. And I found it was really useful to simplify common build tasks with a wrapper script. I found it helped enforce consistency as default values were automatically applied. It reduced steps because there would be a series of defaults that would have to be set and by wrapping it it would just apply these things. So that cut down on the opportunities for mistakes. For me it just made support easier because I had less to tell them on the other end as to what to do. And by wrapping it it also gave me the opportunity to upgrade tools without changing the developer workflows. The default shrewt somebody is supposed to be using has been upgraded. I changed the version number that I'm using with it and nobody knows and stuff still works. Or it doesn't and I can immediately roll it back without disrupting people provided nobody caught me. One of the things that comes up unusually frequently when somebody mentions the repo you need to get clarification on that. Are they talking about the Debian package repo? Is it Git repo? Python repo? Did they really mean Docker registry? Is my car being repossessed? Do I need to ride home? I mean any of this stuff it comes up shockingly frequently and because of the semantic overlap with source code repository and Debian package repository you can be talking about cross purposes for about two minutes before somebody goes wait a minute. You meant the other thing. Okay. Supporting multiple architectures got a little bit tricky. Cumulus Linux supports AMD64 and RML based switches and it's been surprisingly difficult I would have thought I could have just found an RML based server somewhere and built off of that and that just hasn't been the case. So we tried a number of workarounds which I'll skip over here for time but we ended up, one of our engineers figured out had run Jesse on a Chromebook and we built off of that which I believe makes us an official Silicon Valley startup because we built on hardware we bought at Fry's. We did eventually find ARM based servers but this was three years after the fact. Another interesting thing with type all packages that because they run on anything they're not architecture specific if you think of things like bash scripts or Python they can basically be run anywhere and when you're building for multiple architectures you don't want to build type all packages for multiple architectures. You want to pick one architecture and go with it and the problem is that if a package builds an architecture specific type and a type all the binary from AMD64 and the all from AMD64 will upload and then the binary from RML and the type all from upload will, type all from the RML will upload and displace the AMD64 one which leads to check some errors when you try to download AMD64 because there's a mismatch in the repository. It's the same package same version, different checksum. So we just let AMD64 do the builds because it's a faster architecture and prohibit RML from building type all. And that seems to work pretty well. And lastly, I wanted to cover building things that depend on other things that you build. This was a question that had come up I think in Montreal, somebody had asked it in a buff. So in Cumulus Linux there's a lot of packages that depend on the kernel. The build chain is kernel headers and the SDKs depend on those the stuff the SDK builds depend on the SDKs. Cumulus code has to sync up with the SDK build. So kernel changes have to propagate downward and build all of this stuff at once. And I'm sure there's a good algorithm someplace for doing this, but the number of packages affected by this was small enough that I just mapped it in the master build tool to say, hey, if anything in this chain builds build everything below it. And there's also a question though of when you're doing this, the mechanics of getting the dependencies to the next package in the chain. So initially I thought, well this is great the kernel will build, it will upload its build products, the ASIC code needs to build, it downloads the kernel stuff and it puts this stuff up and it keeps going back and forth until finally everything is built. Which does work, the problem is this takes a while and it leaves the repository in a consistent state where you're getting future code in present code and when somebody downloads them they don't work. And so the solution for this which is not an incredibly general solution but something specific that took us a while to find and I just want to share it with everybody. If you're using sbuild there's the additional packages flag that will take locally built Debs and bring them into your build environment. So in building the build chain the locally built Debs are kept around and then they're added to the next build and so on and so forth and at the end everything's built and then we push that all into the repository at once. So in conclusion this should be enough to get somebody started on derivative. It's certainly a roadmap I would have liked to have had three years ago when I was doing the build system. Hopefully it's provided some insight into derivative requirements, ways to use existing build tools, deployment issues ones likely to encounter and how to create a build system that keeps everybody happy. Are there questions? I think we've got two or three minutes left. Thank you. Hi my name is Riku Voipia and I've also worked on derivative builds and very much of the similar stuff that has come in front of my job. We also started with Jenkinsen building our own stuff around it and eventually we ended up replacing all of that with open build service. That is a tool created by OpenSuset but it can also build Debs packages. Cool. It has its own idiosynchronicies but it can handle all these things like having all an AMD64 and ARM64 packages coming out from the same package and not overwrite each others. Excellent. If you're free I'd like to touch base with anybody that's dealt with any of this or any of these issues I'll be at the conference all week but being able to share similar points of view and run into different ways we've solved problems is one of the things I love about these conferences so if anybody wants to look me up please do. Any other questions? Alright I think we're good. Thank you very much.


Hi, my name is Liam Rowe currency trader and trading coach at trading 180.com welcome to this week's supply and demand for us and gold fundamental and technical analysis For those of you who are new or warm welcome to you and the turning Watchers and equally warm welcome to you and for anyone who's watching right now press the like subscribe share button With your fellow colleagues as it helps really get this quality content out to the Traders that need it by helping me boost it. I guess in the rankings for In on YouTube and other social media platforms So again just a bit of an overview in case you're new about the trading 180 process We apply fundamental analysis to establish directional bias and then a technical Analysis of plan demand strategies to time our trade entries risk management and establish profit targets This is not one versus the other we use You know both to make the best trading decisions, right? so getting on to the weekly Week ahead I guess and some of the fundamental analysis from trading economics And I guess a short synopsis would be that the war in Ukraine would again continue to wait on the markets next week With any breakthrough on ceasefire unlikely to happen any time soon. Yeah, there was a recent Think Joe Biden was talking about it. Well, he called The enemy a pew in the butcher, right? So Those are fighting words. I guess Investors also will closely Follow the US paywall report and personal consumption expenditure data for any clues on how fast the Fed will tighten the monetary policy So high-key interest rates and then there's some some in-depth detail in here But actually we'll go over a few of these things. I wanted to make it too long Because I know you guys are here for not only the fundamentals and and some other technical analysis as well as far as supply and demand But it'll be a busy week in the US With all eyes on March paywall reports and personal consumption expenditures Markets expected a 475,000 employment gain and a 3.7 percent rate of unemployment another sign of a tight labor market and strengthening for the case Strengthening the case for the Fed to deliver a 50 basis points hike in May. So that's definitely something that's going to be watched We've got as is elsewhere in America of manufacturing PMIs for Canada Alongside monthly GDP for Canada. That's going to be something to watch Just skimming through this right now in Europe key inflation report will be released We're released in the eurozone including those for Germany France Italy and Spain in the euro area the headline HICP rate, which is a measure of inflation I think it's called harmony Inflation consumer prices. I think that's what it is Rate is expected to surge again in March reaching and Reaching a new all-time high of 6.5 year-on-year compared to 5.9 in February And investors will also keep an eye on the euro business Survey in the United Kingdom Will be publishing the final estimates for fourth quarter GDP growth And manufacturing also incur alongside the current account Bank of England's monetary indicators That's going to be Important as well. And then in Asia the week will be dominated by PMI readings with reports From major economies including China, Japan, Australia and South korea Also as well from a risk sentiment perspective We've got it says the PMI as I said to reveal how manufacturers are being impacted by strong inflationary pressures Intensified by the supply disruptions caused by the war in Ukraine and China's effort to contain widespread virus outbreaks So that's another risk That's on the horizon if if China starts to slow down and it does look like they are again We could you know the case for buying, you know safe haven Assets at least is you know, like gold and silver definitely is if it's not already obvious Then that's definitely something that we should be looking at trading on on pullbacks So with that being said, I think that is pretty much it Yeah, so hold on to his one as well So Australia's retail sales gorge for February will be monitored to access the health of the consumer spending in the country as analysts look for strong cues Of economic recovery to give the green light for a rate hike by the reserve bank of Australia in the middle of the year so again, we'll get on to the Australian dollar, but The Australian dollar price action has been doing really well. I'm not only from a commodities It's a commodity currency and commodities are rising in price, but the fact that they've got The rumour that the australian the rba may look to high crates, right? So that's also going to strengthen the currency. Anyways, let's get on to the technicals and starting off as we usually do on the dx y Which is dollar index dollar index is a measure of dollar strength against major currencies some of them being the the yen the euro and the and the british pound So i'm going to start drawing in some of these demand zones again For what I start off on a bit of a fresh chart Um, so we do have an area Where price has and my bias is really to the long side to be fair Um At the moment there's nothing really of interest from a shorting perspective that i'm interested in looking to short the dollar um There is I hesitate to kind of put that as a as a demand zone There is demand there, but it's it really isn't for me the strongest area of demand I think if it prices come down to that 97 40 area I do like that area as a level of demand And again, that would be confluence why so, you know dollar does have to sell off Before the potential hike then that's really the buying opportunity so um And again, just why would I be looking to buy the dollar not necessarily based on the fact that you know, it's been in an uptrend That makes um, you know, that's not how we trade trading 180 We're understanding why traders and why the markets likely to move higher in the uh medium to long term and uh one of the One of the things or some of the things are the fact that goldman sacks sees the fed hiking 50 basis points at may and june meetings Right, so goldman sacks some of the smartest guys in the world um Now expect the federal reserve to raise rates by 50 basis points at both may and june policy meetings right so Hiking rates generally is supportive of currency appreciates a currency and that's the intention right is to appreciate the currency We also have um fed williams saying that a half a point hike on Is on the table if needed. So, you know, that just confirms What the federal reserve particularly one of the members is actually thinking um So decisions to be made based on economic outlook williams says And says that the medium to long term inflation expectations are stable So medium to long term, but um, I do think that uh again, it just confirms really You know any kind of pullbacks on the dollar for now? Anyway, at least in the short term buying opportunities again, things can change Moving on to uh janet yellen again. Yellen sees no weakness in the u.s economy even with global growth dented so In order for the fed to hike rates The the the economy needs to be able to support it and the economy should be really growing in line with a hike in interest rates, sorry said inflation before in a hike in interest rates So, um janet yellen Said that she expects the u.s economy to remain resilient even as higher energy and commodity costs triggered by russia invasion of ukraine Then the outlook for global growth, right? So globally, um, china could bring that down by the way So we do have to keep an eye on that but for now, obviously the sentiment is you know to the upside and really just to Kind of confirm that as well in the numbers is that jobless rate falls nearly In nearly every u.s state with 12 at record lows, right? So unemployment in a growing economy what you have is high employment and low unemployment unemployment jobless rate Um, you know, you want that that number to go down and that's really you know, what's confirming Uh, the economy, right? So the unemployment rate fell last month in just more than half of us states Including the 12 where jobless rates dropped to record Lows so that's always a positive a positive sign Um, especially if you're obviously living in the in the u.s, right? But for us traders We're looking at um, you know why we would want to buy the dollar, right? So all signs really fundamentally are pointing to a stronger Economy, uh fed rate uh hiking rates, right? So with that being said, it's a no-brainer and it's been a no-brainer For a very long time and the guys in my private, you know, mentoring group would understand exactly why and just a quick reminder I guess as i'm talking about that That we do have uh enrollment will open on the 28th of march So we've got one day nine hours from the recording of this video and not only do you get the supply and demand Zone strategies capture pain relief stop hunts. Um, and and all that good stuff as well. Um, I'm you know, personally You know mentoring, you know the groups. I'm in there pretty much five days a week or six days a week. Um posting, you know videos and helping guys to really get to where they want to be and uh, I have also have You know an interviews playlist where you know, you can look for it on my channel on the youtube channel Where you can listen to traders who are in the group talk about their experience and the results that they've been getting So, um, just a reminder for anyone who is interested 28th of march 2022 So that's uh tomorrow, right? It's it's it opens and uh, i'm only going to be opening it from for really About maybe about five to seven days. I haven't decided yet, but I do want to um, I don't I don't really necessarily keep it open for that long Anyways getting back to the dollar So pretty much my bias is to the buy side any fresh areas of demand Really, you know confidence areas when buying other currencies like for example the dollar yen dollar swiss Talking about the dollar yen. Let's move on to the dollar yen. So dollar yen and um Yeah, we've got uh, this this this literally look at that price action that is literally unsustainable We cannot continue going higher. Um, you know for much longer I know a lot of traders including myself have been waiting to get involved in this But um, when you've got strong fundamentals and sentiment against you well for you matter of fact I shouldn't even say against you But um, you know, we were looking to go long but sometimes, you know, these things happen where you have to wait for a pullback Right, we're always waiting for at least a bargain So we were waiting here and it just didn't come down or anything like that Now we're waiting for, you know certain uh setups to uh to occur Unfortunately, you know, haven't been able to get involved in this trade But we have been fortunate enough to get involved in some others. Um, but anyways, uh dollar yen again, this was uh entirely, uh, I guess predictable not to say that You know, we're predicting where price is going to go But you know, we're talking about just likelihoods of prices doing what they're doing and um If anyone who's been paying attention and watches these videos on a regular basis and goes back over the You know the past year have been pretty much saying, you know, um to buy, you know Me for anyway is buying the the dollar against against the yen, right and uh HSBC In one of their latest reports a weaker yen continue amid uh amid higher us yields and deficits and just one of the key takeaways from this Some of the key takeaways from this is that the dollar yen surge to levels not seen since 2016 Mainly due to dollar yen monetary policy diverges. I talk about this all the time This is what fundamental analysis is is just a trade idea, right? One of one of the trade ideas anyways is is divergences in monetary policy, right? Which central bank is hiking rates, which one is cutting which one is um is is holding rates, right? So Um, fading risk aversion in japan's current account deficit may also contributed to a weaker yen. So, um Interesting, right says but the main driver should be fed yen policy divergence Now we do have to um You know be be cautious and be aware That risk off, you know, if it does intensify that pushes prices down really to where we want to be buyers, right? So if there's risk off sentiment for me anyway, again, I can't tell you what to do at home But uh, wherever you're watching this but for me I'm not looking to short because anyone who's been getting short on this has been getting Absolutely killed right traders would have been getting short at that level there In getting killed, you know looking at, you know zooming out looking at historical levels to try and get short Maybe somewhere around there. They would have been looking at Yep right there They would have tried to get short there thinking. Oh, you know, it's gonna it's gone up high enough, right? And then they've got you know caught in there in that trade, right? Or they've been stopped out or worse not even stopped out They would have suffered something from loss aversion bias Which would have caused them to move and remove their stop loss And now they're in you know, they considered risking one percent on their account down here Now they might be down I don't know 30 percent for example 40 percent 50 percent because they ever because of their inability to To accept the loss, right? So for me if prices come back down to this 119 area That's where I really want to be a buyer also as well just One of the confluences that I use Is is looking at Is moving moving fair value? I don't really want to call them moving averages because an average is really a fair value, right? When it comes to talking about price. So this blue Moving fair value is the monthly Which is the 21 day Because there's 21 days in a trading month. So this is the one month Moving fair value and where fair value at the moment is is at 118s to 117s. That's a simple and the exponential moving fair value Indicator so Anything above this any any when prices go above that what you're actually saying is is that this is an expensive price And you nobody wants to be buying expensive prices, you know Regardless whether it continues to go higher if you do good luck to you on that one But personally i'm a bargain hunter or at least you know the minute fair value Looking at you know this exchange rate to kind of come down because it always does again as I was saying this price action is totally Unsustainable, you know anything that goes, you know up like this when you've got you know several bullish candles in a row Higher closes a little bit tiny bit of a pause there but just continues going higher Eventually has to come down for several reasons, which I won't get into in this video, but Um For me my bias has been to the to the long side anyone who's been just trading technical analysis and going short at levels Historical levels has literally been losing money. Sometimes it's best to sit on your hands And we have to be patient and hopefully you know, I say hopefully it's just you know It will come back down to to certain levels and uh, hopefully that also aligns with you know The the moving fair value as a bit of confluence and yeah, we know that we're going to be if that's an expensive area Right and that potentially is a fair value area right fair value Then at least we do have some upside potential if prices do react around there, right because I'm not saying that they will but That's one of the areas that we're looking towards or looking to take When it comes to looking at, you know, uh, uh buy my bias anyways anyways, let's uh, let's move on spend enough time on the uh on the dollar yen So now looking at the dollar swiss dollar swiss and uh, many traders got involved in this in the group Uh, we're waiting for this pullback. There's always a pullback as I say Um, I was in this from down here. I didn't manage to add in in fact, I was saying to the guys in the group I got in on the uh on the pound um pound swiss on a stop hunt um Which were which actually worked out to be profitable got a nice two to one of one of my positions And now swing trading, uh, you know the the final position Hopefully for a few hundred pips if we can get even more than that But I got in really down here and I was waiting to add in so I'm already in this trade to be fair But I was waiting to add in but I was recording a video and editing a video and uh when I saw the entry I think prices had gone a bit too far for my liking Um, so I didn't want to chase price. So if prices do pull back before going higher I probably will look to look for an entry in and around that zone there Um But let's see what happens. But uh, yeah, so this is no matter what happens This is a profitable trade for me, but I know some guys who missed out on getting long down here This was a nice opportunity to get long and hopefully with the Um the dollar looking to strengthen and the swiss frank not you know, swiss national bank Not really looking to do anything with with rates at the moment, although their economy is doing okay Um I think this should want to move higher or at least, you know retest these the 94s So let's see. Let's see what happens. Um from a selling trade perspective from a supply zone You do have a nice supply zone here If you did want to look for any kind of short trades, I think that's probably the best area Technically we could look at that area there, but um, yeah Because it's lower highs lower lows, but um, again, it's not necessarily the strongest level of of supply for For certain reasons. So if you did want to get short, I do think around this 93 50 Half number would be decent for a short, but I think the best area would be definitely the highs moving on to the dollar cad And the the cad has been really kind of strengthening Over the past, you know, a couple weeks matter of fact against the Against the dollar commodity currencies not really interested in in in trading this pair Simply because you've got two Central banks that are looking to high crates. So it's a bit more of a difficult trade to take in my book Um, I'm really, you know focused more on the divergences So you've got a decent zone here If you are looking to be a buyer In this area and you think that dollar is an absolute bargain against the canadian dollar That's a decent zone to get long if not if you're looking for short trades And looking to buy the canadian dollar then really I think your your supply zone is already going to be all the way up here Um for a for a short trade. So you've got quite a few pips Pull back currently for for any kind of Short trades and looking to buy the canadian dollar versus the u.s. Dollar, but for me, it's not on my Not on my currency pairs to trade New Zealand dollar u.s. Dollar equally. So I'm not really looking to take this currency pair not trading this at all again. You've got two central banks hiking rates But you've had the new zealand dollar Making higher highs at the moment according to our fundamental analysis spreadsheet You've got the new zealand dollar is actually ranked number one, right? One being the strongest eight being the uh weakest the dollar is actually ranked number two Um on here and you can see really the divergence dollar yen two versus eight So that's you know, that's that's not still as long and it's been that way for for a while matter of fact And you can see the others You know, we've got a whole load of currency pairs. We've got all the currency pairs the major currency pairs. Anyway, think about me about 30 on here Which if you join you get access to anyways, um So for but for me, you know, when you've got one versus two ranked Again, it's a harder trade to take is very difficult to know which one should be the clear winner in it You know in a straight fight, right?  Which are equal in strength. It's a harder bet, right? Why not bet on, you know A strong versus a weak team and if you've got two strong teams, you know competing against each other It's very difficult to know who's going to you know, win in the in the short term medium or long term Anyways, any pullbacks if you do want to get long and that should really should be a demand zone Is a decent trade Technically or if you're looking for any kind of sell trades looking at, you know, buying the the us dollar Then that's really where you want to look for any kind of Short trades, but for me again, not really a pair that i'm i'm interested in It's a decent zone down here. I think as well from a demand zone perspective But um, but yeah move on pound dollar pound dollar again, not really my uh, my trade You've got again two central banks looking to High crates or that are hiking rates at the moment You've got a supply zone right here Um, you do have actually a demand zone currently right in there So prices are in that demand zone I think prices should probably end up ranging between this high and this low right here And I say ranging but auctioning between these two Fair value auction So any kind of buy trades if you want to be a buyer of the british pound and you think the british pound is a bargain Fundamentally against the dollar. I could probably say that would be the the best area This is between an expensive area and a bargain area for the pound. This is fair value Fair value is okay, but again between these two very difficult to To kind of derive which where where where uh Where value really is again, I think probably my bias would be more again to the short term Short term trading would be you know just to short the the the the pound against the dollars meaning that i'm looking to buy the dollar Basically Direction of travel should be to the downside the pound though on the other hand One second UK Hot UK inflation spurs biggest bets on rate cuts in 15 years strange one Traders see more than a quarter point reduction in two years bets come after Bank of England warns of weaker growth as cost surge so In fact, it's not really a strange one. Um, what happens is and I was explaining to the guys in the group is that inflation Prices when they get a bit out of control What happens is is that it can hurt the economy? Yeah, and this is one of the things as to why and I guess in the in the medium to long term Why i'm probably going to be a a shorter of the pound is that higher inflation um Basically causes people to stop spending in the economy because You've got rising living costs and living standards start to drop because you're spending more on things like petrol on heating your home and gas and cooking and food etc so In an environment like that People generally think just think about yourself, right? You tend to cut back on more frivolous stuff You know that that you would normally would when you have money in your pocket so There are you know, um Traders actually starting to bet Um on on rate cuts, you know, um, I wouldn't say in any time soon And let me just have a quick read through the article fastest inflation with normally Few expectations for higher interest rates. Yeah, right high interest rates Of course in the uk where price gains are the quickest in three Three decades traders are betting on rate cuts. So again What we would normally see is for interest rate hikes Yeah, but I think traders are getting ahead of themselves Or maybe they're getting ahead of themselves to use that term But they're probably thinking a bit more medium to long term and saying that because inflation is really rising quite rapidly in fact the you know once um inflation The bank of England does does have a cap on inflation get a cap on inflation at some point Then they're going to have to potentially cut rates And this is basically leading to what's known as potential stagflation. So A very very very difficult Situation for the for the bank of England to to be in when you have inflation that is just shooting through the roof And if you don't have the economy Supporting that then that's known as you know stagflation. So, uh, yeah It's uh, it's a bit of a difficult one for the uh for the pound at the moment And I think short term wise there are um The bank of England will have to Or kind of forced by inflation to hike rates But I think and again, maybe by the end of the year you could see You know the potential for um The the bank of England to either hold rates or look to potentially cut rates towards the end of the year That would would be something but um, it's not a necessary trade idea that i'm uh looking to trade just yet But with that on the horizon at some point maybe later this year next year Um, you know, the pound could be a potential uh short But for now for me the pound is uh is a it's still a buy as long as the bank of england or on their hiking cycle. Anyways um moving on to the euro dollar right euro dollar Uh drawing out the uh supply zones demand zones. I want to be a seller with this currency pair supply and some demand here now My bias is to the downside because i'm buying the u.s. Dollar So when he pull backs into a zone is where i'm looking to get short. There is a zone here Their supplies i'm not the strongest supply zone in the world But it's there um I'd probably look for some sort of potential stop hunt in and around this uh one third one one eleven three area or if not A uh short trade if it does come up to the 112s, but it may or may not I know there's a lot of banks forecasting, you know 109s 108s So this this could start to still roll over but any pull backs. I think are uh Are a shorting opportunities and then we also do have the um the ecb president christine lagarde plays down concerns about Euro stagflation again stagflation being when you have rising inflation, but the economy is not growing right It's it's either flatlining or going in the opposite direction More the contraction um phase of the economic cycle. So that's it again It's a difficult situation and the reason why that would be for the economy is because again the russia russia's war in the ukraine Will have consequences for growth. Yeah for for economic growth. So Um, it's it's a hard one difficult one for them to be in and when you contrast where europe are Right in comparison to where for example The us are in terms of, you know, goldman sacks hiking rates fed williams hike rates Janet yellen says no weakness in the economy Jobless rate of falling and then you go to europe Yeah, and christine the guard is saying, you know that there's potential She's even using the word stagflation. There's a divergence there, right? So for me the path of this resistance should be to the downside I'm not saying it's going to go down this week. Right. I have no idea Right that what's going to happen week to week. I just know that in the medium to long term If that continues then prices should be down here in the same way that we've been saying, you know to short Well, you know been You know the guys have been seeing in the group that we've been shorting All pullbacks, you know on the on the dollar why any pullbacks have been Shorting opportunities Right, that's pretty much it. So that hasn't really changed too much fundamentally or risk sentiment wise But that is subject to change depending on if the, you know, europe can get their act together Moving on to the australian dollar us dollar and the ozzy dollar again A bit of a difficult one the um to trade fundamentally the uh, the australian dollar has been in absolutely brilliantly and we've now, you know talks of rate hikes potentially You know the the market is getting ahead of of the narrative, right? And this is basically what fundamentals is all about is You know, you can't if you're looking to buy when they start up when they actually high crates You're already, you know, too late. You've already been left behind And so if you are looking to buy the australian dollar against the us dollar, which I personally am not doing um, then that's a decent zone to get along if you're looking to buy the us dollar against the uh The australian dollar right now is a decent sure, but for me again, you've got um To even though the us dollar is ahead potentially of course potentially, but it is ahead of the australian dollar When it comes to monetary policy I do think this is a Harder trade to take in my opinion There are better trades to you know buy the dollar against as well as even buy the australian dollar like for example You know as we're coming up to the ozzy yen, right the ozzy yen Ozzy yen look at that right again quite parabolic in nature um The yen being obviously the weaker out of the two Right, um, and we've got Uh again just just we we need prices to really kind of have a pullback or show that there's demand somewhere Right, there's got to be some demand and then a pullback into that zone So, uh, let's see what happens because how many pips is that just eyeballed that maybe about what's that 92 to that's about 700 pips 600 pips it needs to pull back and uh, I don't don't know whether it'll do it anytime soon Probably maybe some sort of risk of sentiment may push in some narrative You know may push prices down um To a degree, but let's see what happens and again just think to kind of show you on here You know that's one of the what's that one two three four one of the fifth strongest divergences at the moment according to our spreadsheet Which is three versus eight, right and again, that's been like that for for a for a while so looking at um and back to the um The uh ozzy yen at the moment just no trade setups, right? Zero trade setups from a buy-in perspective even from a selling perspective and a shorting perspective I think we are Yes, there's nothing really there. There's probably some sort of weekly demand supply zone there But it's not even worth putting in anyways, let's see what happens Let you know the market show you where an expensive area is by pulling back and then look for potential demand zones Or the creation of new demand zones and then moving on to finally gold and gold Moving higher as expected Nice buying opportunity within that zone there and then there was a zone here. So probably delete that one there Nice buying opportunity um coming down to that 19 1900 area prices are going higher and again when you've got The only thing really kind of going against gold if you really want to call it going against gold is just the fact that the dollar isn't Is one of the strongest currencies And the gold and the dollar at the moment are moving in the same direction But when you consider everything else when you consider high inflation where you consider risk off tensions, you know, russia Ukraine when you consider a china potential economic slowdowns And and you know some other others that are you know risk events that are on, you know, the The horizon You know, there is I guess Very there's more reasons to buy gold than than than not right and in fact, I think it was Yeah, it was it was the united overseas bank head of market strategy hing koon How now how sorry sees strong safe haven inflows into gold right so lifting to 2200 dollars an ounce right so Again the the smart money At the moment looking at, you know prices being somewhere around here 2200 dollars right and where we are so any pullbacks If you're lucky enough should be potentially buying opportunities Not saying it's going to bounce off of that let that that area there if it pulls back It could go even deeper right it could go even deeper But the point being is that just because prices do something you have to look at, you know This is as value right this is just value and an opportunity to buy for cheap If nothing fundamentally or risk sentiment has changed so any pullbacks Potentially buying opportunities if things do turn around and everything starts to get sorted out with everything in the world inflation starts to come down and You know Russia Ukraine, you know kiss and make up then in fact this would be a very You know decent short trade Technically very nice. I say it's a decent but very nice short trade We are around these highs so that's where we are with gold and Yeah, quite a detailed one this week and just a quick reminder again in case you missed it that the mentoring is open Tomorrow 28th of March depending on when you're watching this if you're watching this on a Sunday, it would be tomorrow but 28th of March The trading when a discord group is open It will be only be open for a limited time So maybe I might do it for me about five days maybe until Friday or Saturday and then That's going to be it for you know a good few months or so Like to keep the group really kind of concentrated and small and it's really a better way to teach traders When it comes to everyone being on the same page if people start, you know Joining at different times, you know, someone joins in March someone joins in April someone joins in May It's not really the way to you know, we don't do that in school right in education So, you know, there's an intake so everyone can be on the same page. So this week It's going to be open then it's going to be closed for again a few months if you missed out You've been waiting to join trading 180 and the mentoring Mentoring group Then this is you know, your opportunity also as well. Please have a look at the playlist that I've created With really the traders in the group talking about their experience and Results and how trading 180 has really helped them to you know, get to where they want to be in their trading Anyways guys, take care. Have a great week and I'll speak to you all soon


Often at university, you'll be told to use scholarly, academic or peer-reviewed sources to support your ideas in your assignments. But what does this mean? Scholarly sources can also be called academic sources. These include journal articles, books, book chapters, conference papers and theses. Sources such as Wikipedia, newspaper articles, magazines, trade journals, newsletters, blogs, social media sites and personal websites, on the other hand, are not. They can be useful for background reading around a topic, but they're not scholarly resources. So, let's look at what a scholarly source is. A scholarly source is written by researchers within a subject area. Reports on research findings. Contains comprehensive in-text citations and a reference list or bibliography. Users specialise terminology and a formal writing style. And is often peer-reviewed or refereed. A peer-reviewed or refereed journal article is assessed by experts within the field before it is published. If the article does not meet publication standards, it can be rejected or sent back to the author for revision. Peer-reviewed articles should include all the details, including the organisation they work for. An abstract which summarises key points, evidence of research findings, in-text citations and a reference list. Many search tools such as databases and the library catalogue allow you to limit your search to scholarly, academic or peer-reviewed articles. Books can also be scholarly. A scholarly book is written by experts within the field, is published by a reputable, well-known publisher, has a table of contents, provides in-depth information on a topic or subject area, includes an index to look at terms, as well as in-text references and a reference list. Look for these clues when searching for scholarly sources. If you need help, just ask.