Welcome to Free Thoughts from Libertarianism.org and the Cato Institute. I'm Aaron Powell. I'm Trevor Burrus. Today we're joined by our colleague Jason Kuznicki, a research fellow at Cato, an editor of Cato Unbound. Today we want to talk about an essay by the philosopher Robert Nozick on what's a pretty interesting question and one that a lot of people have spent a lot of time trying to answer, including many of us here, which is, as the title of the essay, why do intellectuals oppose capitalism? And I should note the essay is up on libertarianism.org and we will put a link to it in the show notes. So in this short essay, Nozick gives what he thinks is the reason, which has to do with schooling. And so to very briefly summarize, the idea is that intellectuals of the kind he's talking about, and we'll get to his definition of intellectual in a second, tend to do very well in school and they tend to receive a lot of praise in school. And as a result, they think that the way school operates is the way that society ought to operate because each of us would like society to be structured in such a way that we come out on top and we receive the prestige that we think we deserve and that the way that schools are structured looks more like command and control economy, more like socialism than it does like capitalism. Therefore, intellectuals oppose capitalism and support socialism. More or less, yeah. I think that the first important proviso I think we should have on this episode is that we are not doing, or at least I am not doing, I don't know about you guys, but we are not doing a pathologizing of our opponents because it's something I always try and warn people that there's a very big tendency in political discussions and political thought to pathologize your opponents to explain the reasons they disagree with you as some sort of like mental illness. And the classic example of this is at least recent is Dinesh D'Souza's The Roots of Obama's Rage, which was trying to explain why President Obama has pretty milk toast leftist policies via some sort of father complex. And I had never read the book, but this is very dangerous. There's reasons people can disagree with you that are valid, that do not require them to have some sort of mental condition. But it is important to try and explain, you know, when there's a disproportionate amount of people who hold one type of belief in one area, then it is something that is worth an explanation. Like it's not an equal distribution of pro-capitalists and the intellectuals we're talking about.  Well, yeah, I mean, it is a phenomenon and it's fair to observe it. It's something that people on the left have also observed. It's not something that we have made up. It doesn't involve necessarily psychological factors. One way that this has been discussed and in particular by people on the left themselves is that what's happening is what might be called a long march through the institutions. That if you want to achieve social change in a leftist direction, the way to do that is to simply insert yourself into positions of power in important institutions in society and take them over.  It doesn't have to be violent. It can simply be a gradual, peaceful takeover. And people on the left have in fact said that this is what's happening. Now, Nozick doesn't seem to agree with that. He doesn't talk about it really. He has a different set of explanations for it. Well, I think that that's an important point too. And it's also important that we don't necessarily have to criticize them for that because you could say that libertarians have had our own strategy for trying to get people into places of influence or influencing people. Kato, we do it with voluntary contributions more often than government money. Every ideology spends time thinking about how can I convince other people that I'm right. Yeah, once you take out the long march thing, I mean, it's really everybody's doing it. It's just a strategy. Just avoid the loaded term there. And yeah, we're all doing it. It's all that anybody does. And I feel like we ought to acknowledge up front there is also the possibility that the reason that intellectuals oppose capitalism, that the reason intellectuals embrace socialism. I don't think this is true but intellectuals are highly educated people. They know a lot. They tend to be very smart. And maybe it simply is the fact that capitalism doesn't work and that socialism does and that the reason that they oppose capitalism and support socialism is because of that. That they're simply right and that the more you know and the smarter you are, the more likely you are to believe true things and therefore they're correct. That's like the argument that they would of course give. And you hear that all the time. Democratic policy proposals, progressive policy proposals are supported by smart people because they're good proposals. Except in economics departments. In economics departments, people tend to be much, much more supportive of the free market than say in sociology or literature departments. Well, I think that the average professional economist, professor of whatever sort is to the right of his colleagues in the humanities department, maybe substantially so when it comes to sociology or anthropology or things like this and maybe slightly to the right of the average person but maybe not. I'm not sure. I think that I've read this. I think that they're slightly to the left of the average like median. But they're not market anarchists. No. No, no, no. But this is a good time to move into. I mean, Nozick has something very specific in mind when he says intellectuals. And so he would actually I think exclude the economics professors that we're talking about from what he refers to as wordsmith intellectuals. I'm not sure he would but continue. I think that economists as usual are right on the edge because he has this thing with wordsmiths which includes poets and journalists and playwrights. But he singles out wordsmiths versus people who work with numbers. People work with numbers are not a part of his explanatory system. He thinks that he's identified something particular to people who work with words. But are economists more like physicists or are they more like sociologists? We can discuss that. And some, I mean, it also will depend on the economists. But we can bracket that issue for now and say that so what he is talking about when he says intellectuals is something he calls wordsmith intellectuals. And Jason, can you tell us what he means specifically by a wordsmith intellectual? A wordsmith intellectual is someone who shapes the flow of words that other people will see. And that can be in any number of different disciplines. It could be a historian. It could be an anthropologist. It could be a philosopher. It could be any number of different academic specialties. And also, professions outside the academy who are in some way shaped by those disciplines. So people who are TV news commentators, people who are journalists, you're going to see them to some degree share the same characteristics. Now, this is the interesting question here, which he doesn't really deal with. But Aaron and I have actually talked about this in the past, too. Excluding the quote numbersmith people, why? It seems like that's an entire thing you could ask, too. Not just why do intellectuals oppose Kaplan, but why only one subset of intellectuals? Why does someone who spends their entire life studying the cell walls of green algae that exist in the Indian Ocean not terribly prone to one type of ideological belief or another? There seems to be something in that. Isn't that partly explained, though, by his ultimate explanation of why intellectuals, wordsmith intellectuals oppose capitalism is the kind of students who do well in school? But that would be true of the physicists. That's the one thing that really... So we were kind of going through the essay, and it's a very short essay, so you can read it very quickly. But yeah, it's basically as was summarized, sort of not seeing their value in society and having experienced their value in schooling and thinking that they should be valuable like that. But that's true of physicists. Well, it is and it isn't. Nozick says that wordsmith intellectuals or incipient wordsmith intellectuals are praised by their teachers. The meritocracy happens on a face-to-face basis, and you are pulled into a network in which perhaps the people who are already there are, to some degree, socialists. They are already highly critical of capitalism. They praise you. They tell you that you deserve a great deal of prestige. They tell you that you deserve honor and whatever is the best in society. And that validates a particular worldview. With the quantitative people, he doesn't really go into explaining how their professional lives work. But I would say that things probably work differently there because your reputation as a quantitative researcher depends on your experimental results. They depend on having results that are interesting, results that are confirmed by other researchers, results that can be objectively demonstrated. It's not about feeling. It's not about personality. It's about whether or not you can deliver results. And results are a thing that can be agreed upon in a more objective fashion. So, Nozick notes in the beginning of the essay that when you ask the question why do intellectuals oppose capitalism, there are two approaches to answering it. And he's really only concerned with one, and I think we should be clear about that up front. So the first one could be rephrased as what reasons do intellectuals give for being opposed to capitalism? So this one would be answered along the lines of capitalism hurts the poor or leads to inequality or isn't as good and wonderful and utopian as socialism. So if you were to ask an intellectual what do you think is wrong with capitalism? These are the reasons they would give. That's not what Nozick is interested in here. He is interested in the second way of looking at this question, which is how do intellectuals come to hold those views? And so that's how this schooling enters in is because the forces at play that he thinks are pushing them towards socialism and away from capitalism are taking effect before we even get to them holding views about socialism and capitalism. This is stuff that's happening from when their elementary school kids on up, that there's certain things they're exposed to that then when they eventually get to the question of what economic system is best influence how they think about it. And he's concerned about this question also because it's more than just an interesting like here's a group of people who seem to disproportionately in a certain way why. He's concerned. He thinks answering this has weight because intellectuals and especially wordsmith intellectuals are very influential in society. They shape the opinions of non-intellectuals of most other people in society in a way that other categories of people don't. I think it's worth bringing up as a point of reference as a footnote to your point that that's also in Hayek's view the intellectuals in society, which was his essay where he explained how he thinks you change the world. He was interested in influencing what he called secondhand dealers and ideas, which were the intelligent people, including lawyers and journalists and things like this. Not necessarily the Nobel Prize winning professor, but the people who talk about the Nobel Prize winning professors work who influence the people around them there and when they go to bars and when they have dinner parties. And he said we have to get those people to believe in capitalism before we can really change the world. And so as a footnote, that's a similar type of thing that people that he's focusing on, I think, in Nozick's essay is similar here. Right. Because how many people actually sit down and read academic papers by Nobel Prize winners? Most people don't. They will read the popular accounts of those papers. And that's where those secondhand dealers and ideas become very, very important. Now, to get back to Nozick just a little bit, he says in this essay that the schools are quote, the major non-familial society that children learn to operate in and quote. And that's I think very, very important because if you learn in school what society is like, then you will presumably think that school is what society should be like when society turns out to be different from school. It will be evaluated as a failure. I want to go more into that because the most interesting thing that I find in this essay is the discussion of the value that the intellectual holds for himself. So you have this part in the value of intellectuals where he says, intellectuals now expect to be the most highly valued people in the society, those with the most prestige and power, those with the greatest rewards. Intellectuals feel entitled to this. But by and large, a capitalist society does not honor its intellectuals, which becomes a sort of weird type of projection of this and say, well, there's something wrong with the society that doesn't adequately value the things that are valuable. And this is the first point in the essay where I kind of wanted to push back at Nozick because I don't think our society fails to value intellectuals. I think they certainly get a very high level of prestige. Maybe it's never enough. I know that Robin Hansen has said that people are fundamentally motivated by prestige and that more than money, more than almost anything, what people want is to be respected and loved by those around them to be told that they're really important, to be told that society needs them. And we say this about our intellectuals. So for example, even in my relatively conservative family, if I were to say, hey, guess what? I got a tenure-track position at Harvard. That would command respect. Now they might say, oh, well, Harvard.  That's prestigious. That's important. And maybe that's not enough, but it's something. And it's certainly something on par with almost any other significant form of prestige in our society. I think of a couple of ways to maybe answer that from Nozick's perspective. The first would be that on the broad level, for every Cornel West or for every Paul Krugman, there are countless intellectuals, countless people who did very well in school and went on to become professors or worse adjunct professors who are on a relative level not getting much acknowledgement. Your family might think it's really neat that you get a tenure-track position at Harvard, but the broader society is not paying attention to that kind of thing. They don't know who the Harvard faculty are except for the handful of very, very famous ones. So there isn't a lot of prestige outside of your small circle. And then along similar lines, the kinds of people who have very broad-level prestige, again, setting aside the handful, the very small handful of super famous academics, are like the guys on Duck Dynasty. Or, you know, they're sports stars or they're actors or they're occasionally... Kim, the Kardashians. Yeah. You know, people who, inexplicably, or they're occasionally like famous businessmen or Steve Jobs or Bill Gates who, to the intellectual, are decidedly not intellectuals and so perhaps not deserving of the kind of prestige. Well, I want to dovetail off that, too, because I think that Nozick's point is a little bit too inward-facing to the intellectual himself about whether or not he's being valued enough. I think that matters, but I think that the other thing is more of what Aaron's point, the question of whether or not an intellectual thinks, regardless of their specific place, is more of a general proposition, is the world valuing the right things correctly? That question, which is less, is the world valuing me correctly? I think that a humble, realistic intellectual who is the world's foremost authority on Rousseau at Harvard doesn't probably think that that should be the most important goal of society is to acknowledge that Rousseau experts are the preeminent top rung, but he thinks that in general that the kind of things that intellectuals like to do, whether it's contemplative life and not watching Duck Dynasty and listening to the symphonies so why do symphonies have to be supported with public money but these stupid screaming bands are supported with the market system? So the general question is whether, and I think this is a very, very important question in free market thought, which is the dichotomy between someone's perception of what's valuable and how they perceive the world as whether or not it is accurately valuing the valuable things. In many ways, one of the reasons that Marxism is like a religion in many ways is because the many religions say essentially the world is run by false values, like the basic core bottom thing is that the world is run by false values. It should be piety and not money. It should be these and not that. Marxism says this too to some degree. You can always feel this idea in a lot of speeches that the point is that the world is being run by false values and we need to figure out some way to correct that. But to pull this back to, so Winozik, he says there's the two ways of answering the question. So what you're describing is that first way of answering the question, like the intellectual can say look, capitalism means a world based on false values. I think these are the correct values, therefore not capitalism. But to pull it back to that second version of the question, which is how did the intellectuals come about having these ideas and attitudes. That's where we get to the school because what he's saying is that because the intellectuals were in this non-familial community, the community they spend so much of their time in, that all of us spend so much of our time in, which is the school system as we're in our formative years, that that is a society that rewards intellectual kind of things. That you get praise from your teacher when you write the really good paper or you ask the smart question or you can articulate the answers in a way beyond what your fellow students can do.  And we tend to think that a society that we like is the right kind of society. The society that makes us happy is the right kind of society. And so you come to think that the values that are important are the ones that happen to be praised, happen to be rewarded within the school system. And so then when you get out and you look at the society outside of it, that's when you start to say, well, that doesn't look like the school system so there must be something wrong. Yes. And one of the things that's I think key in understanding the market process and sort of sociology of the market is that if the market process is doing its job, if it's doing what it's supposed to do, a lot of its results are actually going to look random to any particular observer. They are going to look as if nothing of value has been provided. They're going to look as if money has been distributed in an arbitrary fashion because what the market process does is to discover previously unknown knowledge. And when it does that, you didn't have that knowledge. By definition, you didn't have it. So your judgments about that are likely to be or have a good chance of being false. You're going to think, aha, the market does things that are crazy. And I have knowledge that enables me to make that judgment and I can judge it. Now, what Hayek's insight into the market was is that look, this is what it's supposed to look like. This is what it's supposed to look like when it's doing its job. And to some extent, those judgments are going to look arbitrary, the judgments that are bestowed by the market because what has happened is someone who happened perhaps simply by randomness to have that piece of knowledge when other people did not was able to monetize it. So I wanted to, I think it's a good point, incredibly good point. And we could bring in Schumpeter and all these other things too. But on Aaron's point about my discussion of values was more up at the first question, the second question. I think it sits on the edge because of what I had said. I think that we could say this is maybe the reason, the substantive reason why they think capitalism is wrong. But I think it's also a person personality. I'm not trying to pathologize here. I think it's the question of false values is sort of a way that you can project from your head out to the world about whether or not these are good things that are happening in the world and good things are produced by this. And then you, the intellectuals are raised in has them projecting out. So it's a combination of both the substantive critique of capitalism. It's false values. But also the projection internally from the intellectual of why they would be prone to believe that specifically. And one isn't really necessarily committed to thinking that either value system is false. I found myself thinking and reading this of Jane Jacobs really, really excellent short book Systems of Survival. And she suggests that there are two different sort of moral paradigms. One of them is commercially oriented. It says things like like be honest in trade, be open to new experiences, welcome strangers and treat them fairly as you would treat your friends. Be, be enterprising, be efficient. And she calls us the commercial, the commercial paradigm. Then she also says there's another one, the guardian paradigm in which you are supposed to do things like respect, honor and hierarchy. And to know your place within a system and to value tradition and continuity. And I found myself thinking well of course the market is in the commercial system and the academy is much more like the guardian system. And Jacobs actually does not want to say that either of these are necessarily wrong or evil. But they have particular roles to play in society. So commerce is great at providing stuff. But it's actually not necessarily so great at providing public goods or providing, say, the defense of stuff. We have governments for a reason and governments operate on the guardian morality. To a great extent I think the academy does also. I wonder if what's going on here somewhat when we talk about these conflicting views of how society ought to be organized, the conflicting value systems and then couple it with your good point about the randomness of the market. Because I'm struck by, I should preface this by saying one of the things that we remark on here at Cato a lot is that when it comes down to it what's really frustrating is how few people are actually in favor of free markets. That anti-truly free markets is not limited to leftist intellectuals. So we get mad at the Republican Party for saying, oh, we support markets. But it's like, no, you don't. Like at every opportunity you want to make interventions to stack the deck in favor of different groups. I'm just prohibiting drugs as anti-market immigration. But that everyone, people tend to dislike all sorts of aspects of markets. Win those markets, don't line up. When the results of those markets don't line up with what they want. And so what my question is, is what Nozick's explaining here not something unique to intellectuals, but is instead simply that we want a system, we think that an economic system ought to align with our tastes and our values. Because we think our tastes and our values are correct, otherwise we wouldn't hold them. And so we reject markets whenever they don't align with our values and think that some other system will be better. We support markets when they seem to align with our values. And by we, I mean people who are not genuinely in favor of free markets. And that might explain a bit of the economists being more in favor of free markets because the economist has studied, has more of a big picture view. They see the overall positive effects of markets.  And so you're like, wow, the market has rewarded something that's awesome. Therefore, markets are good. Wow, the market's not rewarding something that's awesome or it's undermining something that's awesome. Therefore, markets bad. And so is this theory, is Nozick's theory basically too limited? Is the problem that markets look random and so they, randomness is ultimately going to conflict with everyone's values at some point? Well, I think that's a Schumpeterian element which is related. Can you explain what you mean by Schumpeterian? Yeah, the idea that Schumpeter was very interested in whether or not a free market society would be perceived as just by those who were in it. And whether or not because of the randomness of the order, an individual person could look at the world and be like, oh yeah, that guy is probably making the money he should be making as a hedge fund manager. Something that they don't have any idea about and whether or not a fully instantiated market society would be fundamentally unstable because everyone would start looking at it with a kind of skepticism or maybe everyone already does on Aaron's point that it's not just limited intellectuals. Maybe intellectuals have done it because they write and speak more often so maybe we just have like a sampling bias and they systematize it more often and use words like neoliberal and things like this. And so maybe they're just out there more. I suspect that there's still a much higher percentage of certain things in intellectuals than there would be at a random sampling of other people I would say. The question that I was talking about which I maybe goes back to intellectuals themselves and this is something that again could be either type one explanation we were saying or type two but there's something that's pretty common amongst the conservatives is that the intellectuals really believe in expertise and so they believe that society should be run according to expertise. Well this gets to – so Nozick after setting up why intellectuals might be frustrated with capitalism we have to move into the question. The next part of the essay is okay but why socialism right because there's no on its face reason why like just because capitalism doesn't reward intellectual virtues. Like we'll stipulate that because I think the intellectuals are largely wrong about that but let's say they're right that doesn't necessarily mean therefore socialism will. We need to have an argument for why and so what Nozick is saying and this gets to the expertise is if we look at the characteristics of the classroom what you have is a single – usually a single authority figure who stands at the front of the classroom and hands out rewards based on what to the intellectual looks like merit and so what you have is a miniature planned economy. There's stuff comes in. We've got school supplies. We've got prestige. Gold stars. That's not necessarily left or right wing. I mean Nozick actually does mention some intellectuals who were right wing. Yates, Elliott Pound he says and I could easily add many more. You could talk about Carl Schmidt or Martin Heidegger. It does not necessarily have to skew left wing. It could skew right wing authoritarian. So yes, why socialism? Well, I think that – I mean one way I can think to answer that question about why socialism because why aren't there as many right wing intellectuals supporting authoritarianism is if we contextualize it. If we say like let's just look at say the United States or let's look at the western world that there's this central planning angle that they seem – that they like in Nozick's story because the teacher is the central planner and distributor of fairness. But at the same time there's an element of what really matters is intellectual discourse which has an element of freedom, right? Like we want a society where people can engage in this freely and openly and we aren't restricted and so that would seem to cut against authoritarianism and the mistake obviously that – I mean we would argue this is a mistake is that they see socialism as central planning plus freedom in a way that strict authoritarianism would be central planning plus no freedom. Well, it's central planning for stuff they would say and stuff is not really important. Right. What's important is ideas. Right. Ideas are what matter and we will have free ideas under socialism while technocratically managing stuff. Now this is a false picture of socialism. This is not accurate at all. In fact, command economies have to have censorship. They can't get by without it. And when you control stuff you also control people. You control individual lives. It can't be escaped. There's no getting around that. But the question I'm asking is so you have intellectuals – if we take it as a general category, one way to get ahead as an intellectual is to tell people that things are not as you perceive them to be. Something counterintuitive is actually true and that's a basic very abstract thing intellectuals do. Well, that makes it very relational. If you're an intellectual society where – let's imagine – and this is a big if for the reasons you just said, but the Soviet Union where intellectuals were free. That's a big if. If you had a Soviet Union where the university was a thriving system and you had an intellectual class that could say anything. Oh, they tried that for a while and they had to crush it. But they couldn't do that in China. But intellectuals in the Soviet Union in that hypothetical situation be capitalist because the way to push back on – they had produced the counterintuitive idea is to say, hey, maybe the social system doesn't work. Well, when they tried that in China that is exactly what happened. They got ideological deviationism all over the place and not necessarily that they all became little miniature mises or whatever. But they had some odd ideas out there and ones that the regime couldn't put up with. And so, yes, there was a period in China that the Hundred Flowers period and they had to crush it. They had no way of dealing with that kind of dissent. Yeah, the idea here is essentially that the intellectuals as contextual is raging against the machine. And the machine is this abstractive concept. The machine is anti-capitalist and socialist, but if they were allowed to be – and that's, again, that's maybe the best, most important point. But if they were allowed to be in the Soviet Union, the machine would be the socialist because just like intellectual juvenile delinquency is contingent upon what the established standard is, perhaps intellectual and maybe called intellectual delinquency is also contingent upon what the standard is. So they're always going to just basically resist what they perceive to be the predominant functioning social order. Could that – I mean, we may be drifting a bit into the psychologizing that we said we were going to try to avoid, but that part of – so you've gone through this school system where you've been praised and you've been told that you're super smart and that you know what you're talking about. And then the problem is that when you get out into the world, people aren't listening to you. They're not – you know, that the world is organized. The people are doing things that you don't think they should be doing. They have tastes that they're listening to that rock and roll instead of the opera. And so what you want is – you want a system where people listen to you. And so – which is necessarily going to be somewhat different than the system you've got, which would explain why – again, we would say that the Chinese intellectuals who are objecting to communism in favor of capitalism are in fact correct. But if we set aside the notion of the truth value of their views, there is some level of rebellion in the – you need to listen to me and so by listening to me, you should be doing something different from whatever it is you happen to be doing. I think Nozick would say that open and closed societies are asymmetric that way because in a closed society, everybody does know their place. And that's actually one thing that is relatively comfortable about the academy. There's a progression of your academic career. You know how it's supposed to go. It's incredibly structured. It's incredibly closed in that way. And in an open society, nobody really does know their place. You could be very wealthy tomorrow. You could go broke depending on choices that you make. There would be more mobility. And mobility not just in money but in prestige, in location, in tastes and values. Those things change in an open society. And when you're not acculturated to that, it does tend to seem weird. I thought a lot of Carl Popper in his talking about how open societies are unstable as regards individual positions. And that necessarily seems threatening to somebody who has lived their life in a system that is much more orderly, that is much more, you know, there's a course that you follow in the academy. I'm reminded of, I had several professors say something along these lines. But one in particular who she would, we talked about the move from status to contract. I remember this. This is Henry Somervain. Yeah. But so the notion that at one time people's position was defined by their, the status that they were born into. You were born into the nobility and you had that level of status or you were born into serfdom. You had that level of status. And then your role in society was very clearly defined.  And that as we shifted to a society based on contract, a society where you could move around, it upset this very strict status-based society. Which is what you're describing the academy as, is basically a status-based society. I mean, your status can change over time as you age into the different levels of, you know, being a professor and getting tenure and all of that. But it's a, it looks like a status-based society and I had a professor in law school who her, much of her intellectual career was based on arguing that we would be better off returning to a society of status. Where people knew what they owed to each other. That it was a good thing that we had, and I'm being perhaps a bit unfair. Yeah, I asked, was she a liberal or a conservative? She was, she was very far left. Really? But the argument was that it was better because there were going to be people, there were going to be poorer people and that in, and this is a mischaracterization of the Middle Ages to say the least, that the kings knew that they owed things to the serfs and that they had to give things to the serfs. And it wasn't just like, well if the serfs can afford it, I'll give it to them and if they can't, they can't. I won't. It was- An obligation. It was an obligation baked into the nature of society. That's as much as the, yes. And the contract of status. That wasn't why it was better. That was a real thing. So that makes me think, like you're talking about, is it possible then that the, if what we want is that structured society, socialism's a version of that, but a return to, I mean this is the neo-reactionaries that we've run into on Twitter, a return to a society of status. Well, Ezra Pound exactly made this point in a lot of his poetry that now we are without clear markers of status. We don't know where to make a home. We don't know where to, how to build straight angles any longer. We are beset by, we call it Yuzura, the goddess of usury and that upsets everything. That upsets the society of status and nobody knows their place anymore. I'm struck as you talk about this of how common this notion of wanting to return to a society where you knew, I mean you knew your place, I guess. Seems to be because it shows up in, so there's the conservatives. There's conservative, the Catholic philosopher Alistair McIntyre talks about this a lot of this need, the communitarians of this notion that you should be deeply embedded in society and know your position in that society, that that's formative of who you are. But I recall reading like an argument for the question was why do nerds wear fedoras? Which is an interesting, like why is it that these nerdy guys wear, Trevor's worn fedoras but he claims it was either he started the trend or it was he decided before. It was ironic. But the argument which I mean at least has, is interesting in light of what you said is that if you are a low status male in the sense that like you're not, girls aren't flocking to you, right? You're not the quarterback. You're the nerdy computer guy before and you haven't gotten to the age where you can start up and become the high status person. Then to some extent like you long for a time when men were men and women were women and you knew what the roles were and that for the fedora wearing set, the fedora represents this, I mean it's the idealized. Idealized Los Angeles of the 1940s, you know when there were tough guys and dames and the tough guys got the girl and there was this and similarly there was this level of like chivalry that we see with, you know, Philip Marlowe. But it's also the popularity of medieval England among this low status or perceived by them to be low status set of again when there were knights and damsels and society was structured. And you knew your place and you didn't have to compete for status and it was very clear and very defined. You know, I don't say I agree with that. I can't because I was that nerdy guy and I just wanted to burn the whole rotten system down. I mean I didn't want to take a place in it. I wanted to escape it. I think the longing for predictability though is an interesting point and whether or not it's better to have, I think it's existential. The unpredictability of a market and the ability for to destroy things including things that you love whether it's Monpa down the street who's going to go out of business because Walmart came in and all this stuff. It seems chaotic and it doesn't preserve the things you love not all the time depending on what you love. And so yes, the desire for predictability. If you want predictability and stability, you really shouldn't be for a market economy is more or less. Right. And so again, this plays out. So the intellectuals who had the stability of school and then in the academy, the very clear stability including, I mean, tenure. But then you see this in the longing for the good old days for the 1950s as an ideal among social conservatives. And you listed it like the lyrics of country music, which are about, you know, things should stay the same and we should stay close to home and we should do things the way that we've always done them. It seems to be this real need among people to have things always be the way they've always been and always be predictable. Yes, I'm going to change the topic slightly to one of their last question is my last question I was pondering which goes back to the expertise point. And this may cut against my point about the relational or conditional attitudes of intellectuals compared to the Soviet Union here. But and this may also push back against Nozick's desire to not my desire not to pathologize them, which which we've been getting close to anyway. But the question of if you're consider yourself incredibly smart, which let's just say intellectuals, let's just say almost tautologically do, then in that is comparatively smarter than most other people. How wouldn't that contribute to you thinking that most people can't be trusted to do a lot of things on their own? I mean, just a very simple, very simple extrapolation. If you don't believe in people, it's very hard to be for free markets. It might correlate. I mean, I could see that but it could, I mean, I could see it cutting in the other direction as well. You could say I have a PhD in economics and I still can't figure out how toothpaste manages to wind up in the bathrooms of every single person in this country. And that's astounding. And the people who make toothpaste are not geniuses. They are somehow probably pretty close to average. They're certainly not as smart as I am with my econ PhD. And yet it happens. How does that happen? That, you know, it could cut the other direction. It could, you know, you could say in fact that society considered as a collective for a moment is actually smart in ways that I can't be. I could see that. But I could also see the attitude coming into place where, you know, when I've talked to people, intellectuals about something like school choice, you say, well, parents go out and make choices for their kids about, you know, what kind of school they want for them. This isn't a crazy idea. And the response is often, I mean, do you really think that most parents are equipped to know what's best for their kids? Like in so many situations, is that really something you want to leave to parents? Like most parents are idiots. And if most parents are idiots, then we don't want them out there choosing. And so again, your perception of other people's relative intelligence, I think goes a lot to whether or not you believe in it. Yeah. But I think if people are on average incompetent to run their own lives, then probably a narrow elite at the top is not going to be able to help them. We're doing it anyway. That's a doctrinal position we have.  Which one's better? I think we can, to bring that back to Nozick's theory, I mean, so in the classroom, it's not just that the classroom is run by an authority figure. It's that the classroom is run by the teacher, right? And the teacher is the person who knows a lot more than everyone else.  But most of the time, the teacher does in fact know more than you do and the teacher seems to run the classroom relatively well.  I mean, I think all these things become consolation of factors. And we haven't mentioned the Long March again. All these things together between Nozick's points are somewhat, I think they're good, but there needs to be more. The Long March is good. I think the relational aspects. And then, of course, you just sort of have like a tipping point type of thing. I mean, if for some reason, the intellectual class, the people who hire and fire universities and the people who hire and fire in newspapers, just for some random reason, whether it was the baby boom of the 60s and the leftism of the 60s. And so all of a sudden they just get swayed to one side. Well, then your career prospects and other things like this start determining whether or not you're going to advance in. You have to have the right beliefs to advance in the sociology department or it's going to be very difficult or the journalism department.  And that was a comment often made about think tanks is that it was people who wanted to be professors, but were conservative and libertarian who fled the universities because they couldn't really say what they wanted to say and expect to get good job prospects. And so that's why conservatives libertarians were the first movers on the think tank front. And so you have all this mixed mass together and I think you start to get a general picture of why this class is. I think future episodes we'd have to deal with like Hollywood and public school teachers and other classes of predominantly anti-capitalist people. Then to close out this discussion, if Nozick is correct, if this story that he tells either explains all of intellectual anti-capitalism or at least a decent chunk of it, if the environment of the schools plays a large role, what can we do about that? What are our prospects for shifting things in a more free market direction? I'm not terribly optimistic about that, but I don't think it means that we're doomed to lose. I think that the universities are very large and at this point path dependent institutions to a large degree unless we can start building up other intellectual classes and other ideas. And that stuff I'm optimistic about. The interesting story of Baldy Harper, F.A. Harper, who started IHS, the Institute for Humane Studies, which is a quite old free market. 1961 is pretty old by these standards, who wasn't allowed to teach Hayek in his economics class at Cornell. So he said, well, screw this. I'm going to go start a nonprofit that teaches Hayek to students anyway. So I'm optimistic about things like that, but I think that the institutions, the universities and the other institutions are pretty set in their ways. And a lot of what Nozick said are some of the reasons and some of the things we've discussed today are other reasons and I don't see it changing much. What do you think, Jason? I would say that if Nozick is right, we ought to consider the currency that academics accept, which is respect and perhaps consider giving them less of it. The idea that one absolutely must go to a four-year college in order to get a good job afterward is one that people have been criticizing lately. And while I still find great value in the academy, and I think that at times it's possible to overplay that criticism, I would also say that things like that should put them on notice. And alternate institutions, the rise of things like think tanks and of alternate educational opportunities ought to make them wonder, you know, is this something that is going to eat into our prestige? Free Thoughts is produced by Evan Banks and Mark McDaniel. To learn more about libertarianism and the ideas that influence it, visit us on the web at www.libertarianism.org. And I should note, if you haven't already, you should check out our new Kato Audio app, available in the Apple App Store for iPad and iPhone. It's a super easy and free way to listen to not just free thoughts, but the other podcasts from libertarianism.org and the Kato Institute.


Good afternoon, everybody. My name is Bob Jelliff. I've got an hour's session here this afternoon to talk a little bit about installing DHS2 on service. So I've got quite a complicated presentation really because I'm going to jump between slides and video clips and typing at the terminal. So hopefully we'll manage OK. A lot to get through. I might have to skip a few bits, but let's see how we go. Right, a little bit of the overview of what we're going to go through this afternoon, a bit of history of the tool set around DHS2 on service. And then a description really of what we're calling the next generation of DHS tools, a bit of the architecture, what the main pieces are. Then I'll go through briefly the setup. I actually recorded the setup and then speeded it up so it's actually less painful to watch than doing it in real time. Then a little tour of the different containers involved, talk a little bit about the logging, the monitoring that's built in and then next steps that we plan to do regarding the tool set. So without further ado, let's, OK, yeah, acknowledgements. I thought I'd better put this up front in case we've missed it at the end. Many people have contributed to the code and ideas which have been incorporated into the tools that I'm going to show you today. I've mentioned a few names there, Stephen Ogavitsa, Stephen Alcaya, Tuzo, Morton, Barnabas, many others. Some of these tools, at least the early version, have been around for a number of years, quite a lot of people using it and all the users generally provide feedback which improved the overall quality for everybody else. A bit of history, now DHS2 as a web-based application, meaning something that's accessed over the open internet, was first implemented. I understand in Caroline, India around about 2007, started to become fairly widespread from 2010. Kenya, Ghana and Rwanda were very early examples of putting DHS2 on the web. And one of the things we very quickly discovered was that many of the people within ministries or within his groups who were responsible for setting up and running the servers were very inexperienced. Often it was their first time learning to do such a thing. And so the challenge was really how to take the best practices that we have written up in the documentation and how to incorporate that into something that was reasonably easy to deploy and probably more importantly easy to learn. With an emphasis on making it easy to do the right thing and making it hard to do the wrong thing. The DHS2 started off the first generation created around about 2014. There was some earlier version I guess but pretty much 2014 as a result of providing assistance to countries who were deploying DHS2. It's essentially a set of scripts, mostly bash scripts at the moment for creating DHS2 instances on Tomcat. It was inspired by, there's a package in Ubuntu that people may be familiar with Tomcat 7 user, Tomcat 8 user, Tomcat 9 user, etc. So it was kind of built on top of that but customised to make a few setup things easier to do. It formed a basis of an early series of server academies starting around about 2016. It's still quite widespread use but it's not really adequate I would say nowadays for managing the security and performance of more complicated systems. Particularly as more and more people now are using Tracker, they've got databases which has got personal identifying information on it. The security of servers have become much more important than arguably five years ago. Also the internet has become let's say a more hostile environment as the years have gone by. So over the past two years we've generated a next generation of these original tool sets with a lot of community supports. And that's what I'm going to describe today. What we call DHS tools, NG, next generation. I know Lars doesn't like that name. We can think of another one we will. But the primary differences with the new generation of DHS tools which is different to previous one. First of all we're using containers. So all of the different components, the proxies, the Tom Cap, the database, things like that, each one in their own separate container. As I've mentioned before we've got a much stronger emphasis on security than before. And I think each of those containers have been defined with reference to security benchmarks, particularly we looked at CIS. We're probably not, definitely we're not 100% compliant with the CIS benchmarks, but we're fairly close. The objective really, the main use of these tools is installing a virtual environment within a single host machine. So it's typically a cloud VPS. They've been tried on Linnodes, AWS. Another tool which I was introduced to by the Hisp Uganda people, SSD node, which is a very low cost VPS. Contabo is another one that's proven quite popular. That's easily and also quite frequently modified to support not just a single host, but often we have an environment where the proxy may be on a completely different machine. In the database, there are often reasons to put it on a separate machine. If you have different disk setups and things like that. So it's a frequently installed on physical infrastructure. There's quite a few of those around the world. It's quite easy to set up, as I hope you'll see in the next couple of minutes. But the fact that it's easy to set up is not really a substitute for strong experience and training. You can't really expect knowing nothing at all about Linux and system administration to start running a complicated DHS to environment, just because you've got a simple setup process. It's also I would say very suitable for probably running less than 20 DHS to instances. If you're a large hosting company like BAO systems or Hisp South Africa or something like that. If you're running hundreds of DHS to instances, you're going to develop your own customized tools for doing that. This is more geared towards mostly it's it's single country deployments. A country may have a half dozen instances. Sometimes their aggregates separate to their tracker and a training instance or a staging instance and that kind of thing. Normally not more than five or six different instances. Okay, so the architecture of the setup. I guess it's aimed at using either Ubuntu 20.04 on 18.04 host. We started with 16 moved to 18. And now I'm largely recommending people use Ubuntu 20.04. Mostly because in fact file system support on Ubuntu 20.04 support something called ZFS. We're not going to have really the opportunity today to talk too much about ZFS. But it's got quite a few cool functionalities relating to encryption and this compression and the like, which are useful in some circumstances.  And those containers are based on Ubuntu 20.04. So even with an 18.04 host, you'd be running the same 20.04 containers. The containerization technology for want of a better word to use is something called LXD. It's been around for a long, long time. It's similar in some respects to Docker but also quite different as well with LXD and LXC containers as they call them, the containers themselves are more like fully fledged virtual machines. So each one is running like a complete Ubuntu operating system on them. But they all share the same common kernels so they start and stop very quickly. The packing density is very good in the sense that you can have many of them on the same host, etc. It's a requirement that you're running a host based firewall on the host. The UFW is what's built into Ubuntu so that's what we tend to use. So all connections in and out of the machine are passing through the firewall. All of these things like the Tomcat instances and Postgres instance and monitor are essentially inaccessible from the outside. All the connections we allow through the firewall are an SSH connection to the host machine and an HTTP and HTTPS connection which gets routed straight through to the reverse proxy. Okay, we can come back to that diagram later if people have questions on it. Okay, so I had a little demo planned. The first step is not really particular to these tools or any other tools. It's a small bit of guidance I suppose for people setting up a raw machine. It's about securing your SSH, enabling some simple firewall rules, etc. Then we're talking about installing the tools themselves which involve basically pulling a set of scripts off JITUP and then running an install script. And then we look at creating a DHS2 instance. As I said, I've prerecorded these so I could speed them up. The problem when speeding them up is then I get a voice like a chipmunk on them. So I've killed all the sounds, I'll just narrate over them so they're like silent movies at the moment. So first of all, setting up what you do with a host machine which is kind of newly born. So we call this a birthing process, a term that arose in February during the training of trainers academy. Let me just stop. So this is based on a linoad. So I've set up a simple linoad. In fact, it's going to be used for Oolove later for running his courses with students. So it's completely fresh out of the box at this point. And what I've run through here in about two minutes is just setting up SSH on it. That's moving on it. Okay, so initially there's no user account on it. So I've logged in as root. The first thing I'm going to do is you'll add a new user. In my case, it's Bob Jane. I added my user and then I would make myself a member of the sudo group. Exit, try it out. SSH in as Bob Jane. Check that sudo works. It does exit again. This time now I'll put my SSH key on there. So that next time I log in, I don't have to use a password. Right now we try again and this time I'm straight in without a password. Now, I've got myself a user. I don't need to log in as root again. I can do a few things to the SSH config. I usually just do three things. I changed the port. It's not strictly necessary. Change root log into no and make sure the password authentication is set to no. Restart the SSH service. Now I should see it's listing port 822 rather than port 22. So I need to enter like that using SSH with minus PA 22. The next thing I would do is start updating packages. Because it's a new machine, I'll do a dist upgrade. But that's all my packages updated. And the next thing I will do is set my firewall rule to allow port 822 in and then I'll enable the firewall. At that point, I can relax a little bit, go off and have a cup of tea. I've essentially secured the machine. I haven't installed anything yet, but we've set up the SSH and the firewall so that the machine is basically ready. Okay, so this next little two minute video, I'm going to show you the installation of the tools themselves. Now there's a lot of ways of doing it. And I guess what I'm doing here is the simplest way of doing it's not necessarily the best. I have a small script called LXD setup which you're going to see you're going to run here, which essentially will set up the LXD hypervisor and then install the basic containers on it. Normally if I'm working, particularly if you're working with physical hardware, I wouldn't automate the setting up of the LXD itself. I'd customize that a little bit depending on the disk type. But let's just watch simple setup. Right, first of all, I didn't do it earlier, so I should do it now. Let's set the time zone on it. This machine is going to be used by Ulavin Oslo, so we'll set the time zone to Europe Oslo. Then there I clone the GitHub repository with the tools. And the first thing I do, I'm going to copy the sample config file and enter a few details, just the top three lines is all we need to do. This config file is going to be used to generate what's required. Okay, so this is much speeded up. Usually this process would take about 10 minutes to run. I think in this little video it takes about two minutes. I've speeded it up four or five times. If you can follow what's flying past there, it's actually creating the three containers that were defined in that config file. Just about to make the Postgres container now. Because this is based on Ubuntu 2004, it'll use Postgres 12 by default. We've used Postgres 12 for some months now. It seems to work very well. Once they're all set up, I'll have an opportunity to show you around them a little bit. The last container to set up is the monitoring system. It's a very simple monitor. I know there's more beautiful ones around. Moonin is a very simple monitor, which gives a lot of very useful day-to-day information to system admins out of the box. Okay, so there's my three containers created. The last step I'm doing here, I'm just installing the SSL certificate onto the proxy. I used to have this as automatically part of the setup script, but people made so many mistakes with it that I thought it was better to do it manually. All right, so that's the setup. I'll put video online so people can have a look at it. It's also described in the documentation of the tools themselves. At this point of the setup, we actually have three containers created, a proxy container with its SSL certificate installed. A database, which hasn't been tuned yet, and the Moonin monitor. So how do we create a DHS2 instance on that? Again, a very speeded up video coming up. Creating an instance. Can I pause there? The first thing I did here, inside this setup directory, I should have done this as part of the previous installation step. There's a number of service scripts, scripts which are used to create new Tomcat instances, scripts for looking at the log files, looking at the database, etc. Which need to be installed. So I run this thing, sudo install scripts that actually copies scripts from this scripts directory and puts them into user local bin so that they're available in your path wherever you are. So currently, before we start creating an instance, we can see there are these three containers that we have here. Command, that's about to come up here. I'm creating a new instance. Create instance. And we give it a name. HMIS, for want of a better word. And often it goes about its business. It's going to do three things really. It's going to create a database on the Postgres container. And it's going to set up a Ubuntu container running Tomcat 9. And this process takes about five or six minutes to do, typically. So I've speeded it up here. So this should run through in less than two minutes, I think. Done. Okay, now you can see there's a fourth container, HMIS container. So we need to deploy a war file to it. So I'm going to put the latest 234 war file to it. There's the command DHS to deploy war. The reason, I'll just pause the video quickly. The reason for having a custom command for deploying the war file rather than just downloading it and putting it in the web apps directory. It's something to do with the way the Tomcat container has been set up. For security reasons. We don't allow war files to be automatically deployed. So that war file is actually unzipped into the web apps directory, which is owned by Roots. So there's a couple of steps involved with deploying a war. So it's very easy just to have a little script to do it. Again, to try and help people make it easy to do the right thing and harder to do the wrong thing. So that's what it's doing now. It's deploying the war file. I've got a little command there, DHS to log view, which will look at the log of the HMS container as it starts up. Get a few exceptions in here, which are related to the rolling file appender. I need to trace down why that happens, but in fact, there's no problem with it. So up it comes and we should be able to now go to our course.dhs2.org and go to our HMS and there it is up and running. Secured behind HTTPS and running 234. Yep, there we go. Okay, so that was a very rapid run through the install. Let's have a closer look now on what actually has been installed. One of the things that I like to do after one of the nice things about running working with containers is you can configure various resource limits side. What I typically would do, I've actually already done this. I'm not going to show you again is set a few resource limits on various containers in particular. The Tomcat container called HMIS, we make sure no stage can ever use more than 90% of the CPU. Sometimes DHS2 can go crazy and start burning up all the CPU. If that does happen, we want to make sure that the host doesn't become completely unresponsive. Same thing with the Postgres. If the Postgres gets extremely busy, we still want to be able to administer the machine. So we'd make sure at no stage you're going to use more than 90%. The other thing I often do is I'll set a memory limit on the Postgres container. We've only got a four gigabyte machine here, so there's not much to play with. But we want to make sure that because Postgres, if it sees all of the memory, if it can, it'll try to use all of the memory. And often that makes it difficult starting if your Postgres container is running already. It can be difficult to start your Tomcat containers because Postgres has already consumed all the memory. So we need to kind of calculate how much memory you think more or less the database needs. And it's a good idea to set a limit on that so that it can't exceed that. There's many, many other configuration options and things which can be limited. A little bit of link to some documentation there. Okay, so we have four containers currently. We do a little bit of a tour through them. As I said at the start, there's quite a lot of attention to the security on this. And one of the things that we've tried to do is make sure that each container is only minimally accessible. So for example, the Tomcat container should be accessible from the proxy because the proxy needs to forward request to it. Similarly, the Postgres container should be available to the various Tomcat containers. So those have been set up automatically, but let's go through them one at a time. So we start off, look at the Postgres container. As I've said, it should be based on Postgres 12 by default. The only thing I'm going to show you here in the interest of time is the configuration file because that's typically what people want to go to. So let's, I'm off the video now, I've gone live. There's our containers. I want to go, I can execute any command inside a container like this. So I'm going to go to the Postgres container and just execute command hash. That brings me in as root into the Postgres container. And the most typical thing people would want to do soon after installation would be to set some configuration items. Now, often people do this directly inside the Postgresql.conf file. I prefer to do it here as a separate configuration file for a number of reasons. First of all, it's easy to see the changes that you've made, but also you can keep different copies of configuration files. For example, you might configure it slightly differently if it was being used to restore a gap or something like that. So these are the most common options actually taken from our implementation guide. Maximum number of connections, people often ask about this. Well, in this case, this machine is going to be used as a training machine for students. By default, if we don't change anything else, Tomcat will try to make 80 connections as part of the connection pool, often a couple more for doing things. So 200 is too high a limit, 100 is much more reasonable. Now, there's not much tuning you can do when you only have two gigabytes of RAM. We definitely want to turn this down to maybe 512 kilobytes, about a quarter of what we have. Workmen, that should be fine. We won't have too many connections. Anyway, maintenance workmen is probably a bit high for the amount of memory that we have. I'm going to turn it down a little bit. And the effective cache size of this machine is not very big at all. Most it's going to be about one and a half gigabytes. These settings generally we can keep. I'll put this in by default. It'll log any query on the database that takes longer than five minutes. That's something that you can customize that if you're troubleshooting. Okay, so before we start in the database to get those connections, I should stop the Tomcat and do that now. I can just go and stop the HMIS container. Stop the Tomcat, restart my database container, and its new settings will be in effect. So let's go and have a look inside the Tomcat container. The Tomcat contains probably the most interesting one. We'll really have a lot of time today to go into a lot of detail. Let's just have a look at the main outlines of it. That's not running because I stopped it. Okay, let's start it. Let's say because these things are containers, they actually start very quickly. It's up already. So because we only have one Tomcat per container, we've based this around the Ubuntu system Tomcat installation, which has got a good thing to do because they've done quite a good job. That's most of the security around things like file permissions and the like. It's a bit of a pain to do all of that manually. So this is just the system Tomcat. So most of the configuration that you're going to get is from places like this, such as default Tomcat 9. That's where you set things like your Java apps and the like. Nothing much else set here at all, but that's where you would set it. The other thing, the place where you might look if you're configuring, if your server.xml file, so that's where that would be. There's a few things that have been done in here. You notice, for example, server port equals minus one. Some of you might know what that does. Basically removes the control port, which again, it just improves a little security thing. This is your server.xml. You can, given that this is going to be a small system, with a very tiny amount of RAM being used by students, I would turn down the maximum threads there from 100 to maybe 20 or 30. But I'll leave it for now. This is the famous Relax Query Chars, which some people might have come across. Otherwise, that's pretty much it in there. There's a few things in. This is interesting, I guess. Here, notice that we've got unpack wars equals false, and we've got auto deploy equals false. The reason for that is that if your web application gets hacked, and it is probably the most vulnerable part of the whole system, if your web application does get hacked, what you don't want is for that Tomcat user to be able to modify its own application. So the Tomcat user has been locked down as far as possible. It's able to run the application, but it's not able to do anything else. Okay, what else is interesting in here? The logging, I'm going to talk about logging later, so I won't bother with that. Yes, the DHS2 home directory. DHS2 home directory is in opt DHS2. That's where your dhs.comp file is, which hopefully has been set with the right permissions. Yep, it's actually owned by Root, but readable by the Tomcat user, which is about as good as we can make it. What I've done in there is, again, just giving you a bit of a template. I'll just take a quick look inside the file. A little bit of a, basically all of these options are essentially taken from the reference guide. So they're all in there, commented out. It's a bit of a mission to keep it up to date every time new versions come out. By default, it's just set up like this. It has its database set up connection string set. That's various things. I would, for example, turn this connection. This is an error in the documentation. The maximum size of the connection pool used to default to 40, I'm thinking now if you're forced to 80, I'm going to specifically set it to 40, et cetera. There's various things that one can set on here. The base URL session timeout generally is too long for most purposes, but I'm not going to tune it now. I'm just showing you where the bits are. Okay, so what else was there? I was going to show you the firewall rules on there. The firewall rules should all have been set up. So your Tomcat user, you can see, it can be accessed by the proxy, which is 192.16802. It can also be accessed from the host. That's not strictly necessary up there because sometimes I run commands on the host to get information from the Tomcat manager. This one here is on all the containers. 4949 is open to the monitor. Monitor uses 4949 to get its information. Okay, what else can we say? Where is my slide? The proxy. The proxy is in an interesting state of flux because as people who know me well will know I was very fond of Apache 2 as a proxy server. I'm still very fond of Apache 2, but I'm transitioning to engine X. I've explained that a little bit. I think in my last one or two slides why at the moment the tools are using Apache 2 by default. That'll be changed over the next week or two. The way that the proxy is configured, and this will be similar on engine X, is that we created a folder in here called upstream. These are the various snippets, if you like. So the HMIS snippet is the little location block, as they would call it. This is the bit where we tell the proxy that if somebody goes to HMIS we need to push them through to the Tomcat running on 192.68010. Similarly, the monitor has got its own little snippet so that if you go slash moon in, it'll get you to the monitor. Any other interesting thing about the proxy, which I'm going to talk a little bit later about these logs is I've configured a particular access log which has got two features to it. It's got, first of all, it's tab separated. We'll see in a minute why that's useful. And also we're recording the request processing time on it. But let's go back to the slides. Yeah, logging. We could have a whole presentation on logging. Unfortunately, we don't have time here. The Tomcat, because it's the system Tomcat, it's actually uses the system journal and syslog for all of its logging. And the journal log on Linux system is access using a program called journal CTL. It's actually a very powerful way of looking at your system logs. It's that useful that in fact I made a little wrapper for it. It's called DHS2 log view for looking at the logs. I'll show you that briefly now. It also logs to syslog. One of the advantages of logging to syslog is it's quite easy to configure centralized logging. DHS2 log view. If I go log view, I want to look at my instance. I can say just show me everything that's happened in the last 10 minutes. Because I remember log files can get huge and they'll show me the log file from 10 minutes ago. You can show me everything that's happened since 9 o'clock this morning until 9.05. There's a particular five minutes that I'm interested in. So I can pull that particular five minutes out of the log when I'm going to look at just what happened between 9 and 9.05, which probably was not very much. Yep, not very much happened at all. It's just sitting idle.  Using DHS2 log view is a very convenient command for looking at your log in all kinds of different ways. The proxy, I'd mentioned that I'd made some changes to the default access log to make it easier to perform for text processing and querying. I don't know if I'm really going to have time now. We are running a little bit to show you a few examples of why that's useful. I've got a file here of little orc one-liners which I used to process the log file, but let's just take you through them one at a time quickly. I think I can just run it from here. One-liners. Okay, so we have a file called perflog. This one is not from this machine. This one is from a production machine. Unfortunately, you'll probably see it in the log file. It's very big. It's only from today, but it's actually half a gigabyte's worth of log. It's compressed. So we're just going to z-cat it out first to show you what it looks like. That's the log format. As you can see, it's pretty much the same as the standard log format. I've separated everything, and I've also got a record of the request processing times here. So one of the things we can do because it's tab separated, using something like orc, orc is just convenient for doing really simple things. If you're going to get fancy, you can use perl or the scripting language of your choice. I think Ben is the scripting language of your choice, Guy. But I'm going to show you a little bit of orc. We know that field number six is the status field. So if I was only interested looking in my log file at the status field, I could run a simple orc program like that, and it's just going to extract the status field. Not really very useful. What's maybe more useful is to know what the requests were and what their status was. So this is just extracting from the log file now. You see, I'm getting all the requests and their status. There's a lot of requests in this file. It's like 1.6 million. So I'm probably just interested in the interesting ones, the ones which are not giving me 200s or 300s, but the 400s and 500s, I can make a little script like this, which says $5.05 and $6 as before, but this time only when $6 is greater than 304, and that's going to give me all the 404s and 500s and 401s and the like. This is often something useful. If you're looking at your system, try to find out things which are misbehaving. You see something here, which is very weird. You see this, get API script, whatever that is giving me a 500 quite often. That's obviously a problem. Something to be looked at on that machine. This particular machine spends most of its time, in fact, putting API events. So if you wanted to look at just everything related to putting API events, it's a little bit like grepping really. You could use a regular expression to only print the lines, which have matched the regular expression and just print $5 and $6 of it. And there we've got all our event puts, and you can see most of them are ending in 200s, about 302s. Not very interesting information the way it is, but what if we look for putting API events, which match an error code of 420, error code of 429. 429, if you look it up, is the HTTP error code where a server has essentially limited to the client. So they're saying that you've exceeded the rate limit or connection limit for this particular server. We can see there's quite a lot of cases where this server has actually returned a 429. How many cases? Well, we can count them. One way to count them, instead of printing the rows out, we just make a variable, increment it and then right at the end, print them. And we see that we get 28,000, which is quite a lot of them. There are 371,000 requests, which ended in 200. Sometimes it's useful to just know the big picture out of all of the requests on the server, just what the breakdown of status codes are. And again, you can do that with a simple one-liner or script. We just say take that $6 in an associative array, clan on the fly, increment it each time we see that status code. And then at the end, after processing all of the lines, we print out the array. And that'll show us all of the status codes that we got when putting API events. And you can see as before, the 371,000 success is 28,000. That's probably about 80% of all the requests were actually rate limited. And then a few other interesting ones that you're getting there. Because we've got the request processing time in here, we can also query on that. That happens to be field number 10. So I can look at all of the requests which took longer than 60 seconds. And there we can see them. There's actually not so many on here. I can see that getting track entity instances is quite a nasty one taking up to three minutes. Possibly needs an index on there. But in general, for 1.6 million requests, the number taking longer than 60 seconds is actually quite short. Okay. So that was a little diversion, I guess, around why it made sense to make a separate performance. Making a performance access log means that it's very quick and very flexible to be able to make interesting queries into your log file. I plan to put together a couple of orc one-liners to do this kind of thing, because it's actually very, very, it's probably the fastest way of processing a log file. Okay, monitoring. There are many ways to monitor a system. Many monitoring solutions out there. We've been using Moon in really for quite a long time. It's not a very beautiful thing. It looks very clunky really. But the main reason for using it is it's very, very easy to configure it out of the box. And it understands already with plugins, Tomcat, Postgres, Apache, Nginx, et cetera. That's very useful for the day-to-day monitoring by sysadmins to get an idea of the health of the system. For more detailed profiling of the JVM, we've found Glowroot is very easy to install and use. Thanks to Dan Cocos for that one, introducing me to Glowroot. We're actually installing it on most of our servers now. Obviously other monitoring solutions are possible because these things are just generic containers really. If you want to monitor them using Prometheus or Nagios or SolarWinds, you can do it. It depends on whether you have an existing monitoring solution in-house. We will need to set up a Prometheus monitoring container sometime soon. Now that we know we have specific Prometheus monitoring functionality in DHS to itself. I'll just show you a couple of pictures of the kind of graphs that are useful. Using Moon, this is showing you the number of threads in the state. Engine X request, engine X status. Postgres connections. This is an example of a very troubled server. All that yellow stuff are Postgres connections, which are ideal in transaction. That's a sign usually that the DHS2 application is run out of CPU. So it's open transactions, but it's not acting on them fast enough. Okay, I'll skip that. That's a little command that we have as part of the tools to look at your non-idle Postgres connections. As an example of the kind of thing you get with GlowRoot, I'll say GlowRoot is not installed by default, but it's quite easy to do, and I will make some documentation around that. Where are we going from here? Well, one thing I plan to do, we've talked about this for a while, just not have the time. Currently, all the installation scripts are rather hairy, nasty, almost bug-free bash scripts. It's going to be much more robust and much more easier to maintain them over time if you just convert them to using Ansible. Ansible plays very nicely with LXD. There's an Ansible LXD connector. So it's not a very big operation to redefine these machines using Ansible, and that's where we'll see it in the next edition, if you like, of these tools. I've not talked about alerting today. Typically, we use email for alerting, but we've had a lot of problems in countries with setting up their email MTAs, mail transport agents, getting banned for spamming and not having the ability to create the right DNS records, etc. So one of the solutions that we're looking at, I know it works, we've got a proof concept already, is using email for alerting, but then instead of actually sending the email, we route all the email to a telegram box so people can get their alert messages via telegram. We had a complete transition from Apache 2 to Engine X. The reason I said that there was a reason I wanted to do this is because there's two modules on Engine X, Limit Rate and Limit Con, which turn out to be really not just useful but actually essential when dealing with huge installations. We've had recent issues with the state of Uttar Pradesh in India. I don't know if anyone from Hisp India is on. But with 28,000 Android devices, now we know DHS2 can't handle 28,000 parallel connections, but we've no way of limiting it. So using Limit Con or Engine X, we can actually quite finely control the amount of parallel clients that are able to get in and thus not bring down the server. Currently, what you've seen defined are just the bare minimum number of containers. There are many other containers could be put together. Central logging container, I've made one of those before. It's quite easy to do, but I haven't automated it. Listening to the presentation yesterday on ICD-11, I can see some value in creating a little ICD-11 container, etc. Also, we plan to have a virtual academy before the end of the year when we get into a lot more detail. Alright, some additional readings. Those of you who want to look further into things, you really want to read the LXD documentation. We didn't get to talk about ZFS, but if you want to do things like encryption at rest, compressing your file system because your short of disk, etc., ZFS is really the way to go. If you like the idea of processing your log files using an ancient language from 1987, that's the classic book to read. That's it. I think all we can fit in in a short hour. Sorry if it's been very, very rapid, but I wanted to leave a little bit of time at the end to answer some questions. Thank you, Bob. We had a few questions coming through on the community of practice. Maybe if you'd like to address a few of those, that would be great. Okay. Oh, I don't have the link open to the community of practice. The link is in the chat. The link is in the chat? Yes. It's convenient. Let me get to the chat. I can share the screen for you. That's also okay. I lost the chat. I got the participants, but where's the chat? All right. Can you see my screen now? I just found the chat. Okay, some questions in the Q&A section. Okay, so starting with Carlos. Well, I did get a quick run through. Okay, there's about four questions. Let's take them one at a time then. So Carlos has asked, hi, thanks for your efforts and time. We're doing a server installation. We're checking since 232. Why do we need to do a post-cursor post year? Yeah, I don't know exactly what you're asking Carlos. It might be that what you're saying is that the documentation around post-cursor installation is not quite up to date and accurate. It might be the case. We'll go back and check that. But as far as I know, certainly if you use these tools, then the installation is up to date and should run 234. I've got a link on one of my slides there, where you get the updated repositories to install the tools. How do we tweak the Windows server? That, Walid, I'm afraid I can't answer. You're going to have to ask a Windows person. I don't think it's very, very common to be running DHS in Windows environments. I know a few people are, but I wouldn't be the one to ask, but I think that's the best way to get good performance out of it. I think one of the biggest problems on Windows probably is that post-cursor. Post-cursor is not designed for a Windows environment. And yeah, it's been successfully ported, but it's probably not going to get the kind of performance that you get on Linux. Morton, does install scripts copy the files to destination to the JIT repository? No, no, it actually copies the files, Morton. Is it recommended from time to time you JIT pool and rerun? Yes, that's exactly what I do often. Pull the updated repository and then rerun install scripts to get the latest set of scripts on there. Any other questions? Anything in the chat? Bob is an interesting guy, since Ben. That's right, Ben is an interesting guy. I'm just running through the chat here to see if anybody has asked me anything. Bob, there was a few more questions on the community of practice that you didn't see. Oh, okay, sorry. John Lewis, we use your server script on training server. Basically run locally without an internet. Yeah, John, this comes back to my very first point. A long time ago, people were very commonly running DHS2 without HTTPS at all. Often they would just run Tomcat on port 8080 and throw it on the internet. There's still a few installations I see do that today. Part of the reason that people were doing that is they didn't really know how to set up the proxy and they didn't know how to get the HTTPS set up correctly. So what I'd done, I guess, in fact, from the previous version of the tools even, was made sure that it was really very easy to set up HTTPS. But unfortunately, the downside of that is now you've got a bit of work to do and you don't want to run it without HTTPS. I can maybe document how you would best do that. The idea is to make that difficult for you. So I'm glad you're finding it difficult. Mamadou, hello, thanks for the presentation. How are backups work with this architecture? All right, that's a very good question. Backups are not configured automatically. The reason for that is because everybody's backup strategy is going to be slightly different. Basically, there is a command, I think it's called DHS2 backup, which will back up your databases. And what I would typically do is to install a cron job to keep a rolling backup, typically keeping seven days of backup and then six weeks of backup and then the first day of every month backup, etc. But where to put those backups is the thing and how to synchronize them with some off-site storage. That unfortunately is going to vary too much from one installation to the next. That's why it's not currently automated. We will provide a bit of documentation or guidance perhaps and what's a good way of putting together a backup strategy. Stelio, thanks for a great presentation. Can you please share it? Yes, I will do that. Why are we moving to engine X? With great sadness. I love Apache 2, particularly mod rewrite on Apache 2. I don't think there's anything as good as that on anything. But as I was explaining right at the end there of the presentation, there's two things on engine X around the rate limiting and the connection limiting, which is just a bit difficult to do on Apache. And they are so essential when you talk about very big installations that we just have to move. Carlos, all apologies for typo. My question is about database installation steps on the implemented documentation. All right, Carlos. I know the implemented documentation often has fallen into a bit of disrepair from time to time. We go back and check them. You're probably correct that there's some problem with the database installation steps. What I could suggest you do is have a look at the DHS tools installation scripts and you can see the steps that are done in there are basically the steps that you need. All right, any other questions? Scrolling, scrolling, scrolling. Ben is talking about me driving a motor home for Ireland to Norway in the winter. I'm hoping then to get it to Norway before it gets really, really icy. So November, Brexit permitting.


Hey y'all, so Dustin Shaper here, the KetoPro. We're going to talk about the three tips to help transition you into a keto diet. So these are really simple, but I promise you if you start googling and getting overwhelmed with all the information out there, you're going to get lost. 20 years have been doing this, and there's a process that you have to go through if you want to maintain and get long-term great results. So this is going to shock you.  What I do know is people, and to get into ketosis is phases. It takes time. It takes about six weeks to get keto adapted. That's why this is a six-week program.  You want to feel good through this whole process. So here we go. These are the three tips. Number one is do not start increasing your fat intake. I said do not start increasing your fat intake. Don't do that. That's a bad idea. The reason why is that a lot of people, you shift your diet with a high fat, you can really affect your GI tract, and you don't want to do that. What you want to do instead, and this is tip two, is reduce your carbohydrates. So the focus is in this week is really get your carbs down. Your goal is to shoot them below 20 grams of carbs. I don't want you to get one of those little calculators and measure everything. If you're a person that has to do that, do it. But know that long term, if you do that, you won't get long term results. You have to get rid of those tools. You can use it from time to time. So tip two is get those carbohydrates down. What do I mean by that? You're going to start cutting out the pastas, the rice, the bagels, the cereals, the chips, the whites, let's call it. So any type of starches, you're going to want to cut down. Any type of, even milks in the white products, I would reduce those because they have a lot of carbs in it. There is some dairy that you can get away with, like aged cheese is fine if you're dairy tolerant. But you want to get those carbs low. And I'm going to give you some tips of what you can eat. So whole type. Also down below, I'm going to have a list of foods that you can choose from. And number three tip is don't worry about protein consumption. Most people need more, especially if you're my audience. You're probably not in your strap in 20s. If you are, appreciate you for following. You're probably a little bit older, above the age of 35, maybe 45 or 50. You need protein. Protein is really, really important. And most of you are not needing to be in ketosis for medical reasons. So don't be afraid of protein. Now at the same time, I don't want you to necessarily start eating a 16 ounce ribeye three times a day because that's just too much. You don't need that much. I don't want you to be afraid of it. If you start Googling, people go, oh, my protein's too high. My carbs are too low. And my fat's not high enough. You won't get results if that's how you're looking at this. So recap again. The three most important things that you want to do when you're in this first phase, getting started, is do not increase your fat intake. We're going to do that in the next couple of weeks. We're going to teach you how to do that. We don't want to let your gallbladder work naturally. We want to let your GI tract not have to go through those ups and downs of too much fat and causing diarrhea and all these irritations. Number two is get those carbs down low. Your focus right now is really low carb. You will still get the ketosis. And that's the most important thing about being in ketosis is elevating your ketone levels, not increasing your fat intake initially. And then number three is do not worry about your protein.  Two to three dexacards of protein a day. If you're a man, you're looking at three to five. On average, if you're an athlete, if you work out, it's going to be a little different. But that's the simplicity of this all. Here's a couple of little added bonuses. Hydration is really important. I want you to make a lot of water. If you use filtered water, we're going to have to use electrolytes. We're going to have to increase and start and be OK with salting your food. You need more sodium when you go ketosis.  That keto flu is really dehydration for most people. And so the other last thing I want to give you is do not stress about this. If you haven't watched the first week video about preparation, definitely watch that first. And so remember, don't worry about fat right now. Start going low carb. Don't worry about protein right now. We'll address that as we go forward, based on your results and how you feel and how you're doing. And then down below, we're going to have a food list of things that you can basically print out. And you can start looking at these are the foods you choose. These are the foods that you're going to avoid. So, y'all, thanks for following. Down below, gets more information. Cheers. This is week two of your keto transformation. Hey, thanks for checking out this video. For the next video of the series, check out right here. And don't forget to subscribe to the channel so that we send out the next video you get notified. And you can check out the latest on the keto journey. Cheers, y'all.


On behalf of the members of the Empowering Development Initiative, I am pleased to launch our first report. As a group, we started our work on Empowering Initiative, essentially started by ITU Development Group over a year ago. We had series of discussions, dialogues and based on our initial discussions, we decided to break this activity into five major groups. Empowerment for education, empowerment for health, commerce, sports and governance. In each of these five areas, we have series of recommendations. We looked at good practices and we have worked out a roadmap going forward because we strongly believe that mobile applications in these five areas are critical to improving quality of life, especially in developing countries. Now, our job is to really begin some concrete work based on this report in the areas of advocacy, innovations, creating platforms and technology and also mobilizing fair amount of resources to really execute on some of these ideas. The key challenge is to reach out to large number of people at the leadership level in many countries also at the level of ministers of education, health, IT, communication, commerce and all that and also sensitize civil society, students, academicians, entrepreneurs and large number of people to really expedite work on mobile applications. Our main focus essentially is on education and health because we believe there are low hanging fruits in these areas so with little bit of effort, we can really gain a lot.


Out with the old and in with the new with Appalachian wireless trading and upgrade to one of the hottest smartphones like the Samsung Galaxy S9 and save $100. Better service, bigger savings. That's today's Appalachian Wireless and East Kentucky Network Company to your agreement required. They'll come up to the big gravel lot at the top of the hill. They'll have a hay ride, hay tractor ride up to the top of the hill where they'll then go through the haunted barn. Then the tractor will pick them up and drop them off at the haunted trail. And then they'll walk through the haunted trail and we've got some pretty mischievous characters out the haunted trail to make sure that they have a good time. So we'll start October the 12th and we'll run the three weekends in October after that on Friday and Saturday nights. All the proceeds from our haunted trail go to our shop of the first responder program. So it goes to purchase Christmas toys, Christmas dinner for underprivileged children in our school system.