Chapter 22 of Book 2 of Rhetoric With respect to nomology, or the doctrine of a sentence, when we have shown what a sentence is, it will then especially become evident about what kind of things, and when, and to what persons, it is fit to employ sentences in orations. But a sentence is an enunciation, yet not about particulars, such as what kind of person Iphikrates was. But about that which is universal. Yet it is not about all universals, such for instance as that a right is contrary to a curved line. But it is about those universals with which actions are conversant, and those things which in acting are eligible, or to be avoided. Hence, because enthamemes are nearly syllogisms about things of this kind, if the syllogism is taken away, the conclusions and the principles of the enthamemes are sentences. Thus for instance, what Medea in Euripides says is a sentence, vis, quote, It does not become a man of a sound mind to educate his children, so as to render them transcendently wise. This therefore is a sentence, but the cause being added, and the why, the whole is an enthamem. As for instance, in the words of Medea, quote, For besides the indolence which they thus acquire, they excite the baneful envy of their fellow citizens. And also, quote, There is no man who is in all respects happy. Close quote. And, quote, There is not any man who is free. Close quote. Is a sentence. But the following words being added, it becomes an enthamem. Vis, quote, For he is either the slave of wealth or a fortune. Close quote. If, therefore, a sentence is what we have said it is, there are necessarily four species of a sentence. For it will either subsist in conjunction with an epilogue or without an epilogue. Those sentences, therefore, require demonstration which assert anything paradoxical or dubious, but those that assert nothing paradoxical are without an epilogue. But of these, it is necessary that some indeed, in consequence of being previously known, should require no epilogue, such as, for instance, quote, health, as it appears to us, is the best of things to man. Close quote. For lest it appears to the many, but other sentences, as soon as they are spoken, become manifest to those that consider them, such as, quote, There is no lover who does not always love. Close quote. Of sentences, however, which are with an epilogue, some indeed are the parts of an enthamem, as, quote, it does not become a man of a sound mind. Close quote. But others are enthamematic, yet are not a part of an enthamem, which also are especially approved. These are sentences in which the cause of what is said is apparent, as in the following, quote, being immortal do not retain an immortal anger. Close quote. For to say, quote, it is not proper to retain anger always, close quote, is a sentence. But the addition, quote, being immortal, close quote, asserts the why or the cause. Similar to this also is the sentence, quote, it is fit that mortal should be wise in mortal and not in immortal concerns, close quote. From what has been said, therefore, it is manifest how many species there are of a sentence and to what kind of things each is adapted. For in things of a dubious or paradoxical nature, a sentence is not to be used without an epilogue. But either an epilogue being added, the sentence must be used as a conclusion, as if anyone should say, quote, I indeed, since it is neither proper to be envious nor to be indolent, assert that erudition is not requisite. Close quote. Or, this being previously said, the former assertions must be subjoined. But in things which are not paradoxical indeed, yet are immanifest, the why or cause must be more concisely added. And to things of this kind, laconic apathems and enigmas are adapted, as if someone should say, what Stesichorus said among the Locrians, quote, that it is not proper to behave insolently, lest the grasshoppers should sing on the ground. Close quote. A sententious mode of speaking, however, is adapted to him who is more advanced in age. But what he sententiously says must be about things in which he is skilled. Hence it is unbecoming for one who is not so advanced in age to speak sententiously, in the same manner as it is for him to mythologize. But for a man to speak sententiously about things in which he is unskilled is foolish and in erudite, of which this is a sufficient indication, that rustics are especially sententious and easily show that they are so. To assert, however, universally that which is not universally true is especially adapted to lamentation and amplification. And in this case, such sentences must be adduced either at the beginning or when you demonstrate. But it is requisite to employ sentences which are generally known and common if they are useful for the purpose. For in consequence of being common, as being acknowledged by all men, they appear to be true. Thus he who exhorts soldiers to encounter danger, though they have not sacrificed, may employ what Hector says to Politimus, quote, without a sign his sword the brave man draws and asks no omen but his country's cause, close quote. And when those are exhorted to fight who are inferior to their opponents in force, it may be said, quote, in battle marrs to either side inclines, close quote. When anyone likewise is exhorted to destroy the children of enemies, though they have done no injury, he may say, quote, he is a fool who, having destroyed the father, spares the children, close quote. Further still, some proverbs are also sentences such as the proverb, quote, an addict stranger, close quote. Sentences likewise are to be sometimes adduced, contrary to those which are generally received. But I mean by those that are generally received such as, quote, to thyself, close quote. And, quote, nothing too much, close quote. And sentences contrary to these are to be adduced when either the manors will from then seem to be better or when the thing is spoken pathetically. But a thing is spoken pathetically if someone, being enraged, should say, quote, it is false that a man ought to know himself, for this man, if he had known himself, would never have solicited the command of the army, close quote. And the manors will appear to be better if it is said, quote, that it is not proper, according to the assertion of bias, to love as if intending hereafter to hate, but rather to hate as if intending hereafter to love, close quote. It is necessary, however, to render the choice manifest by the diction, but if not to subjoin the cause. As for instance, we may either thus speak, quote, that it is necessary to love, not as they say, but as if always intending to love. For the other, i.e., he who loves as if intending hereafter to hate, loves like a treacherous person, close quote. Or thus, quote, what is generally asserted does not please me, for a true friend ought to love as one who will love always. Nor does that saying please me, nothing too much, for it is necessary to hate vehemently, bad men, close quote. But these sentences afford great assistance to orations, one cause of which arises from the arrogance of the hearers, for they are delighted if anyone speaking universally happens to induce opinions which they have formed about some particular things. My meaning, however, will be manifest from what follows, and at the same time it will be evident how these sentences are to be investigated. For a sentence is, as we have said, a universal enunciation, but the auditors are delighted when that is universally asserted which they have preconceived partially. Thus, for instance, if anyone happens to have had bad neighbors or children, he will adopt the sentence, quote, that nothing is more troublesome than vicinity, close quote, and, quote, that nothing is more stupid than the procreation of children, close quote. Hence, it is necessary to conjecture what the opinion of the audience will be about particulars and afterwards to adduce universal sentences conformable to their opinion. And this is one use which speaking sententiously ought to possess, but there is another use of it which is superior to this, for it causes orations to become ethical, and those orations are ethical in which the deliberate intention of the speaker is manifest. All sentences, however, affect this because he who employs the sentence speaks universally about things which are the objects of deliberate choice. Hence, if the sentences are good, they cause the speaker to appear to be one who possesses worthy manners, and thus much concerning a sentence. What it is, how many species there are of it, how sentences are to be employed, and what advantages they possess. Chapter 23 Let us now speak concerning enthememes universally. After what manner they are to be investigated, and in the next place, let us show what are the places of enthememes, for there is a different species of each of these. That an enthemem, therefore, is a certain syllogism and how it is a syllogism we have before shown, and also in what it differs from dialectic syllogisms. For neither ought things remote to be collected, nor are all things to be collectively assumed, since the former will be obscure from their length, and the latter will be nougatory, through speaking of things which are obvious. For this is the reason why the unlearned are more capable of producing persuasion among the crowd than the learned, since, as the poets say, quote, the unlearned speak more elegantly to a mob, close quote. For the unlearned speak of things common and universal, which are adapted to the comprehension of the multitude, but the learned speak of things which they know, and which are near. Hence rhetorical enthememes must not be composed from everything which is probable, but from things of a definite nature, such as are those things which appear probable to the judges, or which the judges admit. Nor is it requisite that these things should be approved by all the auditors, but it is sufficient if they are approved by the greater part of them. It is likewise requisite not only to collect from things which are necessary, but also from things which have a frequency of subsistence. In the first place, therefore, it is necessary to assume that concerning the thing of which it is requisite to speak and syllogize, whether by employing a political or any other syllogism, concerning this it is necessary to possess all or some of the things which are inherent in it. For, if we possess none of them, no conclusion can be made from nothing. I say, for instance, how can we advise the Athenians to engage in war or not, unless we know what their power is, whether naval or land, or both, how great it is, what their revenue is, who are their friends and enemies, and farther still, what wars they have waged, and how they were carried on, and other things of the like-kind? How likewise could we praise them if we were not acquainted with the naval battle at Salamis or the battle at Marathon or the deeds which they performed for the Heraclidae or anything else of the like-kind? For all men praise others from beautiful transactions which exist or appear to exist. In a similar manner also, in blaming the Athenians, we must direct our attention to the contraries to these things, considering what particular of a contrary kind pertains to them or appears to pertain to them, such as that they enslaved the Greeks and subdued those who fought with them against the barbarian, and behaved most intrepidly, viz the egenita and padadita, and other things of the like-kind, and whatever other crime may be laid to their charge. Thus too, both those who accuse and those who defend accuse and defend by directing their attention to inherent particulars, but it makes no difference whether we speak of the Athenians or the Lacedemonians of God or man, for he who advises Achilles, who praises or blames, accuses or defends him, must assume things which are inherent, or appear to be inherent, that from these he may in praising or blaming show whether anything beautiful in conduct or base is inherent. In accusing or defending may show whether anything just or unjust is inherent, and in advising whether anything advantageous or detrimental is present. The like method must also be adopted in every other thing. Thus for instance, in investigating whether justice is good or not, our attention must be directed to what is inherent in justice or in good. Hence, since it appears that this method is adopted by all men, whether they syllogize more accurately or remissly, for their assumptions are not derived from all things, but from such as are inherent in each particular, and through reasoning. Since it is evident that it is otherwise impossible to prove what they wish to prove, this being the case, it is obvious as we have shown in the topics that about each question in things contingent, and the time best adapted to them, it is necessary to have a more select nature. The investigation, likewise, must be made after the same manner in things of an unexpected nature, so that our attention must not be directed to the indefinite, but to things inherent, which are the subject of the oration. Of the things inherent, likewise, the greater part, and those which are nearest to the subject, must be included in the oration, for by how much the greater the number of things inherent is possesses, by so much more easily will he prove that which he wishes to prove, and by how much more proximate the particulars are which he details, by so much the more appropriate will they be, and less common. But I call common things, indeed, such as to praise Achilles, because he is a man, and because he is among the number of demigods, and because he fought against Troy. For these particulars belong also to many others, so that praise of this kind no less pertains to Achilles than to Diamond. Things peculiar, however, are such as happen to no other person than Achilles, such as to of Slain Hector, the bravest of the Trojans. In Cygnus, who, being invulnerable, prevented the Greeks from descending from their ships to the land, and that being very young, he entered into the army, though he was not bound by an oath yet against the Trojans. These and other things of the like kind are peculiar to Achilles. This, therefore, is one place of selection, and is the first topical place. Chapter 24 Let us now speak of the elements of Enthememes, but I call the same thing the element and place of an Enthemem. Let us, however, first speak of those things which it is necessary in the first place to discuss. For, there are two species of Enthememes, and the first species contains ostensive Enthememes, which show that a thing is, or is not, but the other species is adapted to Confutation. They differ also in the same manner, as in Dialectics, and Elincus, and Syllagism differ. But, an ostensive Enthemem is when the conclusion is collected from things acknowledged, and the Enthemem adapted to Confutation is when things not acknowledged are collected in the conclusion. Nearly, therefore, places have been delivered by us about each of the useful and necessary species. For propositions respecting each have been selected. Hence, we have shown from what places it is requisite to derive Enthememes about good or evil, the beautiful or the base, the just or the unjust, and, in a similar manner, places have been assigned by us concerning manners, passions, and habits. Again, therefore, it remains that after another manner, we should assume universally concerning all the three genera of erasions, indicating which of them are adapted to Confutation, and are ostensive, and what are the places of apparent Enthememes, but which are not Enthememes in reality, since neither are they Syllagisms, but these are considered manifest, we shall discuss solutions and objections, and show whence it is requisite to reduce these against Enthememes. One place, therefore, of ostensive Enthemes is from Contraries, for it is necessary to consider whether one Contrary is inherent in another, subverting indeed if it be not inherent, but confirming if it is inherent. For instance, we may less show that to act temperately is good, for to act temperately is noxious, or as in the Messinic oration of Alcidamus, for if war is the cause of the present evils, it is necessary to correct those evils with peace, for as a certain tragic poet argues in Greek Centuries, quote, if it is not just to fall into anger with those who have done evil willingly, neither is it fit if anyone has acted beneficently from compulsion to be grateful to him, close quote. But if to speak falsely is among mortals calculated to persuade, it is requisite to think that on the Contrary many things are true, which are considered by mortals as incredible. Another place is from similar cases, for it is necessary that they should be similarly inherent, or not inherent. Thus, from this place it may be shown that not everything which is just is good. For if everything just were eligible and good, everything which is justly done would be eligible and good, but now to die justly is not eligible. Another place is from relatives, for if someone has acted well or justly another has suffered well or justly, and if the command is just, it is also just to obey the command. As the publican, Diomedon said about the tributes, quote, for said he to the people, it is not disgraceful in you to sell the tributes, neither is it disgraceful in us to buy them, close quote. And if one man deservedly and justly suffers a loss, he who caused him to suffer it acted well and justly. And if he who caused another to suffer a loss acted well and justly, he who sustained the loss sustained it well and justly. In this place, however, it is difficult to paralogize, for if a man died justly, he suffered justly, but perhaps not by you. Hence it is necessary to consider separately whether he who suffered deserved to suffer and whether he who did the thing deserved to do it, unless to infer what is adapted and appropriate. For sometimes the thing of this kind is dissonant and nothing impedes as in the alchemyon of theodectes, quote, did never any mortal hate thy mother? Close quote. To which the answer is, quote, distinctly this must be considered. Close quote. Al-Fasaba also inquiring, quote, how therefore have the judges condemned thee? Close quote. Alchemyon answered, quote, of death deserving she was judged, but I, twas said, could not have slain thy mother. Close quote. Thus, likewise, on the trial of Demosthenes, in those who slew Nicanor, because they were judged to have slain him justly, it appeared that he was justly put to death.  Another place is from the more and the less, such as, if even the gods do not know all things, much less do men. For the meaning of this is, if the more is not inherent in that in which it ought to be more inherent, it is evident that neither will it be inherent in that in which the less is inherent. But this place, that he will strike his neighbor who strikes his father, depends on this, that if the less is inherent, the more also will be inherent. And, this place is useful for both purposes. Viz, whether it be requisite to show that a thing is inherent, or is not. Farther still, if a thing is inherent, neither more nor less. Once it is said, in a certain tragedy, quote, is thy father to be pitied because he has lost a son, and is not envious to be equally commiserated, who has lost his son, Millie Ager, one of the most illustrious of the Greeks, close quote. And, that if thesius did not act unjustly in ravaging Helen, neither did Paris. And, if the tinderida, i.e. Castor and Pollux, did not act unjustly in ravishing the daughters of Lusipus, neither did Paris. Likewise, if Hector did not act unjustly in slaying Patroclus, neither did Paris in slaying Achilles. And, if other artists are not vile men, neither are philosophers. And, if generals are not vile, because they are frequently conquered, neither are Sophists. And, that if a private person ought to be careful of your renown, you also ought to be careful of the renown of the Greeks. Another place consists in the consideration of time, as is exemplified in what Iphicrity says in his oration against Hermodius. For, said he, quote, if before I had done the thing I had demanded a statue in case I did it, you would have granted it to me. And, will you not grant it, now I have done the thing? You would not, therefore, when expecting a benefit, promise a reward, and refuse it, when you have the benefit. And again, for the purpose of persuading that the Thebans ought to permit Philip to pass through their land into Attica, it may be said, that if you had made this request before he sent you assistance against the Fakensis, you would have permitted it. It is absurd, therefore, that because he then neglected to ask permission and trusted you would grant it, close quote. Another place is taken from things said and reported on the speaker. And this mode is eminently useful, and was employed in the tragedy of Teochris by Iphicrates against Aristophan, when he inquired of him whether he would have betrayed the ships for money, and when Aristophan denied that he would, Iphicrates afterwards said, quote, you, therefore, being Aristophan would not have betrayed them, and should I, being Iphicrates, have betrayed them? Close quote. It is necessary, however, that he who is opposed should appear to have acted more unjustly than the opponent, for if not it would seem to be ridiculous if anyone had said this against Aristides, accusing who was in every respect worthy of belief, and which ought to have been said against an accuser who did not deserve to be credited. For in short, the plaintiff ought to be considered as better than the defendant. He, therefore, who opposes another should always reprobate this, and universally that which is said is absurd when anyone who reproves others in things which he himself does, or would have done if he could, or who persuades others to do those things which he himself does not do, nor would have done. Another place is derived from definition, such for instance as quote, that which is demonical, is nothing else than either God or the work of God, but whoever thinks that it is the work of God must necessarily think that there are gods. Close quote. And as Iphicrates said against a certain person named Harmonius, quote, he who is the best of men is most generous or noble, for there was nothing generous in Harmonius and Aristogiton till they had accomplished some generous undertaking. Close quote. He added that he was more allied to, i.e. he more resembled the ancient Harmonius, quote, for my works, close quote, said he, quote, are more allied to the works of Harmonius and Aristogiton than thine. Close quote. And as in the aeration concerning Paris, quote, all men will confess that those who are intemperate are not satisfied with the enjoyment of one body. Close quote. Hence Socrates said that he would not go to Arkelus, king of Macedonia, quote, for it is disgraceful, close quote, said he, quote, for him who has received a benefit, not to be able to recompense him from whom he received it. Just as it is disgraceful, in him who has been used ill, not to return the ill treatment, close quote. For all these, defining and assuming what a thing is, syllogize about the things which are the subjects of their speech. Another place is derived from multiplicity of diction as in the topics. An argument is derived from that which has a rectitude of subsistence being multifariously predicated. Another place is derived from division as if all men act unjustly for the sake of three things. For they act unjustly either for the sake of this vis utility or for the sake of this vis pleasure or for the sake of this vis because they are enraged. But for the sake of two of these they could not do the injury and the opponents themselves confess they did not do it for the sake of the third. Another place is from induction as from the oration inscribed Peparthia in which it is said quote that women everywhere determined truly about the birth of children, close quote. For this is evident from what happened at Athens since when Mantius the rhetoration was dubious about his son his doubts were dissolved by the mother of the child. This likewise happened at Thebes. For when Ismenes and Stilbo contended which of them was the father of the Salicus didonus demonstrated that he was the son of Ismenes and on this account the Salicus was considered as the offspring of Ismenes and again from the law of the Edectes if no one would commit his horses to the care of those who not pay a proper attention to the horses of others nor his ships to those who destroy the ships of others and if the like takes place in all things we ought not to commit our safety to those who have badly attended to the safety of others and as Alcidamus says quote that all men honor the wise close quote for the Perians honor Archilicus though he blasphemed them and the Chians honor Homer though he was not their fellow citizen and the Midlennans saffo though she was a woman the Lacedaemonians also though they were in the smallest degree philologists made Kylo one of their senators the Italians likewise honored Pythagoras and the Lampsini buried Anaxagoras though he was a stranger again the Athenians by using the laws of Solon were happy and the Lacedaemonians by using those of Lycurgus the city of the Thebans also as soon as philosophers were their governors became happy another place is derived from the judgment made about the same or a similar or a contrary thing and this indeed is especially the case if it is the judgment of all men and always but if not if it is the judgment of most men or of all or the greater part of wise men or of good men or if it is the decision of those who are judges or of those whom the judges approve or of those against whom there is no judgment to be given as of princes or of those whose judgment it is not becoming to oppose a father or preceptors but of this place there are many examples and one is what Atocles said against mixedemides quote, if it were well indeed for the venerable goddesses the Furies to plead their cause in the Areopagus can it be improper for mixedemides to do so? close quote another is what Sappho said quote, that to die is an evil for the gods have judged it to be so since otherwise they themselves would die, close quote another is what Aristipus said against Plato asserting something as he thought to positively quote, but our associate said he meaning Socrates affirms no such thing, close quote another example is that of a Jesipolis who at Delphi inquired of the god Apollo having prior to this consulted the oracle of Jupiter Olympus quote whether the son was of the same opinion as the father, close quote as if it were shameful for a son to descend from his father another is that of Isocrates concerning Helen who shows that she was a worthy character because Theseus judged her to be so and who also says the same thing of Paris because the goddesses preferred his judgment to that of other men he likewise asserts that Evigorus was a worthy character quote, because Conan when his affairs were adverse leaving everyone else came to Evigorus close quote another place is from the enumeration of parts as in the topics where it is inquired quote, what kind of motion the soul is close quote for it is either this or that vis, it must either be the motion which is a change in quality or relation or argumentation or generation an example of this place is from Theodex in his oration in defense of Socrates when he was accused by the judges, quote what temple has Socrates violated and what gods has he not reverenced among those whose honors are legally established by the city, close quote another place is from consequent good or evil for since in most things it happens that some good and evil are consequent to them we may employ consequent good for the purpose of persuading praising and defending but consequent evils for the purpose of dissuading blaming and accusing thus for instance we may blame literary pursuits because envy is consequent to erudition because they are attended with wisdom which is a good hence in the former case we may say that it is not proper to acquire erudition because it is not proper to be envied but in the latter that it is proper to acquire erudition for it is requisite to be wise in this place the art of the returition calipus consists to which he added what pertains to the possible and other things of which we have already spoken another place is when about two things and those opposed to each other it is requisite either to exhort or dissuade and to use the before mentioned place in both ways but it differs from that place in this that their casual things are opposed but here contraries only thus for instance a certain priest would not suffer his son to speak in public if you speak what is just men will hate you but if what is unjust the gods it is necessary however on the other hand to speak in public for if you speak what is just the gods will love you but if what is unjust men will love you this however is the same thing with the saying of buying oil and salt this argument may be retorted when to each of two contraries good and evil are consequent each being contrary to each another place is because the same things are not praised openly and secretly but just and beautiful things are especially praised openly and privately men are more inclined to praise what is advantageous one of these therefore we must endeavor to collect for this place is the most principal of paradoxes another place is derived from analogy and was used by Iphicrates for when the Athenians wished to compel his younger son because he was large to engage in public service Iphicrates said quote that if great boys were to be considered as men little men should be decreed to be boys close quote and Theodex in the law said quote you have made mercenaries such as strabuses and charitomous citizens on account of their probity but you have not made exiles of those among the mercenaries who have acted nefariously close quote another place is when in consequence of the same thing following from two things it is shown that the things from which it happens to follow are the same as Wenzhenophonies said quote that those were similarly impious who assert that the gods were generated and those who assert that they die for in both ways it happens that at a certain time the gods do not exist close quote and in short that which happens from each is always to be assumed as the same this place was also used by someone in the defense of Socrates for he said quote you are about to pass sentence not on Socrates but on his pursuit whether it be requisite to philosophize close quote and it may be said quote that to give earth and water is to become slaves and that to participate of common peace is to do what is commanded to be done close quote but whichever of these is useful must be assumed another place is derived from this that the same men do not always choose the same thing in a posterior or prior time but conversely as in this enthamim quote if when we were exiles we fought in order that we might return shall we having returned fly in order that we may not fight close quote for at one time the Athenians chose to fight that they might return to their country and at another time they were unwilling to leave their country lest they should be obliged to fight another place is when we affirm anything to have been done on account of some cause through which it might have been done though it was not in reality done through it as if one man should give something to another in order that by afterwards taking it away we may give him pain hence also it is said in a certain tragedy quote the demon gives great prosperity to many yet not with a benevolent intention but in order that they may receive more conspicuous calamities close quote and in the Meliager of Antiphon who that he might praise Meliager says quote there was a concourse of people from all Greece not for the purpose of killing the boar but that they might be witnesses of the valor of Meliager close quote another example is from the Ajax of Theodex in which it is said quote the diamond preferred Ulysses as his associate in the nocturnal adventure not for the purpose of honoring him but that he might have one to attend him who was his inferior close quote for it is possible he might have thus acted with this view another place is common both to litigants and counselors and consists in considering whatever pertains to exhortation and dissuasion and for the sake of which things are done and avoided for these are such as ought to be done when they are present for instance it must be considered whether a thing is possible and easy to be affected and whether is beneficial either to a man himself or to his friends or whether it is noxious and pernicious to his enemies or is at least attended with greater emolument than loss and exhortations are to be derived from these places and dehortations from the contrary from the same places also accusations and defenses may be derived defense indeed from those which pertain to dissuasion but accusation from those which pertain to exhortation and in this place the whole art of pamphilis and calypus consists another place is derived from things which appear indeed to be done but are incredible because they would not be credited unless they were or nearly were in existence and this in an eminent degree for whatever is done is apprehended to be done either because it has been truly done or is of itself credible and probable if therefore a thing is incredible and not probable it will be true that it has been done for it does not appear to have been done in consequence of being probable and credible thus androcleys pithius accusing the law said the multitude being tumultuous whilst he was speaking must require a law to correct them close quote for fishes also require salt though it may seem neither probable nor credible that animals nourished in salt should require salt and olives require oil though it may seem incredible that those things from which oil is produced should be in want of oil chapter 25 another place which is adapted to confutation is derived from considering things which are not assented to vis from considering if anything is not admitted from all times actions and speeches and this indeed may be done separately in the person of the opponent as quote he says that he loves euthenians and yet he has conspired with the 30 tyrants against his country close quote and separately as to the person himself as quote he says indeed that I am litigious but he cannot show that I ever sued any man close quote it may also be done separately both as to the person himself and his opponent as quote and this man indeed never lent any money but I have ransomed many of you close quote another place is useful with respect to men and things that have been accumulated but which do not appear to have deserved it and this consists in assigning the cause of the paradox for there is something which gave rise to the appearance thus for instance a certain woman was columnated with reference to her son for inconsequence of embracing him it seemed as if she had connection with the lad but the cause of her embracing him being assigned the cause thus to in the Ajax of theodex Ulysses says against Ajax that though he is braver than Ajax yet he does not seem to be so another place is derived from cause which if it exists the effect also exists but if it is not neither does the effect exist for cause and that of which it is the cause subsists together and nothing is cause thus Letimus in defending himself when Thrasybulus accusing him said quote that his name has been branded with infamy on a pillar in the acropolis but the inscription had been erased by the 30 tyrants close quote replied quote that this was not possible for if it had taken place the 30 tyrants would have placed more confidence in him inconsequence of his hatred to the people having been inscribed on a pillar close quote another place is from considering whether it was or is possible to advise or do or have done a thing better than it was advised to be or is or was done for it is evident that if it does not thus subsist it was not done since no one willingly and knowingly deliberately chooses what is bad this place however is false for frequently it becomes afterwards evident how it was possible to have acted better though this was before a manifest another place is derived from considering when something is intended to be done contrary to what has been done thus xenophonies when the aliens asked him quote whether they should sacrifice to Lucothea and lament her or not close quote if they thought her a goddess not to lament her but if a mortal not to sacrifice to her close quote another place is derived from accusing or defending heirs thus for instance in the meta of the poet Carcinus some persons accuse her of having slain her children because they no longer appear bracket for meta aired in sending away her sons but she defends herself by saying quote that if she had intended to commit murder she would not have slain her children but Jason for in not slaying Jason she would have acted wrong even if she had done the other thing i.e. slain her children this place however and species of enthamim formed the whole prior rhetorical art of Theodorus another place is derived from name Carcinus Sophocles of a certain woman named Cydro quote to clear thou iron art and burst the name close quote thus also it is usual to celebrate the gods from the signification of their names Conan likewise called Thrasybulus audacious and Herodicus said of Thrasymachus quote thou art always Thrasymachus i.e. bold in fight he also said of Polis quote you are always Polis i.e. a cult Herodicus likewise said of Draco the legislator quote that his laws were not the laws of a man but of a dragon for they were severe close quote another example is derived from what Hecuba says in the Troids of Euripides when speaking of Venus quote in the name of the goddess i.e. Aphrodite is rightly derived from Aphrosun i.e. Folly and as Coremin the comic poet says quote Penteas was so denominated from future calamity close quote those enthamemes however which are adapted to computation are more approved than those that are ostensive there are short collections of contraries but parallels are more obvious to the hearer of all syllogisms however as well those that are adapted to computation as those that are ostensive those especially excite perturbation in the auditors which manifest themselves as soon as they begin to be annunciated yet not because their meaning is superficial for the auditors are foresaw from the beginning what would follow this likewise is the case with those syllogisms which are understood as soon as they are completely annunciated chapter 26 since however it is possible that one thing may be a syllogism and another not but only appear to be so it is likewise necessary with respect to an enthamem that this should be should not be but should only appear to be an enthamem since an enthamem also is a certain syllogism but there are places of apparent enthamemes one indeed in addiction and of this one part as in dialectics is when though nothing is syllogistically concluded yet at the last it is inferred it is not therefore this or that or it necessarily is this or that what also is said in enthamemes contortly and oppositely appears to be an enthamem though it is not so in reality for such addiction is the receptacle of enthamem and a thing of this kind appears to be from the figure of the diction for the purpose however of speaking syllogistically in the diction it is useful to produce the heads of many syllogisms as quote for each of these is demonstrated from others but from the conjunction of these something appears to be affected another place of apparent enthamemes is derived from equivocation as if someone should say quote that a mouse is a worthy animal for the mysteries are the most honorable of all initiatory rights close quote or if someone making an enthamem on a dog should also comprehend in his enthamem the celestial dog or the god pan because pindar says quote oh blessed whom the olympiad gods call the all various dog of the great goddess close quote or if it should be said quote that it is most dishonorable there should be no dog so that it is evident that a dog is honorable close quote quote that hermes is the most communicative of all the gods for he alone is called common hermes close quote likewise to say quote that logos speech is most worthy because good men are worthy not of riches but of logos speech close quote for to be worthy of logos is most simply predicated another place consists in speaking things which are separated conjunctively or things which are conjoined disjunctively for since each of these modes of speaking appears to be the same though frequently it is not the same it is requisite to adopt whichever of these is more useful to know being in Sicily that there is a three banked galley in the Piraeus close quote another example is that he who knows the elements of a verse knows the verse for a verse is the same thing close quote as the elements from which it is composed another example of this place is quote that since twice so much of a thing is obnoxious neither can the half of that quantity be said to be salubrious for it is absurd if two things are good that one of them should be bad close quote thus therefore this place is useful for the purpose of computation but it is ostensive as follows quote for one good is not two evils close quote in short this place is paralogistic then another example is that of polyquities respecting Thrasybulus quote that he deposed the 30 tyrants close quote for this is conjunctive or what is said in the Orestes of Theodex for it is from division or is disjunctive vis it is just that she who killed her husband should die and it is also just that the son should revenge his father it is just therefore that the mother Klytemnestra should be slain by the son Orestes close quote for if these sentences are conjoined the conclusion perhaps will no longer be just in this Sophism likewise there is a fallacy of defect for it is not expressed by whom it is just that the mother should be slain this place consists in confirming or confuding by exaggeration and this is when a man not showing that he has done a certain deed amplifies the thing for thus he causes it to appear either that he has not done the deed when he who defends the cause amplifies or that he did it when the accuser was enraged hence it is not an enthamem for the hearer falsely collects that he has or has not done the deed the thing not being demonstrated another place is derived from a sign for this also is unsilogistic as if someone should say quote lovers are advantageous to cities for the love of Hormodius and Aristogitan deposed the tyrant Hipparchus close quote and likewise if someone should say quote that Dionysius was a thief for he was a depraved character close quote for this is unsilogistic since not every depraved character is a thief but every thief is a depraved character another place is derived from that which is accidental as in what Polycrates said of the mice quote that they aided the city by knowing the both strings of the enemy close quote or if someone should say that to be invited to supper is a most honorable thing for Achilles in consequence of not being invited was enraged against the Greeks in Tenedos but he as being despised was angry and this happened because he was not invited another place is derived from that which is consequent as for instance in what is said of Paris quote that he was magnanimous for despising an association with the multitude he dwelt in mount by himself close quote for because magnanimous men are lovers of solitude Paris also may appear to be magnanimous and quote since a certain person decorates himself and wanders by night he is an adulterer close quote because adulterers also are men of this kind in a similar manner it may be proved that mendicants and exiles are happy quote because mendicants sing and dance in temples and because it is permitted exiles to dwell where they please close quote for because these things are present with those that appear to be happy those also to whom these things are present may seem to be happy there is here however a difference in the mode on which account this example falls into defect i.e. it is a fallacy of defect another place is derived from which is causeless as if it were a cause as when that which is done together with another thing or after it is assumed as if it had been done for the sake of it and this place is especially used by politicians as by demedies who said quote that the administration of demastinies was the cause of all evils since war happened after it close quote another place is derived from a deficiency in the time when in the manner in which a thing is done such for instance as this quote that Paris just ravaged Helen for the choice was given to Helen of marrying whom she pleased close quote for perhaps this choice was not given to her always but at first and the authority of her father over her extended so far as to this or as if someone should say quote that the strike free men is insolence close quote for it is not entirely so but when he who strikes was not provoked father still another place is when in litigious disputes an apparent syllogism is produced from that which is simply and that which is not simply as in dialectics it is shown that non-being is being for non-being is non-being and it is also shown that what is unknown is the object of science for the unknown is the object of science because it is unknown because it is known that it is unknown thus also in orations there is an apparent and they mean from that which is not simply probable but is a certain probable thing this probability however does not take place universally as Agatha also says quote perhaps someone may say that this is probable that many things which are not probable happen to mortals close quote for that which is unlikely happens hence what is unlikely is likely but if this be the case that which is not probable is probable this however is not simply true but as in contentious arguments a fallacy is produced when a limitation restraining to a part to a place, to a time or a signifying relation is not added so here that which is improbable is not simply probable but is a certain probability but the art of Korax is composed from this place for whether the person be not obnoxious to the crime as he who is weak escapes an action for an assault for it is not likely that he committed an assault or whether he be obnoxious as being a strong man he has the same defense unless a certain probability is apparent and the leg takes place in other things for a man must necessarily be obnoxious to the crime or not both therefore appear to be probable and the one is indeed probable in reality but the other not simply but in the way we have shown and this it is an argument to be superior hence men were justly indignant with what Protagoras professed to accomplish for what he announced is false and not true but is apparently probable and exists in no art but in the rhetorical and contentious and thus much concerning enthamemes both the true and the apparent Chapter 27 it now follows we should speak concerning the solutions of enthamemes but it is possible to dissolve them by contrary reasoning or by introducing an objection with respect to contrary reasoning therefore it is evident that it may be affected from the same places for syllogisms are composed from things that are probable but many probable things appear to be contrary to each other but objections are introduced in the same manner as in the topics in four ways for they are introduced either from the same or from the similar or from the contrary or from the judgment and authority of others by an objection however being introduced from the same I mean as if for instance the enthamemes should be concerning love that it is a worthy thing an objection may be made to it in a twofold respect for either it may be said universally that all indigens is evil or partially that it would not be proverbially said conus love unless there was also base love but an objection is introduced from the contrary when if the enthamemes should be that a good man benefits all his friends it may be objected that neither does a bad man act ill towards all his friends an objection also is introduced from similars when the enthamemes is that those who receive an injury always hate the authors of it close quote for it may be objected that neither do those who are benefitted always love their benefactor close quote and objections which are introduced from the judgments of illustrious men are as if the enthamemes were quote that it is requisite to pardon those who are intoxicated for they air ignorantly close quote the objection is quote that peticus therefore does not deserve to be praised for he should not have legally established greater punishments for intoxication if he who is intoxicated airs through ignorance close quote enthamemes however are derived from four things and these four are the probable example tekmerian i.e. a necessary sign and a sign not necessary but those enthamemes which are collected from things that exist for the most part or appear to exist are derived from probabilities those which are derived from the similar either from one or many similar things bracket when the orator assuming what is universal so logistically collects what is particular close bracket exist through example but those which exist through what is necessary and real are through tekmerian and those that exist through what is universal or particular whether it really is or not are through signs which are not necessary but a probable thing is that which does not exist always but for the most part hence it is manifest that enthamemes of this kind will always be dissolved if an objection is introduced the solution however is sometimes apparent and not always true for he who objects does not dissolve the enthamem by showing that the thing is not probable but by showing that it is not necessary hence the defendant has always the advantage of the plaintiff through this paralogism for since the plaintiff demonstrates through probabilities that the thing is not the same which shows either that the thing is not probable or that it is not necessary and that which exists for the most part is always liable to objection bracket for otherwise it would not be a probability but would be always necessary close bracket hence the judge if this mode of solution is adopted will think either that the thing is not probable or that it must not be judged by him in consequence as we have said of being deceived by false reasoning for it is requisite that he should not only judge from things which are necessary but also from probabilities for this is to judge most judiciously the solution therefore of an enthamem is not sufficient which shows that a thing is not necessary but it is requisite that the solution should also show that it is not probable but this will happen if the objection rather shows that the thing for the most part subsists it is possible however that a thing may happen for the most part or frequently in a two-fold respect is either from time or from circumstances but principally if from both for if things which frequently happen does subsist this is more probable but signs which are not necessary and enthamemes derived through a sign are solved in the way we mentioned in the first book for that every such sign is unsilogistic is evident to us from the analytics enthamemes however derived from examples are solved after the same manner as enthamemes derived from probabilities for if we can induce a contra example in which the thing is not so the enthamem is solved because the thing is not necessary or because many things have happened frequently and in a different manner but if many things have happened frequently and in this manner then it must be contended either that the present circumstance is not similar or is not similarly disposed or has a certain difference tecmeria however i.e. necessary signs and enthamemes which are of the nature of tecmeria cannot be solved in consequence unsilogistic but this is evident to us from the analytics it remains therefore to show that what is said is that certain premises do not exist but if it is evident that the premises do exist and that the enthamem is derived from tecmerian then the enthamem becomes insoluble for all things now become apparent from demonstration chapter 28 to amplify however and diminish are not the elements of an enthamem for i call the same thing an element and a place for an element as also a place is that into which many enthamemes fall but to amplify and diminish are enthamemes for the purpose of showing that a thing is great or small as likewise that it is good or evil just or unjust or anything else and all these are the things with which syllogisms and enthamemes are conversant so that if no one of these is the place of an enthamem neither are amplification and diminution neither are enthamemes which have the power of solving the arguments of the opponent any other species of enthamem than those which are employed in confirmation for it is evident that he solves the arguments of his opponent who either shows the contrary to what his opponent asserts or introduces an objection but he proves the opposite thus if one shows that a thing has been done the other shows that it has not been done and if one shows that it has not the other shows that it has been done so that here indeed there will be no differences for both use the same enthamemes since they introduce enthamemes to show that the thing is or is not an objection however is not an enthamem but bracket as we have shown in the topics close bracket it is to declare a certain opinion from which it will be evident that the conclusion is not syllogistical or that something false has been assumed and thus much has been said by us respecting examples and sentences and in short respecting what pertains to the reasoning power once we may abound with enthamemes and how we may solve them it now remains to discuss what pertains to diction and order end of book 2 recording in memory of Mitchell Edwards


All right. Morning, everybody. Friday, Friday. Everybody ready for the weekend? Exciting weekend spent working on assignment three. All right, so my original plan was that I would finish up talking about performance today. And the next week we would talk about virtualization. But as I started writing these lectures about performance, I realized that there's actually a lot to talk about. So hopefully I'm not really boring people by going too slow. In terms of the material, which I think I described as a pantheon the other day, of fascinating and important information that you're learning in this class, I want to spend some time here because I think this is important stuff. I also think it's likely that you will be able to use this in other places. Yeah, it's a light crowd today. I feel like the room is more echoey. There's fewer warm bodies in here. All right, so today we're going to do lecture two on performance. And then on Monday, I'm actually going to do another lecture on performance where we're going to focus Monday on what do you actually do, like what are ways to improve the performance of your system. So on Wednesday, we talked a little bit about the challenges inherent in improving performance. Today, we're going to continue to talk about some of the pitfalls to performance improvement, some of the things that you might encounter and need to do along the way in order to sort of figure out where to spend your energy.  You only have a certain number of hours from the day. And you want to work on the part of the problem that's going to produce the biggest improvement, the biggest bang for your buck. So we'll talk today about trying to figure out exactly what that is. Work on assignment three, not much else to talk about here. I'm going to send an email out today. I've been meeting with people in my office. And we've talked a little bit about the assignment. And I have some suggestions about ways to approach things in terms of what to implement first. And then we'll also give you some idea of how we're going to test your assignments, right? What programs you should be able to run where and kind of where are stopping points in the assignment that you can get to if you're in terms of finding good places where you might want to get your code to and then maybe tag it there so that just in case you don't get things completely working, you have something working to submit. So yeah, that'll be out today. All right, so any questions about Monday's material on, wait, today's Friday? Friday, right. Wednesday's material, or Monday's material, doesn't really matter, although I can't even remember what we talked about on Monday. Any questions about the stuff we talked about Wednesday? So Wednesday we started to talk about performance analysis, we talked about the difficulties inherent in measurement, and we started to get into a little bit of benchmarking and we're going to keep talking about benchmarking today. But any questions on the stuff we covered Wednesday? Wednesday, Wednesday. All right, so just a couple of questions. So who remembers? I mean, I put this up on the board and we thought this was easy. We could all just go home early at 9.30. What's difficult? Who can remind me about some of the challenges inherent in improving the performance of a real system or even just the performance of any piece of code? I'm gonna start over in the, let's see here. I don't think I knew guys enough. Anybody remember, anyone who was here Wednesday remember what was difficult about improving the performance of a real system? Anybody who's got the slides from last time up on their laptop? And you guess this from here, Alex? I mean, there actually were quite a few things, right? So there should be a lot to pick from here. You don't know. All right, anybody from this side of the room? Left side of the room. Represent nothing, you guys got nothing, right? Ooh, okay. All right, right side of the room. Anybody want to show these guys up? Yeah. Right, so experiments aren't necessarily repeatable, right? That can be frustrating.  So the difficulty in experiment repeatability, okay. I'm gonna, now that the right side has contributed left side, I'm gonna give you guys another chance. Anybody over here wanna throw something into the hat? Something that's difficult about performance analysis and improvement. All right, I'm going back to the well, right side. I feel like this side of the room's a little bit more away today, okay? Anybody else, any other contributions? Timing, measuring time, right? We talked just something as simple as measuring the amount of time that has passed on a system, right, can be itself challenging. Now these guys are really, really giving it to you today. Yeah, Scott. Right, right, right. That's actually kind of what we're gonna talk about today. Finding ways to effect the results, right? Like finding out which pieces of code are actually contributing to the problem and then on Monday we'll talk about some, ah, right, right, yes. The systems measurement uncertainty principle, right? Measuring the system itself, it can create two problems, right? It can either make the system so slow that it's unusable or it just might destroy some sort of subtle effect that you're trying to understand, right? So yeah, okay, great. So yeah, so this was our basic blueprint of how to do things, but we talked about again some of the things that are so hard here, right? How do you measure the system? How do you even measure time, right? And then today we'll talk a little bit about what should the system be doing when you measure it, right? How do you develop good benchmarks that allow you, that, you know, a good benchmark means that if you improve the performance of the benchmark, you're doing something useful. A bad benchmark means that if you improve the performance of the benchmark, nobody will care because nobody will actually notice in real life, right? When they're real at using your system, all right? We'll talk just a little bit today. I mean, this is not a course on probability and statistics but I think that there's some general level of statistical hygiene that's important for computer programmers to have. How many people have taken a course on statistics? Is that required for the major here? To be a computer scientist and engineer at UB must one take a statistics class. Does anybody know? What's that? Oh, whoops, the wrong one. Statistics would have been more useful. Okay, no, no, probability is close, right? But statistics would be more helpful. At least I think so, right? So unfortunately computer systems people are kind of like mentoriously allergic to statistics and we'll talk a little bit today about some of the little minimal amount of statistical hygiene that might help you out when you're trying to understand and do things. And then yeah, so determining why I capitalized a sentence the way I did, but determining what to improve and then finding techniques for ways to improve it. So today we're gonna talk about the kind of what to improve, how do we choose the parts of the code that we're gonna spend our time on and then on Monday we'll go over some sort of classic techniques, different sort of tried and true methods of improving the performance of a piece of code, right? You sit down with a piece of code, it's not performing well, what do you do, right? So that's kind of like that's the last question you have to ask and in some ways one of the more fun ones, all right? Okay, so remember we talked a little bit about the fact that because we started and we got into this whole litany of complaints and problems and challenges about experimenting on real life systems, right? Either real life systems that are in active use, which is very difficult to do, right?  And so we came up with these two other approaches, right? One was modeling, the other was simulation. Who remembers some of the pros and cons of modeling versus simulation, right? Then, right, so simulations will produce repeatable results and that's good, right? That can be in contrast to real systems that are very, very difficult to produce repeatable results on, simply because they're almost never in the same state twice, right? With the simulator, again, you're scraping some complexity off the system in hopes that this will allow you to understand it better and make the necessary improvements. What about models? What's something good about a model? Well, okay, so strong, I would say strong mathematical conclusions, right? Like the conclusions that you get out of a model can be very strong and frequently are provable conclusions, right? They're hard conclusions. What's the problem with models? Unrealistic assumptions, right? Normally, to get the system into a state where we can model it and we can say things about it with mathematical certainty means that we have to make so many unrealistic assumptions that the system that we're modeling, the system that we're using these analytic techniques to try to understand, bears potentially little to no resemblance to the system that we're actually trying to improve in the first place, so that's kind of a challenge, right? What's one of the downfalls about simulations? The simulator, right? I've got to write a simulator, right? Now I've introduced this whole extra piece of code that has to be correct, right? It was hard enough before where I had code that had to be correct. The code that actually had to work that was running on my real system. Now I've essentially asked the person who's trying to improve the performance of something, hey, by the way, go out and write a bunch of other code that also has to be correct and then use that code to experiment with the code that you were trying to work on in the first place, right? So yeah, simulators can have bugs, right? So models, we can make strong mathematical guarantees after making a bunch of unrealistic assumptions, right? Simulations, in the best case, you know, the simulator is simpler and faster to work on than the real system and in the worst case, the simulator, some of the, I mean, there's two things that can go wrong with simulators. Well, there's many things that can go wrong with simulators, right? One is that the assumptions that we make or the complexity that we remove and design in the simulator renders the simulator results meaningless. The other problem is that the simulator itself can have problems, right? All right, any other questions on Wednesday's material before we go bravely onward? Questions about this sort of stuff. You guys feel ready for the first step and a half of performance analysis in Belgium? Maybe we should add a performance assignment as part of assignment three.  Let me just add a little extra. There seems to be no, yeah, okay. I mean, nobody seems opposed. All right, probably not. What's that? He's just in shock.  All right, so what, and okay, so this is a little bit of a review from last time too, right?  So now we're getting to the point where we figured out how to measure, we've decided which approach we're gonna take modeling simulation or, you know, trying to work on a real system. Now the question is, what do we, like what code do we actually have running in the first place, right? What inputs do we use into the system? And, you know, and what metrics do we use as comparison metrics, right? So the first part is, you know, what do we use to run on the system? The second part is, what do we use to evaluate the outputs, right? And we talked about how difficult it can be to compare things, right? I mean, what does it mean to compare to disk drives, right? Or two different scheduling algorithms or I wish I could have eyes in the back of my head so I could remember what was on the slide without looking, two page replacement algorithms, right? Or two file systems. I mean, these are big complicated things that have many different performance outputs, right? And so finding a way to compare these is really difficult, right? And the other thing we're talking about today is frequently, you know, you have to be careful that in improving the performance of one part of the system, you don't make changes to other parts inadvertently that destroy those performance gains.  So what system have we already discussed that potentially had this problem? A system that made a really dramatic improvement to a certain thing that the system did but potentially had this, like, all this dirt that was brushed under the rug in order to do this. Who remembers what we've already talked about that had this potential problem? What's that? Log-structured file systems, right? So remember when we talked about log-structured file systems it was like revelation, right? We could do all the rights of the disk in the same place. Woo-hoo! And then it was like, oh crap, we gotta clean the disk and it's, oh, okay. So again, I mean, sometimes you feel like you're working with the proverbial tablecloth that's too small, right? No matter how hard you yank it into one corner, it pops up somewhere else, right? You just need to be careful when you're improving the performance of your system that you're not doing things that are gonna jeopardize the performance of every other part of this, right? All right, so, and yeah, so I put this up just because, you know, I like to have a picture in every set of slides. All right, so let's talk about benchmarks, right? Usually we can break down benchmarks, right? So first of all, let's just define the term, right? What is a benchmark? What is a benchmark? Anybody? What's that? A standard? Okay, there is something standard about it. I will take the word standard from your answer and I will use it. You contribute it to the right answer. What else? Benchmarks are standard what? What's that? Okay, so it's something that we do use as a point of comparison, but why, right? So it's a standard point of comparison, right? If we do the results carefully, they are comparable and we would hope that they would be objective, right? But why use benchmarks, right? What's the alternative, right? So imagine I've got the file systems community, right? And without benchmarks, what do people end up doing? Yeah, they end up writing their own test. Or they might use some sort of application benchmark, right? But you can't compare things, right? Like, you know, you had some sort of performance problem that led you to implement a new schedule in algorithm and you ran it on your workload and it worked great. You know? And I don't know, Keith had a different performance problem. He wrote a completely different schedule in algorithm and it's so great on his workload, right? But I don't know, you know, I need to pick one of yours and I don't know which one is quote, unquote, better, right? So the ideal benchmark gives us a way to compare apples and apples, right? When you talk about file system benchmarks, the idea is benchmarks are supposed to represent some sort of degree of representative requirements, right? You know, what do most applications file system usage patterns look like, right? Because if I'm trying to evaluate file systems, I want to pick the file system that works best for a specific set of applications. Now, what's the problem with benchmarks for this very same reason? What's the challenge with developing a good benchmark? You need a benchmark for your benchmark, right? So different benchmarks might succeed or fail, but kind of like, what's the existential challenge of benchmarks, right? I mean, on some level, as benchmarks, you know, I would love a benchmark that allowed me to compare every different aspect of file system performance, right? Or let's put it this way, I would love a benchmark that reflected every possible application that could ever use the system, right? But what's the problem with that benchmark? It's almost impossible to cover everything one, right? What's true about applications and processes that use computer operating systems? They're different, right? They have different requirements. And so, you know, it's kind of like, you're like, okay, I had this great idea. I'm gonna come up with a benchmark that allows me to compare any two file systems, right? And then you start looking at different applications. You say, I'm gonna use that application, that application is this. And pretty soon you have something that's so general, that it's meaningless, right? It doesn't really reflect any individual application, right? But it reflects some sort of, like, it's like, well, you know, all the applications do reads, right? Or something, right? So there's some, you know, tension here in benchmarks between generality. Because generality allows benchmarks to be powerful, allows you to say, this, if I do well on this benchmark, it means that I'm gonna prove that lots of different applications performance, right? But as benchmarks get more general, they also stop being reflective of the applications that are contained in them, right? Because the database application is on some level very, very, very different than a gaming application, right? In terms of how it uses the file system. Trying to come up with a benchmark that, you know, improves the performance of both is very difficult, right? All right, but let's talk about these sort of different categories of benchmarks, right? So who knows what a, who knows or can guess what a, what is a micro benchmark? Ben? Right, so micro benchmarks try to isolate, you know, one aspect of system performance, right? Very, very, very specific, but you know, you're trying to look at one, one, one very, very tiny little thing, right? And usually, why are you usually using a micro benchmark? Why would you be trying to observe or isolate the performance of one specific part of the system? Checking for bottlenecks, but what is more likely, what is more likely to be the thing that you are doing at this time, that you are using this micro benchmark? You're working on that part of the system, right? So you wanna be able to isolate your changes from other noise that's affecting other parts of the system, right? Okay, excuse me. Similar vein, macro benchmarks, right? So what's a macro benchmark? Micro benchmark, isolate one specific part of the system, macro benchmark. It tries, yeah, I mean it tries to look at holistic system performance, right? Tries to see, you know, how does the whole system as an entire blob, you know, perform, right? Given some sort of general workflow, right? Okay, now, let's find some sort of uneasy middle ground here. Application benchmark, right? What is an application benchmark? Or an application, or a class of application that you talk about, you know, benchmarks that represent classes of applications. So macro benchmark is trying to like say, you know, oh, okay, over all different types of workloads, this is how the system would respond. Micro benchmark is trying to isolate one specific aspect of system performance. So an application benchmark is doing what? What's the most direct way to measure the improvement provided to an application by a change you're making to the system? What's the most obvious thing to do? Run the application, right? I'm trying to improve the performance of the file server, right? Or I'm sorry, I'm trying to improve the performance of a web server, right? This is my goal, I'm trying to make changes to the file system specifically to better support web servers, right? So run a benchmark or run a specific application that is that type of application, right? Run a web server on the system and measure its performance, right? So you have these application benchmarks that can represent specific classes of applications, right? And, you know, how well would a typical database workload, right? Database is a class of application, right? But you could also just run, you know, a standard version of a particular database with a given workload and look at how well it performs, right? So this is one sort of taxonomy of benchmarks. So let's talk about examples. Let's try to come up with a few examples of each one, right? And let's say, let's do this in the context of we're trying to improve the performance of our virtual memory system, right?  So what are some micro benchmarks that we could develop to focus on specific aspects of the performance of our virtual memory system? Give me one example. What's one sort of atomic unit of operation or specific thing that the virtual memory system might do or a specific part of the virtual memory system we might wanna isolate and be able to? Okay, what about me? Paging is a little broad. So what about page? Page eviction. Page eviction, what, okay. See, I'm gonna, you know, what about page eviction, right? So, okay, so I could measure how long it takes to swap a page to disk, right? I would argue that's not really a VM benchmark though, right? That's more dependent on file, on disk performance, right? But let's think, I'm thinking about page eviction. What are parts of that that are specific to the virtual memory system? What does the virtual memory system have to do before it pages something to disk? It has to decide which one to evict. So what could I benchmark about that? How long it takes to decide, right? Okay, so my first one here that I actually put up was time to handle a single page fault, right? Now this is kind of funny because this, you might argue is not even really micro enough, right? Because this involves lots of other parts of this system potentially. And as we'll talk about a little bit, this, you know, what's the difficult thing about benchmarking sort of overall page fault handling performance, right? If I just ran the system and I measured how long it took to handle each page fault and I computed an average, right? What's the problem with that? Well, okay, we might defend on the performance of other things, but what's another problem? Right, there's so many different things that can happen here, right? There's a fast path where the page is in memory. There's a slightly slower path where I have to load a translation into the TLB. There's an even slower path where I have to evict something. You know, I might have to zero fill pages. So I've started to lump so many different cases together that this isn't really a micro benchmarking, right? So that's it. So time to look up page in the page table. Here's another example of a micro benchmark, right?  How long does it take to figure that out, right? Depending on how you implement your page tables, this will vary, right? And then how long does it take to choose a page to evict, right?  How long does this take? And of course, this is a function of how you do it, right? So here are some example micro benchmarks. What about macro benchmarks? What about macro benchmarks for your VM? Any guesses, any ideas? Yeah, John. You could do the conversation. Okay, yeah, so I would argue that might even be more of a micro benchmark, but you're right, you're starting to combine multiple things in there, right? So there's starting to be, yeah. So, you know, context switch overhead, right? So that's one, like, maybe, media benchmark. Yeah. How long does it take to start at the kernel? That's not a bad thing. That requires a lot of page faults. Okay, what about, can we recycle something from the list we already have up here? What about, you know, again, just aggregate time to handle page faults, right? So I take a system, I run some sort of, you know, heavy benchmark that pages heavily and then I just look at the average time that it takes or the page fault throughput, right? How many faults per second can I handle, right? And then, you know, it got page fault rate, right? So the page fault rate is an interesting indication of how effective the system is at handling page faults, right? The page fault rate can go up for a variety of reasons, right? One of the reasons might be that I'm doing a, you know, a better job of keeping pages in core. I'm doing a better job of selecting pages to move to swap. And so I'm handling more faults because the fault handling path is faster and the throughput of the system is improving, right? The page fault rate could also go up because I'm doing a bad job, right? And I'm evicting things in the TOB that should be evicted, right? So this is why macro benchmarks start to be complex because the fluctuation of them is not necessarily clear, right? All right, and then application level benchmarks. So, right, you guys have some of these, right? I can't claim that these are really like super realistic, useful applications, right? Like triple sort, right? I'm gonna sort three large arrays of numbers, right? I mean, I guess that's a benchmark, right? I don't know why you would do three all at once, but anyway. So we've given you actually some application level benchmarks and these can be used. And you can use these, right? If you guys want to experiment with your system, you can run these several times and you can look at the results, right? All right, so what about challenges using these various types of benchmarks, right? So what would be, and I think people have already kind of hitched it at some of these, what about the problems with micro benchmarks? What's one problem with a micro benchmark? Yeah, sorry, go ahead. Yeah, so there is some overlap between macro benchmarks and application benchmarks, right? The difference is that macro benchmarks are supposed to reflect some higher level component of system behavior that you can argue would be potentially important to all applications, right? Where application benchmarks are allowed to zero in on system performance, you know, components of system performance that are specifically important to a particular application, right? This is why you have these classes, you know, for example, database workloads, right? Because databases use operating systems in many ways that are kind of unusual. Actually, it's been kind of a running complaint by the database community for years and years that operating systems don't do a good job of supporting database workloads because databases do kind of weird things, right? Like databases will, you know, because of how databases store information, they really kind of almost want raw access to the disk, right, they don't want a file system in the way, right? So even just providing them a file abstraction, they start to get annoyed about, right?  You know, I'm like, imagine if you have your own data structure that you're laying out in a file, right? Well, now that file is gonna be in different parts on disk and what you really wanted is just give me a partition, right? Just give me a chunk of the disk that I can lay out things myself. I know better than you do, right? So this has been a source of kind of research and continued conversation between the database and operating system communities, right? Yeah, there is overlap between those two classes of benchmarks, definitely true. All right, you guys have time to think. Problems with microbenchmarks. Kind of obvious problem with the microbenchmark, right? I've got a microbenchmarked, I've zeroed in on some very, very specific part of the system, I've doubled its performance, and what? Right, so as I think you guys basically said the same thing, which is that I don't know what I did to the rest of the system. Actually, there's two problems with this. We're gonna get to these later. The first is, I don't know what I did to the rest of the system. The second is, who cares? Maybe that part of the system doesn't even matter, right? So yeah, this is my way. You may not be studying the right thing and the thing that you're improving may actually have deleterious effects on other parts of the system. So you may be, your small performance improvement to something that doesn't matter may cause a significant performance degradation to something that does matter. So that's kind of the worst possible case, right? Macrobenchmarks, you're kind of talking about the opposite side of problems, right? So you're at such a high level that it's not clear exactly what fluctuations are caused by and like we talked about page fault rate, right? Where page fault rate can go up, the page fault throughput can go up and down for a variety of reasons. Just, right, what actually happened? Please put me on the spot here. So it's probably gonna be difficult to come up with a convincing example. But I'm trying to think about something in the context of VM, too, because that would be easiest, right? Well, okay, I mean, here's an example, right? It's possible that you, you know, it's possible that you're trying to improve the page eviction, like the speed at which you can find a page to evict, right? So you've identified locating a page to swap to disk as a bottleneck in your system and you're trying to improve that, right? And by doing so, you come up with that, you use a different algorithm that potentially evicts pages in a different order, right? And it turns out that the ordering that you choose is poor, right? And so what happens is you're moving the wrong pages to disk, right? And the overhead now of doing IO back and forth because you're evicting the wrong pages overwhelms the small improvement that you made, right? So there's a case where there's a balance between two components of the system, right? You know, the right thing to do, if I wanted to improve overall performance, might be to write a smarter page eviction algorithm that takes longer, right? So, you know, we talked about this a little bit, we talked about VM, it's possible that the extra, you know, thousand, 10,000 cycles that I spend on the page eviction path, if that prevents me from doing one additional IO, it's worth, right? So yeah, so that's a case.  But if you get too myopic, right? If you get too focused on that one thing and you're not running, you know, higher level benchmarks, you might miss the fact that, oh yeah, by the way, you know, the system is significantly slower now because of the improvement that you made, right? So we'll talk about that in more in a sec, all right? So yeah, so macro benchmarks, there's so much going on that it can be difficult to actually come to real specific conclusions about the results. And so what a lot of analyses do is they start with micro benchmarks. They use micro benchmarks to draw conclusions about some of the changes they've made, and then they use macro benchmarks to kind of prove to you that they haven't, you know, totally destroyed the overall performance of the system as a result, right? And then hopefully they've made some improvements, right? So it's kind of the application, the macro benchmark is I've improved the system. The micro benchmark is to prove to you that the changes that I made to a specific component of the system are actually the source of the improvement and not some other thing, right? Or some other weird interaction between parts of this, right? And then application benchmarks always have this kind of a provinciality to them, right? I mean, that's just your application, right? And who cares, right? So, you know, if I'm, you know, trying to improve web server performance and I make all these changes to the file system and the web server performance goes up, you know, by a factor of two or three, that's great, right? But if, you know, 90, if most of the users that are logged into your system are trying to read email and the performance of the email clients just went down, then, you know, they may not care that the pages are loading somewhat faster from your web server. What they probably care about is that, you know, pine is really slow, right? So this is an important issue, right? All right, and I think, you know, it's worth pointing out a little bit of the psychology of how benchmarks work out in practice too, right? One of the biggest problems with benchmarks, I mean, or anticipate a problem with benchmark, right? You are, you're working on improving the performance of your system and, you know, you've figured out how to measure things, you've chosen a benchmark, you run some tests, the benchmark improves and, you know, you're ready to brag about your results, right? So what's the, you know, and I'm not trying to accuse you of anything, but what is it possible that has happened along this path? What do people do when they select benchmarks or what people, what might people do? Yeah, John. Right, now I mean, no one would ever do that, right? And they don't necessarily have to be like hacks, right? They might, like the problem is you get too wetted to your own benchmark. The first thing is you pick the benchmark, right? So that's kind of like, for example, you know, that would be like me saying, hey guys, how many people would like to have a final where you write the exam, right? And then actually maybe that's what we'll do. Maybe I'll have you guys submit exam questions and then I'll create an exam from the questions you submit it, right? That will give people an incentive to submit questions too because if you submit a question then you should probably know the answer. Yeah, that's just an idea. Anyway, but if I asked you to write, like I think we can do this in a crowdsourced way and actually have it be cool. If I actually asked you to write your own exam, right?  I would hope you guys would do well. But don't write an exam that you can't get 100 on, please. But yeah, so I mean it's kind of like people are, and you know, I mean you have to, people who have good ethics and good scientific practice try to do the right thing, but there's actually, there's a lot of evidence and I wish I'd looked this up. From earlier scientific eras that this is always a problem, right? And it's not the function of just people who are trying to pass off bad results. It's just a function of people who get so embedded in a problem, right? And so determined to improve things that they start to see things in their data and they start to choose experimental methods that are really tailored to what they're doing in ways that render their conclusions very, very difficult to reproduce, yeah. Yeah, of course, right? I mean no, the answer's not hard, right? I mean running a bunch of benchmarks is usually a good idea, right? But again, I mean you'd be surprised at how many papers get published because running benchmarks takes a lot of time and it's also equally frustrating when you run another benchmark and the system doesn't improve.  Maybe I won't use that benchmark in my paper. So, I mean I remember reading just recently actually that it's something like, I don't know, there's this shocking number of studies in psychology and medicine that produce really exciting results the first time they're performed. But every time they're reproduced the results diminish, right? And actually there's people who are studying this in the medical community because they don't know why this is. It's something very strange. But there's all these different claims that people have made that looked fantastic. The first time they were published and that is people have tried to reproduce them. They get harder and harder to reproduce, right? So again, it's not necessarily people with bad intentions. Sometimes it's just people who are, it's just some feature of how this happens, right? All right, so yeah. Again, people choose benchmarks that try to justify the changes they wanna make and also people choose benchmarks and then work on the system to the point where their work becomes coupled to that benchmark, right? Starts to lose a little bit of its more in reality, right? And again, I just wanna point out that this isn't just like human weakness, right? So what can we say? I mean, this is interesting because if we get to virtualization, which I hope we will, we'll talk a little bit about virtualization as a solution to this problem, right? But operating systems have this tough job, right? They're trying to support a lot of different applications, right? So what is, if you're an operating system designer, what's one way to make, and you've got a client, let's say you're, I don't know, you're Microsoft or something, and your big client is Oracle, right? And Oracle's databases, right? Like they are running a specific application on your system and that's what they care about, right? So what is one way to make them happy, right? What is one way to make your operating system better from the perspective of Oracle? To what? Right, so okay, so my maxim here is that the fastest system is a single purpose system, right? And to the degree that people continue to work on single applications, again, like the best way to, or one way to improve performance of the system is just to tailor it more and more and more to a specific application, right? Unfortunately, at least for a large class of what operating systems are still used for. For a lot of cases, this is okay now, right? Because a lot of times, operating systems are running environments where they only support one application, right? That's kind of an interesting modern twist on the general purpose operating system. But for the types of things you guys do, right? I mean, you probably, you might, you may or may not get really upset if the next Android phone that you got was incredibly tuned for Angry Birds, right? And that slowed down everything else on the system. Like mail was slow, the web was slow, Angry Birds was super fast, right? So, or you might get equally upset the web was really fast and Angry Birds was slow, right? Whatever, I don't, Angry Birds is kind of my catch-all thing to pound out. I don't really use Angry Birds or play it. All right, so, yeah, so we're going even slower than I want to, which is fine. So I think that, so let's talk a little bit, you know, we've spent a lot of time for the first two lectures on performance talking about what not to do, right? What not to do? You know, all these mistakes that you can make. So I want to sort of finish today by talking more about what to do, right? What are good techniques to follow, right? So the first thing that's kind of critical is having a goal, right? Having a specific goal, right? I mean, usually, that's the problem with performance. Even vaguely defined performance seems like something that's worth doing, right? I'm just gonna make my system faster, right? That actually seems like a worthwhile goal, right? You don't have to be that much more specific. If you actually want to make your system faster, however, it is really helpful to be more specific, right? So it's helpful to say, you know, I want to improve this particular piece of code. And I know we're coming to it very slowly, but we are gonna talk about how to choose what to improve, right? But the more specific you can make your goal, the more likely it is that you can actually improve things, right? And improve things for real, right? Not just happen to run the benchmark on a set of warm caches and have it look faster and then go home, right? And the reason is, you know, the more specific you can make your goal, the more likely it is that you can identify changes that you're making to the system, right? If it's just faster, right? And you make some changes to the code, then the change, you know, I mean, here's one way of thinking about it, right? Any, you know, any changes you make to your code base are likely to affect the performance of the system. Unfortunately, it's like flipping a coin, right? It might get faster or it might get slower, right? You know, absent of any sort of analytic techniques. So one way of improving performance without paying any attention to this is simply to make a change, run the system. If it's faster, commit that change, right? If it's not faster, revert that change and try making another change, right? And in theory, you could do this for long periods of time, right? I don't think that that would work very well over a long horizon, but if anyone wants to try that as a research project, I would love to have somebody experiment with that. It'd be kind of interesting.  Make change to code, commit. If code works, evaluate performance. If code is faster, you know, continue. So if you don't want to do that, then being specific helps, right? So when you're using models and simulators, and you guys will probably use models and simulators, validating them before you start is really important, right? And especially with simulators, this can be really frustrating because when you write a simulator, you spend time writing the simulator, you know? Like, you wanted to improve the performance of the system and now you ended up writing code, extra code, that you didn't want to write in the first place, that's never going to ship with the product, that's not a part of the system that no one is ever going to know about.  And you'll be like, of course I did, I wrote it. Right, you know? Like, I understand the code, of course it works, right? So before you make changes, making sure that the simulated behavior matches the real behavior of the system is really critical. It's not something that people do very often, but it's really important, right? Because otherwise, you've created a completely different thing, right? And changes that you make that improve the performance of the simulator, what evidence do you have that they're going to improve the performance of the real system, right? If the simulated performance doesn't match up before you start making changes, then the simulator is really not that helpful, right? And you can do the same thing with models, right? And particularly, I mean, simple models, you know, should produce results that match your intuition, right? Okay, right? So, you know, another important thing to do is to pick the right techniques, right? And there's no real, I'm not sure there's a science to this, right? I mean, this is kind of like part of the art of performance improvement, you know? When do you use modeling? When do you come up with analytic techniques? When do you write a simulator? When do you try to do experiments on real code? When do you get a benchmark, right? And use that? When do you use the workload that the customer was running, right? I mean, these are the things that are, you know, over time, as systems engineers and as programmers, you guys will start to get a feeling for, right? But you should at least know that these techniques are out there and think about them when you're approaching a problem, right? Think about it. You know, should I model this technique, right? Is this system one that I can model, right? Maybe that's a powerful approach, if you can do it, right? Does a simulator already exist for this system? Can I use it to evaluate this technique? And, you know, I mean, and on some level, I'm not totally sure that I believe what I wrote in my own slides here, but, you know, but there's some, you know, there might be some hierarchy of these things, right? For example, if you can't convince yourself analytically that a new algorithm is gonna improve the performance of your system, then don't bother simulating it, right? And if it doesn't work in the simulator, don't implement it on the real system, right? I mean, I think sometimes people, including myself, get so wedded to an idea that they are determined to implement it on a real system and try it out because maybe it'll work, right? But, you know, what people have found out over time is that, you know, things that don't, things that don't work repeatedly usually don't work in your real system either, right? So it's just wasted effort. All right, so I think this is a good stopping point. Does anybody have any questions about this stuff? I'm going more slowly than I thought I would, but I'm going as fast as I want to, so. Are people bored by this? Anybody? It's okay, I'm not, and we'll take it personally. Am I going too slow? I feel like I'm going too slow. Anybody want to claim I'm going too slow? Other than me. Okay, what's that?  Okay, that's cool, that's cool, I'll speed up on Monday. All right, so Monday, Monday we will get through the remainder of this material and we will talk about a few hints of ways to make your system faster, all right? Good luck with assignment three and I will see you on Monday. Have a great weekend.