The idea of a learning machine may appear paradoxical to some readers. How can the rules of operation of the machine change? They should describe completely how the machine will react, whatever its history might be, whatever changes it might undergo. The rules are quite time-invariant. This is quite true. The explanation of the paradox is that the rules which get changed in the learning process are of a rather less pretentious kind, claiming only an ephermal validity. The reader may draw a parallel with the Constitution of the United States. — “Computing Machinery and Intelligence,” Alan Turing (1950)

Table of Contents


Wikipedia entry


The paradox on which I will focus arises from the question whether the clause of a constitution that authorizes amendments may authorize its own amendment or repeal. May a rule that permits the change of other rules also permit its own change, especially its irrevocable change nnnninto a form inconsistent with its original form? This paradox does not have a strict counterpart in logic, for it pertains to changing the rules of the system by means of a rule within the system. In logic rules are traditionally thought to be either immutable and eternal, or arbitrary postulates changeable by logicians but not by the postulates’ own authority. (http://www.earlham.edu/~peters/writing/psa/)

Unsupervised learning

  • No feedback about true labels; only (often imperfect) measurements
  • Often group entities by their similarities to each other
  • Not easy to evaluate performance (since we don’t have any true labels)

Supervised learning

  • A set of data is labeled with true labels; this is the “training” set
  • Learning is based on measurements and true labels; how do certain measures provide evidence for certain labels?
  • Easy to evaluate performance: % of correctly-labeled cases on a “test” set (which is distinct from the “training” set; possibly, the original training set is split, with one small subset saved for testing purposes)

