Tag Archives: method

Successive over-relaxation

Successive over-relaxation

From Wikipedia, the free encyclopedia
In numerical linear algebra, the method of successive over-relaxation (SOR) is a variant of the Gauss–Seidel method for solving alinear system of equations, resulting in faster convergence. A similar method can be used for any slowly converging iterative process. It was devised simultaneously by David M. Young and by H. Frankel in 1950 for the purpose of automatically solving linear systems on digital computers. Over-relaxation methods had been used before the work of Young and Frankel. For instance, the method of Lewis Fry Richardson, and the methods developed by R. V. Southwell. However, these methods were designed for computation by human calculators, and they required some expertise to ensure convergence to the solution which made them inapplicable for programming on digital computers. These aspects are discussed in the thesis of David M. Young.[1]

The candle problem

Uploaded on Aug 25, 2009


The candle problem or candle task, also known as Duncker’s candle problem, is a cognitive performance test, measuring the influence of functional fixedness on a participant’s problem solving capabilities. The test was created [1] by Gestalt psychologist Karl Duncker and published posthumously in 1945. Duncker originally presented this test in his thesis on problem solving tasks at Clark University.

The test presents the participant with the following task: how to fix a lit candle on a wall (a cork board) in a way so the candle wax won’t drip onto the table below.[3] To do so, one may only use the following along with the candle:

  • a book of matches
  • a box of thumbtacks

The solution is to empty the box of thumbtacks, put the candle into the box, use the thumbtacks to nail the box (with the candle in it) to the wall, and light the candle with the match.[3] The concept of functional fixedness predicts that the participant will only see the box as a device to hold the thumbtacks and not immediately perceive it as a separate and functional component available to be used in solving the task.


Many of the people who attempted the test explored other creative, but less efficient, methods to achieve the goal. For example, some tried to tack the candle to the wall without using the thumbtack box,[4] and others attempted to melt some of the candle’s wax and use it as an adhesive to stick the candle to the wall.[1] Neither method works.[1] However, if the task is presented with the tacks piled next to the box (rather than inside it), virtually all of the participants were shown to achieve the optimal solution, which is self defined.[4]

The test has been given to numerous people, including M.B.A. students at the Kellogg School of Management in a study investigating whether living abroad and creativity are linked.[5]


Glucksberg (1962)[6] used a 2 × 2 design manipulating whether the tacks and matches were inside or outside of their boxes and whether subjects were offered cash prizes for completing the task quickly. Subjects who were offered no prize, termed low-drive, were told “We are doing pilot work on various problems in order to decide which will be the best ones to use in an experiment we plan to do later. We would like to obtain norms on the time needed to solve.” The remaining subjects, termed high-drive, were told “Depending on how quickly you solve the problem you can win $5.00 or $20.00. The top 25% of the Ss [subjects] in your group will win $5.00 each; the best will receive $20.00. Time to solve will be the criterion used.” (As a note, adjusting for inflation since 1962, the study’s publish year, the amounts in today’s dollars would be approximately $39 and $154, respectively.[7]) The empty-boxes condition was found to be easier than the filled-boxes condition: more subjects solved the problem, and those who did solve the problem solved it faster. Within the filled-boxes condition, high-drive subjects performed worse than low-drive subjects. Glucksberg interpreted this result in terms of “neobehavioristic drive theory”: “high drive prolongs extinction of the dominant habit and thus retards the correct habit from gaining ascendancy”. An explanation in terms of the overjustification effect is made difficult by the lack of a main effect for drive and by a nonsignificant trend in the opposite direction within the empty-boxes condition.

Another way to explain the higher levels of failure during the high-drive condition is that the process of turning the task into a competition for limited resources can create mild levels of stress in the subject, which can lead to the Sympathetic nervous system, otherwise known as the Fight-or-flight response, taking over the brain and body. This stress response effectively shuts down the creative thinking and problem solving areas of the brain in the prefrontal cortex.

Linguistic implications

E. Tory Higgins and W. M. Chaires found that having subjects repeat the names of common pairs of objects in this test, but in a different and unaccustomed linguistic structure, such as “box and tacks” instead of “box of tacks”, facilitated performance on the candle problem.[3] This phrasing helps one to distinguish the two entities as different and more accessible.[3]

In a written version of the task given to people at Stanford University, Michael C. Frank and language acquisition researcher Michael Ramscar reported that simply underlining certain relevant materials (“on the table there is a candle, a box of tacks, and a book of matches…”) increases the number of candle-problem solvers from 25% to 50%.[4]


  1. ^ Jump up to: a b c “Dan Pink on the surprising science of motivation”. Retrieved 16 January 2010.
  2. Jump up ^ Daniel Biella and Wolfram Luther. “A Synthesis Model for the Replication of Historical Experiments in Virtual Environments”. 5th European Conference on e-Learning. Academic Conferences Limited. p. 23. ISBN 978-1-905305-30-8.
  3. ^ Jump up to: a b c d Richard E. Snow and Marshall J. Farr, ed. (1987). “Positive Affect and Organization”. Aptitude, Learning, and Instruction Volume 3: Conative and Affective Process Analysis. Routledge. ISBN 978-0-89859-721-9.
  4. ^ Jump up to: a b c Frank, Michael. “Against Informational Atomism”. Retrieved 15 January 2010.
  5. Jump up ^ “Living Outside the Box: Living abroad boosts creativity”. April 2009. Retrieved 16 January 2010.
  6. Jump up ^ Glucksberg, S. (1962). “The influence of strength of drive on functional fixedness and perceptual recognition”. Journal of Experimental Psychology 63: 36–41. doi:10.1037/h0044683. PMID 13899303. edit
  7. Jump up ^ Inflated values automatically calculated.

Cognitive bias

The notion of cognitive biases was introduced by Amos Tversky and Daniel Kahneman in 1972 and grew out of their experience of people’s innumeracy, or inability to reason intuitively with the greater orders of magnitude. They and their colleagues demonstrated several replicable ways in which human judgments and decisions differ from rational choice theory. They explained these differences interms of heuristics; rules which are simple for the brain to compute but introduce systematic errors. For instance the availability heuristic, when the ease with which something comes to mindis used toindicate how often (or how recently)it has been encountered.

Cognitive bias is any of a wide range of observer effects identified in cognitive science, including very basic statistical and memory errors that are common to all human beings (many of which have been discussed by Amos Tversky and Daniel Kahneman) and drastically skew the reliability of anecdotal and legal evidence. They also significantly affect the scientific method which is deliberately designed to minimize such bias from any one observer.

cognitive bias is a pattern of deviation in judgment that occurs in particular situations, which may sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, or what is broadly called irrationality.[1][2][3] Implicit in the concept of a “pattern of deviation” is a standard of comparison with what is normatively expected; this may be the judgment of people outside those particular situations, or may be a set of independently verifiable facts. A continually evolving list of cognitive biases has been identified over the last six decades of research on human judgment and decision-making in cognitive sciencesocial psychology, and behavioral economics.

Some cognitive biases are presumably adaptive, for example, because they lead to more effective actions in a given context or enable faster decisions when timeliness is more valuable than accuracy (heuristics). Others presumably result from a lack of appropriate mental mechanisms (bounded rationality), or simply from a limited capacity for information processing.


Some biases affect decision-making, where the desirability of options has to be considered (e.g., Sunk Cost fallacy). Others such as Illusory correlation affect judgment of how likely something is, or of whether one thing is the cause of another. A distinctive class of biases affect memory,[12] such as consistency bias (remembering one’s past attitudes and behavior as more similar to one’s present attitudes).

Some biases reflect a subject’s motivation,[13] for example, the desire for a positive self-image leading to Egocentric bias[14] and the avoidance of unpleasant cognitive dissonance. Other biases are due to the particular way the brain perceives, forms memories and makes judgments. This distinction is sometimes described as “Hot cognition” versus “Cold Cognition”, as motivated reasoning can involve a state of arousal.

Among the “cold” biases, some are due to ignoring relevant information (e.g. Neglect of probability), whereas some involve a decision or judgement being affected by irrelevant information (for example the Framing effect where the same problem receives different responses depending on how it is described) or giving excessive weight to an unimportant but salient feature of the problem (e.g., Anchoring).

The fact that some biases reflect motivation, and in particular the motivation to have positive attitudes to oneself[14] accounts for the fact that many biases are self-serving or self-directed (e.g.Illusion of asymmetric insightSelf-serving biasProjection bias). There are also biases in how subjects evaluate in-groups or out-groups; evaluating in-groups as more diverse and “better” in many respects, even when those groups are arbitrarily-defined (Ingroup biasOutgroup homogeneity bias).



For other noted biases, see List of cognitive biases.

A 2012 Psychological Bulletin article suggests that at least 8 seemingly unrelated biases can be produced by the same information-theoretic generative mechanism.[17] It is shown that noisy deviations in the memory-based information processes that convert objective evidence (observations) into subjective estimates (decisions) can produce regressive conservatism, the conservatism (Bayesian)illusory correlationsbetter-than-average effect and worse-than-average effectsubadditivity effectexaggerated expectationoverconfidence, and the hard–easy effect.

However, as recent research has demonstrated,, even scientists who adhere to the scientific method can’t guarantee they will draw the best possible conclusions. “How could such highly-educated and precisely-trained professionals veer off the path of objectivity?” The answer is simple: Being human.

As the fields of psychology and behavioral economics have demonstrated, homo sapiens is a seemingly irrational species that appears to, more often than not, think and behave in nonsensical rather than commonsensical ways. The reason is that we fall victim to a veritable laundry list of cognitive biases that cause us to engage in distorted, imprecise and incomplete thinking which, not surprisingly, results in “perceptual distortion, inaccurate judgment or illogical interpretation” (thanks Wikipedia), and, by extension, poor and sometimes catastrophic decisions.



The biases that helped us survive in primitive times when life was much simpler (e.g., life goal: live through the day) and speed of a decision rightfully trumped its absolute accuracy doesn’t appear to be quite as adaptive in today’s much more complex world. Due to the complicated nature of life these days, correctness of information, thoroughness of processing, precision of interpretation and soundness of judgment are, in most situations today, far more important than the simplest and fastest route to a judgment.



  • The bandwagon effect (aka herd mentality) describes the tendency to think or act in ways because other people do. Examples include the popularity of Apple products, use of “in-group” slang and clothing style and watching the “The Real Housewives of … ” reality-TV franchise.
  • The confirmation bias involves the inclination to seek out information that supports our own preconceived notions. The reality is that most people don’t like to be wrong, so they surround themselves with people and information that confirm their beliefs. The most obvious example these days is the tendency to follow news outlets that reinforce our political beliefs.
  • Illusion of control is the propensity to believe that we have more control over a situation than we actually do. If we don’t actually have control, we fool ourselves into thinking we do. Examples include rally caps in sports and “lucky” items.
  • The Semmelweis reflex (just had to include this one because of its name) is the predisposition to deny new information that challenges our established views. Sort of the yang to the yin of the confirmation bias, it exemplifies the adage “if the facts don’t fit the theory, throw out the facts.” An example is the “Seinfeld” episode in which George Costanza’s girlfriend simply refuses to allow him to break up with her.
  • The causation bias suggests the tendency to assume a cause-effect relationship in situations in which none exists (or there is a correlation or association). An example is believing someone is angry with you because they haven’t responded to your email when, more likely, they are busy and just haven’t gotten to it yet.
  • The overconfidence effect involves unwarranted confidence in one’s own knowledge. Examples include political and sports prognosticators.
  • Finally, the granddaddy of all cognitive biases, the fundamental attribution error, which involves the tendency to attribute other people’s behavior to their personalities and to attribute our own behavior to the situation. An example is when someone treats you poorly, you probably assume they are a jerk, but when you’re not nice to someone, it’s because you are having a bad day.

Memory bias — Memory biases may either enhance or impair the recall of memory, or they may alter the content of what we report remembering. There are many memory …  > read more

Anchoring bias in decision-making — Anchoring or focalism is a term used in psychology to describe the common human tendency to rely too heavily, or “anchor,” on one trait or piece of …  > read more


  • Bandwagon effect — the tendency to do (or believe) things because many other people do (or believe) the same. Related to groupthinkcrowd psychologyherd behaviour, and manias.
  • Bias blind spot — the tendency not to compensate for one’s own cognitive biases.
  • Choice-supportive bias — the tendency to remember one’s choices as better than they actually were.
  • Endowment effect — “the fact that people often demand much more to give up an object than they would be willing to pay to acquire it”.[2]
  • Exposure-suspicion bias – a knowledge of a subject’s disease in a medical study may influence the search for causes.
  • Framing – drawing different conclusions from the same information, depending on how that information is presented.
  • Loss aversion — “the disutility of giving up an object is greater than the utility associated with acquiring it”.[3] (see also sunk cost effects and Endowment effect).
  • Obsequiousness bias – the tendency to systematically alter responses in the direction they perceive desired by the investigator.
  • Planning fallacy — the tendency to underestimate task-completion times. Also formulated as Hofstadter’s Law: “It always takes longer than you expect, even when you take into account Hofstadter’s Law.”
  • Selective perception — the tendency for expectations to affect perception.
  • Unacceptability bias – questions that may embarrass or invade privacy are refused or evaded.
  • Von Restorff effect — the tendency for an item that “stands out like a sore thumb” to be more likely to be remembered than other items.


  • Anthropic bias — the tendency for one’s evidence to be biased by observation selection effects.
  • Gambler’s fallacy — the tendency to assume that individual random events are influenced by previous random events. For example, “I’ve flipped heads with this coin five times consecutively, so the chance of tails coming out on the sixth flip is much greater than heads.”
  • Hindsight bias — sometimes called the “I-knew-it-all-along” effect: the inclination to see past events as being predictable, based on knowledge of later events.
  • Optimism bias — the systematic tendency to be over-optimistic about the outcome of planned actions. Found to be linked to the “left inferior frontal gyrus” section of the brain, and disrupting this section of the brain removes the bias. Article summarising this finding
  • Overconfidence effect — the tendency to overestimate one’s own abilities.
  • Recency effect — the tendency to weigh recent events more than earlier events (see also ‘peak-end rule’).

Most of these biases are labeled as attributional biases.

  • Actor-observer bias — the tendency for explanations for other individual’s behaviors to overemphasize the influence of their personality and underemphasize the influence of their situation. This is coupled with the opposite tendency for the self in that one’s explanations for their own behaviors overemphasize their situation and underemphasize the influence of their personality. (see also fundamental attribution error).
  • Dunning-Kruger effect — “…when people are incompetent in the strategies they adopt to achieve success and satisfaction, they suffer a dual burden: Not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it. Instead, …they are left with the mistaken impression that they are doing just fine.”[5] (See also the Lake Wobegon effect, and overconfidence effect).
  • Herd instinct – a common tendency to adopt the opinions and follow the behaviors of the majority to feel safer and to avoid conflict.
  • Illusion of transparency — people overestimate others’ ability to know them, and they also overestimate their ability to know others.
  • Just-world phenomenon — the tendency for people to believe that the world is “just” and therefore people “get what they deserve.”
  • Beneffectance – perceiving oneself as responsible for desirable outcomes but not responsible for undesirable ones. (Term coined by Greenwald (1980))
  • Consistency bias– incorrectly remembering one’s past attitudes and behaviour as resembling present attitudes and behaviour.
  • Egocentric bias – recalling the past in a self-serving manner, e.g. remembering one’s exam grades as being better than they were, or remembering a caught fish as being bigger than it was
  • Confabulation or false memory – Remembering something that never actually happened.
  • Hindsight bias – filtering memory of past events through present knowledge, so that those events look more predictable than they actually were; also known as the ‘I-knew-it-all-along effect’.
  • Selective Memory and selective reporting


Baloney Detection Kit


  1. How reliable is the source of the claim?
  2. Does the source make similar claims?
  3. Have the claims been verified by somebody else?
  4. Does this fit with the way the world works?
  5. Has anyone tried to disprove the claim?
  6. Where does the preponderance of evidence point?
  7. Is the claimant playing by the rules of science?
  8. Is the claimant providing positive evidence?
  10. Are personal beliefs driving the claim?


This is the first video by RDFTV.
Presented by The Richard Dawkins Foundation for Reason and Science
Directed by Josh Timonen
Produced by Maureen Norton
Animation by Pew 36 Animation Studios
Music by Neal Acree
Post Production Sound by Sound Satisfaction
Supervising Sound Editor/Re-Recording Mixer: Gary J. Coppola, C.A.S.
Sound Editor: Ben Rauscher
Production Assistant: Graham Immel
Copyright © 2009 Upper Branch Productions, Inc.

Baloney Detection

How to draw boundaries between science and pseudoscience,

By Michael Shermer


pseudoscience–I suggest 10 questions to ask when encountering any claim.

1. How reliable is the source of the claim?


2. Does this source often make similar claims?

Pseudoscientists have a habit of going well beyond the facts. Flood geologists (creationists who believe that Noah’s flood can account for many of the earth’s geologic formations) consistently make outrageous claims that bear no relation to geological science. Of course, some great thinkers do frequently go beyond the data in their creative speculations.


3. Have the claims been verified by another source?






Power iteration

Power iteration

From Wikipedia, the free encyclopedia
In mathematics, the power iteration is an eigenvalue algorithm: given a matrix A, the algorithm will produce a number λ (theeigenvalue) and a nonzero vector v (the eigenvector), such that Av = λv.
The power iteration is a very simple algorithm. It does not compute a matrix decomposition, and hence it can be used when A is a very large sparse matrix. However, it will find only one eigenvalue (the one with the greatest absolute value) and it may converge only slowly.

Arnoldi iteration

Arnoldi iteration

From Wikipedia, the free encyclopedia
In numerical linear algebra, the Arnoldi iteration is an eigenvalue algorithm and an important example of iterative methods. Arnoldi finds the eigenvalues of general (possibly non-Hermitianmatrices; an analogous method for Hermitian matrices is the Lanczos iteration. The Arnoldi iteration was invented by W. E. Arnoldi in 1951.
The term iterative method, used to describe Arnoldi, can perhaps be somewhat confusing. Note that all general eigenvalue algorithms must be iterative. This is not what is referred to when we say Arnoldi is an iterative method. Rather, Arnoldi belongs to a class of linear algebra algorithms (based on the idea of Krylov subspaces) that give a partial result after a relatively small number of iterations. This is in contrast to so-called direct methods, which must complete to give any useful results.
Arnoldi iteration is a typical large sparse matrix algorithm: It does not access the elements of the matrix directly, but rather makes the matrix map vectors and makes its conclusions from their images. This is the motivation for building the Krylov subspace.

Krylov subspace

Krylov subspace

From Wikipedia, the free encyclopedia
In linear algebra, the order-r Krylov subspace generated by an n-by-n matrix A and a vector b of dimension n is the linear subspacespanned by the images of b under the first r powers of A (starting from A0 = I), that is,
mathcal{K}_r(A,b) = operatorname{span} , { b, Ab, A^2b, ldots, A^{r-1}b }. ,
It is named after Russian applied mathematician and naval engineer Alexei Krylov, who published a paper on this issue in 1931.[1]
Modern iterative methods for finding one (or a few) eigenvalues of large sparse matrices or solving large systems of linear equations avoid matrix-matrix operations, but rather multiply vectors by the matrix and work with the resulting vectors. Starting with a vector, b, one computes Ab, then one multiplies that vector by A to find A2b and so on. All algorithms that work this way are referred to as Krylov subspace methods; they are among the most successful methods currently available in numerical linear algebra.
Because the vectors tend very quickly to become almost linearly dependent, methods relying on Krylov subspace frequently involve some orthogonalization scheme, such as Lanczos iteration for Hermitian matrices or Arnoldi iteration for more general matrices.
The best known Krylov subspace methods are the ArnoldiLanczosConjugate gradientGMRES (generalized minimum residual),BiCGSTAB (biconjugate gradient stabilized), QMR (quasi minimal residual), TFQMR (transpose-free QMR), and MINRES (minimal residual) methods.


  1. ^ Mike Botchev (2002). “A.N.Krylov, a short biography”.