Book review: Value-Focused Thinking: A Path to Creative Decisionmaking, by Ralph L. Keeney.

This book argues for focusing on values (goals/objectives) when making decisions, as opposed to the more usual alternative-focused decisionmaking.

The basic idea seems good. Alternative-focused thinking draws our attention away from our values and discourages us from creatively generating new possibilities to choose from. It tends to have us frame decisions as responses to problems, which leads us to associate decisions with undesirable emotions, when we could view decisions as opportunities.

A good deal of the book describes examples of good decisionmaking, but those rarely provide insight into how to avoid common mistakes or to do unusually well.

Occasionally the book switches to some dull math, without clear explanations of what benefit the rigor provides.

The book also includes good descriptions of how to measure the things that matter, but How to Measure Anything by Douglas Hubbard does that much better.

I recently got Bose QuietComfort 15 Acoustic Noise Cancelling Headphones.

I had previously tried passive earplugs and headphones that claimed 30 dB noise reduction, and got little value out of them.

The noise cancelling headphones suppress a good deal more train (BART) noise, enough that I’m now able to read nonfiction while on the train.

It won’t help with the situations where noise bothers me most (multiple conversations nearby) because it mainly eliminates predictable noises. It makes speech sound more distant without affecting the speech volume a lot. But reducing the cost of train and plane travel is valuable enough that I feel foolish about not having tried them earlier.










Book review: The Depths: The Evolutionary Origins of the Depression Epidemic, by Johnathan Rottenberg.

This book presents a clear explanation of why the basic outlines of depression look like an evolutionary adaptation to problems such as famine or humiliation. But he ignores many features that still puzzle me. Evolution seems unlikely to select for suicide. Why does loss of a child cause depression rather than some higher-energy negative emotion? What influences the breadth of learned helplessness?

He claims depression has been increasing over the last generation or so, but the evidence he presents can easily be explained by increased willingness to admit to and diagnose depression. He has at least one idea why it’s increasing (increased pressure to be happy), but I can come up with ideas that have the opposite effect (e.g. increased ease of finding a group where one can fit in).

Much of the book has little to do with the origins of depression, and is dominated by descriptions of and anecdotes about how depression works.

He spends a fair amount of time talking about the frequently overlooked late stages of depression recovery, where antidepressants aren’t much use and people can easily fall back into depression.

The book includes a bit of self-help advice to use positive psychology, and to not rely on drugs for much more than an initial nudge in the right direction.


I recently tried larger-than-normal methylfolate supplements. I had known that my genes caused problems with processing folate (MTHFR T/T), but hadn’t noticed any effects from supplementing at 800mcg/day.

Due to a report that several milligrams/day helped with depression, I changed from 400mcg/day to 2mg/day. I felt like my mind started working better within hours (although I haven’t seen much change in behavior). I saw a clear and large improvement in my heart rate variability starting after one day.

I’ve experimented for 5 weeks randomly altering my dose from 1 to 3 mg/day. On the week when I switched from 3 to 1 mg, my mood slowly got worse. I felt much better withing hours of switching back to 3 mg. I haven’t noticed any clear difference between 2 and 3mg.

Book review: Superintelligence: Paths, Dangers, Strategies, by Nick Bostrom.

This book is substantially more thoughtful than previous books on AGI risk, and substantially better organized than the previous thoughtful writings on the subject.

Bostrom’s discussion of AGI takeoff speed is disappointingly philosophical. Many sources (most recently CFAR) have told me to rely on the outside view to forecast how long something will take. We’ve got lots of weak evidence about the nature of intelligence, how it evolved, and about how various kinds of software improve, providing data for an outside view. Bostrom assigns a vague but implausibly high probability to AI going from human-equivalent to more powerful than humanity as a whole in days, with little thought of this kind of empirical check.

I’ll discuss this more in a separate post which is more about the general AI foom debate than about this book.

Bostrom’s discussion of how takeoff speed influences the chance of a winner-take-all scenario makes it clear that disagreements over takeoff speed are pretty much the only cause of my disagreement with him over the likelihood of a winner-take-all outcome. Other writers aren’t this clear about this. I suspect those who assign substantial probability to a winner-take-all outcome if takeoff is slow will wish he’d analyzed this in more detail.

I’m less optimistic than Bostrom about monitoring AGI progress. He says “it would not be too difficult to identify most capable individuals with a long-standing interest in [AGI] research”. AGI might require enough expertise for that to be true, but if AGI surprises me by only needing modest new insights, I’m concerned by the precedent of Tim Berners-Lee creating a global hypertext system while barely being noticed by the “leading” researchers in that field. Also, the large number of people who mistakenly think they’ve been making progress on AGI may obscure the competent ones.

He seems confused about the long-term trends in AI researcher beliefs about the risks: “The pioneers of artificial intelligence … mostly did not contemplate the possibility of greater-than-human AI” seems implausible; it’s much more likely they expected it but were either overconfident about it producing good results or fatalistic about preventing bad results (“If we’re lucky, they might decide to keep us as pets” – Marvin Minsky, LIFE Nov 20, 1970).

The best parts of the book clarify many issues related to ensuring that an AGI does what we want.

He catalogs more approaches to controlling AGI than I had previously considered, including tripwires, oracles, and genies, and clearly explains many limits to what they can accomplish.

He briefly mentions the risk that the operator of an oracle AI would misuse it for her personal advantage. Why should we have less concern about the designers of other types of AGI giving them goals that favor the designers?

If an oracle AI can’t produce a result that humans can analyze well enough to decide (without trusting the AI) that it’s safe, why would we expect other approaches (e.g. humans writing the equivalent seed AI directly) to be more feasible?

He covers a wide range of ways we can imagine handling AI goals, including strange ideas such as telling an AGI to use the motivations of superintelligences created by other civilizations

He does a very good job of discussing what values we should and shouldn’t install in an AGI: the best decision theory plus a “do what I mean” dynamic, but not a complete morality.

I’m somewhat concerned by his use of “final goal” without careful explanation. People who anthropomorphise goals are likely to confuse at least the first few references to “final goal” as if it worked like a human goal, i.e. something that the AI might want to modify if it conflicted with other goals.

It’s not clear how much of these chapters depend on a winner-take-all scenario. I get the impression that Bostrom doubts we can do much about the risks associated with scenarios where multiple AGIs become superhuman. This seems strange to me. I want people who write about AGI risks to devote more attention to whether we can influence whether multiple AGIs become a singleton, and how they treat lesser intelligences. Designing AGI to reflect values we want seems almost as desirable in scenarios with multiple AGIs as in the winner-take-all scenario (I’m unsure what Bostrom thinks about that). In a world with many AGIs with unfriendly values, what can humans do to bargain for a habitable niche?

He has a chapter on worlds dominated by whole brain emulations (WBE), probably inspired by Robin Hanson’s writings but with more focus on evaluating risks than on predicting the most probable outcomes. Since it looks like we should still expect an em-dominated world to be replaced at some point by AGI(s) that are designed more cleanly and able to self-improve faster, this isn’t really an alternative to the scenarios discussed in the rest of the book.

He treats starting with “familiar and human-like motivations” (in an augmentation route) as an advantage. Judging from our experience with humans who take over large countries, a human-derived intelligence that conquered the world wouldn’t be safe or friendly, although it would be closer to my goals than a smiley-face maximizer. The main advantage I see in a human-derived superintelligence would be a lower risk of it self-improving fast enough for the frontrunner advantage to be large. But that also means it’s more likely to be eclipsed by a design more amenable to self-improvement.

I’m suspicious of the implication (figure 13) that the risks of WBE will be comparable to AGI risks.

  • Is that mainly due to “neuromorphic AI” risks? Bostrom’s description of neuromorphic AI is vague, but my intuition is that human intelligence isn’t flexible enough to easily get the intelligence part of WBE without getting something moderately close to human behavior.
  • Is the risk of uploaded chimp(s) important? I have some concerns there, but Bostrom doesn’t mention it.
  • How about the risks of competitive pressures driving out human traits (discussed more fully/verbosely at Slate Star Codex)? If WBE and AGI happen close enough together in time that we can plausibly influence which comes first, I don’t expect the time between the two to be long enough for that competition to have large effects.
  • The risk that many humans won’t have enough resources to survive? That’s scary, but wouldn’t cause the astronomical waste of extinction.

Also, I don’t accept his assertion that AGI before WBE eliminates the risks of WBE. Some scenarios with multiple independently designed AGIs forming a weakly coordinated singleton (which I consider more likely than Bostrom does) appear to leave the last two risks in that list unresolved.

This books represents progress toward clear thinking about AGI risks, but much more work still needs to be done.


I finally found a way to buy insects in enough quantity to satisfy my desire for nutrition from insects: World Ento, which sells dried crickets and cricket flour at a price per gram of protein comparable to seafood. (H/T Holden Karnofsky.)

According to Organic Value Recovery Solutions, crickets have impressive amounts of the nutrients I’ve found the hardest to get good amounts of. Here are examples of how much I’d get if I got my 2000 calories a day from crickets:

  • B12: 20 times the RDA (more than 3 times that of eggs)
  • Folate: 5 times the RDA (3 times that of eggs)
  • Zinc: 9 times the RDA (5 times that of eggs)

(using data for eggs from pastured chickens).

They have plenty of fiber and good amounts of most minerals and B vitamins.

The cricket flour tastes ok in brownies, but I’ll want some other recipe for regular use.

Update 2015-01-05: ThailandUnique has a better selection of insects. My favorite so far is the Big Cricket.

Book review: The Hidden Reality: Parallel Universes and the Deep Laws of the Cosmos, by Brian Greene.

This book has a lot of overlap with Tegmark’s Our Mathematical Universe

Greene uses less provocative language than Tegmark, but makes up for that by suggesting 5 more multiverses than Tegmark (3 of which depend on string theory for credibility, and 2 that Tegmark probably wouldn’t label as multiverses).

I thought about making some snide remarks about string theory being less real than the other multiverses. Then I noticed that what Greene calls the ultimate multiverse (all possible universes) implies that string theory universes (or at least computable approximations) are real regardless of whether we live in one.

Like Tegmark, Greene convinces me that inflation which lasts for infinite time implies infinite space and infinite copies of earth, but fails to convince me that he has a strong reason for assuming infinite time.

The main text is mostly easy to read. Don’t overlook the more technical notes at the end – the one proposing an experiment that would distinguish the Many Worlds interpretation of quantum mechanics from the Copenhagen interpretation is one of the best parts of the book.


This book does a good job of explaining how barbaric practices such as feuds and honor killings are integral parts of clan-based systems of dispute resolution, and can’t safely be suppressed without first developing something like the modern rule of law to remove the motives that perpetuate them.

He has a coherent theory of why societies with no effective courts and police need to have kin-based groups be accountable for the actions of their members, which precludes some of the individual rights that we take for granted.

He does a poor job of explaining how this is relevant to modern government. He writes as if anyone who wants governments to exert less power wants to weaken the rule of law and the ability of governments to stop violent disputes. (His comments about modern government are separate enough to not detract much from the rest of the book).

He implies that modern rule of law and rule by clans are the only stable possibilities. He convinced me that it would be hard to create good alternatives to those two options, but not that alternatives are impossible.

To better understand how modern individualism replaced clan-based society, read Fukuyama’s The Origins of Political Order together with this book.