Sunday, November 26, 2006

Some Thoughts on Haskell

I have been reading quite a bit about the programming language Haskell. It looks like Haskell is succeeding in the goal of consolidating the attention of a lot of programming language researchers into a single language, or at least a single core language with a variety of extensions.

Haskell is extremely expressive and dense, but so are lots of other languages. What interests me most about it is that it makes you wear the "hair shirt," as Simon Peyton-Jones describes purely functional programming in this presentation. I have used Dylan and NewtonScript for real-world projects, but only dabbled in Scheme; one reason is that I tend to have trouble coming up with the functional way to do things. I've got too much imperative programming under my belt. I've learned to think in terms of lambdas and closures and recursive functions, but I have not really gotten into the purely functional mindset. So my thinking goes that if I learn Haskell, then programming using the idioms of map and fold and all that will start to make sense. In fact, I might not bother going back to Scheme.

Haskell is an even "redder" pill than Lisp or Scheme in that regard. And it supports type inference, which has interesting implications for compilation and run-time efficiency. It also supports a programming idiom I am still trying to wrap my head around: monads.

I've worked through some very elementary category theory in the book Conceptual Mathematics: a First Introduction to Categories by Lawvere and Schanuel. You can find more about Monads in programming in this paper. John Shutt has a tech report here which looks promising in that the text outweighs the pure math notation; I've set as my goal a complete understanding of Schutt's paper. And Schutt's paper doesn't go very deep; it's an introduction. But even that will be hard enough for me for now. The rabbit hole goes much deeper!

Tuesday, November 21, 2006

I Do Read Other Stuff

I do read more than science fiction and fantasy. Recently I've read:

Ryu Murakami: In the Miso Soup. I really liked Ryu Murakami's novels Almost Transparent Blue and Coin Locker Babies. In the Miso Soup is a story about an American serial killer and sex tourist in Japan. It's a weird tale, but I didn't find it entirely convincing. Perhaps some stories about Japan just don't work that well in translation, because it is hard to believe a country could really be like that?

Also by Ryu Murakami: 69. This is a coming-of-age story about the author's high school experience in 1969. A little smoother going down than In the Miso Soup. If you felt alienated in high school you'll be very familiar with the narrator's persona, which is a layer of carefully crafted artificial sophistication on top of tremendous insecurity. The revelation of what happens when the narrator actually gets the girl is also very funny and feels utterly true. The book is a little short, and the author has an annoying habit of name-dropping the names of bands and books from the period which, unless you know them, might feel a little meaningless. Still, I recommend it.

Richard Bird: Introduction to Functional Programming Using Haskell, Second Edition. OK, maybe this should count as science fiction. Because it enforces lazy evaluation, referential transparency, and a lack of side effects, your program becomes a sort of idealized representation, nearly a mathematical proof. Whether I can actually apply such a thing of beauty in the real world -- that is, if the practical implementations really live up to the theory -- remains to be seen. I have only completed a few pages, but already Bird's exposition is helping to improve my understanding of functional programming. The style is a bit dry and this is an overpriced, under-designed (read: ugly) paperback, but I'd consider it required reading if you want to understand how Haskell is meant to be understood and used.

Leper Outcast Unclean!

So, the day I started reading Lord Foul's Bane, I noticed this weird open sore on my leg. I'm not sure how I got it. Maybe an infected spider bite? Maybe something that started out with a scratch from a two-year-old's dirty toenail?

It didn't seem very infected, and didn't seem to be getting any worse, but it just wasn't making any progress towards healing, either.

It is finally shrinking now that I'm in the last few pages of Lord Foul's Bane and Covenant, the outcast leper, will soon be returning to his own world.

Coincidence? I think not!

It makes me wonder whether my system is ready to handle The Illearth War!

Donaldson will be on Fantasy Bedtime Hour!

From Stephen R. Donaldson's site:


What a sport!

I still have not quite decided quite what I think of the show. I find the Fantasy Action Sequences very funny, but most of the humor of the analyses turn on deliberate misreadings of the text, turning it scatological or pornographic, and then hammering on each particular gag until it is not only dead, but smashed unrecognizably flat.

The misreadings are sometimes funny (I'd love to attend an annual Ritual of Desecration!), but it can get a little tedious to watch the girls play dumb. Surely they've learned something about the novel by now? Still, I have to laugh at lines like:

"He's Salt-lick Foamfollower! A legume from the seaweed giants!"

Rules for Parenting: No Nose-Biting

Parents! Here's a handy tip!

If you regularly play a little game that goes something like this:

(You): Can I bite your nose?
(Your baby): No.
(You): Can I bite your nose?
(Your baby): No.
(You): Can I bite your nose?
(Your baby): Hmmm... OK! (You bite her nose, producing much giggling).
(You): Want to bite my nose?
(Your baby): No.
(You): Want to bite my nose?
(Your baby): No.
(You): Want to bite my nose?
(Your baby): Hmmm... OK! (She gently bites your nose, with much giggling).

You may live to regret it once said baby girl has big teeth.

Grace tells me that she didn't actually damage baby Sam's nose, although it made him pretty upset. I guess I'm still learning...

Monday, November 20, 2006

Clench Count

I thought I might be able to search for the number of occurences of "clench" in Lord Foul's Bane using Google Books, but that book doesn't seem to be part of the database yet. However, I was able to do it using Amazon's "search inside this book" feature.

The results: "clench" appears 4 times. "Clenched" appears 41 times (wow!) And "clenching" appears 8 times.

There might be other variants of the word, but just those gives me 53 occurences.

In The Wounded Land, "clench" appears 9 times; "clenched" appears 41 times, and "clenching" appears 12 times. That's 62 occurrences. That's a lot of clenching. Covenant should talk to his dentist or about that, or maybe his therapist.

Just for comparison, I did a search on David Foster Wallace's novel Infinite Jest. "Clench" appears once; "clenched" appears twice, and "clenching" does not appear.

Just for kicks, I note that using the same tool, The Best of H. P. Lovecraft: Bloodcurdling Tales of Horror and the Macabre contains 105 instances of the word "fear," 51 instances of the word "vague," 32 instances of the word "blood," 36 instances of the word "nameless," 9 instances of the word "shapeless," one instance of the word "faceless," 22 instances of the word "dread," 41 instances of the word "terror," 5 instances of the word "tentacles," and 4 instances of the word "ichor." In particular, ichor is always green:

"... on its side in a foetid pool of greenish-yellow ichor"

"... foetid greenish-yellow ichor"

"... monstrous tracks and that foetid green ichor"

"... Their blood was a sort of deep-greenish ichor..."

So clearly if Donaldson is guilty of something in particular for his excess use of "clench," Lovecraft must be guilty of something similar.

Interesting. I just discovered another feature: the searchable books on Amazon give you a concordance. You can look at the most commonly occurring words! These are the top 100 words by frequency in Lord Foul's Bane:
across again against air arms atiaran away bannor behind between bloodguard came come company covenant darkness day done down drool earth end enough even eyes face fear feet felt fingers fire first foamfollower foul found gave giant go hand head heard heart held help high himself horses keep know land last left lena life light long look looked lord man mhoram moment moved must night nothing now old people power prothall ranyhyn reached ring river rock saw see seemed shoulders side soon staff still stone stood things thought time took toward tree turned two voice wall warriors went without word
Of those words, these are actually names or titles:
Atiaran Bannor Bloodguard Covenant Drool Foamfollower Foul Lena Lord Mhoram Prothall Ranyhyn
Comparing it to the general Science Fiction and Fantasy category tells us that with a fog index of 9.1, 37% of Amazon's science fiction and fantasy is harder to read than Lord Foul's Bane. 41% of Amazon's science fiction and fantasy books have more complex words, 58% have more syllables per word, and 37% have more words per sentence. This confirms my intuition that it is not actually the style, vocabulary, or sentence structure that makes Lord Foul's Bane difficult reading for some people.

The length statistics place Lord Foul's Bane quite high, which I found a little surprising. It is about 162,000 words. Only 14% of books classified as science fiction or fantasy are longer than Lord Foul's Bane. That is interesting to me, because I've never thought of Lord Foul's Bane as a particularly long novel.

Let's compare it to Gravity's Rainbow by Thomas Pynchon. Gravity's Rainbow weighs in at 325,000+ words. Infinite Jest by David Foster Wallace weighs in at almost 480,000 words. Infinite Jest contains 10 instances of the word "repellent," 10 instances of the word "pathos," and one instance of the word "presbyopic," and an amazing 29 instances of the word "dissemination."

The Memory Keeper's Daughter by Kim Edwards contains ten instances of the word "impulse" -- perhaps the protagonist is impulsive? The word "limestone" appears five times. "Surge" appears ten times, as does "diaper." The word "damp" appears fifty times, and "wet" twenty-six, while there is not a single occurence of the word "moist." I find that "odd," which appears nine times; it makes me "uneasy," which also occurs nine times; however, "nine" occurs only seven times, while "seven" occurs six times and "six" occurs twenty-nine times despite the fact that "occurs" occurs only twice. If we count up the appearances of the word "twice," we get twenty, which itself makes eleven appearances.

At the present time I do not wish to hazard a guess as to exactly what this all means.

On Clench Racing

Hmm. I read the essay by Nick Lowe that contains a description of clench racing.

Actually, Grace and I read it together in the car on the way to Saginaw and discussed it. Her comment was "I'm glad I have no idea who this guy is, because I can feel free to criticize his writing."

I did hunt through a few chapters of Lord Foul's Bane for "clench." I found it four times so far, enough to tell me the game probably would work reasonably well. But I have a couple of comments about Lowe's essay.

I took the Leonard Nimoy poetry challenge and I'm happy to report that I wrote concluding lines which were in all cases nearly identical to Nimoy's lines, except that I left out the word "love." Mocking Nimoy's poetry is not much of a challenge, sort of like killing fish in a barrel using a stick of dynamite instead of a gun, but let's take it as evidence that I have enough of an ear for phrasing to be able to knock these out without breaking a sweat. I can't really complain about this portion of the essay. Mocking bad writing is a long-standing form of entertainment at cons.

No, my first complaint is that he attempts to jump from a funny observation about Donaldson's use of "clench" -- which is about word choice and style -- to a point about predictability in word choice, to a point about plot devices.

First off, I don't buy his argument. Once you've read a novel, you can remember the words the author used a lot. In Jack Kerouac's novel On the Road the word "beat" pops up again and again. Does that mean Kerouac's prose is predictable? I wouldn't say that. Does it mean he was bad at plot? "Predictable" only works if you can predict it before you've read it, like you can with Nimoy. Is Donaldson really predictable in his word choices? Let me ask a related question -- do people's habits make them predictable? Covenant says "Hellfire and damnation." Speed freaks twitch. It makes sense to associate words with people; nervous people clench. I used to clench. My dentist gave me a bite guard and told me to avoid unnecessary travel to far-off Lands.

My next observation is that Lowe seems to be a bit dishonest about what writing is. Writing is artifice, genre writing perhaps even more so. It's fundamentally an unnatural activity. Writing seems "natural" only to the extent that the reader goes along for the ride with the writer.

Let's say I'm going to write a novel. What is it that I actually write about? Well, in general, unless I'm David Foster Wallace, or I'm writing something weird and postmodern about my childhood, I'm going to have to give my characters something to do. That's plot. It's an artifact. It is fundamentally artificial.

In a short story the plot can be vestigial, or seem very organic. I short story can even by primarily a character sketch, or about a place, or a memory. In a postmodern "blob" of a novel where there aren't any rules and there is almost no structure, it may seem like there is no artifice. The "plot" can feel seamless. In a genre novel -- particularly in a genre novel -- you have to build a scaffolding of plot. You do this whether you are very deliberative about it -- using note cards, or a timeline, or an outline -- or whether it evolves mostly in your head.

Lowe seems to be griping that some writers leave pretty obvious seams. But I want to be clear on this -- if you're a close enough reader, the seams are there in every novel, because ever novel is constructed, and can thus be deconstructed. Genre novels in particular follow conventions and thus have more overt structure. Make no mistake: looking for, and cataloging, plot holes and seams can be quite entertaining. But it is not necessarily the best way to enjoy a novel.

Lowe honestly doesn't seem to know that much about the history of the novel. To say that elaborate plotting first appeared in comic novels by writers such as Barth and Calvino is to ignore pretty much the whole history of the novel, including all of Russian literature. I like Barth and Calvino, but Barth and Calvino are very cerebral and self-conscious and ironic and self-referential. That's all well and good, but sometimes people just want to read a story. And they read genre fiction.

Lowe really rubs me the wrong way when he makes statements like this:
I'm a big fan of Sturgeon's law, so I have to agree that this is true, although I would place most books into a kind of no-man's land of mediocrity. It's important to keep in mind, though, that if Sturgeon's law is true, and I believe it is, then the only way to get more good books is to get more books. And that means more bad books -- a lot more bad books, or at least more mediocre books. That's how it works, and that's OK.

Is Lowe seriously coming from such an elitist position that he thinks he can convince us that Tolkien -- author of the best-selling novel of all time -- is a hack writer who churned out work that centered around plotting tricks? And even that Donaldson, whose works were huge sellers, was turning out turgid crap? Yes, that's exactly what he's saying. And he doesn't leaven his criticism with any light-heartedness at all. It might have been entertaining to hear him give his talk at a convention. I'm assuming that he actually was smiling when he said some of these things. On the page, though, the finished essay doesn't read as if he has any actual admiration for the genre underlying his funny, and mostly valid, criticism of the ways that authors create plot.

No admiration? What exactly is someone who hates most modern writing doing writing criticism of science fiction, exactly?

The author's elitism is revealed as a pretty hollow value system. He doesn't offer much to like. His attitude seems to be, at its core, "if people actually like it, it's crap." I'm reminded of a t-shirt I saw for sale online once. It says only "Your favorite band sucks."

In fact, Lowe's entire essay reads like something I might have written when I was sixteen. It has some superficial but funny insights but the overall tone is very elitist and obnoxious. In Lowe's world view readers don't rise up and demand something different because they are too stupid to understand that what they are reading is poorly written. Yes, that's really what he is saying. At the end of his ranting he comes right out and says it:
I hope that in revealing to you, for the first time in cosmic history, these precious secrets of how to tune and play your very own plot devices, I've given you some idea of the opportunities that exist for the talentless hack to abuse, short-change and exploit the mindless masses who put up with this garbage.
Huh. This is a bit like Arthur Dent confronting the Nutrimatic drink machine on the starship Heart of Gold. The machine attempts to convince Arthur that the drink it is producing for him is perfectly tailored for his nutritional needs and enjoyment, because it knows best. "But it tastes filthy," Arthur retorts. "I'm a masochist on a diet, am I?" Just what kind of tasteless gruel does Lowe think we should be enjoying if not the work we actually know and love? Personally, I'd rather settle down with a nice cup of tea.

And then:
The only thing that could possibly stand in your way would be a united resistance from those contemptible snot-gobbed arthropods the readers themselves, crying out against cheapskate exploitation fiction and demanding stories that can hold the road without the author stepping in every five pages to crank the bloody things up. Small chance of that, eh?
This is supposed to be funny, I suppose, but it reminds me why I will never be caught reading one of the "for Dummies" books. Why am I reading an essay by a writer who calls me "contemptible?" This essay should be entitled "Science Fiction for Contemptible Snot-Gobbed Arthropods."

I certainly have had a lot of fun mocking various writers. I love writing pastiche and parody. Parts of Lowe's essay are funny, but it left an unpleasant taste in my mouth. Maybe that's the snot in my gob. Or maybe it's the snottiness in Lowe's essay.

Imagine that this sentence is a spirited defense of Gene Wolfe's work. If you've read Wolfe, you can write it for me. Although Lowe mocks some of the rare weak spots in Wolfe's writing in The Book of the New Sun, it is safe to say that now in 2006 Wolfe's reputation is firm. It remains to be seen what people will think in 2026.

I'd like to throw out a few closing thoughts inspired by a writing teacher of mine who loved Mark Twain and tried to remind me that James Joyce, whose work I love, was not a valuable model for a writer attempting (as I was) to actually complete stories that readers might actually enjoy reading.

- Readers are not fools. They appreciate work like Tolkien and Donaldson despite, not because of, its flaws. Both these writers told stories that people enjoyed reading.

- Writers are not fools. Especially not writers who have made far more money than I will ever make.

- To clarify: there is no shame in writing a book that stays on top of the New York Times bestseller list. These writers are doing something right. They may not share my priorities, but they aren't fools.

- I say this even about books I don't like very much. I could easily mock The DaVinci Code. I don't think it is a lasting monument of literature. Most works aren't. And yet:

- At one time I might have thought that I would never, ever want to write a book like The DaVinci Code. But as I think more about writing for money myself, I find I can no longer afford to hold such an elitist view. I can't afford to call the readers who bought The DaVinci Code idiots. If I myself aspire to write a novel and earn money with it, I'd do far better to call them something more like "a potential crossover readership demographic."

- About that "lasting monuments of literature" thing? A work is not best judged by its contemporary critics. It is actually not simple to predict what will be considered great and what won't be.

- Recently I've been really, really enjoying reading the complete works of William Hope Hodgson. He sold upwards of a hundred short stories, all written in about ten years, for money. To eat. Not for art's sake. He barely earned enough to eat, but did manage to make writing his job. A lot of the stories are really quite good.

- Hodgson also wrote poetry. I'm reading it. It's a mixed bag: a lot of it is wretched, but some of it is really pretty good. Some of it is coming into print now, for the first time, over 100 years after it was written. Remember what I said about how "a work is not best judged by its contemporary critics?"

- Conversely, it's hard to predict what will be considered crap and what won't be. There must have been an anti-Donaldson backlash about the time that Lowe wrote this essay, which seems to have been in the mid-eighties. But people are still reading Donaldson. He's in print, and his new series is selling well. But another writer he mentions, Susan Cooper? I've never heard of her.

- Lowe mocks Donaldson for his word choice, but the fact that he uses obscure and archaic words is seen by many readers, including me, as a strength. As readers who think of ourselves as smart and literate, don't we consider it actually fun to learn new words (or old ones)? I know that, personally, it is one of the things I enjoy about reading.

- Although in English classes we are trained to put different kinds of works into boxes, there is actually no sharp division between serious work, homage, pastiche, and parody.

- The category of "unintentional self-parody" exists mostly in the mind of the newly minted English major. It doesn't exist in the mind of most readers.

- This isn't actually because most readers aren't capable of being critical. Sure, they may not have training in criticism. They may not use the language of the academic. I'm starting to think that's a good thing. No, it's because most readers take what they enjoy out of a work and leave the rest behind. They don't start out from a pedantic, self-aggrandizing, mocking point of view.

- You can believe that there's always a better book. You can believe each year that the books you enjoyed last year are not worth reading, and that you only enjoyed them last year because back then you were young and naive and didn't know any better. But that way lies a lot of bitterness. Eventually you've cleared your shelves for everything except whatever undiscovered obscure Afro-Carribean novelist is being discussed by the literati this week. Nope, I refuse to go there. I prefer my shelves to be bulging with books I enjoy, but which I ought to know better than to waste my time reading.

- The difference between a bad work and a good work is often that the bad work doesn't know what it is actually doing. But I'd hazard that even Lin Carter knew what he was doing. And Donaldson certainly did. The readers said so, and his own work says so. I'd like to cite the following in defense of Lord Foul's Bane:
...he had tasted the consequences of allowing the people of the Land to treat him as if he were some kind of mythic figure. With an effort, the replied gruffly, "Nevertheless. I'm not used to such things. In my own world, I'm -- just a little man. Your homage makes me uneasy."

Softly, Mhoram sighed his relief, and Lithe raised her head to ask in wonder, "Is it possible? Can such worlds be, where you are not among the great?"

"Take my word for it." Covenant drank deeply from his flask.
And, even more to the point. Donaldson might have imagining his critics saying these words:
"That was a joke. Or a metaphor." Covenant made another effort to turn his sarcasm into humor. "I can never tell the difference."
And one more, which I'll let stand as my last word on Lowe's essay :
Watching him go, Winhome Gay breathed, "He dislikes you." Her tone expressed awe at the Warhaft's audacity and foolishness. She seemed to ask how he dared to feel as he did -- as if Covenant's performance the previous night had exalted him in her eyes to the rank of a Ranyhyn.

"He has good reason," answered Covenant flatly.

Gay looked unsure. As if she were reaching out for dangerous knowledge, she asked quickly, "Because you are a -- a 'leper'?"

He could see her seriousness. But he felt that he had already said too much about lepers. Such talk compromised his bargain. "No," he said, "he just thinks I'm obnoxious."

Friday, November 17, 2006

Free Empire




Followup: the prize has been claimed! Thanks for reading.

Two Recent Netflix Movies, and an Observation on Lord of the Rings

Two of our recent rentals from Netflix were House of Flying Daggers and Memento. Aside from the inherent difficulties of trying to concentrate on a movie with a two-year-old running around the living room, both of them were fantastic films. I'd rate them both as among the top ten movies of the last few years and will probably wind up buying both of them.

Memento is a story that comes together through the accumulation of detail. It is assembled backwards in time as a series of short vignettes, with each successive sequence slightly overlapping the start of the previous sequence. Given a chronology of events A, B, and C, Memeto shows me all of C, then B followed by the very start of C, and then A followed by the very start of B.

The main character in Memento has suffered a head injury and exhibits a form of anterograde amnesia: he can remember his life prior to the date of his injury, and thus knows his name and has a general knowledge of his surroundings, but he can't form new short-term or long-term memories. This means whenever he falls asleep, or is distracted, or his mind wanders, he forgets what he is doing. He has acquaintances, but each time he meets them he does not remember meeting them before. They may be lying to him; they may be using him for their own purposes. Each time he meets one of them, he has to pull out his collection of Polariod photographs and review his collection of notes.

As the story unfolds, we have the feeling that it is up to us to keep track of the details the character can't keep track of. It's an uncomfortable feeling; I wound up taking notes. However, the mystery of Memento is not really to be solved like a puzzle; it comes into focus when we understand the way that the main character creates his own reality by choosing the exact nature of the bread crumb-trail he leaves for himself. And although the main character is allegedly attempting to solve a murder mystery, we come to realize that his deeper purpose may be to give himself a reason to continue living the next time he has forgotten everything.

Last night we watched House of Flying Daggers. This movie operates on several levels: as an adventure film filled with impossibly elegant and inhumanly precise martial arts moves, it is without parallel. The fighting and dance sequences are stunning. It also has some of the best foley work I've ever heard in a motion picture, bar none. The score is Eastern and has a period "feel" without resorting to musical cliche. The costumes are gorgeous, the outdoor sequences lush and impossibly lovely.

There is more going on than just a fight film, though; for a good fight film, you could go watch Jackie Chan's Rumble in the Bronx or one of my personal favorite action adventures, The Transporter. House of Flying Daggers also involves complex political intrigue. No one seems to be trustworthy; no character's motivations appear to be what they ought to be. They don't trust each other, and with good reason. The characters are constantly making unexpected turns, and it requires considerable concentration to keep up. You'll be constantly guessing and then changing your guess as to what is really going on. And you'll be slightly wrong, again and again, which is refreshing given the utter predictability of a typical film script.

The film turns into a road movie on horseback and in the canopy of a bamboo forest, a surprising love triangle, and a devastatingly brutal fight sequence. This scene, which ought to be studied by any filmmaker aspiring to shoot a fight, begins with the stereotypical glamourization of violence, then passes into a work of acrobatic art, then then reaches a plateau of violence and suffering that completely transcends the trivialization of violence in the film so far. Watching it awakens your senses and sympathies and re-sensitizes you to violence and pain all over again. You realize that even beautiful violence is still violence. And the more the characters fight, the more they regret the necessity of fighting.

Which brings me to the motivation of the characters. They are drawn by duty: oaths and loyalties which impel them, and ultimately a desire to choose their own destinies, even if that destiny is also a doom. They are not modern characters; they do not have complicated anxieties and phobias and selfish needs. They have iron loyalties and almost unlimited personal abilities and a level of discipline that is almost impossible to comprehend.

Which brings me back to Peter Jackson's Lord of the Rings.

I've been reading at a book of essays on the films, called Tolkien on Film: Essays on Peter Jackson's the Lord of the Rings. It's quite a worthwhile book, and with a few exceptions, I've enjoyed all the essays in it. It is said that to fully understand an argument you have to initially alter your world view to accept the argument. As I read these essays, most of which are strongly critical of Peter Jackson's adaptation, I find myself agreeing with each author -- provisionally. I then undergo a kind of mental snap, and begin playing devil's advocate, coming up with reasons that the author is wrong. It's an interesting exercise.

One of the arguments that recurs in several of the essays is that in modernizing several of Tolkien's characters -- especially in giving them frailties and weaknesses and self-doubt -- Jackson made his adaptation untrue to the books. In particular, Jackson's Gimli, while still heroic and strong, is given lines and behavior which Tolkien never wrote, making him a character of comic relief. Some of this is true to the book -- Gimli did have a friendly rivalry with Legolas, as they both kept count of the number of orcs that each killed. But some of it is not true to the book. Tolkien's Gimli was not a buffoon. He is not overcome by tears when he encounters Balin's tomb in Moria. Jackson takes similar liberties with Aragorn -- his Aragorn tells Arwen that things are not going to work out between them, and tells Elrond of his ambivalence towards assuming the kingship. Jackson's Faramir is temporarily seduced by the ring and takes Frodo and Sam with him to Osgiliath with the intention of taking them the rest of the way to Minas Tirith. Tolkien's Faramir, with some of the blood of Numenor in his veins is, like Aragorn, not so easily seduced. Tolkien's characters may be flawed -- Jackson's Boromir is actually quite true to the text -- but they are not ambivalent and plagued by self-doubt.

It is possible to make the case that in modernizing the portrayal of Aragorn and Gimli, and even Gandalf, Jackson worked around a barrier to acceptance of Tolkien's tale by a modern audience. This barrier is that a modern audience would not understand, and thus would not empathise with, characters that have uncomplicated loyalties and clear, lucid motivations.

This may be true for some audience members, but the success of House of Flying Daggers and Memento seems to prove otherwise. Although most of us never take blood oaths and would not fight to the death to uphold them, we feel a piercing sympathy for the characters that do have these qualities. In fact, their simple motives makes them even more meaningful and easier to appreciate. And while most of us will never suffer from a complicated amnesia, any thoughtful person grown to wisdom will eventually understand the necessity of self-delusion and the way that we choose to shape our own realities every day in order to avoid paralyzing indecision.

So, while Jackson's films succeed on some levels, I believe the alteration of Tolkien's characters was primarily an unnecesary exercise and ultimately weakens the films. However, Jackson clearly believed it to be necessary: this may have been more of a function of the compromises he had to make to get the film released by New Line. I'm not privy to the negotiations involved, but such an expensive endeavor is inevitably compromised and a studio will generally insist that a project of such magnitude take few chances. Someone along the way apparently believed that the application of modern screenwriting tropes to Lord of the Rings increased its chances for popular success, and so it was done. This unfortunately turned several of Tolkien's heroic characters into weaklings, and so we were not given the chance to see if a truer film adaptation, in which a hero remains a hero and does not need psychoanalysis, would also have found an appreciative audience.

Followup: this is far from my last word on the subject of the Lord of the Rings films. They are complicated creations, and I find them to be worth watching again and again. While there are many points in the film in which Jackson undermines the story, there are also many points in which Jackson places the characters in scenes Tolkien never wrote, but in which the characters' behavior is perfectly in keeping with Tolkien's vision. And while Jackson's love of monsters and horror-movie tropes tilted his vision in many places towards an action film, I don't find this interpretation to be invalid, just different than my own. I will have more to say about Jackson's films in the future!

Lord Foul's Bane

So, I have taken a break from reading The Runes of the Earth while I backtrack and re-read Lord Foul's Bane. First published in 1977, I was probably thirteen or fourteen the last time I read this book. It is interesting to find out how much I remember, and also how well the novel holds up.

Donaldson's book has some superficial resemblances to Tolkien's work. There's a ring of power, goblin-like creatures, dwarf-like humans who live in stone buildings, and elf-like humans who live in a big tree (Woodhelvin, which sounds like "wood elves.") The antagonist is a disembodied spirit who speaks through others, and rallies armies of deformed creatures to bring forth a blight upon the countryside. There's a stone keep (Revelstone) that bears a superficial similarity to one of the cities of stone in Tolkien, Helm's Deep or Minas Tirith. There's a dreadful mountain, Mount Thunder, slightly comparable to Mount Doom. There's a giant with a personality a bit like Tolkien's ent Treebeard, who laughs deeply and loves long tales and urges others not to be hasty.

However, it would be a big mistake to think of the Chronicles of Thomas Covenant as a pastiche or homage to Tolkien, even though they make passing nod to some of the physical elements of his story, which were not necessarily original to his work.

Aside from an occasional, forgivable lapse into a more traditional "high fantasy" mode in which characters recite their lineage by way of introduction, Lord Foul's Bane is a stripped-back, pared-down fantasy almost entirely free of grandiose sentiment and heroism of any kind. Donaldson does not attempt to be a "cultural completist" like Tolkien; his world is not a re-creation of the lost history of ancient England. The Land's history and legends are not intact: in fact, it is a dimished world, and most of the great lore of its past has been lost, and the inhabitants are struggling to regain it. When his characters recite the occasional fragment of verse or epic tale, these things are not written in a faux-ancient style; they are, for the most part, in modern un-rhymed verse. Indeed, they occasionally lapse into a slightly dated 1970s Rod McKuen tone, but only occasionally; most of the work still reads as if it had been written yesterday. And although some may consider it to be tedious and hard going, I am impressed by just how how economically the story is told. There's no Tom Bombadil, and no happy drinking songs. Almost every word very pointedly and urgently propels the story along.

The protagonist, Thomas Covenant, is just about as unlovable a man as you could possibly construct. He was a successful writer, stricken with leprosy through no particular fault of his own. Embittered, he has dedicated his efforts to his own survival, which has required him to strip himself of all consoling illusions about life's loftier rewards and goals; his goal is to survive each day, and he has little to look forward to except the avoidance, for one more day, of reactivated leprosy, infection, numbness, and amputation. His wife has abandoned him, taking with him their infant son; his community shuns him, and he has thrown his unfinished novel onto the fire. In short, his virtue consists of a carefully cultivated selfish will to his own safety and survival; his primary impulse is self-protection and withdrawal. But however much the world recoils from him, he is keenly aware that he needs to continue to go into, and interact with, the wider world, so that he does not retreat entirely into isolation and self-pity.

When the story opens, he is walking downtown to pay his electric bill. He is angry and bitter: someone has been paying his bill for him, apparently to keep him from venturing downtown to pay it himself. A woman drags her child away from his path, telling him that he ought to be ashamed. He feels panic and vertigo, and looking at a pair of giggling teenaged girls, a lechery born of the impotence the disease has imposed on him. I don't think we're told exactly how old Covenant is, but I place him at about thirty; old enough to have left behind some of the careless illusions of youth, but not old enough to have developed wisdom and equanimity.

Telling the clerk that "it isn't catching," and constantly scanning himself for scratches, since his extremities are numb, Covenant succumbs to an anxiety attack and stumbles into the path of an onrushing police car. Falling, he believes the car will strike him, and he loses consciousness. When he regains consciousness, he is in the Land, listening to a blithering goblin-like creature, Drool Rockworm, announce gleefully that he has summoned Covenant by using his new found-mastry of the Staff of Law. The voice of Lord Foul intrudes and Covenant finds himself given a grim message to take to the lords of the Land.

The story, and Covenant's motivations, here become paradoxically confusing. One the one hand, Covenant must believe that everything that is happening to him is a kind of dream. It seems to him that the temptation to succumb to his fantasies is a kind of suicide, a complete retreat into unreality. But if it is a dream, then why not act out his own self-gratification? He meets a teenaged girl, Lena, who believes him to be the reincarnation of a legendary hero from the Land's distant past, Berek Halfhand. She looks at him with both trust and overt attraction, in a way that would provoke a more mature man's protective instincts. But Covenant's nerves are miraculously healing, and his sexual potency returns.

This healing drives him to believe even more strongly that what he is experiencing cannot possibly be happening in reality. He succumbs to his lusts and rapes Lena. And so now as readers we are forced to decide whether we will throw the book down in disgust or attempt to maintain our sympathy with a man who must surely be one of the most unpleasant protagonists ever set to paper. Or, as Julie puts it in Fantasy Bedtime Hour, "I hate him so much!" For Donaldson's part, he does not indulge our prurient interest -- the rape is described only vaguely, not like a modern romance/pornography. But there it is.

If we continue, we are faced with a series of complicated questions. If I indulge a debased fantasy in a dream, have I done anything wrong? Couldn't such a thing actually be healthy, if it prevents me from acting upon impulses like that in my waking life? But what if my dream is, in any sense at all, real? If you believe that nothing you do matters, does it still matter what you do? Earlier in the book Covenant was handed a pamphlet which asked a similar question, and declared that this was the central question of ethics.

Covenant then reacted with disgust, because in his reality, the choices and options he faced were dramatically limited, and centered around his own survival. But he is forced to confront the question written on the largest possible scale. One could rewrite the question as a choice between moral relativism and a belief in unversally applicable moral principles. That's a big topic for a work of fiction to take on, and it only works because Donaldson does not provide a single, simple answer.

As if all this wasn't enough, Donaldson has also written a book which takes on the complex question of the relationship of human beings to nature. One of Tolkien's themes was the madness of war and the destruction that inappropriate use of technology wreaks upon the earth. The land is a wonderfully beautiful place, and Donaldson grants Covenant a "health-sense," the ability to directly feel the health or sickness of the Land's ecosystem itself on both a small or large scale. In such a world failure to take care of the land has a direct and potent effect on one's own senses. It seems a consummation devoutly to be wished. But would you like to feel the earth's pain in an open-pit mine, or a poisoned watershed?

So Covenant is embroiled in a paradox. All his senses -- because they are fully functional, and indeed, more functional than they ever were before -- are a constant reminder to him that what he is experiencing can't be real. He doesn't want to succumb to what feels like madness. But the vitality he sees and feels around him -- the beauty, and the strength and devotion of the Land's residents -- convinces him that he must fulfill the mission that Lord Foul has given him, to warn the Lords of the Land of their impending doom and try to preserve what he can of the beauty he sees around him. He calls himself "Unbeliever," even as he sets off on a journey to the Lords with Atiaran, Lena's mother. He does not atone for the rape of Lena; he does not ask forgiveness, and it is not preemptively given to him. Atiaran learns of his act she does not renounce her Oath of Peace and take his life, because the threat to the Land is greater than her own need for revenge.

And so the story proceeds. It is intellectually challenging, and morally challenging, in a way that Tolkien isn't. The two obvious options Covenant is faced with -- absolute disbelief, and absolute acceptance, of the reality of the Land, feel to him like forms of madness. And so his only alternative is to remain on the knife edge between belief and unbelief. And so he is torn. He could fall into crippling guilt for Lena's rape and for his own inability to save the land. Or he could veer into complete contempt for the alluring and fantastic lie laid before him. He's not a hero, although everyone he meets seems to treat him like one, but they also expect miracles and wonders from him. He doesn't know how to use the white gold ring that everyone he meets tells him is an artifact of immense power, both creative and destructive. And so all for all his apparent freedom his options are agonizingly limited. This makes Lord Foul's Bane not just a potent work, but a profoundly ethical work.

Spam Record

My spam intake is reaching critical mass. I had 64 messages in the last twelve hours.

For the most part they are caught by my filter, which is configured as a whitelist. However, every once in a while one gets through. I'm not sure why, since the address given is not in the whitelist. My best guess is that sometimes the filter process on my host crashes and then mail goes through unfiltered for a while.

Worse than the unwanted messages is the fact that, in using a whitelist, I inevitably lose wanted mail. I just don't have time to examine each message so I have to skim subjects and addresses. I get dozens of messages with the subject "Hi Paul." Are they spammers or a long-lost friend from college? I don't have time to try to figure that out. Also, the flood of random words that spammers use in their subject lines ensures that at some point I'll see something that looks like it really was written specifically to me.

There are also messages from companies I've placed orders with, and mailing lists I've signed up for. The anti-spam measures they take with their return addresses (using a changing no-replay address) ensures that many of these wind up in the filter. So I'm probably missing an occasional order update or mailing list posting as well. Or a notice about a job.

Add to this the spam we had to contend with on our Wiki, now closed, and the comment spam I'm getting via Blogger, in bursts of a half-dozen identical messages. I'm getting fed up!

Monday, November 13, 2006

More Blog Spam

User "prasannab" -- you're next on my shit list. Another spam comment and I start dedicating myself to getting you kicked off Blogger.

Blog Spam

User "Askinstoo" is spamming this blog with comments containing "make-money-fast" links, although since this blog is moderated I have been rejecting the posts. I have reported this abuse to blogger. The contact link is here.

If you have similar problems with bad blogger users please report them and help get them kicked off Blogger!

Saturday, November 11, 2006

New Photo Galleries

I have put some new photo galleries on our family web site here. Of special note: photos from Sam's first week, and a gallery of photos of both Sam and Veronica from their first two days after birth, showing how similar they looked. These photos don't have any annotations yet. I'll try to get some notes attached to the pictures as soon as I get a chance.

Thursday, November 09, 2006

A Lost Change to Star Wars?

Apparently, in the scene where Luke throws his grappling hook across a shaft in the death star and carries Leia across to safety, some people recall the hook coming unstuck the first time, requiring a second throw.

That sounds familiar, but apparently there are two throws in the novelization. And I've learned not to trust my memory on these things.

If this change is real, I wonder why it was made. The failure of the first throw actually increases dramatic tension, and I wouldn't expect an editor to want to reduce the tension at that point in the film.

On Slashdot, there's an unintentionally ironic quotation from Lucas, although it is unsourced and so I can't really verify he ever said it.

Did he really say this? If so, it seems a bit hypocritical!

And here's an interesting bit of trivia: some videodisc players existed that did not use a laser pickup. (They did not use a "needle," precisely, but they used a physical pickup that touched the disc. Very strange. I had never heard of these until a former boss mentioned them to me. I always wondered how a vibrating needle could carry a signal of sufficient bandwidth to reproduce video).

From Wikipedia here:
SelectaVision used a special medium known as a Capacitance Electronic Disc (CED). The VideoDisc was a 12 in (305 mm) platter housed in a special caddy. The video and audio signal is stored on the Videodiscs via peaks and valleys in the grooves, similar but not exactly like a phonograph record, of both sides of the discs. To play a Videodisc, you inserted the caddy into the player and the platter would be extracted. A keel-shaped needle with a titanium electrode layer would ride in the groove with extremely light tracking force, reading the electrical signal from the groove where it is decoded back into its FM state.

Unlike a phonograph record, where physical movement (vibration) of the stylus in the groove of the platter led to an audio signal, the stylus in a SelectaVision player slid along the crests of the groove, at a constant rotational speed of 450 rpm. The varying undulations of the peaks and valleys in the groove provided differing amounts of capacitance between the stylus and the conductive carbon loaded PVC disc. This varying capacitance was measured by the player circuitry, providing an audio/video signal.

If that's not weird enough for you, read about PixelVision.

Very strange! But an object lesson in obsolete media formats. What would you do with one of these discs, if you found them? Somewhere I have a rare 7" analog videodisc containing several music videos by They Might Be Giants. It's a really pretty coaster!

Wednesday, November 08, 2006

Laser Monks and Inkjet Printheads

The HP Business Inkjet 1100 had lost its mind again and after I took all the cartridges and printheads out, cleaned everything, and put it back together, it decided the magenta printhead was bad.

So I ordered a new one from Laser Monks. Although their prices are not particularly competitive for the HP inkjet supplies, it arrived very quickly, so I'm pretty happy with their service. I'll let you know my opinion for sure when we get the chocolate-covered caramels I also ordered from them!

The printer is doing better, although continuous-tone images are still kind of streaky. I do know that two of the other printheads (black and cyan) are looking pretty spotty, so I should probably replace those next. (The printheads are failing not because I print too much, but because I have hardly printed anything on it in the last six months to a year).

Weirdly, the printer thinks the year is 2007. At some point it gained a year. This means it thinks that I installed the new printhead on 10/08/2007. Does that mean I get an extra year out of it because the printer doesn't know when to consider it expired, or does it mean it is going to think that all my ink cartridges are expired a year early? I don't know. The printer also seems to think that the magenta ink cartridge is empty, even though it is brand new, and still feels and sounds full; it thinks the yellow one is empty also, although it is probably more than half full.

Logically, I would not spend any more money on supplies for this printer. The printheads and ink cartridges cost about $30 a pop. A set of all eight goes for $240. The whole printer with starter cartridges only cost $200, and that's really expensive for an inkjet these days. However obscenely expensive the supplies are, though, throwing away a whole printer seems even more obscene. So, for now, it stays.


Santorum and family as he concedes.

I almost feel bad for him (and them).


Stephen R. Donaldson's Newest (and Oldest)

I became aware of Stephen R. Donaldson's Chronicles of Thomas Covenant sometime in junior high or high school and read them, or at least the volumes of the Second Chronicles, as they came out. Donaldson graduated from the College of Wooster, and this was at least a small factor in my decision to go to Wooster myself.

I majored in English and wrote a collection of short stories for my senior independent study project; for one of my stories, published the The Wooster Review, I was awarded with that year's Donaldson Prize for Fiction. It was not that many years earlier that Mr. Donaldson wrote his own senior independent study project: a fantasy novel.

I'm not actually trying to draw any kind of parallel between my career and that of Mr. Donaldson; although I wrote some publishable short stories almost twenty years ago, and had aspirations towards writing as a career, I also had many other overlapping interests -- I was also taking all the computer science courses I could manage -- as well as a "butterfly mind." I've sold several non-fiction articles and occasionally been paid to write book-length technical documentation, but I haven't written a complete original story since 1989, and I've never managed a novel. I've begun to plan some possible writing projects, but they will move only sporadically for the near future.

When I won the prize I got a letter from Mr. Donaldson as well, asking me about the state of the English department and their attitudes towards genre fiction such as science fiction and fantasy. I was slow in answering him and probably didn't give him much useful feedback. I think he wound up calling me, but I don't remember for certain. At the time I was not thinking much about the reputation of science fiction and fantasy in academe. The English department offered an occasional course on science fiction literature taught by Professor Thomas Clareson, although it was only offered irregularly and I never actually took a course from Clareson. I was focused on medieval literature with Paul Christensen and fiction by women with Joanne Fry and creative writing with Michael Allen and then my thesis work with Dan Bourne, while also taking all the computer science classes I could manage; I was also spending a lot of my time engaging in a less academic pursuit: debauchery.

Anyway, I always felt like I should have been more helpful for Mr. Donaldson. According to a little Googling the Donaldson prize seems to be alive and well -- in fact, there seem to be several of them now. It might be interesting to round up the past Donaldson prize winners and find out what they have to say about the experience, and to come up with a group "thanks" to Donaldson himself.

Meanwhile, it has been decades since the completion of the Chronicles double trilogy, but I was interested to note that Donaldson has started another series. Although he killed off Thomas Covenant, Linden Avery is still alive and well and she is the hero, or anti-hero, of the latest book.

I don't get very much time to read these days, with three children at home, one of them only three weeks old. However, when I discovered that William Hope Hodson wrote more than one famous novella, it awoke some dormant interests in me, and I've begun to think much more about genre writing: the overlapping demands of money and art, how writers have reconciled them, and how I might do the same.

Donaldson's book The Runes of the Earth had been in one of my to-read piles for several months. I have several such piles. But a couple of days ago, for no clear reason, I finally plucked it out and started reading. (It is sad reality in my life that the only uninterrupted time I get to read is generally early in the morning, in the bathtub; I'm sure I'm not the only bathtub reader out there; and no, I've never actually dropped a book in the tub; but just in case, I never read my rare or out-of-print books that way).

So far, The Runes of the Earth is quite engaging. I'm getting reaquainted with Donaldson's trademark style (at least, the style he uses for the Covenant books; like William Hope Hodson, he has several at his command). It's safe to say that readers are not neutral on the issue of writing style. It's grim, but beautiful; it will put off some readers and fascinate others. Donaldson must have a well-thumbed copy of the OED in his office; he loves to pull out weird, wonderful words. I've always felt that I have a big vocabulary, but Donaldson always surprises me by pulling out words I've never read before. I try to force myself to make mental notes of these words and look them up later. To give you a taste of what I mean here's a passage from The One Tree:
Suddenly, power seemed to flash around her as if she had been dropped like a coal into a tinderbox. Bells clanged in her head – chimes ringing in cotillion on all sides. Bubbles of glauconite and carbuncle burst in her blood; the air burned like a thurible; the world reeled.

Stunned and gaping, she panted for breath. She had been translated by water and travertine to another place altogether – a place of eldritch astonsihment. An opalescent sky stretched over her, with the suggestive evanescence of night and the specificity of day. And under its magic, wonders thronged in corybantic succession. Nearby grew a silver sapling. Like flakes of precious metal, the leaves formed a chiaroscuro around the tree, casting glints and spangles as they whirled. A furry shape like a jarcol went gamboling past, and appeared to trip. Sprawling, it became a profuse scatter of flowers. Blooms that resembled peony and amaryllis sprayed open across the glistening greensward. Birds flew overhead, warbling incarnate. Cavorting in circles, they swept against each other, merged to form an abrupt pillar of fire in the air. A moment later, the fire leapt into sparks, and the sparks became gems – ruby and morganite, sapphire and porphyry, like a trail of stars.

And these were only the nearest entrancements. Other sights abounded; grand statues of water; a pool with its surface woven like an arras; shrubs which flowed through a myriad elegant forms; catenulate sequences of marble, draped from nowhere to nowhere; animals that leaped into the air as birds and drfted down again as snow; swept-wing shapes of malachite flying in gracile curves; sunflowers the size of Giants, with imbricated ophite petals.

And everywhere rang the music of bells – cymbals in carillon, chimes wefted into tapestries of tinkling, tones scattered on all sides – the metal-and-crystal language of Elemesnedene.

Now, that passage isn't quite typical -- Donaldson uses so many rare and beautiful words here specifically because Avery has jus passed into a very strange and beautiful place, a bit like Galadriel's stronghold, Lothlorien. But it isn't completely atypical, either. It reminds me faintly of E.R. Eddison's style in his novel The Worm Ouroboros. Compare this passage:
At the end of the hall upon a dais stood three high seats, the arms of each composed of two hippogriffs wrought in gold, with wings spread, and the legs of the seats the legs of the hippogriffs; but the body of each high seat was a single jewel of monstrous size: the left-hand seat a black opal, asparkle with steel-blue fire, the next a fire-opal, as it were a burning coal, the third seat an alexandrite, purple like wine by night but deep sea-green by day. Ten more pillars stood in semicircle behind the high seats, bearing up above them and the dais a canopy of gold. The benches that ran from end to end of the lofty chamber were of cedar, inlaid with coral and ivory, and so were the tables that stood before the benches. The floor of the chamber was tessellated, of marble and green tourmaline, and on every square of tourmaline was carven the image of a fish: as the dolphin, the conger, the cat-fish, the salmon, the tunny, the squid, and other wonders of the deep. Hangings of tapestry were behind the high seats, worked with flowers, snake's-head, snapdragon, dragon-mouth, and their kind; and on the dado below the windows were sculptures of birds and beasts and creeping things.


I'm partway into The Runes of the Earth; Linden Avery has just been drawn back to the Land. She's just had a series of visions and her perceptions are darkened and confused. Several of her acquaintances and co-workers have just been murdered. She herself has apparently been shot in the chest, but can't feel the pain of the injury; maybe she's in shock, or maybe it is something stranger. I'm drawn back to the Land as well. I only hope that I don't neglect my family any more than necessary while trying to find some brief moments of peace and quiet in order to get through this novel! And once again I'll be in the long-familiar state of waiting for Mr. Donaldson's next book. But, meanwhile, I've ordered copies of the original two trilogies from ABE, so I'll have the original books to re-read.

It will be interesting to see how they read. I can't remember for certain, but I think that I first read the books out of order. I may have started with a signed hardcover copy of The One Tree (that copy is long gone), then gone back to the first trilogy and caught up, then read White Gold Wielder when it came out in hardcover. But I could be wrong about that; my first encounter with the books was at least 23 years ago.

Meanwhile, if this all seems a little too high-brow for you, you can watch two California girls and their friends attempt to make sense of Lord Foul's Bane in a community-access cable show available as a vodcast (for iTunes, but it may work with other RSS feed readers here (note: this is a link to the XML file for feed).

Anyway, Mr. Donaldson: if you wind up reading this, thank you for writing another novel of the Chronicles of Thomas Covenant lineage. And thank you for the Donaldson prize! If it ever comes to pass that I get my act together and finish a novel, or begin submitting short stories again, your work and indirect encouragement will have been partly responsible.

"The Negatives of the Movies were Permanently Altered"

Over at there's the text of a reply from Lucasfilm about releasing the original 1977 Star Wars on DVD. It's all pretty much what you'd expect until you get to this paragraph:

Now, I don't really know anything about filmmaking or film restoration, but the phrase "the negatives of the movies were permanently altered" brings me up short. What the hell does that mean? Does it mean the negatives were _restored_, as in cleaned or whatever else an archivist would do to them? That's wonderful. Or does the publicist really mean "altered," with Special Edition changes cut out or printed on?

Like I said, I don't really know much about filmmaking, but I can't imagine a scenario in which any archivist or preservationist would make destructive changes to the original source material. Most restoration work (on paintings, on sculpture, etc.) is inherently conservative, operating under the fundamental dictum "do no harm" and perhaps the informal creed "don't do anything that will cause future generations to curse your name and piss on your grave." I'm quite certain that no reputable institution would start correcting spelling errors in their Gutenberg bible in permanent ink or brightening up the colors of a fading Picasso painting with magic markers.

Lucas's own feelings towards archiving are pretty well summed up in comments he made about the Special Editions. See the original quote on Wikipedia here:

So DVD media and digital file formats are the artifact you want to preserve, and give you "perfect quality?" Huh. That's news to me. I'm sure the people who worked on the BBC Domesday Project thought that LaserVision videodiscs were "archival" as well, but in just a few short years they were proven wrong, and that media was only salvaged by heroic effort. A lot of modern digital formats and media are even more ephemeral. Much of the things I've personally worked on: multimedia projects that use videodiscs, or documents in obsolete text or image file formats, or projects stored on casettes or 5 1/4 inch floppies or in compressed archives for which there no longer exists decompression tools that can run on modern computers, are long-gone, even when the media themselves are still readable. It's a big, big problem. It's hard to believe that an alleged science-fiction visionary whose career has been spent embracing new technology would somehow fail to see this.

One can only hope that Lucasfilm has at least one genuine preservationist or archivist working for him who actually cares about the integrity of the original source material. If not, we can only hope that those materials will wind up in the hands of a University library with a mission to preserve them, so that the legacy of the original films will somehow last beyond Lucas's own short-sightedness.

Tuesday, November 07, 2006

Restoration versus Cartoonization

Here is a slightly revised version of comments I posted to Amazon about the 2006 reissue of Star Wars on DVD, with the 1977 edition included as a "bonus disc." I gave it 3 out of 5 stars. It is not everything I feel I want to say on the subject, and actually needs to be shorter rather than longer to achieve that goal. But is starting to turn into an essay in its own right.


I am one of those geeks who was ten years old when Star Wars came out (note: it was not originally called "Episode IV.") I watched it in the theater perhaps a dozen times. It was the most amazing thing I had ever seen.

This release contains two DVDs: the version that Lucas has been tinkering with, and on a bonus disc, the original movie in 4:3 letterbox, taken from the best-available videodisc masters.

About that "tinkering." The 2004 version of Episode 4 looks, for the most part, quite gorgeous. The _restoration_ that Lucasfilm did is impressive: the blacks are blacker, the whites whiter, the color richer, the contrast improved all around, and the soundtrack is great. The dirt and scratches are gone, the shaky color very solid.

However, at some point Lucas crossed over from "restoration" into making a new movie. That's fine; he has the right to do so. But for him to say that the original Star Wars is not really what he had in mind, when it was one of the most famous and popular movies in history and became entrenched in the culture -- well, I find that weirdly arrogant. And when he says, in effect, that his altered version _is_ "Star Wars" and the original _isn't_ -- well, hmmm. A movie is a historical artifact. There's a difference between preservation and tinkering. Mainly, that tinkering mostly is there to gratify the artist, while preservation serves the art -- and the fans of the art. Artistic creation is a fragile and uncertain process. For Lucas to assume that he knew exactly what made Star Wars great and presume to make it better misstates the amount of control that artists actually have over how their creations are received by the public.

Where you draw this line is slightly unclear. I think the cleanup of the backgrounds is fine. The improvement of the soundtrack to use the latest technology available is wonderful. I'm not sure why Kenobi's weird cry that drives away the sandpeople was replaced with a slightly different weird cry. He's redone some of the explosions twice now. I'm not sure why they all became pink in 2004. Why did he feel that Alderaan and the Death Star needed to explode in giant rings? Why do the lightsabers now give off blinding green flashes when they collide? Most of these changes are not improvements; they are just distractions. They tend to stick out like a sore thumb to fans who have seen the movie many times.

But there are more than just the small arbitrary changes, I'm not really happy with the addition of dinosaur-like creatures in Mos Eisley, and I'm really not happy with giving these little bits and pieces cartoonish _sound_ that sounds like it came from Episode 1. Star Wars, the original, had a different tone, a different mood, than Empire and Jedi and the whole prequel trilogy. It's a little darker. It's a world where rebels and stormtroopers are violently killed and Han Solo shoots first. Lucas is free to make that world happier and more cartoonish in his later films, but altering, and in some cases censoring violence from the original, is a very strange thing to do.

So, although I really admire the improvements to the image and sound in the 2004 edition, I generally prefer watching the original 1977 cut. For that, I'm sorry to report that the digital transfer, from the videodisc master, is only adequate. Many fans are griping that it is 4:3 instead of anamorphic 19:9. This means it isn't full-width on a widescreen TV. That doesn't particularly bother me, but I'm viewing it on an old TV, not a widescreen TV. It looks like a very good analog videotape, but we've recently -- and rather abruptly, in terms of years -- gotten used to DVDs of films that were transferred to the digital realm and mastered there. The audio is good, but again we now tend to compare it to all-digital productions. Negative comments on Amazon about the black level are on the mark; some of the space scenes make black outer space look brown, or gray. This is particularly evident when we see Vader's helmet in his tie fighter; his helmet is blacker than the black background of space. But that is true in the original film; it was noticeable in the theater on opening day. A number of the desert scenes have poor contrast and faded color; some of this is film deterioration, and some is because the contrast and color in some of the outdoor Tatooine scenes were never that good to begin with. There are noticeable scratches. The color is shaky in some scenes, particularly outdoor scenes, and it flickers a bit. It looks like a film that is considerably older than it is. I've seen older restored films that look a lot better than this one does.

Here's the thing: it didn't have to be this way. We would have considered it to be a fairly good video rendering at one point in time. But our expectations have been raised considerably -- and, in fact, Lucas himself is largely responsible for raising those expectations, because of his constant embrace of new technology for delivering films to audiences. The 2004 DVD release has all those black level problems fixed. There aren't any visible scratches. The contrast is excellent. The colors are vivid. The missing dialog is restored.

So which version do I want to watch? Well, the answer is neither. I want to watch a version that doesn't exist: call it "Star Wars: the Nostalgia Edition." That version would be fully _restored_, but not _altered_. And it would have things like Han Solo's scene with Jabba available as a "deleted scene" special feature, along with all the other so-called lost footage such as the scene at Anchorhead, which introduces (and makes sense of) Luke's relationship with Biggs. It would have been presented with respect for the original work, not as a bonus disc given no special treatment. Instead, Lucas has disowned that picture.

And here's the thing: I'd be shocked if Lucas didn't have every scene, unaltered, from the first film in beautifully restored digital form. After all, wouldn't a restored original film have been the starting point for this whole process of alteration that led to the 1997 and 2004 versions?

So, I'm not actually advocating that we give up the advances in restoration that are evident in the 2004 release. But don't bother with the tinkering. The fans don't care about it. Keep the original death star cell block footage. Sure, the tunnel behind the actors is obviously a matte painting, and the perspective is off kilter when the camera angle changes. But you know what? I saw Star Wars at least ten times in the theater and I never noticed the problem -- because it isn't a "problem," it's an artifact of the budget and technology that existed in 1977. You notice it if you are looking at the frame cynically, not when you are immersed in the story. It might be an irritant to Lucas now, but it is the effect we grew up with. For the "Nostalgia Edition," let Star Wars be Star Wars.

Then Lucas can go on with his director's cuts, turning Star Wars into a 3-D cartoon until the sun explodes, for all I care. Just don't make me watch Greedo shoot first!

Monday, November 06, 2006

Burning a DVD on Tiger

So, I have an unencrypted VIDEO_TS folder (a non-commercial video project) obtained via download. I know this is a valid set of files because Apple's DVD Player application can open the VIDEO_TS folder and play it like a DVD. I've got a Mac Mini (G4, not Intel processor) with a SuperDrive. I've got a few blank DVD-R discs. So how do I burn it?

Allegedly, you can make a disk image containing this folder using Apple's disk utility. But if you don't do it right, you'll have an HFS image, not a UDF image. Allegedly, to do it right you tell Disk Utility to make a file system that is a "DVD/CD Master." This results in a .cdr file instead of a .img file. Allegedly, you can then burn that to make your DVD.

There's only one problem: this process doesn't work. I burned 4 coasters before I gave up. What I'm getting is an HFS file system on a burned DVD-R. The Mac can read it, but my DVD player can't, and neither can my PC. By comparison, if I put a commercial DVD in the Mac, the Finder's "Get Info" shows the file system type as UDF.

So, how do I get a usable UDF file system burned? Preferably, with no, or at least little in the way of extra software?

It's surprisingly hard to find a coherent explanation online. Apple provides some command-line utilities (hdiutil and drutil). The hdiutil command can do a huge variety of things. The drutil command seems more targeted around burning to and providing information from CD and DVD writers. The GUI Disk Utility seems to be mainly built on top of hdiutil. The descriptions of these command-line tools make it sound as if, between them, they ought to be able to do everything you need.

There's only one problem -- as far as I can tell, they can't!

There are two parts to the problem.

The first part of the problem is that I need to get a real UDF file system written. The GUI Disk Utility does not do this when I tell it to burn a .cdr file. It seems like perhaps hdiutil or drutil ought to be able to do it, although each of these has such a plethora of command-line options that it is not exactly obvious at first glance how to do this. When I say a plethora, I mean a plethora. And how often do you get to use the word "plethora" in a sentence and mean it?

The second part of the problem is that even if I can get a UDF file system written, apparently set-top DVD players aren't capable of playing just any old UDF file system. The file system has to use 8.3 uppercase filenames. And, apparently, the .VOB and other files have to be laid out in a specific order, in contiguous sectors, and possibly with padding. I don't know (and don't want to know) all the grisly details, but it appears that the players really just get a starting sector from the filesystem and then play video data from contiguous sectors. I suppose this makes some sense when you consider that you don't want the DVD player to have to seek to a new point on the disk in the middle of a chapter.

So, to build a UDF file system image which obeys these rules, it appears that the only option, besides using Toast or some other commercial tool, is to use a tool called mkisofs, passing it the -dvd-video option. You start with a directory containing your VIDEO_TS directory (and optional AUDIO_TS and JACKET_P directories and perhaps extra files) and use the tool like this:

mkisofs -dvd-video -o test.iso test/

However, this tool is not bundled with MacOS X 10.4, like the hdiutil and drutil tools are.

So, you can just download a binary, right?

Well, mkisofs doesn't seem to be available as a binary for Tiger on PowerPC through either DarwinPorts or Fink, although a source build looks feasible. It also seems like you can't get mkisofs by itself. Instead, it comes as part of a package called cdrtools. Or, you might see it as part of the port of xcdroast. I've used xcdroast under Linux, and it works fine, but I don't want to have to build or install that whole toolchain to get that one command-line tool. My Mac Mini is a bit lacking on both hard drive space and processing power. Although I've used them before, I have not yet installed either the developer tools, DarwinPorts, or Fink onto my Mac Mini. So I decided to try another approach.

I thought maybe I could use the mkisofs tool under Linux, since I already have the VIDEO_TS directory in question on a FAT32 partition on my PC. My PC is configured to dual boot Windows 2000 and Fedora. Allegedly I could mount the FAT32 file system under Fedora and run mkisofs to build the image file, then send it to my Mac, right?

Well, that got me nowhere. The mkisofs tool failed with a generic error message: "Unable to make a DVD-Video image." It might also have something to do with the way filename case is handled; under Windows 2000 the filenames are all uppercase, but when mounted under Fedora, the ls command shows the filenames as all lowercase. There's probably a solution to this, but again I was just trying to get something done, so I decided to look for another way.

After many Google searches I came across a little application called DVD Imager for MacOS X, available here:

This application is basically an AppleScript UI wrapped around a binary of the mkisofs tool. (I am not sure whether it is a universal binary which would run on Intel-based Macs or not). You can drag and drop your VIDEO_TS directory onto the UI and then tell the tool where to put the resulting image file. The tool produced an .img file (not .dmg), which the Finder calls an NDIF disk image. What happened when I mounted it? It showed up as a UDF file system. That looked promising, so I burned it to a DVD-R using the Apple Disk Utility.

It appears to work fine on my Sony DVD player, so now I have my disk image file!

However, I didn't _really_ want a GUI with limited options; I'm accustomed to working with command line tools. The GUI tool created its own temp directory for assembling the file system and didn't pick up the JACKET_P directory next to the VIDEO_TS directory I dragged onto it. Can't I tell it exactly what I want?

Yes! The little .app package actually contains the mkisofs tool. You can right-click on it and choose "Show Package Contents." Inside the package, under Contents/Resources, is the mkisofs executable file. You can copy it and paste it elsewhere. For now I just put the mkisofs tool in my working directory, rather than with other Apple-supplied command-line tools. I ran it like this:

./mkisofs -dvd-video -udf -V PROJECT -o project.iso ./PROJECT

(where PROJECT is the name I want for the DVD filesystem, and ./PROJECT is the directory containing the VIDEO_TS and JACKET_P directories), and project.iso is the name of the image I want to generate.

It looks like it worked! It created another mountable file system image file, which the Finder will happily mount. The next thing to do was to try burning the DVD from the command line as well:

drutil burn -udf ./project.iso

And, it worked as well! So there you are. Note that I didn't have to give it any special options; it just did the right thing. If you have an external DVD burner or more than one drive you might have to provide some more options to tell drutil exactly what you want to do. If you want to add more special content to your DVD to be available on MacOS X or Windows or both, you might have to look into making some kind of hybrid file system. I'm sure that's all possible but I'm not going to go there for now!

I hope this information has saved you a little bit of time and effort!

Friday, November 03, 2006

Episode IV

So, we watched the original 1977 Star Wars on DVD. This edition was not called "Episode IV." I'm sorry to report that the digital transfer, from the videodisc master, is only adequate. Many fans are griping that it is 4:3 instead of anamorphic 19:9. This means it isn't full-width on a widescreen TV. That doesn't particularly bother me, but I'm viewing it on an old TV, not a widescreen TV.

Some of the Amazon reviewers make it sound like the transfer is horrible. It isn't. It looks like a very good analog videotape, but we've recently -- and rather abruptly, in terms of years -- gotten used to DVDs of films that were transferred to the digital realm and mastered there. It's actually taken from the master for the analog videodisc.


Here's the thing: it didn't have to be this way. We would have considered it to be a fairly good video rendering at one point in time. But our expectations have been raised considerably -- in fact, largely by Lucas himself. The 2004 DVD release has all those black level problems fixed. There aren't any visible scratches. The contrast is excellent. The colors are vivid. The missing dialog is restored. And here's the thing: I'd be shocked if Lucas didn't have every scene, unaltered, from the first film in beautifully restored digital form; after all, wouldn't a restored original film have been the starting point for this whole process of remastering that led to the 1997 and 2004 versions?

So, I'm not actually advocating that we give up the advances in restoration that are evident in the 2004 release. The restoration work is great. But why can't we buy a restored version without any of the "artistic" changes? No new matte paintings, no new animated creatures carrying stormtroopers, no altered creatures in the cantina in Mos Eisley. No cartoonish sound effects from episode 1 need to be added. We just want a beautifully restored version of the original film, with the best soundtrack possible, with cheesy aliens here and there intact, and the occasional poorly rendered special effect. We could debate just what constitutes restoration: leftover black bars in a matte should probably go, but in the scene where a rebel scout is tracking the Millenium Falcon as it approaches the rebel base, but no ship is visible should the ship be added? For the restored original, the "Nostalgia Edition," I think the answer should be "no." We don't need green flashes when the red and blue lightsabers collide. We don't need pink explosions or strange rings added in the explosions, which Lucas inexplicably decided look more realistic. Keep the original death star cell block footage. Sure, the tunnel behind the actors is obviously a matte painting, and the perspective is off kilter when the camera angle changes. But you know what? I saw Star Wars at least ten times in the theater and I never noticed the problem -- because it isn't a "problem," it's an artifact of the budget and technology that existed in 1977. It might be an irritant to Lucas, but it is the effect we grew up with. For the "Nostalgia Edition," let Star Wars be Star Wars.


Followup to "Through the Binoculars"

There's a fan-produced documentary called "Deleted Magic" which assembles a variety of the Star Wars deleted scenes, together with outtakes, and puts them in context with bits of the release film. It's a labor of love.

After watching portions of this, I've come to realize that no, I never did see the "deleted scenes." Why? Because they are pretty different from my "memories" of them. These memories are probably conflated from my reading of the novelization, hearing audio versions, and seeing images from the deleted scenes (for example, the Star Wars bubblegum cards had some images that weren't in the actual film release, such as shots of Luke wearing his poncho and shots of Biggs on Tatooine).

Many of the things in the deleted scenes are actually referenced in scenes in the theatrical release: for example, the blue milk shows up on the dinner table, Luke makes reference to Toschi Station and Anchorhead, and also talks about Biggs. I probably conflated the exploding red droid with the one Luke calls "Treadwell" in the unused footage.

What's frustrating is that it has been left up to the fans, who love this work, to preserve the lost footage and put it in context. That should have been Lucas's job. If he had an actual appreciation for what was great about the original Star Wars, it would have been a labor of love for him, something he relished doing. "Deleted Magic" would have made up the bonus disc to the "Nostalgia Release" of Episode IV. Instead we have Lucas redesigning CGI explosions, and then redesigning them again.

I recommend "Deleted Magic," if you can find it. (Hint: Google "star wars deleted magic.") You'll get to see Koo Stark as one of Luke's friends, the voice of David Prowse as Darth Vader, and other amazing bits and pieces!

Star Wars: Through the Binoculars

So, I've bought a copy of "Star Wars: A New Hope" on DVD. This is the first DVD version I've purchased. This version comes with a bonus disc that contains the 1977 theatrical release in 4:3 letterbox. We'll watch it tonight. I don't think I've seen Star Wars since the 1997 theatrical re-release. And I wasn't a terribly big fan of the second trilogy; I still haven't seen episode 3, although I guess I'll have to bite the bullet sometime.

For many years I have had some confused thoughts about Star Wars. It certainly hasn't helped that Lucas has released so many slightly different tweaked versions. I'm not going to talk about his more dubious changes, like the alterations to the scene where Han kills Greedo with a blaster. There were a number of minor changes prior to the 1997 Special Edition release. Some things are obvious, like the addition of "Episode 1: A New Hope" to the opening crawl, which was not there in the 1977 release. There were other changes, mostly very minor, such as a missing line of dialog. One of the more noticeable changes features a stormtrooper yelling "close the blast doors!" and then after Han Solo escapes through them, yelling "open the blast doors! open the blast doors!" In some versions "close the blast doors!" is missing, which fails to set up this minor joke.

I'm not talking about those changes. I'm talking about the lost scenes. Scenes that were shot, but perhaps never finished, and which according to every source I've seen never made it into the released film.

Is it possible that some of the "lost scenes" actually made it into prints of Star Wars shown in theaters? Or have I just managed to delude myself about scenes I remember seeing in 1977?

I first saw the movie at, I believe, the Millcreek Mall in Erie. Erie, PA was not one of the theaters that got Star Wars on opening day. I don't know what the date was. To the best of my recollection I didn't know anything about Star Wars prior to seeing the first film. It was actually my uncle Ted, who was, I believe, 19 at the time, who took my brother and I to see the movie. At least, that's the way I remember it. And I remember being very, very excited about the film -- absolutely blown away -- and afterwards laughing, yelling, just screaming in utter amazement at what I had seen.

I went on to see Star Wars many more times; I think I may have seen it as many as ten or twelve times, since it ran for over a year at the Eastway theater in Harborcreek. I saw a lot of dollar matinees. I noticed, and recall thinking _at the time_, that the film seemed to have been slightly recut.

There are a number of these "lost scenes." In one of them, Luke observes a sparkle in the sky, and sees ships in orbit through his binoculars. He urges a small droid with treads to come with him, but it blows up and he abandons it, thus setting up the need for new droids. Luke's friends insist that the ships are just docking, but Luke tries unsuccessfully to convince them that he has seen shooting. He has a conversation with Biggs about the rebellion, setting up the scene where he meets Biggs just before the final battle (this scene was restored for the recent DVD release and, perhaps, for the 1997 theatrical release, although I am not certain about the latter).

If that scene never appeared in the theatrical release, why do I have such a clear memory of Luke examining the sparkle in the sky through his binoculars? Why do I vividly recall the droid blowing up? And why do I remember thinking, when I saw the movie again later, that the scene with Biggs was mysteriously missing?

There are other "lost scenes" that I have no memory of whatsoever, such as Han Solo flirting with a woman in the cantina.

The Star Wars Cut Scenes web site here claims that the scene I'm describing never made it into a release of the film, and so does Wikipedia.

I was a big fan: I owned a vinyl record dramatization with audio from the film called "The Story of Star Wars," the John Williams soundtrack, complete sets of bubblegum cards, and even a wonderful LP by the Electric Moog Orchestra with an energetic and funny rendition of the Cantina Band theme. I read the novel; I thought it was amazing that George Lucas could write novels, but now I realize that it was ghost-written by Alan Dean Foster, who wrote the spinoff Splinter of the Mind's Eye. Which I also read, naturally.

Since I was also exposed to the novelization, the comics, the trading cards, and probably other materials such as publicity photos and magazines -- and that scene is either described or alluded to in all those sources -- I'll acknowledge that it is possible, even likely, that my memory has become muddled, and I didn't actually see the scenes in the theater. But I can't shake the feeling that the first print I saw was _different_. Allegedly, a scene on Tatooine involving Aunt Beru filling a tupperware-like container up with some kind of blue milk also never made it into the film. Does anyone remember the blue milk? I do! And I don't think I've through about that blue milk in 30 years.

I'm looking through Luke's binoculars backwards. I'm looking back thirty years at my younger, smaller self. It is hard to admit that I'm now middle-aged. I'll never have that visual acuity back. The colors will never be as bright. The soundtrack will deafen me, instead of exciting me. Every time I see the movie, the thrills will be a little less thrilling. It's a hard pill to swallow. And I can't even trust that my own memories are correct.

Well, we'll see the original tonight. On Amazon fans are complaining that the 1977 film re-release was not given the respect that it deserved. It's a 4:3 letterbox. This leaves viewers with widescreen TVs with bars on the left and right edges of the film. What we have is a transfer of a version that was made for videodisc. No features (fine by me; I'm not really interested in interviews with Lucas), but no real restoration effort, and, sadly, no deleted scenes. Lucas missed an opportunity to create "Star Wars: the Nostalgia Edition," targeted at... well, me.

The Nostalgia Edition would be the 1977 theatrical release given the Criterion Collection treatment. No bullshit, just a restoration, getting the finest possible quality out of what is already there. Lucas doesn't have the inclination to do it; he thinks the movie doesn't represent his "vision" of the movie. That's weirdly arrogant even for a film director. Lucas doesn't seem to understand anymore, and perhaps never did understand, what made the original Star Wars mean so much to its original fans. (Hint: it isn't the quality of the special effects or gorgeous digital images). It's a little something called exuberance; it's about a world that looks rusty and lived-in. It's about the brilliant artists and designers and painters who lovingly made magic on a shoestring, and it looks gritty.

The deleted scenes exist, in various degrees of degraded quality, because apparently no one has been interested in preserving them. That shows you where Lucas's heart really lies. The scenes could have been on the disc I've got in my hand. They're not. Lucas doesn't seem to have an interest in preserving and restoring his original creation, which is what those of us who went to see his movie in 1977 love and remember. There was nothing wrong with the original Star Wars. It never needed extra digital imaging. The explosions didn't need to be re-animated; Greedo didn't need to shoot first. It just needed to be treated with love and respect. And I'd dearly love to see the lost scenes brought back to life and made available to the original Star Wars fans, before they deteriorate any more -- and before we do.

Note: allegedly Lucas is preparing some kind of massive restored edition for release in 2007. A 3D verion -- 3D! -- of the original trilogy. It sounds nauseating. And his "archival" version of the originals with lots of extras, perhaps including deleted scenes, which will probably be released on Blu-ray disc. Let's not go there. Now I really do feel old, and so does my DVD player!