Awk one-liners

Peteris Krumins has written a fine little book Awk One-Liners Explained. It’s just 58 pages, and it’s an easy read.

As I commented here, I typically try to master the languages I use. But for some languages, like awk and sed, it makes sense to learn just a small, powerful subset. (The larger a language is, the harder it can be to just learn part of it because the features intertwine.) Krumins’ book would be good for someone looking to learn just a little awk rather than wanting to explore every dark corner of the language.

Awk One-Liners Explained is exactly what title would lead you to expect. It has 70 awk one-liners along with a commentary on each. Some of the one-liners solve common specific problems, such as converting between Windows and Unix line endings. Most of the one-liners are solutions to general types of problems rather than code anyone is likely to run verbatim. For example, one of the one-liners is

Change “scarlet” or “ruby” or “puce” to “red.”

I doubt anybody has ever had to solve that exact problem, but it’s not hard to imagine wanting to do something similar.

Because the book is entirely about one-line programs, it doesn’t cover how to write complex programs in awk. That’s perfect for me. If something takes more than one line of awk, I probably don’t want to use awk. I use awk for quick file filtering. If a task requires writing several lines of code, I’d use Python.

You can get an idea of the style of the book by reading the author’s blog post Famous Awk One-Liners Explained, Part I: File Spacing, Numbering and Calculations.


If you’d like to learn the basics sed and awk by receiving one tip per day, you can follow @SedAwkTip on Twitter. If you don’t use Twitter, you can subscribe via RSS.

Read More

Code to slice open a Menger sponge

Last month the New York Times ran a story about a sculpture based on cutting open a “Menger sponge,” a shape formed by recursively cutting holes through a cube. All the holes are rectangular, but when you cut the sponge open at an angle, you see six-pointed stars.

Here are some better photos, including both a physical model and a computer animation. Thanks to Mike Croucher for the link.

I’ve written some Python code to take slices of a Menger sponge. Here’s a sample output.


Read More

Gritty coordinate systems

Check out The Calculus of Grit by Venkat Rao. This article is somewhat similar to my Jack of all trades post but goes into far more depth. It is about 20 times longer than my article and well worth reading.

Venkat Rao compares discipline boundaries to extrinsic coordinates and one’s career to intrinsic coordinates. You don’t have to understand the mathematical significance of these terms to read The Calculus of Grit, though it helps. Extrinsic coordinates describe a surface as it sits inside a larger space. Intrinsic coordinates describe a surface as it would be experienced by a bug crawling around on it. A line that is straight in one coordinate system will typically not be straight in the other coordinate system.

For some background on the technical use of the term “grit,” see the Psychology Today article The Winning Edge. (The math in the first paragraph is annoying because the superscripts were stripped in the online version of the article. It says, for example, 32 + 42 = 52.)

Thanks to DavidC for pointing out Venkat Rao’s post.

Read More

When rejected thoughts coming back

I was struck by this quote from Ralph Waldo Emerson, even though I’m not sure I understand what he meant.

In every work of genius, we recognize our own rejected thoughts: they come back to us with a certain alienated majesty.

Maybe Emerson was referring to that why-didn’t-I-think-of-that feeling when you see that someone else connected one or two more dots than you did. You thought about a challenge, and maybe you were close to resolving it, but you lacked a key insight to pull it all together. You decided your approach wouldn’t work, but someone did make it work.

If that’s what Emerson had in mind, it’s puzzling that he speaks of “every work of genius.” It would be incredibly arrogant to think that you almost came up with every great idea you see. Maybe he means that we recognize genius best when it relates to something we’ve struggled with.

What do you think Emerson meant? When have your rejected ideas come back to you?

Read More

Wires and air

Tim O’Reilly recalls a remarkable prediction:

I remember, must be 20 or 25 years ago, hearing a talk given by Nick Negroponte of the MIT Media Lab, in which he made a prediction … Everything that today goes through wires will go through the air, and everything that goes through the air today will go through wires.

From an interview with John Donovan, CTO of AT&T.

I find it interesting that Negroponte didn’t just predict a move to wireless but that there would be an exchange, wireless things becoming wired.

Read More

Microsoft developers need not apply

Last night I shared the article Why we don’t hire .NET programmers by David Barrett on Twitter. Some of the responses I got said the article was

  • A load of rubbish
  • Amazingly successful trolling
  • So narrow minded it hurts

The article contains some provocative criticisms of Microsoft’s tool stack. But it also has high praise for the same tools.

Here’s what I believe the article is saying at its core:

The Microsoft tool stack is not optimized for the kind of development we want to do, so we doubt that people who have chosen to make a career using that tool stack will be a good fit for us.

I’ll let David Barrett decide who is or is not a good fit for his company, but this much seems undeniable: Microsoft’s tools are optimized for a certain market. All tools are optimized for some market, at least tools that are successful. I would take Microsoft’s enormous financial success as evidence that their tools are indeed optimized for some market, and a large market at that. The article says

[.NET is] the most modern platform for application development on the planet.  Microsoft has always produced the best tools for building internal business applications, and .NET is their masterpiece.  There’s a reason why they own that space; they earned it.  That space employs millions of people, and those people are unquestionably the masters at what they do.

That’s quite an endorsement. Microsoft should quote that in their marketing literature.

I assume .NET developers don’t take offense to what Barrett says .NET does well but rather what he thinks it does poorly.

Barrett’s main criticism of .NET is that it makes it easier to solve common problems at the expense of making it harder to solve uncommon problems. And that seems clear. He makes his point in an inflammatory way — implying that Microsoft wants to entrap developers, and that .NET developers are happy to let Microsoft think for them — but I agree that Microsoft has designed its tools for developers working on common problems. They’ve aimed at the profitable heart of the developer market.

I don’t agree with Barrett’s argument that start-ups are necessarily working on unusual problems that are not well served by Microsoft tools. A start-up may have a unique product or service and yet have mainstream software needs. For example, suppose you develop a kit that lets people run their car on oatmeal. A web site for selling your kits might not be very different from a web site selling T-shirts.

Related post:

Usability versus composability
Termites and programmers

Read More

Jack of all trades?

Whether a person is a “jack of all trades and a master of none” depends on how you define trades. The same person could be a dilettante or a specialist depending on your mental categories.

Take an expert programmer back in time 100 years. What are his skills? Maybe he’s pretty good at math. He has good general problem solving skills, especially logic. He has dabbled a little in linguistics, physics, psychology, business, and art. He has an interesting assortment of knowledge, but he’s not a master of any recognized trade.

Is a manager a master of one trade or a jack of several trades? Obviously if you recognize management as a profession, then someone who is good at it is a master of that trade. But if you don’t have the mental category of manager, what is a manager good at? She knows a little psychology, a little sociology, a little math, she has good communication skills, etc. But she’s a jack of all trades and master of none unless you have a name for her trade.

Calling someone a jack of all trades could be a way of saying that you don’t have a mental category to hold what they do.

Related post:

Too much time on their hands

Read More

Odd numbers in odd bases

My wife told me about someone on the radio yesterday discussing voluntary water rationing. People in odd-numbered houses are being asked to water their yards only on odd-numbered days. This person said “I suppose they’re talking about the last digit.”

When she told me about this, my first two thoughts were:

  1. Yes, that’s what it means to be odd.
  2. Nearly every house number in suburban Houston starts with 1, so going by first digit would be a bad idea.

Strictly speaking, it’s a theorem that odd numbers are those that end in odd digits. The definition of an odd number is one that leaves a remainder of 1 when divided by 2. And in base 10, a number is odd if and only if it ends in an odd digit.

But what if you were using a base other than 10? If the base is even, then a number is odd if and only if the last digit is odd, just like base 10. But what if you’re using an odd base, say base 7? Then the theorem doesn’t hold. For example the number 122 in base 7 is odd, and the number 33 is even. And it’s not just the opposite of the rule for base 10 because 43 is also odd in base 7.

In an odd base, a number is odd iff it has an odd number of odd digits.

(In case you haven’t seen “iff” before, it’s an abbreviation for “if and only if.”)

So, for example, in base 7, the number 642341 is even because it contains two odd digits. And the number 744017 in base 9 is odd because it has three odd digits.

Why does this rule work? Suppose, for example, you have a 4-digit number number pqrs in base b where b is odd. Then pqrs represents

p b3 + q b2 + r b + s

All the powers of b are odd, so a number like p times a power of b is odd iff p is odd. So every odd digit in the number contributes an odd number to the sum that expands what the number means. Even digits contribute even terms. A sum is odd iff it has an odd number of odd terms, so a number in an odd base is odd iff it has an odd number of odd digits.

Similar posts:

Divisibility by 7
Divisibility rules in hex
Casting out z’s

Read More