Meta Programming

A metaprogram is a program that manipulates other programs (or itself) as its data. The canonical example is a compiler.

Metaprogramming is the act of writing metaprograms.

Examples of metaprograms are: Most uses of the term these days refer to a particular special case. Metaprogramming is what you do with CeePlusPlus templates or CommonLisp macros: defining the content of (part of) a program by writing code that generates it. The code in question usually consists of a template and some instructions on how to fill it in; but the instructions can potentially be rather complicated. With CommonLisp macros, you can transform or generate code in arbitrarily sophisticated ways. C++ templates are rather more painful, but also very powerful. Other languages that support this kind of metaprogramming pretty well include Scheme and Dylan.

Does ForthLanguage count? In Forth you generate the bytecode rather than source, but other than that it works in a similar way.

I don't know who wrote the above, but I think it is wrong. That is the subset of MetaProgramming called GenerativeProgramming. MetaProgramming in CommonLisp is about the MetaObjectProtocol. In Smalltalk, it is about hacking DoesNotUnderstand: and subclassing Behavior. It is about programming at the "meta" level, i.e. about changing your interpreter or changing how code is compiled. -- RalphJohnson

Some of us consider the above to be ThreeStarProgramming

In TeX, you can change category codes to change the way the file is read. You could do this to make it read and interpret CSV files or XML files, or whatever.

A lot of languages do not allow MetaProgramming in the meaning that RalphJohnson explained. This is why there exist such a lot of PreCompilers (e.g. AspectJay). --HaskoHeinecke

The ForthLanguage allows MetaProgramming. A programmer can extend or change the compiler.

Forth allows changing and extending the compiler, as well as code generation, reading with a different syntax, and a lot of other things that were not actually invented until later programming languages were invented to invent those new features!

MetaProgramming seems closely related to the more powerful notions of CategoryReflection. How are they distinct?

I think it's the other way around: both GenerativeProgramming and CategoryReflection are subsets of MetaProgramming. CategoryReflection is probably simpler when the language allows it, but GenerativeProgramming is probably more efficient since the work is usually done at compile-time. --SamuelGelineau?

Two other interesting questions are: Why is metaprogramming done, and should it be done?

Note: In this section, the discussion is limited to "application-specific" metaprogramming, where a project team or organization hacks up some custom language, along with implementing a translator for it (the translator can be anything from a runtime interpreter, macro hackery (either for an internal preprocessor such as CPP or an external one such as EmFour?), or an external custom tool (whether a quick-and-dirty Perl script or a complete compiler). Writing a translator for a general purpose programming language is outside the scope of this discussion--we all know why this is done, and agree that it should be. :)

Why is it done?

Is meta-programming a good idea?

Depends on two things: a) how domain-specific the meta-language is, and b) how complex it is. Generally, domain-specific meta-languages are more justifiable (if you need domain independence, there are many fine choices available off-the-shelf; designed by folks whose domain expertise is programming languages), and simpler is better.

However, there are many StumblingBlocksForDomainSpecificLanguages (remainder of content moved there).

The highest form of metaprogramming is the MetalinguisticAbstraction?, usually implemented with a MetaCircularEvaluator. The StructureAndInterpretationOfComputerPrograms book has a great section on this. Essentially, it moves language design decisions to the programmer from the language designer. Whether this is good or bad depends entirely on the programmer in question.

The basic problem is that any general-purpose programming language without a metaprogramming model that is designed to be as pleasant and expressive as the language itself will eventually frustrate the user with it's limitations. No-one can design the perfect language, and yet allowing users to extend the language in generic ways is avoided with a BondageAndDiscipline mentality. Consider how JavaLanguage lasted so long without GenericObjects? (practically a necessity for typesafe OOP) and no usable metaprogramming model to allow users to do it themselves. Consider how popular CodeGeneration and Reflection are in CsharpLanguage - effectively a substitute for metaprogramming - although it relies on the language (or the codegen system) being expressive enough to prevent the developer from having to manipulate the actual codegenned (and thus unstable) code.

Even CeePlusPlus templates were made with a BondageAndDiscipline mindset. They were intended to be a limited subset of macros primarily designed for allowing type-safe but (compile-time) dynamically typed code... and very little else without using convoluted, hidden, functionality. How does one iterate across the members of a class in order to define a new class that wraps all of the existing members of the previous? That's a fairly standard action in OOP languages, and yet rarely provided within the language's metaprogramming facilities. Or create an enum and a corresponding list of strings where the contents of the string are identical to the symbolic names of the Enum values, without violating OnceAndOnlyOnce.

This is why the LispLanguage has remained in use, dispite being ugly, anachronistic, and divided - simply because the LispLanguage community has always focussed on avoiding confining abstractions and BondageAndDiscipline mentalities. From this perspective, even PythonLanguage is a BondageAndDiscipline concept, as GuidoVanRossum fought long and hard against adding syntax that would allow for user-defined block and flow-control structures.

Or maybe the reason LispLanguage has remained in use is because you can do procedural programming in Lisp (you can easily create side effects since it isn't pure).

Unfortunately, it doesn't look like this mentality will change in mainstream programming - CsharpLanguage version 3.0 will further avoid a non-painful MetaProgramming approach by adding everything but the kitchen sink to the language. --MartinZarate


View edit of February 7, 2012 or FindPage with title or text search