Channels ▼

Open Source

Porting D to the Mac


The D language is statically typed and compiles directly to machine code. It's multiparadigm, supporting many programming styles: imperative, object oriented, and metaprogramming. It's a member of the C syntax family, and its appearance is very similar to that of C++. For a quick comparison of the features, see this comparison of D with C, C++, C# and Java.


Multiple compilers exist for the D programming language, including the Digital Mars compiler (dmd), the gnu compiler (gdc), the upcoming ldc compiler, and even a .net D compiler is in the works. They are all based on the same open source front end code.



From a text window, the Mac is just another Unix machine. I put all the source code to the compiler on it and set about trying to compile it. Most of that consisted of finding all the conditional compilation:

#if linux

and changing it to:

#if linux || __APPLE__


The first big problem is that dmd can generate both Intel OMF and ELF object file formats, but the Mac used the Mach-O format. The best way to learn a file format is to write a dumper for it (called "dumpobj" for object files). Object file specs are usually wrong in some detail or other, and the mach-o spec is no exception. The Mac comes with an object file dumper called otool. Unfortunately, otool doesn't give a clue as to the structure of the object file, nor will it disassemble any code that is not in a __text segment. So I also got the Digital Mars disassembler, obj2asm, converted to work with mach-o files.



Some other differences:




It Works -- Sort Of

I had "hello world" working, but I was using D as a C compiler to do it. All that existed was a main() function. There was a long way to go -- functions, data, the runtime library, the test suite, etc. First I'd try and build the runtime library.








Finally, tonight, I got the runtime to start up, run, and shut down successfully. Next, I'll try to get the test suite to run.

It Works!

The short version -- it's done and out!

I had thought retargeting the D programming language to the Mac was just an object file format change -- a week, maybe two. It turned out to be six weeks. I feel like Yosemite Sam walking through a mine field, managing to step on every single mine in it.

So let's pick up where I left off at the end of my last entry, where I had just gotten the library to compile and a couple sample pieces of code to link to it and run. Now it was time to run the test suite.

It's essentially impossible to develop a compiler without some sort of test suite. The one I use is an ad-hoc collection of every fixed bug and other assortments of things. The beauty of putting in every fixed bug is that the bugs stay fixed, and the quality of the compiler steadily ratchets forward. Over time, this makes for a formidable test suite. If it passes the suite, I know it's at least as good as the previous version, and if it shows to not be, that failing gets added to the suite for next time. (I've discovered that just throwing volumes of code at the compiler is fairly useless as a test suite. The test code has to be crafted to test specific things and verify correct results.)

The first thing that failed was exception handling. But surprisingly, it only took me about an hour to get that to work. Exception handling is complicated and hard to understand, and I expected a tough slog. The EH design in the back end dates back to when it was a 16-bit compiler (Digital Mars C++ was the only C++ compiler to ever implement exception handling on DOS). The support code moved to 32 bit DOS extenders, then Linux, and now OSX with very little change. The OSX support needed to tweak the assembler bits to keep the stack 16-byte aligned. (Exception handling for Win32 is completely different, using Microsoft's Structured Exception Handling scheme.)

The dmd exception handling system is completely independent of g++'s. The two do not interact with each other. D is binary compatible with C++ name mangling and single inheritance, and g++ on OSX uses the same protocol for this as on Linux which dmd is already compatible with, so that was easy.

My next problem was the floating point failed miserably. Some investigation showed that, uniquely, OSX aligns the CPU's 10-byte reals on 16-byte boundaries. This is 6 bytes of pad for each. I don't know the reason for this; Linux uses 12 and Win32 uses 10. It's just that if you have large arrays of reals, it's going to chew up 60% more space. Oh well, it was easy to account for in the code generation.

The worst problem I had was my own fault. The front end is only few years old. But the back end code generator is about 25 years old (it may be the oldest code generator still in professional use!). Although it is well debugged (sporting a thorough test suite), fast, and generates great code, it uses a lot of global variables to communicate. Problem after problem was traced back to the use of global variables. Over time I'd eliminated a lot of them, but there's a lot left. It's hard to change how a function works if there's a back channel of globals passing state around. Globals break encapsulation, making code difficult to understand.

Of course I wouldn't write it that way today, hopefully I've learned something in the last 25 years. You might ask "why not just rewrite it" and certainly that thought comes to mind. The problem is a code generator is about a zillion special cases, most of which interact with each other. Getting that all adjusted, tweaked and working right across the broad spectrum of code that it must generate is years of work. It's not something you throw away and rewrite lightly. But there is some hope. The Mach-O generating part is a lot nicer than the Elf generating part, which itself is much nicer than the very old OMF generator. And if I'm doing open heart surgery on a particular section, I'll refactor it and rely on the test suite to make sure the patient recovers.


Thanks to Sean Kelly for his invaluable help with the more complex OSX system library work. Thanks to Jason House, Andrei Alexandrescu, Bartosz Milewski, Sean Kelly, and Cristian Vlasceanu for reviewing this.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags


<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags


<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

