So the following talk is about a very relevant piece of technology legacy of our human race. The first piece of computer that landed on our moon and actually it became a metric. People started to compare other architectures, other computers in volumes of multiples of processing speed of this computer. It's rocket science but it's even harder it's computer rocket science. So I'm very happy to have Christian Hessmann or Hessie on stage who is actually a rocket scientist and for the computer part we have Michael Steyer who is the founder of the export linus project and has gathered with this project and many others lots and lots of experience around architectures of computers. So please give a warm round of applause for the ultimate Apollo guidance talk. Is this on? Can you hear me? Yes. Welcome to the ultimate Apollo guidance computer talk aka a comprehensive introduction into computer architecture and operating systems and space flight. My name is Michael Steyer and I'm Christian Hessmann. This talk is number six in a series by various people. The idea is to explain as much as possible about a classic computer system in 60 minutes. The Apollo guidance computer is a digital computer designed from scratch specifically for use on board the Apollo spacecraft to support the moon landing between 1969 and 1972. Developed at MIT between 1961 and 1966, a total of 42 AGCs were built at a cost of about $200,000 each. The base clock is about one megahertz, all data is 15 bits and there are two kilowatts of RAM and 36 kilowatts of ROM. It's about the size of a large suitcase, weighs 32 kilograms and consumes about 55 watts. Its user face is a numeric display and keyboard. Some historical context. In the mid 1960s you can just take it off the shelf computer and put it into a spacecraft. The first mini computers were the size of a small fridge, too heavy, too power hungry and too slow for real time scientific calculations. Even though the industry had come a long way since the previous decade. 10 years later, microcomputers with highly integrated circuits started outclassing the AGC hardware in many regards. There are many reasons that make the AGC especially interesting. The architecture is very 60s and feels very alien to us today. The hardware is very innovative for its time. It has some very interesting and unusual peripherals. Its operating system was revolutionary for its time and the mission software has all the bits, too, with the right hardware attached flyer to the moon. In the Apollo program, the Apollo guidance computer was used in two unmanned test missions where it was remote controlled from the ground in three manned test missions and in the seven manned landing missions. Astronauts hated the idea of giving up any control to a computer, but they wanted to be in charge. And while as a fallback most of the mission could also be flown manually, the mission planners got their way. To understand the purpose and the responsibilities of the Apollo guidance computer, we need to first look at the Apollo mission. The core strategy of the Apollo program was instead of landing the complete spacecraft on the moon for which an extremely large rocket would have been required to only land a much smaller lander while the larger part with a few for the way back stays in moon orbit. So the Apollo spacecraft can be separated into the lunar module, the command module and the service module. The Saturn V rocket launches it and three astronauts from Cape Kennedy into Earth orbit. By accelerating at the right time, the trans-lunar injection moves the spacecraft into a so-called free return orbit, but just coasting it would travel around the moon and back to Earth. Right at the beginning of this three-day journey, the command and service module extracts the lunar module and docks with it. By breaking on the far side of the moon, the spacecraft enters a lunar orbit. After two of the astronauts have climbed into the lunar module and after undocking, the lunar module breaks. This is called powered descent and lands. After taking off again, the lunar module rendezvous with the command and service module and the two astronauts from the lunar module climb into the lunar module and the lunar module is jettisoned. The remaining command and service module accelerates at the far side of the moon for a trajectory towards Earth. For entry, only the command module remains. By the way, these excellent visualizations are from Gerrit Owens how the Apollo spacecraft works videos which we can highly recommend. The command and service module and the lunar module each contained one AGC. It was the same hardware, but attached to partially different IO devices and with the software adapted for the specific spacecraft. The astronauts interact with them through the display and keyboard units which are mounted alongside these hundreds of switches. The computer's responsibilities during the mission are to track the position and speed, the so-called state vector of both spacecraft, stabilize the spacecraft's attitude, calculate and control engine burns and monitor or control the Saturn V during launch. In order to understand how the Apollo guidance computer does all this, we'll look at its architecture, the hardware implementation, some of its interesting peripherals, the system software as well as the mission software. The architecture of the AGC can be described as a von Neumann accumulator machine with 15-bit once complement big Indian arithmetic. So we'll talk about the instruction set, the arithmetic model and instruction encoding as well as the memory model, IO operations and counters and finally the interrupt model. Machine code instruction sets very widely, the instruction set of a modern ARM processor which is mainly optimized for runtime performance consists of about 400 instructions. Subleg is a language mostly of academic interest that shows that a single instruction can be enough to solve the same problems as all other Turing-complete languages. While a more complex construction set can achieve higher code density and contribute to higher performance, it also generally means that the CPU will be drastically more complex. A computer from the early 1960s consisted of only a few thousand transistors as opposed to today's billions which is why this is the sweet spot for the AGC. 36 instructions provided just about the performance that was required for the mission. These are the 36 instructions, some load and store instructions, arithmetic and logic, control flow instructions, IO instructions and instructions for dealing with interrupts. The memory model is the cornerstone of the instruction set. Memory consists of 4,096 cells numbered in hexadecimal 0,000 through FFF. Each cell contains a 15-bit word, number between 0 and 7 FFF. Almost all changes in data in memory go through the 15-bit accumulator, also called the A register. A program can copy words between the accumulator and a memory cell but also add, subtract, multiply and divide values as they are moved around. The data in memory can have many meanings depending on how it is interpreted. These values may represent integers while those three words are meant to be decoded as machine code instructions. Code and data in a single address space make the AGC a so-called von Neumann machine. The CPU's program counter, PC, always holds the address of the instruction to be executed next. The load instruction copies the contents of a given memory cell into the accumulator. The PC goes on to the next instruction. The add instruction adds contents of a given memory cell to the accumulator. And the store instruction copies the value in the accumulator into memory at a given location. The generalized version of these instructions we just saw used K as a placeholder for a memory address as an argument. These are cards, quick reference of instructions. This is the generic syntax of the instruction, a short description, the exact operations in pseudocode. This one takes memory address K and adds it to A, the accumulator. The encoding of the instruction in memory and the number of clock cycles. The original syntax is the name the original design as gave to the instruction. For this talk I have chosen a more modern syntax here on the right, which makes it much more easier, much easier to describe the CPU both to people with and without a background in machine programming. Let's have a look at the instruction set in detail. Here's an example of the load instruction. Load A, indirect 200. On the left you see the set of registers of the AGC. Most operations work with the accumulator, so we will be ignoring the other registers for now. While executing this instruction, the CPU looks at memory at location 200, reads its contents and copies it into the accumulator. This is the store instruction. In the store, load indirect 200, A. Like with all instructions, the first argument is the destination, memory, and the second one, the source, the accumulator. It looks up the rest to a zero zero in memory and copies the contents of the accumulator to that cell. There's also an exchange instruction which can atomically swap the contents of the accumulator and the memory cell. The add instruction will look up the contents of a given memory address and add it to the contents of the accumulator and store the result back into the accumulator. And there's a subtract instruction. It takes the contents of the memory address and subtracts it from the contents of the accumulator and stores the result back into the accumulator. The result of a subtraction can be negative, so we need to talk about how negative numbers are expressed on the AGC. Let's look at just four-bit numbers. Four-bit unsigned integers can express values from zero to 15. With sign and value encoding, the uppermost bit corresponds to the sign, and the remaining three bits represent the absolute value. Consequently, there are separate values for zero, plus zero, and minus zero. This encoding is hard to work with since zero transitions need to be special cased. Once complement encoding has the order of the negative numbers reversed. The zero transitions are simple now, but there are still two representations of zero. Modern twos complement encoding only has a single encoding for zero, and it's fully backwards compatible with unsigned addition and subtraction. In the 1960s, computers designed for scientific calculations are usually once complemented, and so is the AGC. This can express values from zero to 15. In once complement, the values zero through seven match the unsigned values zero through seven, and the negative side is organized like this. Unlike twos complement, the two sides are perfectly symmetrical. So negating a number is as easy as complementing it, that is flipping all the bits. So the two representations of zero are plus zero with all zero bits and minus zero with all one bits. Addition in the positive space is the unsigned version. Same in the negative space when mapping signed negative numbers to their unsigned counterparts. It gets interesting when we have a zero transition. Signed six minus four is six plus minus four, which is unsigned six plus 11, which in module 16 is one. We have a carry. In once complement, a carry needs to be added to the end result. So we get two, which is correct. An overflow means that the signed result does not fit into the number space. Signed seven plus one would result in signed minus seven, which is incorrect. The same happens when overshooting negative numbers. After applying the end around carry, the result of signed seven here is incorrect. The CPU detects this and flags the result. The accumulator has an extra bit to record the information about an overflow. We call it V. So if we have code that reads the value of seven FF from memory and adds one, the result is zero and an overflow is detected. So the accumulator is flagged. The store instruction in addition to writing A to memory does extra work if there's an overflow condition. It clears the overflow condition, writes plus one or minus one into A depending on whether it's a positive or a negative overflow and skips the next instruction. This way the program can detect the overflow and use the plus one or minus one to apply the signed carry to a higher order word. By storing A to memory, there's a double word result. In one's complement negating a number is as easy as flipping every bit in a word. So there's a dedicated instruction for loading and negating a value. LDC, which stands for load complement, reads a word from memory, negates it by inverting all the bits and writes it into the accumulator. Incrementing that is adding one to a word is such a common operation that there's a dedicated instruction that increments a word in memory in place. There is no corresponding decrement instruction. Instead, there are two similar words, augment and diminish. The increment instruction adds one to the original value. The augment instruction adds one to all positive values and subtracts one from all negative values. It effectively increments the absolute value retaining the sign. The diminish instruction decrements positive values and increments negative values. Optimized for scientific calculations, the CPU has dedicated multiplication circuitry. The model instruction reads a word from memory and multiplies it with the accumulator. There are two signed 15-bit words. You need up to 29 bits. That is two words for the result. The complete result will be written into two registers, the upper half into A and the lower half into B. B is a separate 15-bit register, which is mostly used together with the accumulator with instructions that deal with 30-bit data. Double-word values are expressed with the uppermost bits in A, or if in memory at lower addresses, and the lower bits in B, or at higher addresses, making the AGC a big engine machine. Assuming the normalized number of words with matching signs, the effective value is the concatenation of the 2x14 bits of the values. Division also works with double-words. It takes the combination of the A and B registers as the dividend and a word from memory as the divisor. There are also two results, the result and the remainder. The result is written into A and the remainder into B. Some other instructions also allow using A and B as a double-word register. Load A, B, come indirect to 0, 0 looks up address to 0, 0 in memory and reads the words at A and B. The load complement variant does the same but inverts all bits during the load. There is no instruction to store A and B in a single step, but there is a double-word exchange instruction. And finally, there is an add instruction that works in double-words. And to load and store just the B register, there is an exchange instruction for that. For working with tables, there is the indexed addressing mode. Any instruction that takes an address as an argument can use it. This example, load A, write 0, 0, 0 plus indirect A to 0, first looks up address 0, 0, 0 adds it to the base of 7, 0, 0 which results in 7, 0, 2, reads from that address and writes the result into A. What does this mean? There is a table in memory at 7, 0, 0. In the example, it contains multiples of 3. And an index to that table is stored in memory at 0, 8, 0. Which in the example is 2. So we read the entry at index 2 of the table, which is 6. Without a base address, we get the syntax in this example. Load A, double indirect 8, 0. The base is effectively 0 in this case. The CPU will look up the value at 0, 8, 0 in memory, add it to the base of 0, so the value is still the same and read from that address. In this case, memory at 0, 8, 0 effectively stores what C programs know as a pointer. And 3, A, 0 is the pointer's destination. The instruction performed in indirect read. By default it is executed sequentially. The program counter PC, increments as instructions are executed, always pointing to the next instruction. Control flow instructions, like jump and conditional jump, change that. When the CPU hits a jump instruction, it will load its argument into the program counter, which means that execution will continue at that address. JZ, jump of 0, only jumps if A is 0, otherwise it continues with the next instruction. Similarly, JLEZ only jumps if A is 0. It is a four-way fork for execution. Depending on whether the value in memory is positive, negative, plus 0, minus 0, it will jump to one of the next four instructions. If you know the value is positive or 0, you can ignore the other two cases and just fill the first two slots. And if it's supposed to be only negative, you have to skip the first two slots. They should never be reached, but it's good practice for them to fill them with error handlers. Since CCS A is the absolute dimension value of the memory location into A, special case of CCSA can be used for loops that count down A. The call instruction is meant for calling subroutines, a.k.a. functions. It's like a jump instruction, but it saves its origin so the callee can return to it later. For the call instruction, the program counter is incremented first and then copied into the link register, LR. Finally, the argument of the call instruction is copied into the program counter and the register now contains the return address. At the end of the subroutine, the red instruction effectively copies the contents of the link register into the program counter so execution resumes just after the call instruction. If the subroutine wants to call its own subroutine, the program has to save the link register before it's restored afterwards. There's an exchange instruction specifically for this. For additional levels, a stack can be constructed manually using the indexing syntax. So far we've seen the following registers. The first register is used for memory accesses and all arithmetic. It is combined with a B register for double width arithmetic. The program counter keeps track of what to execute and the link register remembers the return address when calling a subroutine. We haven't seen the zero register yet. It always contains zero so when we read from it we get zero and when we write to it the value gets discarded. There are three more registers that we'll talk about later. The eight registers are numbered. That is, they are assigned memory addresses. They can be accessed using the addresses and all instructions that take a memory address. This allows for much greater flexibility in the instruction set. We can load A with the contents of the B register by reading the contents of memory at location one into A. The constant of zero can be loaded into A by just reading from memory at seven, which is the zero register. A can be incremented by incrementing memory at A and B can be used as a pointer by reading from double indirect one. Let's look at memory more closely. Memory is 4,096 words and goes from 0,0,0 to FFF. The registers are located at the very bottom of memory, including them there are 1,024 words of RAM, random access memory and three kilo words of ROM read only memory. The AGC was originally architected to only have this little RAM and ROM, but there's actually more. Let's look at the RAM area. The upper most quarter is banked. The area is a window through which one of eight different banks can be accessed with 250 words in size. The erasable bank register, EB points to one of these banks. If EB is zero, bank zero is visible in the banked area. If EB is five, bank five is visible. Addresses in the fixed area always represent the same RAM cells, but these are not additional cells, but the same as banks zero, one and two. This means that there's a total of eight times 256 words of RAM two kilo words. ROM is organized similarly. The lower kilo word is banked. The fixed bank register selects one of the 32 banks. Support for more than 32 kilo words of ROM was added at the last minute. The super bank bit can switch the uppermost eight banks to a second set so that a total of 40 kilo words are supported by the architecture. The fixed ROM area will always show the same contents as two of the ROM banks. The designers chose banks two and three to simplify addressing coding. In practice, fixed ROM contains core operating system code and fixed RAM core operating system data that have to be available at all times. The remaining functionality is distributed across the different ROM and RAM banks. Switching the RAM bank can be done by writing to the EB register. This is not a separate instruction, but can be expressed by writing A to memory location three. If A is five, writing A into EB will make RAM bank five visible at three zero zero. The same store instruction could be used to write to the FB register at memory location four to switch the code for a common case. If code in one bank wants to call code in another bank by first switching the ROM bank, load FB, and then doing the function call, writing the bank number into FB will switch out the bank the code is currently running on, so it won't be able to execute the call instruction. Instead it will continue running some unrelated code that happens to be the same address on the other bank. To call code on a different bank, FB and PC registers need to be changed atomically. Call F is only a synonym for the same exchange instruction. Code first has to load the bank and the program counter into A and B. Which then call F can automatically move into FB and PC. The same exchange instruction can be used for a far return. It moves the original values back into FB and PC. The two bank registers only hold five and three bits respectively. The other bits are zero. And there's a third bank register, BB, both banks, which merits the transfer from both other bank registers. The call for both banks synonym is a double word exchange instruction that updates the program counter and both banks. Subroutines usually have their private variables on particular RAM banks. Call for both banks passes control to a function on a different ROM bank and also directly switches RAM banks so that the colleague can immediately access its variables. Return for both banks returns to the caller restoring its RAM bank configuration. The unusual ordering will allow for a double word exchange of FB and PC as well as for a double word exchange of PC and BB. Now we've seen all eight registers. And there's eight more special locations in memory above the register, the shadow area, which we'll talk about later. And above those, there are four so-called editing registers, which make up for the missing shift and rotate instructions. When writing a 15-bit value into the ROR editing register, it will be moved to the right by one bit and the lowest bit will be cycled to the left by one bit. ROR can then be read back. ROL rotates left. SHR shifts to the right duplicating the top bit and SHR7 shifts to the right by seven bits filling the top with zero. This is needed for the interpreter system software component that we'll learn about later. We have seen that the CPU is connected to RAM and ROM, the memory bus. But computers also talk to peripheral devices. That is the IO bus. We've already seen the address space to talk to devices. There are 512 IO channels, numbered 0000 through FFF. Each channel is 15 bits and the in and out instructions can read words from and write words to IO channels. For many devices, a channel contains 15 individual control bits. A control bit can, for example, toggle a lamp on a display. The out or instruction sets individual bits and out and clears individual bits. So our instructions can work on the whole board or do Boolean operations on them. And, or, and XOR. To make Boolean operations also usable between registers, channels one and two are actually aliases of the B and LR registers. Which allows for these instructions. For end, there's also dedicated instruction that works on A and memory. After the registers, the shadow area and the editing registers, there's another special area that counters. Like IO channels, they connect to external devices but they don't send bits or whole words back and forth. Instead, they are controlled by hardware pulses or cars hardware pulses. On every pulse time one gets incremented, for example, while other counters take the numbers stored into them by code and count down generating pulses. When IO devices need to signal the CPU, they can interrupt normal execution. Next to the program counter which points to the next instruction, there's the instruction register which holds the current time. When an interrupt happens, the CPU copies PC into a special memory location, PC prime and IR into IR prime and then jumps to a magic location depending on the type of interrupt, in this example 814. When the interrupt handler is finished servicing the device, the IR instruction will copy PC prime and IR prime back into PC and IR so execution will continue at the original location. Memory locations 8 through hex F are shadows of the A registers. PC and IR are automatically saved and interrupts and the remaining registers need to be saved by software if necessary. The overflow condition flag cannot be saved or restored. So while there's an overflow condition until the next store instruction which resolves the overflow interrupts will be disabled. The 11 interrupt handlers have to reside in fixed ROMs starting at 800. There are four words for each entry. Typical interrupt entry code saves A and B. Loads B with a bank and PC of the actual handler and jumps there. Interrupt zero is special, it's the entry point on reset. Next to the interrupt return instruction there's an instruction to cause an interrupt in software and instructions to enable and disable interrupts globally. There's one more special memory location at hex 37, the watchdog. This location needs to be read from or written to at least every .64 seconds, otherwise the hardware will decide the system software is responsive and car to reset. Now we see an instruction set and in the examples we've seen the codes that represent instructions in memory. Let's look at how the encoding works. For the load instruction the upper three bits are the opcode representing the load A and the remaining 12 bits encode the address. This allows for a total of eight instructions, but there are more than eight instructions. RAM addresses always start with 00 and ROM addresses start with anything but 00. So the store instruction, which only makes sense on RAM, only needs to encode 10 address bits instead of 12. Making room for another three RAM only instructions. The same is true for the increment instruction which makes room for three more as well as CCS which shares an opcode with jump which only works on ROM addresses. Since jumps to the bank register don't make much sense, these codes are used to encode STI, CLI and extend. When the code is pre-fixed it changes the meaning of the opcode of the next instruction allowing for a second set of two word instructions. There's one more special call instruction called two, which is callLR which is the return instruction but the CPU doesn't special case this one return is a side effect of calling memory at location two. It executes the instruction encoded in the LR register the 12 bit address with the leading zero decodes into another call instruction which transfers control to the return address. Indexed addressing is achieved by using the index prefix. An indexed instruction consists of two instruction words index and the base instruction. The addressing code in the base instruction is the base address and the index instruction encodes the address of the index. Indexed is an actual instruction. The CPU reads from the given address zero at zero in the example then adds its value three to the instruction code of the following instruction three seven zero zero which is stored in the internal IR register. The resulting instruction code three seven zero three for the next instruction which is a load from seven zero three the effective address. If an interrupt occurs after the index instruction that is no problem because IR contains the effective instruction code which will be saved into IR prime and restored at the end of the interrupt handler. Finally there's one index encoding with a special meaning when the address looks like it's referencing the shadow instruction register is an interrupt return instruction. Looking at the instructions there are many quirky and unusual features when compared to modern architectures. It uses one's complement instead of two's complement it has no status register. The overflow flag can't even be saved so interrupts are disabled until the overflow is resolved. The store instruction may skip a word under certain circumstances the CCS instruction can skip several words and can be outright dangerous if the instructions follow in it use prefixes. There are no shift or rotate instructions but magic memory locations that shift and rotate when writing into them. Most Boolean instructions only work on IO channels indexing is done by hacking the following instruction code and the architecture has no concept of a stack indexing has to be used if one is needed. This was the architecture of the Apollo guidance computer now let's look at how this architecture is implemented in hardware. The hardware implementation runs at one megahertz is microcoded and uses integrated circuits core memory and core rover memory. We'll look at the block diagram and how instructions are implemented in microcode and then about how the schematics map to integrated circuits on modules on trays. This simplified block diagram shows the AGC at the hardware level. Each box contains on the order of 500 logic gates the dotted lines are wires that move a single bit of information and the solid lines are 15 wires that move a data word. These units deal with timing and control and these are the central units. The central register unit stores A, B link register and program counter and the arithmetic unit can add and subtract numbers. The memory components deal with RAM and ROM. The main clock of about one megahertz feeds into the sequence generator which keeps cycling through 12 stages which is one memory cycle MCT. Instructions usually take as many memory cycles as they need memory accesses so load add and store take two cycles and jump takes one. The sequence general contains a collection of 12 step micro programs for each MCT for each instruction like this one for the load instruction. In each step the entries send control pulses to the other units which are connected through the right bus. The control signal WA for example instructs the register units to put the contents of A onto the right bus and RA makes it read the value on the bus into A. Memory is also connected to the right bus. WS will copy the bus contents into the memory address register and RG and WG will read and write the G register which buffers the sales value after read and before write. So in stage 7 for example RG puts the memory buffer onto the bus and WB writes the bus contents into the temporary G register. And in T10 B gets put on the bus and it gets read into the A register. At the beginning of every memory cycle the hardware sends the memory address S and G to memory and copies the contents of that address into G. In the second half of the MCT it stores G back into the same cell. So if we show memory timing next to the microcode as well as the pseudocode version of the load instruction which is easy to read, we can see it loads the value from memory into G copies it into B and then copies it into A. More interesting is the exchange instruction. It saves A to B reads memory into G copies the resultant to A copies the old value into G and stores that G into memory. Division for example takes several MCT and its micro-program is way more complex. But there are more micro-programs than the ones for the machine instructions. Since there is only a single adding unit and the whole computer incrementing and decrementing the counters is done by converting the pulses into special instructions that get injected into the instruction stream. There are 14 of these so called unprogrammed sequences with their own micro-programs some counter shift, some are interacting with debugging hardware and these two control interrupt and the interrupt and reset sequences. The complete schematics are publicly available and fit on just 49 sheets. The whole implementation only uses a single type of gate, a three input NAND gate. Two of these are contained in one integrated circuit. And about 100 of these ICs form a logic module. 24 logic modules and some interface and power supply modules are connected together in Trey A which also contains the IO and debug connectors. Trey B contains various driver amplifier modules as well as RAM and ROM. RAM is implemented as magnetic core memory which stores bits in magnetized torus. Reading a bit clears it so the memory sequencer makes sure to always write the value again after reading it. Without mass storage like tape the AGC has an unusually high amount of ROM. Core rope memory encodes bits by wires that either go through just a ferrite core. The 500,000 bits per computer were woven completely by hand. Trey A and B are put together like this and termitically sealed making for a rather compact computer. This is its orientation when installed in the spacecraft with the six ROM modules accessible so they could in theory be replaced during the mission. And that was the hardware part. Next let's look at the devices connected to the computer. We will look at the quad devices that allow the Apollo guidance computer to maintain the state vector. Some quite special devices you don't see on many other computers and the peripherals used for communication with astronauts and mission control. The gyroscope is the core peripheral that the Apollo guidance computer was originally built around. The Apollo guidance computer rotates it into a certain base position with the CDU command counters and then the gyro detects rotation around the three axis of the spacecraft that can be read from the spacecraft always knows its attitude that is its orientation in space. The accelerometer measures acceleration forces on the three axis. The three values can be read from the Piper counters. The optics on the command module are used to measure the relative position to the celestial bodies. The computer uses the opt command counters to move the optics to point towards the general direction of a star and will read in the astronauts fine tuning through the opt counters. The landing radar sits at the bottom of the lunar module and the radar interrupt will be triggered whenever a new measurement is available and the R and R counter contains the new value. The lunar module's rendezvous radar measures the distance of the command and service module during rendezvous. After setting the two angles in the CDU-T and CDU-S counters to point it towards the other spacecraft it will automatically track it and cause radar interrupt interrupts when new data is available which can be read from the R and R counters. The command module, the service module and the lunar module all contain reaction control that emit small bursts for holding or charging the attitude. On the lunar module there's one bit for each of the 16 jets. Setting a bit to one will make the jet fire. The system software uses a dedicated timer time 6 and it's interrupt T6 interrupt for timing the pulses. The user interface is provided by the so-called disk key which stands for display and keyboard. It has 19 keys, 15 lamps and several numeric output lines. Keys generate the keyrupt interrupt and the key number can be read from the key in IO channel. The numeric output line can be read from the digital radio communication on S-Bunt between mission control and each spacecraft at a selectable speed of 1.9 or 51 kilobits per second. Data words from mission control show up in the in-link counter and trigger interrupt upropped. Data words to be sent are stored in the IO channel DNTM1 and the download on top will signal the program when it can load the register with the next word. These were some of the interesting peripherals. The AGC software makes it a priority-based cooperative but also preemptive, real-time interactive fault tolerant computer with virtual machine support. The topics we'll talk about are multitasking, the interpreter, device drivers and the wait list as well as the user interface and mechanisms for fault recovery. The AGC has many things to do. It does mathematical calculations that can take several seconds and it does IO with its devices. It services interrupts when a device wants the computer's attention, for example it does regular servicing of devices like updating the display and it supports real-time control like flashing a lamp or firing boosters at exactly the right time. There's only single CPU so it must switch between the different tasks. Batch processing, multitasking computers work on long-running jobs one after the other but if some jobs have higher priorities it makes more sense to run a job for only say 20 milliseconds then check the job queues and keep running the highest priority job in the queue and it's removed from the queue. Then keep picking the highest priority job. Jobs have to manually check at least every 20 milliseconds whether there's a higher priority job in the queue by doing a so-called yield which makes the AGC a priority scheduled cooperative multitasking computer. A job is described by a 12 word data structure in memory that contains the PC and both banks register that point to where the job will start or continue running. As well as a word with a disabled flag in the sign bit and a 5-bit priority. The core set consists of 7 job entries minus 0 in the priority word means that the entry is empty. Job 0 is always the currently running one. When a new job cut is created with a higher priority the yield operation will exchange the 12 words so that new job is job 0. Negating the priority will put a job to sleep so yield won't switch it again. Negating it again will wake it up. The first 8 words in the job entry can be used for local storage of the job. Since it's always job 0 that is running these words are always conveniently located at the same addresses in memory. The executive has a set of subroutines that control the job data structures. You can create a new job pointed to by a pair of PC and BB registers of a given priority, change the priority of the current job, put the current job to sleep wake up a given job and terminate the current job. Heal does not an executive function but a two instruction sequence that allows the new job variable in which the executive always holds the idea of the highest priority job. If job 0 is the highest priority job there's nothing to do. If there is a higher priority job it calls the change job subroutine which switches to that job. New job isn't just a variable in memory but also the watchdog word. If it isn't access regularly that is cooperative multitasking stuck the hardware will automatically reset itself. A lot of the code on the AGC does scientific calculations. Calculating two products of a scalar and a vector would require hundreds of instructions in AGC machine code. There's library code that provides all kinds of operations on single double or triple precision fixed point values, vectors and matrices. It also provides a software multi-purpose accumulator M-PAC which can hold a double triple or a vector depending on the mode flag. In C-like pseudocode we would load the vector into the M-PAC, multiply it with a scalar, save it, do the other multiplication and add the result to the saved value. Formulas like this one need to store intermediate results so a 38 word stack is provided. If a job uses math code the M-PAC, the mode field and the stack pointer will be stored in the remaining fields of the core set entry. The stack will be part of a data structure called VAC which will be pointed to by the core set entry. A job can be created with or without a VAC depending on which subroutine it is created with. The machine code version of the example code would be used by many function calls passing pointers. The designers of the AGC software decided to create a new and compact language that will be interpreted at runtime, a virtual machine. The interpretive language is Turing complete and in addition to the M-PAC has two index registers, two step registers and its own link register. The encoding manages to fit two seven bit op codes in one word which allows for 128 op codes and explains why there is a shift right by seven function in the CPU. The two operands are going 14 bit addresses. 14 bit addresses means interpretive code doesn't have to work with this complicated memory layout anymore. It allows addressing about half of the ROM at the same time. At the lowest kilowatt of each half RAM is visible so interpretive code can pick between one of these two memory layouts. This is the complete instruction set. Regular machine code and interpretive code can be mixed and matched inside the job. The exit instruction will continue executing regular machine code and then switch to interpretive mode. In addition to long running math tasks, the system software also supports device drivers. When a device needs the computer's attention, for example, in case of a disk key press, it causes an interrupt. The current job will be interrupted and the interrupt handler will read the device data and return as quickly as possible. If there is more to do, it can schedule a job for later. Some devices need to be serviced regularly. A 120 microsecond timer is triggered. The timer interrupt cycles through eight phases, which distributes the device drivers across time to minimize the duration of one interrupt handler. Some devices need to be driven at exact times. If, for example, a job decides that it needs to flash a lamp twice, it would turn it on immediately and schedule three weightless tasks in the next few days. If a device needs to be serviced regularly, schedule three weightless tasks in the future at specific times. The first one will turn the lamp off, the second one will turn it on again, and the third one will turn it off again. The sorted time deltas of the weightless tasks are stored in the data structure LST1, with the first entry always currently counting down in a timer register, and LST2 contains a pair of PC and BB for each task. There are subroutines to create a new task and end the current time. The timer that controls the weight list has a granularity of 10 milliseconds. Other timers can fire at the same rate, but are offset, and the work triggered by them is designed to be short enough to never overlap with the next potential timer triggered work. This is complicated by device interrupts, which can come in at any time. The duration of an interrupt handler causes latency, and the maximum duration will reduce the allowed time for the timer handlers. The core system software makes no guarantees that it's up to all the components to operate, so the real-time goal can be met. The pinball program is the shell of the AGC. He press interrupts, schedule a job that collects the digits for the command and updates an in-memory representation of what should be on the display. The 120 millisecond timer triggers the display update code. When the command is complete, pinball schedules a new job. Mission control has a remote shell in front of it, and it's connected through the S-band radio. System software that supports human life has to be able to communicate malfunctions and be able to recover from them. The alarm subroutine takes the following word from the instruction stream, displays it, and illuminates the prog light. This should be interpreted as a warning or an error message. The AGC software is full of validity and plausibility checks that help to find bugs during development and help better understanding potential issues during development. In addition to showing the error code, they also cause a hardware reset. But the system offers recovery services. A job can have recovery code for its different phases. During execution, it sets the respective phase, and if an abort happens in any job or task, the currently set up recovery routine gets executed. Which could, for example, clean up and try to work again, or skip to a different phase, or cancel the job altogether. The phase change call sets the current phase for a job in the recovery table. For example, phase 5 for job 4. Each phase is associated with a descriptor of a task or a job with or without a vac. So during normal execution, with several jobs and tasks scheduled, if an abort happens, the core set and wait list are cleared. The contents of the recovery table are activated, scheduling tasks and jobs for all jobs that set up recovery code. Sometimes of failure, though, like corrupted memory, are not recoverable. They cause a fresh start, meaning a full initialization of the system without running any recovery code. And that was the AGC system software. As we now have a good overview on architecture, hardware peripherals, and system software of the Apollo guidance computer, it's time we'll be reviewing its practical use on a mission to the moon. We will look at the user interface, the launch sequence, and once in orbit, the attitude and orbit determination. Further, we will understand how the digital autopilot works and how powered flight is being performed. As soon as we've reached the moon, we look at the lunar landing and the lunar rendezvous of the lift off. And finally, re-entry into Earth's atmosphere. Last but not least, contingencies, or as we like to call them, fun issues. Let's start with the user interface. It is like any command line interface, but since there are only numbers and no letters, keyboards have to be encoded. On a normal system, you might say, display memory, enter. Display is the verb, memory is the noun. On the Apollo guidance computer, display verb 01, which means display noun 02, memory, enter. Subprotein asks for an argument. On a normal system, it might display a prompt, you enter the number, press enter. On the Apollo guidance computer, flashing verb and noun indicate that it is waiting for input. So you type 25, enter, an octal address, and the Apollo guidance computer displays the result, the memory contents at the address octal 25. The Apollo guidance computer uses the same concept of verb and noun when it proactively displays the verb. The Apollo guidance computer asks for the CSI ignition time. CSI means coalyptic sequence initiation. We'll come to that later. A special case is when the Apollo guidance computer asks a yes or no question. The astronaut has the engine ignition with the proceed key. The astronauts have a complete reference of all verbs announced on paper, as well as cue cards with the most important information. Let's now go through each of the faces of the mission, starting with a lift off. The Apollo guidance computer has a passive monitoring mode. With the cutting of the umbilical cables, which you see right about now, it has started the mission clock. In case this trigger fails, one disk key is always prepared with verb 75 and just waiting for enter to manually start the mission timer. We can display the mission elapsed time at any time with verb 16, noun 65. During the flight with the Saturn 5, the Apollo guidance computer is only performing passive monitoring of the flight. Control of the Saturn 5 is with its own system and the instrument unit ring. The disk key automatically shows verb 16, noun 62, which is velocity in feet per second, altitude change rate in feet per second and altitude above pad and nautical miles. Note that the units at the position of the decimal point are implicit. And yes, the whole system was working in metric internally, but for the benefit of the American astronauts, the display procedures converted everything into imperial units. In case of problems with the Saturn computer, we can take over full control of the launch vehicle. In extreme cases, astronauts could even steer the whole stack into orbit themselves with the hand controller. In case you ever wanted to manually control a 110 meter tall rocket with more than 30 million Newton of thrust, this is your chance. In less than 12 minutes, we've gone through the first and second stage and are using a small burn from the third stage to get us into a 185 kilometer orbit which circuits the Earth every 88 minutes. But how do we know where we are in the right orbit? We need to know where we are. To be able to navigate in space, we need to maintain our three-dimensional position and our three-dimensional velocity, the so-called state vector. Let's start with the determination of the position. For this, we need a telescope and a space sextant. The space sextant is very similar to an 18th century nautical sextant. Position is determined by measuring the angle between the horizon and the celestial body. As a horizon, we can either take that of Earth or Earth in our bodies. We are surrounded by them. Luckily, the Apollo guidance computer already knows the position of 45 of them. The optics hardware in the command and service module can be moved to point in the general direction. With the launch of program 52, we command the Apollo guidance computer to rotate the spacecraft to point one axis of the sextant, the so-called land-mark line of sight, LLOS, to the nearest body, the Earth or Moon. The astronaut then uses the optic systems astronaut looks for one of the known stars, points the starline to it and let the Apollo guidance computer read the trunnion and shaft angle. Repeating this one or more times in a different plane gives a three-dimensional position of the vehicle in space. In the lunar module, on the other hand, the whole optics hardware was trimmed down for weight reduction. Any alignment requires rotation of the lunar module. This is mostly used to determine the landing site and support the rendezvous maneuver. It even lacks the software to perform positioning in trans-lunar space. As we're moving, our position changes all the time. But after two location fixes, as long as we're coasting, we're able to establish our speed and can determine future positions by that reckoning. As position and velocity are known, future positions can be extrapolated. Unfortunately, the near-exploitation doesn't work in space as we have gravitational forces which bend our path. Thankfully, there are two mathematical models implemented in the Apollo guidance computer, conic integration based on the Keplerian orbit model on the left, which assumes one perfectly round gravitational body influencing our flight path and anchors integrating method for perturbation, considering multiple bodies with gravitational imbalances. I think this helps to understand why we need a computer on board and can't just fly to the moon with a hand controller. As we see, the Apollo spacecraft is perfectly capable to fly on its own, but in the end, NASA decided that the primary source for state vector update shall be mission control in Houston, measured with three ground stations. Remote programming is done with the Apollo guidance computer in idle and running program 27. Mission control can use its link via S-band to update the state vector. But there's one thing mission control doesn't know better than us, and that's attitude. Attitude is the orientation of the spacecraft in its three axes. Starting from a known attitude, we have to ensure that we can measure any rotation on any axis. That's what gyros are for. They are one of the major components of the IMU, the inertial measurement unit. Three gyroscopes, one per axis, measure any rotation and provide their data to the Apollo guidance computer to keep track of the attitude of the spacecraft. Before we leave Earth orbit, let's quickly discuss the digital autopilot. It is the single biggest program in the Apollo guidance computer with about 10% of all the source code, both in the command and service module as well as the lunar module. The implementations for each vehicle are significantly different, though, due to different flight modes, thruster sets and symmetry of vehicle. As there's no friction in space, the tiniest event would constantly make the spacecraft rotate. The digital autopilot of the Apollo guidance computer uses the jets to maintain the attitude within certain thresholds, so-called dead bands. The autopilot is also used in case the astronauts ever need to use their hand controllers for thrusters, basically both the command and service module and the lunar module have fly-by-wire control. As any thruster could break at any time, the autopilot is capable of calculating the ideal burn mode even with a reduced number of thrusters. It has some simple algorithms for center of gravity and weight distribution as well, which are taken into account in calculating thruster maneuvers. It can do more than that, though. Give it a new attitude and it will calculate the most efficient transfer vector to reach the new attitude. In certain flight modes, it might be required to have a stable rotation, be it for temperature control, monitoring in the landing site or other reasons. The autopilot supports stable constant rolling, which can be directly activated. The autopilot does not only control attitude, it also supports the crew in performing powered flight maneuvers. It calculates the potential solution, which obviously can be overwritten by ground as usual, but still after confirmation, the autopilot automatically fires the engines and keeps a timer for the correct length of time. It does not measure the results of the burn, though. For powered flight, obviously dead reckoning isn't correct anymore, so the Apollo guidance computer contains a subroutine called average G, which takes the input from the IMU, meaning gyro and accelerometer, to compute the change to the state vector. Now that we know how to orient ourselves and how to control the spaceship, it's time we fly to the moon. Usually, the trans-lunar ejection happened in the middle of the second orbit around the Earth, so around two hours 45 minutes into the flight. This is still performed by the third stage of the Saturn V, so the Apollo guidance computer once again should only have a passive role here by monitoring the trans-lunar ejection with a dedicated program P-15. After separation from the S-4B, we are on our way. Since the next externalizing phase is the lunar landing, let's get right through that one. Once in lunar orbit, separation between the combined service module and lunar module happens four hours and 45 minutes before the ending. On the lunar module, directly afterwards, rendezvous equipment like radar, strobe, and VHF are tested, as well as the IMU, which is realigned. Additionally, there's lots of preparation work on the lunar module. One of the main tasks is to prepare the abort guidance system, AGS, which is another more simpler computer that is able to get the lunar module with the astronauts back into orbit and safely dock with the CSM in case of an emergency. Let's get back to power descent. The lunar module, AGS, has a special program for that one, P-60C, breaking phase. The landing radar is switched on and updates the state vector. The Apollo guidance computer controls the burn to reach the correct corridor towards the surface with a minimal amount of fuel. This is fully automatic. The astronauts just sit along for the ride. The lunar module is oriented with its descent engine towards the moon. Visibility for the astronauts is close to zero. The second program, P-64, starts automatically at around 8,000 feet. The lunar module is pitched so that the astronauts can actually see the ground, and the lunar module commander is getting a better understanding of the landing site and can search for a suitable spot. The third program, P-68, keeps the lunar module in a stable attitude above the surface, and the commander manually adjusts the height in one feet per second increments to slowly descend to the surface. Ideally, at that point, the horizontal movement of the lunar module should be zero. After touchdown, the crew manually activates program 68, which confirms to the Apollo guidance computer that, yes, we have indeed landed, ensures that the engine is switched off, terminates the average G-routine, and sets the autopilot in a very forgiving setting to avoid any corrections when it measures the rotation of the moon. The autopilot is not completely switched off, though, as the astronaut might need it in case of an emergency ascent. Well, we're on the moon, we do the usual stuff, small step for man, jump around, plant a flag, and we then skip directly to the interesting bits with this lift off and rendezvous. The rendezvous technique was developed in the Gemini project. Here you can see the Agena rendezvous target in Earth orbit. It follows the principle of an active vehicle, in this case the lunar module, which follows the command and service module and approaches it from below on a slightly faster orbit. There were actually two different ways for rendezvous. A more conservative method called coalyptic rendezvous, which required one and a half orbits for the lunar module to reach the command and service module, but gave ample opportunity for monitoring progress, mid-course corrections, and orbit scenarios. And a more risky direct rendezvous method, which directly aimed the lunar module towards the command and service module, taking less than one orbit until docking. This one was used starting from the Apollo 14 mission, as mission control had more experience and aimed for the shorter, less fuel-intensive method. Preparation had to start two hours before lift off. We have to align the IMU and we visually monitor the orbit of the CSM and calculate the rendezvous data. The Apollo guidance commuter has program 22 CSM tracking for this purpose.  The Apollo guidance commuter performs the countdown, asks for confirmation, we proceed and we have lift off. The trip into orbit takes only seven and a half minutes, but depending on which method for reaching the target orbit was used, it takes us either one and a half or three and a half hours to come up behind the command and service module. During that time, program 20 is running all the time, measuring the state vector of the other vehicle, the command and service module, via various peripherals, like RONDU, RAIDA, VHF antenna, and the optic system for visual alignment. It calculates the necessary corridor and respective maneuvers required to get the lunar module into an interception course. Multiple other programs run in parallel to perform the necessary mid-curve burn maneuvers. On the command and service module, the pilot is actively tracking the lunar module the whole way up to orbit. The command and service module's computer is calculating the state vector of the lunar module to take over the role of the active vehicle in case anything goes wrong. The approach of the lunar module stops at 50 meter distance, at which point it rotates to point its docking target on top towards the command and service module. At that point in time, the command and service module takes over the active role and activates program 79, final RONDU, which slows down the command and service module to close the distance until docking. Seconds before contact, the autopilot on both spacecraft is switched off to avoid both trying to correct the attitude of the combined spacecraft. So far so good. Time to go home with a trans-Earth injection. We feed the Apollo guidance computer with Earth orbit parameters and let it calculate the burn, which is then activated and controlled. Any kind of potential mid-course corrections are performed the exact same way. Once in orbit around Earth, re-entry parameters are calculated on ground and transferred to the Apollo guidance computer via S-band uplink. The first entry program, P61, entry preparation, starts at entry minus 25 minutes. Barriers landing parameters are requested like latitude and longitude of the splash zone, as well as the velocity and angles to enter the atmosphere. Entering and confirming these values completes program 61 and starts program 62, which basically asks the astronaut to perform a checklist for manual command module, service module separation, which is not controlled by the Apollo guidance computer. After that has been performed, it switches automatically to program 63, entry initially zinged the aliization. At that point, the autopilot is taking care of thruster control to break the command module out of its orbit into Earth's atmosphere. The main programmer for re-entry is program 64, entry, which starts automatically. Program 64 monitors the trajectory and splashes down location and determines the best entry solution and potential velocity reduction by invoking two specific programs, either P65 entry up control, which basically makes the command module serve on the atmosphere to reduce speed and extend the range, or program 66, entry ballistic, throwing us through the atmosphere like a cannonball. The right mixture of the two is decided by program 64. The last program, program 67, final phase performs the final maneuvers to the splash zone. The following steps like parachute deployment and so on are not done by the Apollo guidance computer, but by the ELSC, the Earth Landing Sequence Controller. The job of the Apollo guidance computer is done before deploying the parachutes. So this was a beautiful nominal mission. What can go wrong? Well, let's start with Apollo 11, which had a 1202 program alarm during power descent. Normally, programs during power descent use about 85% of the processing power of the computer, but due to an incorrect power supply design, the roundover of the roundover radar generated an additional 12,800 involuntary instructions per seconds, ironically amounting to the exact additional 15% load. Due to the cooperative multitasking, a queue of jobs built up, which resulted in an executive overflow and the 12.02 alarm. The operating system automatically performed a program abort. All jobs were canceled and restarted. All of this took just a few seconds and landing could commence. Next, Apollo 13. They had an explosion of the oxygen tank in the service module at 55 hours, 54 minutes, 53 seconds, and yeah, correct. 320,000 kilometers from Earth. Fortunately, they could make use of the free return trajectory to get the astronauts back to Earth, but they had to move to the lunar module to survive, as the command and service module was completely shut down, including its Apollo guidance computer. The IMU settings needed to be transferred to the lunar module system first, adapted to the different orientations of the spacecraft. The manual burns and the mid-course corrections were actually done with the abort guidance system on the lunar module due to power constraints with the Apollo guidance computer. Successful reboot of the command and service module computer was luckily done hours before reentry. And last but not least, Apollo 14, which had a floating solder ball in the abort button, which might lead to an unwanted activation of abort, therefore putting the lunar module back into orbit. This was solved within hours by reprogramming the Apollo guidance computer to spoof the execution of a different program, which was not listening to the abort button during the power descent. Real abort activation, though, would have to be manually activated via the disk key. So this was an overview on how the mission software was used on a flight to the moon and back. Now you probably want to run your own code on a real Apollo guidance computer, so you need to know where to find one. 42 computers were built total. Seven lunar module computers crashed onto the moon. Three lunar module AGCs burned up in the Earth's atmosphere. 11 command module computers returned. They're all presumably parts of museum exhibits. And 21 machines were not flown. Little is known about those. One is on display at the Computer History Museum in Mountain View, California, but it is missing some components. Luckily, several emulation solutions are publicly available, as well as a tool chain. And the complete mission source, originally the size of a medium-sized suitcase, is available on GitHub. It takes a village to create a presentation. We would like to thank everyone who helped and supported us. This includes the indirect contributors who wrote the books, the original documentation, the websites, and the software. Thank you very much for your attention. Thank you. Wow, that was a densely packed talk. Thanks, Michael, and thanks, Christian, for this amazing information overload. Please give a warm hand of applause because we can't have a Q&A, unfortunately.


Great. Okay. I'll make a start. Thank you very much to everybody for joining. Either joining us live or catching up on YouTube at a later date. Today's bite-size is just a short one. It's going to be about half slides and half live demo. And I'm going to talk through some of the tools that we use for code linting. I'm specifically code style linting. So I'm not going to talk about the NFCore lint commands in this talk. That's kind of as other bite-size talks specifically about that. This is more about the general use tools that we use for linting and code style. What do I mean by that? So as I'm sure anyone who's done even a little bit of development work will know how you format your code is a contentious issue. I started looking for a good like gift for me to put in, of course, XKCD gave up the goods with some rather excellent insults about code style. And once I started, there's just it's a pretty deep trove. Actually, there's a lot of a lot of good comics out there about about edition crises about code formatting. And any of you who watched Silicon Valley on TV or other TV programs will be familiar with the tabs V spaces argument. Anyway, the point being here is that everyone has quite individual views on how code should be formatted, not the content of the code, but just the white space and how things are formatted and how things are done. This is where code format is come in. And the idea of code format is or lint is that they set the style you're going to use. They decide how things should be formatted. And that means that you don't have to. It's good because they mean that you kind of free up a little bit of space in your brain for it. You're not kind of having to think about whether to use single quotes or double quotes, how much indentation you should be using, things like this, because it just happens. And the other thing is that they're really good for projects where there are lots of contributors like NF core, because then we have lots of different people coming in and working on the same single code base. Everyone has their own idea of how things should be formatted. And if ever you've come in and read someone else's code, you'll find that sometimes it's very difficult to read through that code if it's not in a style that you're used to. Code format is forced everyone to write code in exactly the same way. And that makes it much easier to collaborate together. And also means that when you change things, the differences in the code, the code diffs that we see on GitHub are just the important things, the actual code that you're changing. And we don't get lots of spurious diffs where white space is being changed. So yeah, code format is think about style so that you don't have to. I've put a little asterisk down in the corner here that they're highly addictive. I think code format is a bit like wearing a seat belt in a car or gloves in a lab or using git to manage source control. Once you start using code linters, you'll find that it's very difficult to work without them. I've been using them for a year or two now, and now I find it really irritating whenever I'm in a project that doesn't have them. I notice always little inconsistencies and I find myself wasting time on thinking about this. So once you start using code linters, I suspect that many of you will find it difficult to go back. I hope so anyway. So these are the different languages within NFCore that you'll come across.  Also some scripts within pipelines. We've got a lot of documentation written in Markdown. We primarily use YAML for configuration files, also JSON. And then on the website, we've got a few extra languages, websites written with PHP, CSS and NHGML. And these are the main languages there that I'm going to talk about today. And there's not very many tools that we use. We use basically two tools, maybe three. Python, we format using a tool called Black, which is the most popular and commonly used code linter for Black, code formatter for Python, used by a lot of big projects now. And it was Black that got me into this in the first place. And then we use a tool called Prettier to do everything else. And so recently, until we last released the tools, we used to use a tool called Markdown lint for Markdown and YAML lint for YAML. And we had numerous different individual lintes here, but we replaced those in the last major release of tools. And now we just use Prettier for everything. So if you did a pipeline sync recently to a template, you'll see quite a lot of minor changes, things like quotes and things like that in your YAML files. And that was because we switched using Prettier and now using that standard. And then the little mouse down in the bottom corner is a tool called edits config, which is basically a generalized, but it's a standard where it tells your code editors what the code style is for a project. And both of these other tools tie into that config as well. And it's also a standalone tool that's generic for any language. Now that one does stuff like indentation should be at multiples of two. So if you try and indent any code with the multiple of three, then it will complain, stuff like that. Okay, so these are the three projects we're going to talk about and these in their web sites. They all come with command line tools. So they all come with a command line package that you can run that look very similar. With black, you can do black, minus, minus check. Prettier, minus, minus check. And then edits config checker, that tool is specifically for checking.  So normally it would be asterisk for all of the files in the projects. Black and prettier especially will just ignore any files that they don't recognize. And they will go through and they'll check up with syntax of all of your different files and they'll throw a warning if anything doesn't conform to the style that they like. What is particularly good about these tools and one of the main reasons we switched to prettier recently is that these commands also will fix it for you.  Now you can just do prettier, minus, minus, right to go and prettier will run through and it will format everything for you and just fix it. That's great, that saves a lot of time. Okay, so before I go on to the live demos I'm just gonna walk through some stuff. There's three main ways that you're likely to come into contact with these tools. For many of you, the first way you'll come into contact with them is through the continuous integration tests. So you'll push a commit or you'll open a pull request and you'll have a failure on the CI. You'll have no red cross and it will say that prettier failed. And it probably says something like, oops, forgot to format your files with prettier. And usually pull requests will not be merged until that turns into a little green tick. So that might be the first time you see this and that might be how you ended up on this talk. One of the first places that we might go to to then to fix this problem is you can run these tools on the command line. Like I just showed they have commands so then you can go back to your code place run prettier minus minus rights which will fix the problem and then push another commit. And then hopefully that red cross will turn to a little tick. But really the best way to use these tools is to have them set up in your text editor in your code editor. And when that's done properly they will run automatically every time you save a file every time you edit anything. And you don't ever have to think about running them because just everything is automatically formatted properly. So that's the best way to work really. And once you've set that up once you can just forget about this and it just works. Right, so let's see if I can screen share a little portion of my screen over here. Hopefully everyone can see that bit of screen and see it throws everything around my window. So wave at me if you can't see what the website I'm looking out for the black documentation. Good. So just to show you the websites for these tools this is the black documentation. You basically don't ever need to look at this. But just so you know this is kind of what it looks like called the Google Python black, you'll find it. This is a website for prettier. And again, it kind of talks about how it works and with different languages that supports and for different code editors that you have integrations with which is kind of basically all of them. So if you're using something other than VS code which is what I'm gonna show you in a minute you can go along here and, you know install a plugin for Atom or whatever else. Yeah, and it's got quite good documentation. And the final one is this editor config which is just like I say it's a way of sort of standardizing config files across editors and projects for things like what type of new lines to use character encoding, indent size use and stuff like this. And certainly prettier will find these editor config files and load these settings from that. So they kind of integrate together. A lot of editors have built-in support for editor config and some you need a plugin for. So again, you can come here and get your Vim plugin for editor config. And basically then that this if this file is in your project it will override your normal local settings. Right, and then I said it ran on continuous integration. So just to give you a look of where this is this was within the pipeline template and there's a file in the GitHub workflows called linting.yaml. So just so you can see this is running editor config here and this is the exact command it runs. So if ever you want to emulate the CI tests that are running on GitHub you can run these commands yourself on your local system and you should hopefully get the same results as long as you've got the same versions of the tools installed. So you can see it's just running prettier check and editor config. And if we look in the main tools package where we've got a load of Python code you'll see it's also running black. Python black, you can see it's running but it's actually running GitHub action for running black. Okay, so just to do a quick kind of live demo now I've got the RNA-seq pipeline open here and I've just made a couple of basic kind of dummy changes I do get diff, I wonder if I can do diff dunk so you can see here that I've just made a few changes I've added something to a change log file here and I've just made some white space changes to this YAML file just for the purposes of this demo. Now right away you can see that series of changes here, the GitHub changes you can see it looks a bit weird so it's probably not surprising that this is gonna fail but this is a valid YAML file. This is the NFCore config file so it's customizing a bit about how the NFCore tools work the linting tests and stuff but all of this is valid YAML I haven't actually changed any of the real kind of meaning of this file and it should have run fine but you can see my indentation is a bit wonky here and I'm kind of using some single quotes here and whatnot. So if I, I've turned everything off at the moment so if I hit save then I could do prettier check and this will check all the YAML markdown files and sure enough it says, there's something wrong here the NFCore.YAML file has got a warning and also a change log where I added a little note about what I've done here. So I know something's wrong and if I had pushed my code anyway I would get a warning on GitHub, some GitHub actions. So I'm just gonna commit this now. So now we'll be able to see what changes after this. Okay, so I'm in BS code so I can run prettier, right? That goes through all the different files it recognizes and you can see if I do get status that has actually modified these two files. So you'll see I did get add first I committed those changes first and I think that's good practice before you run code linens is to commit your actual changes what you're thinking about first and then if you're gonna make big linting changes you can do that as a separate commit and it's very easy to see the diff of what's changed in case you're nervous about it. Later on you get used to running it and you won't need to do a separate commit but just at the beginning. And then if I look at the diff you'll see that both these files have been changed and you can see it's basically kind of messed around with some of the formatting and whitespace and sure enough in this file now all the indentation is fixed or using two space indentation everywhere. My single quotes appear converted to double quotes and all the extra line rates were removed and if I'd had any like trailing spaces at the end of lines like that then they would have been removed. Over in the change log you can see my markdown now correctly has a blank line after this heading and my bold text has been changed from double underscores and italics with asterisks to the standard that we use which is underscores for italics and double asterisks for bold. Minor changes didn't make any difference to the actual contents of the file but now we have a nice consistent usage of markdown and YAML. Okay, great. If I kind of... Hang on, if I just did it, where was it? And then here we go. So if I just put this back to how it was and I will show you the other way of doing it. So, say this... I said, so that's on the command line but I said the best way to do it is with the browser with the code editor. Now to do this I'm going to use a couple of plugins with VS Code and like I say, you can do this with basically any editor. So for VS Code, if you go to the marketplace or you just go to the extensions tab here you can search for prettier and there's a few ones but it's the obvious big one that's got 20 million installs. And basically you just hit install there and things should work. The same goes for editor config that there's a plugin for VS Code. You may not need this. And then black is kind of a bit more complicated because it's actually built into VS Code. So you need the Python add-on and then if you go into settings and type black you'll see that the Python formatting provider you want to set that to black so that it runs black whenever you edit Python code. If you're never going to edit Python code then it's all packaged, don't worry about that. Okay, so I've got them installed and set up. Now, if you're doing this on some other projects that's outside of NFCore, it's important to note we have a few files in the roots of the repo here which are helping us out. For the edit config tool this guy we have an edit config flow that sets everything up. Says what the indent line should be and things. And for prettier we have a prettier ignore file which tells it, it's like a get ignore file tells it to ignore certain stuff. And a prettier RC file which has some settings. Here we just specify that we want the width to be 120. I can't remember what the default is but it's quite narrow. So we don't change very much. So we have some config files in there. And then once that's installed I can go to the command palette here. So that was command shift B for me because I'm on the Mac. And you can see there's an option for saying format document. So I can run that and boom, done. So that's a bit better but command line it's still not great because I still need to actually remember to run that command all the time and we're trying to get away from having to remember anything. So if I go back into settings here if I look for format on save there's the magic tick button here editor format on save. And this tells VS codes to do what it says to format your files whenever you hit save. So now if I go to the change log markdown file and make some change if I hit save you'll see that prettier ran and it fixed this extra line break and it fixed the markdown here. I can put for the bunch of extra training spaces in here which are there and if I hit save they disappear because prettier runs every single time I hit save. And this then is the state that I recommend you run in where every time you hit save everything just is formatted automatically and you don't need to worry about it and all the tests should pass. Okay, final, I'm running a bit late so final little bit. Sometimes prettier breaks or doesn't do what you want it to do or there's some specific reason that you want to ignore stuff. I said that there's this pretty ignore file. One of the things that we ignore is actually like in the new release it's gonna come out for the template any minute is this email template file because this is actually broke because of prettier. So we've got groovy code mixed into HTML and so that confuses it. So there are sometimes legitimate reasons to ignore the code lenses. If you want to ignore an entire file you can stick the file name into the prettier ignore file and if you look at the prettier documentation you will see that there's the section about ignoring code. It talks about that file that I just mentioned and it also says that you can use the keywords within the file. Basically you make comments and you say prettier ignore and that will just ignore a chunk of that file basically. So there's a way to do this. This just ignores that one line and then prettier will continue to pass. But of course this is kind of exceptional use case only. Most of the time you don't need to do this and you should just let prettier do this thing. Right, final couple of slides just to wrap up then. Screen, come on. All right, very good. Yes, the final mention. I mentioned the YAML and Markdown who's the two of the biggest offenders here and also like Jason and a few others. But of course wouldn't it be amazing if we could do this with next flow code? And we had a standard about indentation after inputs and output blocks and script blocks and everything. Those of us who have worked for a while and especially who are used to code and using them and like them would absolutely love this. But it doesn't exist yet. However, prettier can handle plugins such as you have a website does the PHP code is like a sort of semi-unofficial plugin. And there is some interest within NFCore and next flow communities to build something like this. So the other day we were talking about this and Edmund actually kicked off a new Slack channel called Prettier Plugin Next Flow. So there's nothing really to see there yet but if you're interested and especially if you have any experience in doing it so would like to help out or just get involved go and check out that Slack channel and join in because please, please I wanted. I imagine you'll make a lot of friends if you can make this work and make all NFCore is all about standardization and best practices and this would really be icing on the cake for that. Right, with that I'm happy to take any questions be it about Lindsay or anything else who can hit me. There's no one moderating today so I would just keep an eye on the Slack chat there's a couple of things in there and or just unmute yourself, I believe. So it's a question about what do I use for next flow at the moment? Yeah, editor config, that's one of the main reasons that editor config is in there that's a general use tool is very, very simple. Like I say, it just checks like the indentation basically kind of two space for space and that's about it. So it's still possible to kind of do pretty variable formatting, but at least it's there's something. Like I say, hopefully we'll have a prettier plug in one day. And this asks, have I ever come across that black introduces bugs in my Python code? No, I quite like that actually when I hit save, if I don't see things move around that usually means that my Python code is invalid and there's a syntax error somewhere because black can't run on it. So it's actually the other way around. I think black probably saves me from bugs by alerting me early to the fact that there might be something fishy going on. I've never seen it introduce a bug. No, sometimes you might not agree with the choices it makes, especially black. It defaults to very narrow column width. So it breaks loads of stuff over lots and lots of lines quite quickly unless you change that default. But I've never seen it actually introduce errors or bugs. Moritz says that he likes the shorter 80 or 88 character line length. This is personal preference. I think that we've got it set to 120 at the moment partly because that's what I have everything else set up to when I came into this. There's quite, with the configs, so you'll see there's kind of a couple of too weirdish bits. I don't know if who was paying attention might have really spotted it. I'll go back to it again in this window. So if we go to the pipeline templates and go to the editor config file, you'll see here that we've got some stuff to be set up to be indent size four. So that's all files. So that would be like net flow and config files and everything indent size four. And then some stuff we've got indent size two. And then some stuff we've got like modules with kind of unsetting a bunch of stuff so that it's not clobbered, but yeah. We're doing some stuff here. And some of these choices you might be like, well, why are we using two for some files and four for some files? That doesn't seem very consistent. And it's the same for line length. There's a pretty simple answer with that. It's because we tried to minimize how many changes would be introduced into the code base when we started using these files. So Jason was already set as four, I think. So we tried to stick with four so that it didn't break everything. So we tried to actually minimize the number of changes which were introduced by these tools when we started using them. And stick, if we already had some kind of standard we tried to stick with that. Yeah, line length you can be set in editor config. I think that's what might be where we're setting it, I'm not sure. Okay, that's nice actually. If we can set the pristier line length in editor config then maybe we can just completely get rid of a prettier config file. That'd be nice. Any more questions? She checks that. Great, well thank you for sticking with me on what could be a bit of a dry topic but hopefully a useful set of tools for everybody. And as always, if you have any questions or any questions or any questions please jump into it and of course Slack and we'll be more than happy to help you out.


Hello, hello, hello, I'm Matthew Horky and I'm Sherin Tan. Together we are Exotic Wine Travel. We've spent the last three years traveling around the world writing and speaking about unique and exciting wines. Our journey has taken us to both lesser known and established wine regions. In 2016, we came to Croatia for the first time. We fell in love with the country and the wines. Prompting us to write the book Cracking Croatian Wine, a visitor-friendly guide. Now we're at it again. We're going to re-tour the country and bring our book to life. We aim to take you deep into the Croatian food and wine scene. So sit back, relax, and join us as we go Cracking Croatian Wine!