A nice new brother WP1 word processor. I could connect the Raspberry Pi up to the UART. Okay, that's interesting. You can see that I'm successfully interacting with the Raspberry Pi sort of. I am actually going to take the time to write a whole new terminal emulator. I think we have most of what we really care about. And of course, once we've done with the state machine, we then have to start working the actual implementation. That will be fun. So I think I might actually be overthinking this slightly after looking into the way the various terminal standards work. For our purposes, we're probably okay with VT52, which is a much simpler protocol than VT100. The biggest reason why I want to switch away from the ADM3 is the ADM3 has some nasty limitations, like the fact that the cursor keys actually produce these control keys, which if you've ever used VIM will look strangely familiar. This is not a big problem, but it does mean that application programs can't distinguish between, for example, Control-L and pressing the right cursor key, which is irritating. But we can get all that from VT52. So we probably don't want to go the full VT100. However, the extended ANSI sequences that we actually want to implement aren't supported by the VT52, so we'd have to add the support for parsing these anyway. So I think I'm just going to go ahead and push on with this VT100 state machine. We can always cut it down to a VT52 later if needs be. I think this is going to end up being too big to realistically fit into a CPM BIOS. But as a standalone program, it's fine. Over here, I have the actual digital research VT102 documentation from the excellent VT100.net. And it appears that there are more instruction sequences than were in the man page I was looking at, either that or they're just not in the same order. So we've done G, we've done J and K, but we haven't done H, which is the cursor motion command. So let's just do that. And this is called cup. Now I also happen to know that F does exactly the same thing. Lowercase F does exactly the same thing. So let's put the lowercase ones here to CSI cup. And this wants to go after char. OK, so this is actually, let me see. What are the order? Cursor to line PL column PC. So the line goes first. I'm going to refactor this because we can reuse all this code. So this is going to be a routine too. So our char here is just going to be like this. And likewise, we want a set Y, which is exactly the same but in the Y direction. So to move to a particular location, we do this. So this is pulling the two parameters out of the parameter buffer, and then it sets them as X and Y. If no parameters are provided, then we go to 0, 0. Now the origin is actually configurable by a mode switch, but I don't think we're going to implement that. I don't believe anything actually uses it. Most actual console programs these days, if they do anything at all complicated, they use something like cursors, which abstracts over your terminal details. We're going to need a custom term info anyway, so we can just tell cursors not to use origin mode even if it tries. OK. So we've got to J, and GH, is there an I? S-I. It'd be nice if this was in alphabetical order, to be honest. Not sure there is one. Here's setting origin mode, and you see there's a question mark in the command, which means it's private to deck, not technically part of the standard, although I wouldn't be surprised if this is now part of the de facto standard. So let's look at the CSI set. These are here. OH is in this. I just failed to spot it yesterday. Excellent. There's no I, so we've done J and K. Right. L, insert the specified number of blank lines. Interesting. This isn't mentioned here. That looks like describing VT52 mode, to be honest. Oh yeah, the VT102 can be set to VT52 mode. We're not going to implement that. That just seems like a waste of time. So there should be a summary in Appendix C. It's not very. I really want just a global index showing all the commands in a row, rather than having them grouped into categories. Anyway, cursor motion commands. I think we've actually done all these. Tab stops. I don't know if we need those. Erasing. We've done these. L and M. Insert and delete. OK, what I wanted to know is actually what they did, but this turns out to be fairly obvious. So also I called this the wrong thing. So we want to do EL to do CSI EL. So we have do CSIL insert lines. So this is a fairly straightforward loop around the insert line primitive. So we need to load the parameter. If the parameter is 0, increment it to make it 1. Stick it in B. Insert a line. And just loop around that many times. If you try to insert lots of lines, it will spend a reasonable amount of time here. So what we could do is compare against the screen height. If you try to insert more lines than that, then of course you might as well just clear from the beginning of the current line to the end. Let's not worry about that. No one is ever actually going to want to do that. You're going to be inserting or deleting one or two lines at most for scrolling up and down. OK, delete line. Again, the same but different. And we could probably refactor this. We've done this logic several times before. So having a routine that just you give it a address of a routine and it just calls that routine tty parameters times would shorten this. But let's stick with this. OK, delete the number of characters, indicated number of characters on the line, and erase the number of characters on the line. OK, what's this called? ECH. Not in the ET52 documentation. DCH, delete characters. This is something that we are actually going to need to implement because things all want to use it. The trouble is we can't actually do this without adding a new primitive because the state machine here doesn't know what text is on the particular line. Only the back end knows that. And I'd rather not add too many primitives. You know what? Let's ignore this one. This is only going to be a subset of the ET102 protocol. And it always was going to be. OK, and likewise, there's a whole pile of lower cased ones. So HPR, move cursor right. But we already have move cursor right, C-U-F. There is no HPR documented there. Lowcase C-D-A. Yep, implement that. Do we have terminal ID string? This is the right string. So we can pull this into its own routine. And this is called dA-identify. And let's put the string up by the code that uses it. That way we don't need to define a symbol. So this then becomes dCSI-DA, OK. VPA, move to the indicated row current column. Not in that documentation. HPR, not in that documentation. HPP, horizontal and vertical position. Yeah, we've done that one. Clear tab stop. Implementing tab stops is actually straightforward. We would want an array of 91 bits through each column. So advancing to the next tab stop, we just work forwards until we find a set bit. Or we reach the end. But I'll ignore that just for the time being. I strongly suspect that this is not going to be completely functional the first time around. OK, modes. This is H, set mode, C below. As you can see, it takes a numeric parameter, possibly a query private parameter. The two big ones that I don't know if anyone uses is IRM, which sets insert modes that if you over type into a line, the text shifts right. The other one is new line mode, where if you send a line feed character, that's ASCII 10, does it move the cursor over to the left or not? Normally not. But again, I think we can get away without that for the time being. And again, we've got reset mode, which is the same thing. You know those character attributes. This is actually the sequence that was causing us problems. There are lots of character attributes that should be defined here. So at some point, we are going to want to implement them. This particular video chip actually supports a whole bunch of these, including underscore, reverse video. It may do half bright and bold. But for now, we can ignore it because it will get parsed and discarded. Scrolling region, now, when the terminal scrolls, you can actually tell it to only scroll within a certain area. This is useful for programs that have a top bar or a bottom bar, and you want to scroll the text without affecting those. Without this, then the program would have to redraw the top and bottom bar each time. Excuse me, I've got some tea here. So this would give a much nicer experience for editor programs, which is exactly what I want to use this for. But it's not required because cursors should be able to cope without it. So ignoring, if we've done index and reverse index, that's a yes we have, right? No, yes we have. Right, but we haven't done seven and eight. So this says that it saves the cursor and any attributes. We don't have any attributes yet. So this is straightforwardly loadCursorXY, saveCursorXY, and the attributes can go in there as well. Restore cursor and attributes. We simply do this in reverse and go to Update Cursor. OK, mediaCopy. So these old terminals had the ability to plug a printer in so that you would have your terminal plugged into the main frame in another room. But you would have a local printer plugged into your terminal. And the main frame is capable of printing on your printer by talking to your terminal, which is what these are for. Now, my brother does have a printer, which I'm not using because it's broken, so ignore. It's also really annoying to use, so I would have to figure out all the brother OS system calls to do it. Reports, you can ask stuff about the terminal. So these we've done are reports the position of the cursor. Whoops, what did I just do? So sending an N with a parameter of six causes the terminal to echo back where the cursor currently is. So we're going to want that. There's our CSI set, lowercase N, so device status report. So we fetch the parameter. If it is five, then we print terminal OK. So this sequence here, so s square bracket zero N. These are private and do with the printer. Right, report cursor position. Right, now this one is tricky because we actually need to write back decimal values. And I found the easiest way to do that is to go and find somebody else's code and use that. What's the simplest routine we can find? There's this. Yeah, that seems to be plausible. It may be possible to do something with decimal mode, but let's just go for this. So we want to, this is going to be a generic routine for printing the current value in A, which this is doing by repeated subtraction. And the TTY type back A routine is a primitive that is going to add A to the output keyboard buffer. It's the equivalent of this routine but for a single byte. Right now this is just printing stuff to the console. So in fact we can replace this with C-pop-HL. And then this, we're going to make generic as this. So now the back end only needs to implement TTY type back A to push one character into the output keyboard buffer and everything else is dealt with by the state machine. And we're going to want to change this to this. Good, and it compiles. Right, report cursor position. We want to output an escape followed by a square bracket. Now we want to output the cursor Y position incremented by one to convert it back to terminal numbering, one base numbering. Print it out in decimal. Output a semicolon. Print the Xordinate final character of the report. OK, so let's just do a quick test. We want a status report is 6n6n. So we run it. And we get type back escape. Type back 5b, which is going to be this. Then we get a digit. Then we get a semicolon. And then nothing happens. So what's the wrong with this code? It has actually emitted a zero here. Wait a minute. It is emitting leading zeros, which we don't really want, to be honest. The issue is that we call n1. Oh, right. Yeah, this works by repeated subtraction. We start by with the left-hand side. So each time, yeah, so it mutates a along the way. It doesn't have to actually save the value of a. So this seems to work. Is 6e the right? 6e is an n. Yeah, there is no reason why this should not be printing multiple characters at this point. I mean, it should be flowing through this code here. Wait, an n, it shouldn't be emitting an n at all. If it gets an n, it's right. It's done this. That's what it's doing. This is why we write the tests. OK, 0 0 6 semicolon 0 7 6, which is correct. We should probably do something about the leading zeros, but I don't think it matters. I mean, anything that is parsing numbers should be able to cope. So we will just leave that for the time being. Load LEDs, we don't have any. This is vt52 mode, which you don't care about. OK, I think that's our state machine. Oh, all right. So what's next? Well, we need to put this into a actual program. And I do have one. OK, how big is our program? 12 39 bytes. Wow. Over a kilobyte of Z80 code. That really adds up. Well, we can actually make it smaller by replacing a lot of these JPs with JRs. OK, well, this is our test framework running under CPM. I do actually have here a framework that produces bootable floppy images for the brother. So let's copy this code over. And it was in CPM ish, brother, tools, brother, p1, tools, pterms, and 80, into vt102.z80. Get rid of this.  It compiles with this makefile. And makefiles are the worst build tool ever. The only thing they have going for them is that they are everywhere. But we can, I prepared this earlier. What this does is it compiles the main program here into bterm.image. And then it compiles the bootloader and the file system image, which refer to these two binary files. Hopefully that made sense. So make. OK, and we do actually want to do some adjustment here. So this is now in relative mode. So we don't want our test program. OK, it compiled correctly. So if we open the file system image in hex editor, here is our state machine code. These zeros are interesting. I would expect these to be at the end, honestly. This has done a modular compile and then is calling the LD80 linker to link them together. So here you can see code segment for main.z80, which is empty, apart from one ret, which is the c9. You can see it at the top of the file here. And our vt102 state machine then goes below it. So that's why this stuff appears below the zeros. I never put this back into cseg mode. OK, well, everything from here down is our faked up test framework, not test framework, backend. So let's create a new file for that at display.z80, which we also want to compile, like so. And of course, it instantly fails to assemble because we have not imported and exported all our symbols. Now, one of the downsides of ZMAC is that this is a direct descendant of the assembler and linker from CPM machines. And it supports seven-bit external identifiers, which is nice. So we're probably going to want to do some renaming here. And we also want to copy our magic string, like so. So we're actually going to want to rename a lot of these symbols. So that's three, six, that's seven. TTY cursor for Update Cursor is, I think, global is correct. So global here exports a symbol. Over here in VT102, we are going to want to import it, like so. And Update Cursor becomes TTYCurs. And that has indeed solved the problem. And we also want, let's move, add AHL into main. And this is why add AHL was in capital letters because it's already configured for this. OK, display. Dell line, insert line, clear line. In fact, I think TTY is wrong because this is not part of the terminal. This is all part of the back end. So let's replace that with DPY for display. So DPY, insert line, delete line, clear line. TTY, insert line, becomes DPY, insert line. TTY, delete line, becomes DPY, delete line. And TTY, clear line, becomes DPY, clear line. That didn't work. TTY, clear line, DPY, clear line. OK, that's better. We have fewer problems. Typeback. Typeback does not actually belong in display because this is part of the keyboard. But we're going to leave it here for the time being. Typeback. This is going to become keyboard type. The seven character identifies as actually an extension. Originally, it was six keyboard type. So now we're just missing put hex eight and put hex 16. Which, why is this calling those? Put hex eight is, ah, yes. The print routines, which I completely forgot about. So the print routines write a character and increment the cursor. So they want to write the character C. So DPY print, actually print ASCII, I think. Because we're going to have a print unicode as well. We now want to move write one with line wrapping. So now we're going to do this. So increment cursor X. If it is at screen width, that means we've just passed the right hand screen edge, then there's nothing else to do. All we need to do is update the cursor position. So once we've passed the right hand edge, we want to move on to the next line. Actually, but this is slightly different. So, right, we actually have the code we need to do that here. So if we, this is all we need. So to print UTF-8, this is going to be almost exactly the same code. So let's do, let's put a label here. Print UTF-8, that lives down here. Print unicode, advance the cursor. All our unicode characters are going to be a single character cell wide. It's extremely shoddy unicode support. OK, so print unicode, print ascii. OK, the cursor X and Y positions are actually owned by this code here and need to be exported. So the question is, do we want to have the back end have to read them directly? Or do we want to pass them in as parameters to these routines? Honestly, if they're passed into parameters, then it makes some of our existing logic simpler. So let's do that. So where are we calling clearL? You see here, for example, we have to reset cursor X and fiddle with it. So let's pass the cursor position in in DE into all of these. These go away because we don't have anywhere to print things anymore, apart from to the screen. They're just all red. This goes away as does all of that and that. So this is our back end currently, very simple. Oh, yeah, and we also need print ascii, print unicode. OK, back to our state machine. dpy clearL, right. Here is our erase code. We're raising from lines d to e. So we don't need any of this. We can replace this with some much simpler code. d is actually the y location. So it's already in the right place. So clearLineD, incrementD, compare it with e, like this. It assembles arrays, the current line. So here we want to load DE with cursorX. This is a 16-bit read of two 8-bit values. So X is the low byte and goes into E. Y is the high byte and goes into D. And those are the only two places it's used. Curses, right. This one is actually a little bit more complicated because we're calling this in lots of places. So let's actually replace this with updateCurses throughout. Put this one back again. So this then becomes this. So whenever we want to update the cursor, we call updateCursor and that loads the appropriate parameters. dpy del, deleteLine. So this deletes this many parameters. Simple enough. Scroll up. This can be simplified. We want to scroll up. We want to do deleteLine0, like this. To scroll down, you want to delete the bottom line. All right, this. IntsLine. So again, this is the same as below. Now, intsLine is only actually going to use D. It's not using E. However, we can't load D from a 16-bit. We can't load D from memory directly on the Z80. Due to the rather weird unorthogonal instruction set, you can load 8-bit values from memory into A. And you can load 16-bit values into memory from memory into any register pair. But you can't load an 8-bit value from memory into a register that's not A. So loading x and y into D and E here is actually cheaper. Now I come to think of it. Is it cheaper? So in order to load D from a 16-bit value, we have to use this instruction, which is 4 bytes, ED5B into address bytes, and 20 cycles. In order to do it through A, we would have to load this. So that's 13 cycles. And then do ALDD comma A, which is here, 4 cycles. So it's actually cheaper to do it through the accumulator. Not a lot cheaper, but it is cheaper. We could push and pop D to save it. But that is 10 cycles ago. It's shorter code, but it still uses up 20 cycles in both cases. Anyway, let's just go with that. Where else is Dell used? There, insert line, yeah, print accumulator. So we put the character in A. And this time, we do want both x and y. So we print that and fall through. OK, I think we're good. How big is our program? Our program can occupy one track of the disk. And each disk is 12 256-byte sectors. So this is the first sector of the second track. This is our boot sector. So we actually have quite a lot of space. And we have absolutely loads of RAM. OK. So let's do some of our main code. Now, so this is my old pterm. This has got the code in it to actually read from our interface. So we're going to copy some of that. So on entry, we want to reset the interface. And then we start the main loop. And here we go. Right, this is the code for testing to see whether there's anything in the interface. So I can cut and paste this. So this read, this is the bit that reads the character. Then we acknowledge it. And then we wait until we're back in the done state. And then we acknowledge the done state. So instead of calling CPM here, we're actually going to call into our state machine. We're going to call ttyputc. So this actually needs to be exported. Then this code is as it was, like so. And then we loop. And we need to import it as well. OK. So that builds. So we should now have a working terminal that is reading characters from the outside world and pushing them through the state machine. I am just actually wondering, this can probably be simplified. Yeah, let's make another file. We're going to put all the interface code in one file. So process from the interface. There's a reason for doing it like that. Because that lets us do just ret z to mean that if there's nothing return, OK. Let's go back to our make file and add this. And likewise, there's going to be a keyboard file as well. So interface, we want to global int read in it. This is going to be. So this is all slightly longer code than if we would just inline it, but somewhat easier to read in it. Int read. TTY put C. It's not called there, but it is called here. OK. Undefined symbol, pre-new, that should go. Right, we haven't exposed these, pre-new. OK, it compiles. Good. Now, it's not actually going to do anything. Because we haven't done any of the back end yet. Do I want to do that now, or do I want to do the keyboard? Let's do the keyboard. That should be straightforward. So again, this is going to export keyboard type here. I'm going to add it to our list of files. Why is it not liking that? External label defined. Oh, yeah, that should be global, not external. OK, in. Yeah, keyboard type is redefined, but it doesn't say where, which is kind of annoying. OK, now, so keyboard read will, hmm. Keyboard read is going to read the character from the keyboard and snatch it into our input into our output buffer. There's also going to be another routine that takes from the output buffer and pushes it into the interface. I think we want to rename IntRead to IntExec, because it's not just reading from the interface. It's reading and processing a byte from the interface, because it's calling Putzi. Hmm, no, actually. We're not going to do this at all. One of the good things about the VT102 protocol is that zero bytes are always ignored. They're just used to waste time. So we read the byte into C, and then it stays there until the routine exits. So here in main, we are just going to call IntSee and then immediately print it. Now, this is safe because we're going to test it for zero here. Keyboard type undeclared. Interesting. Oh, yeah, there we go. Keyboard init undeclared, yep. And these need to go into our keyboard routine. And we need to export these with global, and it assembles. How are we doing? Loads of room. OK, now how's the keyboard actually going to work? Well, it turns out I've done all this work, which is nice. So it's in cpm-ish arcbrotherwp1-keyboard.z80. It's this code here. So in fact, we're going to stash it on the end here. So this is the code that actually initializes the keyboard produced from reverse engineering. What these are doing is poking various output bits to, as far as I know, enable the keyboard. I forget what RE stands for. But yeah, the keyboard and the brother is wired up to the synchronous serial I.O. So that's clock and data. So these ports turn on the keyboard itself. And I think this initializes the serial I.O. controller, which is straightforward. We have a, the way the cpm keyboard driver works is we have a buffer containing a single pending key, which we're going to have to expand on for this code. So let's do some of that. Now we're going to have a, we're going to use a ring buffer. We have tons and tons of memory. So we're going to use a 256 byte ring buffer because that makes a lot of the maths easier. It means we don't have to mask stuff when wrapping around. But we are going to have a read pointer. How am I going to do this? So on the 6502, then I would use an 8-bit offset into the keyboard buffer. But on the Z80 indexing is like that is kind of annoying. That's why we had to do that add AHL thing. So there may be a cleaner way to do this. Let's put a variable in the data segment. This means that it will be initialized to the address of the keyboard buffer. The low byte is going to be the offset into the... No, I was overthinking this. You know what, I'm just going to use 8-bit bytes. There's a cheaper way to do this. But I'm only going to do that if necessary. OK, so when we call keyboard read, what we're actually going to do is test the keyboard hardware, attempt to read a key, and then push it into the ring buffer. There's then going to be a different piece of code, which is going to read keys from the keyboard buffer, the ring buffer, and push them out to the interface. The reason for this is so that we can use keyboard type to insert keys into the keyboard buffer. So keyboard poll here is actually doing most of the work. Yeah, returns the ASCII key code in A, or zero if nothing is pending. Trouble with this, yeah, this is combining scan codes and keyboard codes. The keyboard returns scan codes, which you then need to turn into ASCII. For the ADM3, because all keys are one byte long, then we can just deal with ASCII throughout. On the VT52 and VT100, key reports can be more than one character long. For example, the cursor keys return longer, more complicated values. Here we go. They return these. So every time you press a cursor key, you're going to have to push three bytes into the keyboard buffer. So what we're actually going to do is, so this is going to read a scan code, or that's going to be a keyboard exec. Read the keyboard scan code, and if one's present, converts it and pushes it. So the first thing we want to do is to read the scan code, which is this, names, names, keyboard pull, pull into the buffer. So test to see if there is anything in the synchronous serial IO buffer. If there isn't, do nothing. If there is, read it into A and re-enable the interface. It needs to be enabled each time through. So now we want to convert it. This involves keeping a pile of state, which is what these modifier bits are for. I forgot that there is a keyboard table. Yes, we can actually use most of this. So let's get rid of this. Keyboard modifiers is our set of modifier bytes. This is our converter, which we will pick here. Let's move this. Move all of this. Now, for the keys that need to return multiple values, our keyboard table, which I will deal with in a moment, is going to return high bit set characters. So if the high bit is not set, then this means the value is really ASCII, and we just type it. Typing it pushes it into the keyboard buffer. If the high bit is set, then this means it's an extended key, and we're going to have to convert it to a multi-byte sequence. So let's just leave it at that for the time being. Now, this will not compile. Wow, for many reasons. One is that we don't have the keyboard map. We don't have the brother include file with all this stuff in it, and the Z180 extended characters. So let's copy those. Oh, we've already got the Z180 include file. So let's just active Z180. OK, so now everything works except for the keyboard map. The keyboard map in CPM-ish is generated by this C program. This just, you give it the scan code, and you give it the normal and shifted ASCII representations, and it will then generate the include file. So we're going to copy this, and then we need to add a rule to actually build it, like so. And then we have to add a rule to generate the keyboard map, like this, like so. And we need to add a dependency to say that keyboard.Z80 requires a key map, and then we have to include the key map. Let's see what this does. It worked. Well, I've got all that syntax right first time. So here it compiles our key map tool, and then generates the key map, which is in here.  The scan codes from the keyboard use the top bit to decide whether a key is pressed or not. Here are the tables. Here is our keyboard buffer. That should really not be there, but we've got lots of space, so I'm not going to worry about it for now. OK, so what's next? Type. This is going to want to write the key into the keyboard buffer. And the keyboard buffer is that ring buffer here. The way ring buffers work is that you have a right pointer and a read buffer that point into the ring buffer. When you write, well, whenever you write or read, you read the character pointer to do by the pointer and advance the pointer. You know when there is nothing to read when the pointers are pointing at each other. And you know when there is nothing. Yeah, let me start that sentence again. A ring buffer is a FIFO queue where you can keep adding stuff on the end and reading from the beginning so that you get out all the data you put in in the right order. And it's implemented by having two pointers into the ring buffer. And they are annoyingly subtle. So one of the pointers points at the byte. One of the pointers is post increment, and the other is pre-increment. And I'm trying to remember which way around they are. No, actually they are both post increment. So yeah, when the two pointers are the same as they are here, this means that the byte address 0 has yet to be written to and has yet to be read, which means that there is nothing to read. So when we write, we write to this address and increment that to a 1. Then when the read comes along, we can see that 0 is not the same as 1. Therefore there's a byte to read. When things get complicated, it's when the ring buffer fills up. Because then the right pointer has to be less than the read pointer. Or rather, not the same as the read pointer. Because if the right pointer advances to such an extent that it hits the read pointer, you run out of space. So, slash the character in C, read OK. So what we want to do is to check to make sure that the buffer isn't full. So to do that, we want to see whether the right pointer can be incremented. If it is equal to read pointer plus 1, so decrement our local copy of the read pointer, not the real one, and compare it against the right pointer. So take the right pointer, find the location in the keyboard buffer, write the key, and write back. We are only writing the right pointer because here we corrupted our local copy of the read pointer. We can do this in a less confusing way. So take a local copy of the right pointer, increment it, and compare it against the read pointer. That's actually a little bit more clear. Because what that's saying is if the right pointer cannot be incremented any more without colliding with the read pointer, then the buffer must be full. And this then allows us to do ink D. It'll probably be a bit slower, but it's clearer. Does that compile? We need to fetch this. OK, so we should now be reading ASCII keys from the keyboard and pushing them into our ring buffer. Next job is to push the keys from the ring buffer to the interface. And in fact, we're going to do it like this. And I think I've got pull and push the wrong way around because it's natural to talk about the ring buffer. So this is going to be pushing keys into the ring buffer and pulling keys out of the ring buffer. So we want to change this. Keyboard pull is going to read a value out of the ring buffer. OK, so we read our pointers, return 0 if the two pointers are the same, calculate the address of the read pointer, and return it. Undefined symbol. But yes, we have gotten once more to expose. OK, now we want int write. So check for nothing. But of course, we only want to write it to the interface if the interface is writable. So we are actually going to have to query the interface state and only pull the value from the ring buffer and write it to the interface if the interface is actually writable. So this is actually straightforward. It's this code here. Let's toggle that so that we're going to leave the state. In the flag register, we're going to leave z to be true if that should be writable, if the interface is writable. So over here in main. So if it is not z, skip this. OK, that's a fairly simple bit of code. Right, writable code. And once again, we're going to copy our existing code from pterm, wherever it's got to. Interesting. Oh yeah, writable 0. So that's not actually necessary, because the zero flag will be set automatically by the previous and. So this is the code here. So we know the interface has to be writable from the console. Write it to the interface. Our key is in A. So we can just do output. And then we just copy this code here like so. So I think we are mostly there for at least this bit, which brings us to the last bit of code, which is the important bit. In other words, the code that actually does the work. Now, this is going to be irritating. So once again, we go look at the brother code, tty.z80. This is the code needed to actually write to the screen. The big thing we're going to need is our character table here. The brother video memory's character set is kind of funky. So this does the conversion. However, the CPM-ish code, it writes directly to video memory, which, if you remember correctly, was causing us no end of issues. And so we can't do that. So we are going to define a back buffer, which we're going to write to. And this is going to be flushed to the real video memory whenever it's safe to do so, so that we only ever update the screen during the vertical interval refresh time when we know that the video memory is not being read from. So we want our screen width and height, which are here. I'm actually going to add yet another file, not ink but lib, because this will allow us to include it from multiple places, matlib constant. And here in our back buffer, we can do screen width time, screen height. Now we will actually need two bytes per character for this because we need a byte for the character attributes, that is which font to use, whether you want it reversed, underlined, et cetera. But we're going to ignore that for the time being. So just going to leave it at that. Now how much space do we have left? I think we're OK for space. Let's see. Let's just do one of these in. So I'm not sure why it's insisting on putting these data segments in line. It's probably, wait, I know what it'll be. I know what it'll be. My flags to LD80 are probably not correct. What's the help like? Not good. Give me a sec while I just go and find the documentation. OK, I sorted it out. It was, in fact, LD80 was doing everything correctly. It's supposed to put the data segment after all the text segments, this way that you don't need to store the data segments on disk. I mean, they're always going to be initialized to zero. However, I had managed to, I put this in the code segment rather than the data segment. So that big block of zeros I was seeing was the keyboard buffer. But now it's not there. So we should be fine for space. OK, so this is our back buffer. It's just a simple array of bytes, which we're going to copy into the real video memory. As our screen is 91 wide, we are actually going to have to create a lookup table to find rows. Because multiplying by 91 is annoying. The Z180, which this is, actually has a multiplier, but it's only 8 bits wide. It's capable of doing 16-bit multiplication by doing a little bit of basic maths. But I am not going to, because again, we have lots of space, so we are going to create a table. That's actually going to be back buffer address table. So we should be able to see that somewhere. I think the display comes. Oh yeah, here is our character map table, which is there. All of our blank display routines that just have a ret in them is this row of C9s here, which means that our line table is then over here somewhere. And we should be able to see lots of that's not right. Back buffer plus line number. Line number is increasing from 0 up. So we should be seeing these numbers are all garbage. Something's not right. I mean, this is wrong anyway. This should be like this. So xyzcc9bcd8fod. Then these should now be the addresses into the back buffer. 803e41d3a, wait a minute, wait a minute. I've broken something here. Looking at this, it's rebuilt this, but it's then not done anything with it. So fs.sim depends on fs.z80s. fs.z80 depends on boot sim be termed image. So why hasn't it rebuilt fs.sim, what does this do? OK, here's our table, which is all the same addresses. I think I got my makefile wrong. So let's just update this again. No, that's not rebuilding. Oh, well, that's wrong for a start. That's better. That has actually gone and rebuilt everything. This is one of the reasons I hate makefiles. So let's just do this, make sure it rebuilds. Yes, it's done it. It's gone all the way to the end. Good. So after our character table, we now have 66ad66c566dd66f5, et cetera. So what are we going to do with this? Well, in order to print a ASCII value, we want to convert the ASCII value to brother character set. So we subtract 32 because our table is offsetted by 32. And then we call our good old friend adahl. It's not the scan code. It's the screen code. OK. So now we want to calculate the address of the line. So to do this, we load hl with backbuffer address table, put our y value into a and call. Yep, is adahl again, cursor. We now want to add on our x. So we put our x location in adahl in a and add it on again and write a screen code to video memory like this. Prenew will ignore for the time being. Right, dpycurse. This will move the cursor to a particular address in video memory, not the backbuffer, but video memory. And I do mean the actual address. So we are going to want to copy another table. So over in constant, we want to do around base. I believe that's 500 and screen stride is 192. Yeah, we're using a different memory map. So this isn't the same here. Oh, yep. And constant, constant.lib. So in order to put the cursor somewhere, we're actually going to have to do some of this, this bit, to calculate the address of the cursor. It's the same math, but using a different table. The video memory screen layout is quite different from the backbuffer screen layout, which is why we need two tables. So here is CPM is just code. So here calc cursor you see is basically the same thing. Nope, nope, nope. This is all wrong. So this does not if this calculates the address into the backbuffer table.  And we're going to have to do the same thing here. And for the first time, we're going to start poking the video chip using code stolen from CPM ish. So this will write HL to the video cursor address in the video chip. And it doesn't like that because I need some of these. These are the addresses of the two ports where the video chip lives. I need to also include z180. There we go. And it compiles. So we should now be able to print characters onto the screen. So we're almost at a point to test the thing. I'm actually going to copy all abstract this out because we're going to want to use it a lot further down. So we pass in the location in DE and it does the calculation there. So here to delete, let's actually go for here to clear a line. That calculates the address. We're then going to use our L dear trick to clear the memory. It's HL2D. So it's going to be like that. OK. Insert line and delete line, we're going to leave blank for the time being. We're going to want to do that next. Actually, now I'm not going to memorize them because we actually have we can't use the CPM ish code because this is working with video memory addresses. OK, so to delete a line, we want to copy up. Now, I don't think we can use LDIR for that. LDIR always copies in increasing memory order. Can we use LDDR? Yes, we can. OK, the problem is here that in order to scroll the screen, we have to move a whole chunk of video memory around. To delete a line, we want to move everything up, which means copying everything from line one to line zero. Then from line two to line one. And then from line three to line two, et cetera. It has to be done in that order to avoid overwriting stuff that we need. So you want to read from a higher address and write to a lower address. So you want to do that in increasing order. OK, so we now have the address in HL. Just trying to think how we're going to do this. Put that into DE, which I'm looking for 16 bits of tracks. So we want to calculate how much we want to copy. So we've calculated the address, our destination address. We take our SOAR, so the end of video memory minus that address is slightly more than the total amount we want to copy. Because of course, there is nothing to copy into the last line. So we now want to do the subtract. So I think we need to do that. And then we stash the value into BC. OK, our line address is still in DE. So copy it into HL. HL is the source. So we actually want to add a screen row to that. I run out of registers. So by adding on the screen width, hang on, we don't, the screen width is an 8 bit value. So we can use our old friend, add AHL again, which doesn't touch any registers so that we can actually put our size into BC. So this should move everything up from the cursor line down. But it won't clear the last line. So what we want to do here is, are we on the last line? If so, don't do any of the copy because there's no point. Now we want to blank the last line. And this is straightforward because it's basically what we've done down here. But we don't need to actually do any calculations. So get the address of the last line, address of the last line plus 1, screen width minus 1, blank the first character, copy. So the same code happens here. If we're on the first, now actually the same thing applies. So if we are, yes, if we're inserting and we're on the last line, so it is in fact the same code, then we just blank the last line. So up here, we're then going to want to do all this stuff again just differently. So in fact, yeah, source HL target DE, swap HL and DE, put that into a miami of the direction because this comment is wrong. So this is going to be. So that's copying from line plus 1 to line. We actually want to copy line to line plus 1. But in reverse order, actually, this makes life a bit easier because these values are fixed because we're copying backwards with LDDR. So the bottom of the screen is always going to be at a fixed address. So source is going to be the last line of the screen. Destination is going to be the second last line of the screen. And BC is the amount to copy. But we actually, we want to start on the last character of the line because everything's going backwards. And do I have my insert and my delete backwards? No, no, I think that's correct. And it can be even assembles. OK, then. So there are two more important things to do, which is we actually want to initialize our TTY, which is to reset the cursor position and the state and clear the screen. And to clear the screen, all we're going to do is call doedall and then updates the cursor position. OK, and the second thing we need to do, which is the really important thing, is to actually flush the back buffer onto the front buffer. And in our main loop here, we are going to do this whenever we see that we're in the vertical blanking interval. So you see, I've got them commented out code here from CPM-ish that's doing the test. So call dpy flush. There's display dpy flush. OK. So we need to read the status register of the video chip and check the vertical blanking bit. So if the vertical blanking bit is not set, return, do nothing, because it's not safe to update the video memory at this point. However, if it is set, then we can actually do our copy. And how are we going to do this? Because this kind of needs to be as fast as possible. It's actually probably worth putting in a bit here to detect for a dirty back buffer so that we only do the update if the buffer has changed. But I'd have to go through and actually, there aren't that very many places to put it. But let's try it without first and just see what happens. OK. So for every line, we need to copy from the back buffer to video memory, according to these two tables here. So DE is destination, HLS source. I'm running out of registers. So we want to repeat this screen height times. So there's a loop. We wish to copy this many times. Copy. So this is advanced DE and HL. HL has got our lines packed tightly together. So after we've called LDIR, HL is in the right place. However, DE needs an adjustment. Let's put one of these in, I think. It's only a handful of bytes and it simplifies things. E, E, D. So this will advance DE to point at the next video memory line. And we want to repeat this 14 times. 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. So what that is, this is the same construction I was using down here for our lookup table. Rept causes the assembler to repeat the contents that many times. So by just unrolling this loop, we avoid the machinery needed to actually count. That's not a lot, but the fact that we're using all the registers means that we also need the machinery to save and reload our count. And this needs to be as fast as possible. So this should actually help. And it assembles. Size-wise, comfortably within the limit, this piece of repeated code here is going to be our flush routine. I can actually go and find the listing. No, I didn't tell it to make a listing. Never mind. Listing, listing. Yeah, listing in ZMAC is kind of annoying because it's very opinionated about where it wants to put things. Yeah, I would need to add subs. I want to change sent.rel for sent.list in the output file. That's actually worked. Amazing. So here we go, display.list. Here is the assembler listing of our routine. Here's our flush. And here's the repeated code. OK, well. Now this will almost certainly not work, but it should give us a starting point. This is the boot code. Where do I set the cursor to flash? I think it's CPMishers job. Yeah, it's here. We need to initialize the video chip. So we actually want a dpy init, display. So what this is doing is it's disabling the windowed mode, which for some reason the brother leaves the video chip when you boot stuff and sets the cursor to be enabled and flashing. OK, so I think that is actually ready to test. So I'm going to stop on a cliffhanger and get on to the testing next time. It also means that the people who don't want to watch multiple hours worth of coding can skip straight to the good stuff. Anyway, see you then.


I'm going to talk today about the Meta-P package. This package, which is available from C-RAN, does the meta-analysis of significance values, also known as P-values. This is useful if, in fact, you don't have effect sizes. For instance, you have a number of old studies which just report a P-value, or the primary studies have reported incompatible tests. If you do have effect sizes then obviously using them would be preferable. And there are a number of packages on C-RAN which enable you to do this, perhaps the most popular to our metaphor and meta. What methods do we have available in Meta-P? Here I show the various methods, the eponym for it in normal type, after the name of the function in Meta-P in parenthesis. There is a group of methods using some transformation of the P-values. They rely either on the inverse chi-squared distribution, inverse normal distribution, inverse students T, or the inverse logic. And there are also a number of methods that work directly with the P-values, either with the sum or the mean of them, or just using a single selected P-value. I think of these the most well known methods are Fisher's method, which relies on the sum of logs, which is, in fact, an inverse, with two degrees of freedom and inverse chi-squared, Stouffer's method, and possibly Tippitt's method, the minimum P, although that's not usually, not often known by that name. Now how do you use them? Well, all the functions have as their first parameter a vector of the P-values, and they have some additional parameters. They all return an object of class Meta-P for which there is a plot method, and a function-specific class which supports a print method. The example shows you loading the package, extracting the particular data set from the collection of data sets provided. It's one set of teacher expectancy ratings, and then using Stouffer's method on that, which is sum Z. That returns the relevant statistic and also the P-value. As you can see, this rejects the null hypothesis. You'll see the values of P for the teacher expect when we come on to the graphical display section. I said that show that it was rejecting the null hypothesis. Well, what is the null hypothesis? Well, it's well defined. It's that all of the P-values, the P-i, have a uniform distribution on the unit interval. However, there are two classes of alternative hypothesis. One, that all of the P-i have the same unknown, non-uniform, non-increasing density, and the second, that at least one, possibly more as an unknown, non-uniform, non-increasing density. If all of the tests being combined come from what are basically replicates, then the first one of these is appropriate. If they're different kinds of tests or different conditions, then the second is appropriate. It's also appropriate when you imagine that the signal in the data will only be located in one or two of the primary studies which you're combining. Now, does it make a difference which method you choose? Well, yes, of course it does. Otherwise, one would wonder why on earth we have so many different methods. One feature of the methods is that they don't all behave in the same way. If you have a number of values significant in both directions. As a very artificial example, we take these four one-tailed P-values. Two of the methods, fissures and tippets method say reject the null hypothesis because of the two very small ones. But most of the others suggest that in fact this is neutral as to whether you should reject or not and return an overall P-value of 0.5. The maximum P-method which isn't very widely used gives the inverse to the minimum P-method as you might expect. Another way in which we can look at these is what happens if all of the P-values you had were equal. What happens for fixed P-sub-i if we increase K? Well, if it's above a function-specific limit the return value tends to 0. If it's below that then it tends to 1. The picture for the maximum P is that we show here the minimum P-value which for constant PI rapidly tends to 1. As K gets larger and larger. Below it the logic P and the mean P and some others as well show the return value being 0.5 if all the PI are 0.5. If it's above that then it tends to 1. If below it tends more or less rapidly to 0. The fissure is an unusual case but the specific limit is not 0.5 it's 0.3679 approximately. The mean Z-method has the rather strange and undesirable property that for equal PI-sub-i it always returns either 0 or 1 and never an intermediate value. It also can have some other rather strange properties. This is because it relies on the deviation of the PI to get the significance value and if they're all very close together then you're dividing by a very small number and vice versa. Now Loughan carried out a simulation study and although he didn't include all of the methods which are in metopey his suggestion was after you've considered the cancellation issue because obviously you don't want a method that cancels if that isn't what you wanted to do and similarly you don't want a method that does if you don't. He suggests that if the emphasis is to be placed on the very small p-values then you want to use Fischer's method or Tippitt's method. If your emphasis is on the large p-values then you want to use Eddington's method or the maximum p-method. If you are interested equally in all sizes of p then you should use Stouffer's method or the logic p-method. He in fact recommends overall logic p is probably as good as anything. Now I mentioned that the package provides some graphics and this is a QQ plot of the data you've seen used in the example earlier. And using a QQ C RAM package I use their function for providing a confidence interval about QQ plot and this is a simultaneous confidence interval. If any of the points falls outside the ellipse then you can reject the null. In this case it's actually quite hard to see this maybe but right down in the bottom left hand corner they're very close to the edge of the ellipse. Are they on it? Are they inside? Are they outside? It's quite hard to tell. However fortunately the authors of that package provide another option for plotting the scales on a log scale. The direction of axes here is reversed so the small p-values have now gone up into the top right hand corner. It's now much clearer to see what's happening there and one of them is well outside the QQ plot. The advantage of either of these plots is that you can tell and see where the signal lies in the range of the p-values. Is it the case that all of the p-values are tending towards lying outside the confidence ellipse or is it just a few? Another plot which I provide is the albatross plot so-called. This plots on the y-axis the transformation of the sample size on the x-axis the transform p-values and the contours here show constant effect sizes. In this case I've chosen standardised mean differences. In this particular example there are three different sorts of trials being considered here and they're labelled in the plot A, B and C and I think you can see that on the whole the A's lie rather towards the right hand side and the C's rather towards the left hand side. There's some evidence certainly that the trials are of different types. This plot's also been suggested by its authors that may be useful if you have the situation which does often arise where in your primary studies for some of them you've got effect sizes for others you've only got the p-values. In that case if you plot all of them and mark two different sets with different symbols you can see whether the ones only with p-values are broadly compatible with the ones where you have the effect sizes. I should say that despite the name that doesn't look to me remotely like any albatross I've ever seen but that's the name the author gave it. There is stator code available for this but using some advice from him in his blog I've programmed it in R. Just to wrap up the albatross plot is suggested by Harrison and colleagues. The methods, the truncated fissure methods which I haven't discussed today our wrapper is a wrapper for two functions one from CRAM package T Fisher and another MUTOS. There are other packages available from CRAM particularly about the PULAR package which provides methods for correlated p-values and there are a number of other methods which I haven't covered here but which are available and for details of those see the CRAM task view on meta-analysis. Citations for everything I've suggested and the data sets and the full documentation are available either in the package manual or in the vignette so I'm not going to put them separately in the slides.