Welcome back. Oh God. Do we have an echo? Is the echo gone? Yes. Okay, great. Hello. Welcome back to the second day of Bang Bang Con. I had such a great time yesterday. I'm so excited that we're back. I'm so excited for all the talks today. And before we get started, I wanted to do a very quick recap of yesterday's incredible talks. All right, so very, very quickly. Here we go. We had this incredible keynote where we got to see this adorable elephant watering can among many other wonders. There was the taming of the clue, teaching a pen plotter to solve crossword puzzles. With this pen plotter, I actually impulse bought a pen plotter after the last been made. So beware. It's a real danger. We had little printing for everyone where we got to see this adorable tiny printer make these adorable photos. And they put their pen plotter on the internet. Sorry, their little printer on the internet. We had printing floating-point numbers is surprisingly hard. As you see in this slide, we had bringing back my 1998 by bringing an by building an Arduino that can recover a PlayStation memory card. I think one of the lessons here was to use your brother's first so that you don't ruin yours. But don't tell him something like that. Trapped in code points no more and bring Chinese characters with this really amazing demo that I want to play with to learn a little bit more about Chinese characters. We have reverse your ski goggles for fun and profit with some famous last words of the reverse engineering hardware. We had playing break out inside a PDF where we learned how to write JavaScript to play break out with a very large number of text boxes. We had Oval-esque and the unknown unknowns or the art of fumbling through your side project to create something incredible, which was a really lovely meditation on side projects. We had punch card love, a very personal history of computer dating. We had let's implement DNS to learn history where I learned about the chaos class instead of the internet class. We had recreating photography of the 1850s in a digital world Cowboys principle component analysis and one vengeful volcano spirit, which really delivered on its title. We had email with the best ASCII art, which definitely won the best ASCII art award. Maybe there'll be more competition today. Who knows? We had Supercharged Dijkstra as computing shortest paths on large road graphs in microseconds where we learned about Dijkstra's algorithm and how to improve it and make it go orders of magnitude faster. And the last thing we had yesterday before stage talks was Quebec's 735 kilovolt power lines can survive the apocalypse, but can they run TCP where we learned that the answer is yes, but that you should not. So those were yesterday's talks. It was incredible. And I just want to say a couple of words. I introduced today's keynote speaker, which was yesterday was so amazing. And I was a little worried honestly before the conference as conference organizers often are because like we made the call to run this conference online. And I think like we really hoped it would work, right? And we worked hard to create an environment where it would be a fun conference. And where like, so to me, Bangkok is a little bit like programming Christmas, like it's somewhere where you get to come and enjoy like all of these amazing talks and celebrate with this group of people at the conference. And I didn't know who would feel like that and if it would be like that. But we tried to make the space and we invited you all to come and it has been kind of like programming Christmas, like it's been wonderful. And so thank you so much for everyone who came to the conference for making it what it has been. And of course, to all the speakers for for giving these incredible talks and to everyone today who is going who are going to give more incredible talks that we can all be excited about. And if you're watching on the live streaming, you don't have to take it, please. I hope you enjoy all these talks as much as I have. I really had a great day yesterday. Okay, so we're going to start our first keynote presentation of the day today. So I'd like to introduce Allison Parish. Allison Parish is a computer programmer, poet and educator and game designer who's teaching and practice address the unusual phenomenon, which that blossom when language and computers meet. She's an assistant arts professor at NYU's interactive telecommunications program, where she earned her master's degree in 2008. And perhaps less importantly, she is also a friend of Bang Bang Con and has spoken at Bang Bang Con many times before. And I'm so excited to see another one of her incredible talks. So welcome, Allison. Okay, thank you. Let me get my screen going here. I'm hoping everyone can see my slides now. So my name is Allison Parish. And this talk is called computational stochography or connecting lines. I am an assistant arts professor at New York University in the interactive telecommunications slash interactive media arts program. And I am overjoyed to be here today. Bang Bang Con is an amazing conference. And this is actually my fourth time speaking here, believe it or not. And it is an honor to deliver one of the keynote talks this year. I'm looking forward to luxuriating and the extra time that I have instead of just 10 minutes. Thank you to the organizers and to oops. Sorry, I'm trying to move this like weirdo little thing. Okay. There's like a little dude out on the screen that's in the way of my notes. Okay. So thank you to the organizers and to all the other speakers and to everyone in attendance for helping to carry on this really important event this year. So I'm a computer programmer and a poet. And I'm here today to talk about something that I'm enthusiastic about, which is lines, lines of text in particular. In fact, an interest in lines of text is one of the things that poets and computer programmers have in common. And this talk, I'm going to share a little bit about why I think the line is so interesting, and then show some of the projects I've been working on but engage with the line as a unit of text. So line one between poets and programmers. The importance of the line to poets may already be clear to you. It's widely understood as the fundamental unit of poetry as the poet James Longenbach puts it. Poetry is the sound of language organized in lines. More than meter, more than rhyme. Line is what distinguishes our experience of poetry as poetry rather than some other kind of writing. Poets make use of the line in a remarkable variety of ways. Just a few examples. Emily Dickinson's lines that carefully interweave rhyme clause and punctuation. William Carly's Williams lines that have tense and dramatic enjambment that breaks syntax across lines. Nervaise Phillips near ejection of the line altogether in favor of more free and expressive layout in song. You can make rough taxonomies of how poets use the line. But no formalism can entirely capture all of these techniques. Just as a quick review of terminology, so I don't lose some of you. The word that poets use for the way that poem is divided into lines is called is lineation. Frustratingly, this is also a term of art in geology. So whenever you search the library database for lineation, you get a lot of results about sandstone. Another common term is enjambment, which is a technique for lineation that involves introducing line breaks in unexpected places like in the middle of a sentence or the middle of a word. While researching this talk I discovered from the Oxford English Dictionary that you can also say this word in a fancy French way, enjambment, which is a nice way to feel learned and important. So I might say it that way for the rest of the talk. Programming isn't all about lines, the way the poetry is, but programmers do deal with lines a lot in their course of work. The line is, of course, an important mathematical extraction used in both computer graphics and statistics. But the line as a unit of text is actually equally important to computer programming. Code is conventionally written in plain text files, which are sequences of characters delimited with the line break character. Version control software like Git uses the line as the atomic unit of change. When you apply a patch to a file with Git, you're applying that patch based on lines of text, not some other unit like characters or bytes or functions or parts of a syntax tree or anything like that. Dozens of UNIX data processing tools like grep, sort, head, tail, cut, all use lines as their fundamental unit of processing. So poets, programmers have in common this interest in the line as a unit of text. Line number two, ASCII and stichometry. The code isn't the only part of programming where the line pops up. The line is also the go-to unit for organizing data. Many file formats, both official and ad hoc, make use of the line break, the ASCII line feed character as a record delimiter. This is on the slide you see an example of data in CSV format in which the data is stored in a plain text file and each data, each line of data, is stored as an individual line. The line feed character is arguably the most popular way to delimit records in data, despite the fact that the ASCII standard actually has included from its inception a control code specifically set aside for separating records. Hexadecimal1e, the record separator control code, which I've highlighted in this list on the slide. But nobody actually uses that record separator control code to separate records. And I think a reason that people use the line feed character instead is instead even when there's this like tool that's been set apart specifically for that purpose, I'm not like ambiguously the way that the line feed character is. The reason that they think that's the case is that the spatial metaphor of the line is just too well established, like we already think about data in terms of lines, so it's difficult to resist using the line feed character for that purpose. The visual line, as much on a sheet of paper as on a computer screen is already used everywhere as a way of organizing information. The convention of having rows of data arranged vertically from top to bottom by time, dates back to pen and paper ledgers and even as far back as ancient Mesopotamia, there's on the right there, you see a cuneiform tablet that's showing tabular data from 1295 BCE. That looks exactly like a spreadsheet. And arguably, maybe it even goes back to the to the origin of writing itself. So the line break is already a record separator, we didn't really need a separate character for that. Lines are a way of organizing information. But they're also likely the earliest unit used to measure text itself. Ancient Greek scribes divided text up into lines and then counted those lines up in order to better catalog text and to make it possible to refer to sections of text by their parts, sort of the original page number. And of course, that would be useful with scrolls where there aren't pages. This process of counting lines is called stichometry from the ancient Greek stikos, which means a row of soldiers or a line of poetry. One scholar I read argues that the stichometry was the birth of text analysis and corpus linguistics saying that segmentation of text into formal semantic and later syntactic units not related to the physical data carrier was at that time a great achievement and allowed future generations of philologists and grammarians to create first real corpus and statistical tools, by which I mean concordances and indexes. So the line made in like the beginning of, you know, being able to do statistical analysis of text that leads us to things like TFIDF and things like that today. The photograph in the slide shows a line number in the margin of a manuscript copy of a text by Plato that's from the ninth century. In ancient Greece and medieval Europe, books were often cataloged by name and line count and book buyers use the line counts to ensure that the copies they were buying were complete, sort of an ancient checksum technique. The image on the left is a list of books in the Codex Claremontanus from the fifth or six centuries CE common era in which the titles of various Christian texts are listed next to the number of lines in the corresponding text in Roman numerals. To me, this looks a lot like the output of the Unix word count command WC hyphen L, the line count option enabled. So stichometry is alive and well in the 21st century on the command line. Another like interesting fun parallelism between ancient scribes and computer programmers is that according to this often cited paper by J. Randall Harris, scribes in ancient Rome were paid by the line. And also programmers, of course, their productivity has in the past been evaluated based on the number of code, number of lines of code that they write. And this image just shows like a paper that I found about using lines of code as productivity metric. Lines are one of the two ways that language becomes data. And wherever you have data, money isn't far behind. Next line is between linen and cannabis. Earlier I mentioned the etymology of the ancient Greek word for line stichos, which also is the word for a row of soldiers. And that actually comes from the same proto Indo European Indo European route that gives us the English word for stairs because you're walking up on you're climbing up them. And in alternate reality, the word for line poetry in English might have been a stair of poetry, which I think is interesting. But the word for line in English, ultimately derives from the word for the flax plant, which produces a fiber that remains rigid without having to be held taught, like other fibers need to be. And that is a flax fiber is on its own a straight line. And to me, this etymology of the line explains the main metaphorical difference between a line and say a rope, a cord or a string. Straightness in nature is the exception and not the rule a rope or cord or a string becomes a line when someone holds it taught and rigid, like a linen fiber, thereby removing the slack. Likewise, with the ancient Greek word, a row of soldiers stays in formation only through discipline, you have to exert effort in order to keep the line straight. And in computer programming, we use both the words line and string to refer to sequences of characters, but they're not interchangeable. And I think there's a reason that we call it the command line and not the command string. A string can contain an arbitrary sequence of characters, but a line is text that has already been set apart as having some intention, a record in a file or user input in response to a prompt or something like that. While I'm on the topic of etymology, one of my favorite topics, I'll point out that the English word read originates from a word that means govern or counsel. The word file comes from a lot and word for thread. So a fair etymological and metaphorical translation of reading a line from a file is governing a thread by pulling it taught.  So the next time you are using JavaScript, say to put a line on a canvas, you can think of it as like gently setting a flax plant on top of a cannabis plant, which I think is a nice image. The next line that I want to draw is between oxen and typewriters. Getting back to literature and writing more specifically. In the history of the written word, you can actually distinguish two different kinds of line, the graphic line and the poetic line. The poetic line is also called a verse. A graphic line is just what you call on a writing surface when you see a number of glyphs placed in sequence, sharing the same baseline as they progress across the writing surface. In some writing systems, glyphs and lines are arranged from left to right and others from right to left and others still on top to bottom and then moving across the page from right to left or left to right. This image shows an inscription on a fragment of stone plate in archaic cretin script written in Bustra Fiedin style in which characters and lines alternate between being arranged from left to right and right to left and you can actually see that the orientation of the characters change as the writing progresses from the top to the bottom. Bustra Fiedin means in the manner of oxen plowing as you get to the end of one row and then you move back to the next starting from where you were. The graphic line I think originates just as kind of like a side effect of the need to adapt media of different dimensionality to each other. We tend to perceive speech as a one-dimensional medium, a sequence of sounds and words. Most writing surfaces on the other hand are understood to be two dimensional. So when you're writing speech onto a writing surface like a scroll or a sheet of paper, you have to find some way to fold the one-dimensional sequence onto the two-dimensional surface in as efficient a manner as possible and that's what a line is. It's a way of like folding one-dimensional data onto a two-dimensional surface. In this slide I'm showing Hilbert Curve as an example of another way to fill a 2-key surface with a line. No writing system that I'm aware of uses Hilbert Curve's to layout text on a page but I think it would be cool if there was one that did do that. In purely oral literature by which I mean literature that isn't written down, poetry also has lines. In this case usually where a line ends is determined not by space limitations of the writing surface of course but by metrical, phonetic, syntactic and semantic characteristics of the words themselves. What's interesting to me is that when poems from the oral tradition were first written down in antiquity and in the Middle Ages in Europe for example, apparently there was no consensus that one poetic line should correspond to one graphic line. Instead you see a variety of conventions for separating poetic lines from manuscripts including things like placing a point after the last word in a verse, putting spaces in line between verses. We'll see an example of that later or using capital letters at the beginning of each verse. Catherine O'Brien O'Keefe points out that the technique of starting verses flush with the left margin and leaving blank space up to the right margin for a single verse was maybe used only as a last resort in medieval manuscripts because it was a waste of the expensive writing material leaving all of that expensive like all of that expensive vellum you're leaving at blank. So that that thing that we think of as as being the usual way that lines of text are written especially lines of poetry are written where you have one graphic line per poetic line is actually like you know not the only way that you do it and it's kind of something that's only become the standard in a contemporary modern context. In fact we still use graphical conventions like putting a point after the last word in a verse when we use the slash character to separate lines and poems or lyrics when you're quoting a poem or a song in line in prose. As far as I can tell it wasn't until centuries later that the conflation of the poetic line with the graphic line became as universal as it is today and I might argue that the conflation was only truly complete after the invention of the typewriter a technology that allowed poets to essentially typeset their own texts and quickly experiment with how lineation affected the process of reading and composing poetry. The poet Charles Olson in his well-known 1950s manifesto projective verse basically claims this and says the line is equated with the breath and the typewriter is the first technology that the poet has had that allows them to indicate their breath as precisely as they want to on the written page. The next line is between tidiness and fleshyness. So taking all of that into account we've seen the line in many different forms and contexts so what makes the line such a powerful and persistent unit of organization and text? On this question I think the poets themselves have some good answers. The most elegant formulation about the line I've come across in my reading and research comes from the poet Lynn Hyginian who writes the line affixes detail to time. The writing of the line begins as an act of observation and it is completed by recognition of the thought that it achieves there. I understand this to mean that in the process of translating sequences to surfaces the line is how a writer notes detail and anticipation of how the reader will approach and perceive the text as it unfolds in real time in the process of reading. The line is how a writer draws attention to particular characteristics or in Hyginian's terms particular observations of the language and question and how it operates. The poet and scholar Johanna Drucker emphasizes not the temporal aspect of lines in time but instead their visual and spatial aspect. She writes, as visual forms the lines create their own order in the text a frame which introduces the struggle of hierarchy into the words the brakes become a force against which the hole must be recovered or against which the hole can be fractured it dissolved let go the visual line strains the literary authority with its vulgarity its crudeness its fleshiness which pollutes the material of cure language. Drucker is arguing here that the spatial layout of text on the page has its own significance which demonstrates that written language is never simply a straightforward transcription of speech but something more that has its own affordances and operates with its own connections. In other words a text always carries traces of the fleshy physical world with it. Not just the decisions the writer made in adapting language to the visual form but also the material of the page and the physical gestures of the writing process whether that means the writer's penmanship or their keystrokes or any of the other things that go into producing the physical text. Finally Ron Silliman the poet Ron Silliman wrote this which I think is like important for talking about the line as it regards computer programming. The line is the sole unit of punctuation whose whose use historically has not been determined by its potential for submitting chains of words to the hierarchy orders of logic descended from the classical grammars of Greek and Latin which had become a normative contemporary model for clarity and writing. In other words one role of the line is specifically to cut across grammar and syntax especially when grammar and syntax alone don't give us the expressive tools to do what we need to do with the text. All of these statements about the use and lineation of poetry apply equally to the use of lineation in computer programming. Most programming languages understand their input as just an unbroken stream of tokens from which any characters representing visual layout like spaces tabs and line breaks are stripped before the program is compiled or interpreted. Nevertheless programmers make use of visual layout in their source code and line breaks in particular in order I think to affix detail to time and make the source code more readable for themselves and for other programmers. What's considered readable code is however highly conventionalized and often algorithmically enforced with tools like linters and autocomplete. In this animation I'm using the T5JS web editor to make this creatively formatted code more tidy in the word of the editor. In my experience as a programmer and maybe in your experience I found that noddle code is made more readable by adhering to convention and sometimes there are bits of code that require unconventional lineation in order to make their inner workings more evident. Sometimes as a programmer you have to enjamb your code and many programmers also know that readability can be an arbitrary concern which is among the reasons that the international obfuscated C code contest exists in which contestants often make use of line breaks and layout towards more aesthetic ends. This is an Edward Giles entry from the 2019 competition that has some little waves. The code is broken up into these little waves which actually is a program that applies a reverb to a to a wave file. Even in a language like Python which is somewhat unusual by virtue of the fact that white space is actually a part of the language's syntax, Python still provides so many resources for creativity and expression with line breaks that entire PEPs have to be written in order to guide people in the right way to do things. As with other languages there's even a module of probably many different modules for automatically reformatting your code in order to conform to these conventional standards. Personally as a poet and a Python programmer I would never use one of these libraries because I have too much fun playing with the layout and making the layout to do exactly what I want it to do inside of a Python program. So that's some historical background. With that in mind I want to talk and speaking about Python programming I want to show off some of the stuff I've been working on recently the concerns lines of poetry. As a poet I've been interested in putting the idea of lines as free-standing textual units test, removing lines of poetry from the original context and then figuring out ways to juxtapose them in unusual ways. This sort of line of research started a few years ago when I made something that I call the Gutenberg Poetry Corpus. That corpus consists of around three million lines of poetry generated from Project Gutenberg which is a database of texts that are in public domain around like 70,000 books that are in public domain that are all available in plain text format that makes them really easy to manipulate for text analysis and creative purposes. To build the Gutenberg Poetry Corpus I wrote a program that goes through all of the metadata and Project Gutenberg finds all the books that are listed as poetry and then I scan through those my program scans through them line by line extracts line of texts that look like poetry as opposed to introductions or commentary or quotes or other other paratests and then collect them together and it ends up being a 52 megabyte file that contains three million lines of poetry from all of the included books in Project Gutenberg or at least the books that were in Project Gutenberg a couple of years ago as of a couple of years ago along with a number that identifies the volume that it originates from. This is actually stored in JSON format but the particular kind of JSON format where you separate JSON records with a new line which I don't even know if that format it's used all over the place but I don't know if it has an official name. This is a diagram of the process of creating that corpus. The most sophisticated part of the code is the rules to determine what counts as poetry which is just a bunch of ad hoc criteria that I'd eventually like to replace with something more well-reasoned but this program works pretty well and does what I want it does what I want it to do. The corpus just to give you an idea of what it looks like this is an animation of what happens when you make a program that just prints out lines at random so it's just like a big stream of lines of poetry that have been taken out of their context. Actually the reason that I made the Gutenberg poetry corpus in the first place was because I was teaching a creative writing class in 2015 and I wanted to give my students an assignment to write autocomplete poetry which is usually you go to google and you type in some words and then you make a poem from what google suggests after what you typed but I didn't want my students have to use google to do this if they didn't want to and so I made a quick tool with nginx and redis that sort of works like google search autocomplete accepted autocompletes your typing with a line of poetry from the project Gutenberg poetry corpus the search is the same letter so you can search through it and it completes it with the line that matches that prefix. This is actually believe still helping and it's five years old which is an eternity in online art project terms but I think you can still go there and use it. Once I had that corpus other applications started to suggest themselves the poetic line as you've discussed is often determined by sound meter rhyme assignments alliteration these are things that like set one line of verse from set apart one line of verse from another. Starting with this insight I wanted to make a project that could sort cluster and juxtapose poetic lines based on their phonetic characteristics. Basically what I wanted to do was first of all find a way to represent the sound of words as vectors so that I could draw like a scatter plot like this where words with similar sounds like octopus and apocalypse would be closer together on the scatter plot and words that are dissimilar like kinky and kinky would be further away even though they would be clustered together. So finding a way to like turn a word of the phonetic information from word into a vector. To do this I made the program using the CMU pronouncing dictionary which is a database of English words and their pronunciations which you need in English because the way that words spell the way the words are spelled doesn't really have much with how they sound. Then I wrote sort of like a weird kind of ad hoc feature detection manual feature extraction algorithm that uses bigrams of phonetic features in order to produce a big a big matrix of counts of features used principal components analysis so that I ended up for every word of English a single 50-dimensional vector that represents the sound of that word. I used that same algorithm then to determine to get a 50-dimensional vector for every line of poetry in the project Gutenberg corpus. So I had for every line of poetry I had a number a coordinate in space that indicated the sound of that poem. And with that I made this book called articulations. This is a book of computer generated poetry. It was published by counterpath press two years ago as part of Nick Monfort's Using Electricity series which consists all all of the books in the series are computer generated. And I've included an example of the output here. I don't have time to read it even though I love reading it. So the book is composed of a random and automated random walk through the phonetic space of the project Gutenberg poetry corpus. I start with a random line of poetry and then I find the line that is most phonetically similar to it based on its calculated vector and then add that to the output find the line that's most phonetically similar to that add that to the output and so forth. Excluding any line that has previously been in the output. The end result is this kind of poetry where cohesion is achieved only through phonetic similarity. And one key decision here that has to do with the the topic of this talk is that I decided to format the output as prose by by running the lines together into into paragraphs instead of retaining the original formatting of the lines and my reason for doing that is I wanted to blur the distinction between lines and draw attention to how they're similar to each other. So that's one experiment with that corpus. A more recent project which i'm calling variational sesdenoid spirals. This is less about the sound of words and more about the meaning and syntax of lines of poetry and the ways that machine learning can facilitate the invention of new poetic forms. A computer-generated poetry of course is a whole field of practice and computational creativity. You can get on Google Scholar and find hundreds of papers about generating poems with computers. But the question I started out with was not how do I generate new poems but how do I generate new lines of poetry? How do I make a generator that knows how to produce just a single line of poetry? What does it mean to write a generator that produces a single line? To do this I made use of a kind of neural network architecture called a variational autoencoder sort of akin to again in the sense that it is a model that kind of essentially learns how to compress data. The encoder produces a fixed length vector for each item the train set and then the decoder learns how to produce the original data from that fixed length vector. In a variational autoencoder the values in the fixed length vector are constrained to conform to particular statistical distribution and the benefit of this is that the model becomes generative. You can generate new data that looks like the original data by picking random numbers and feeding them to the decoder. Similar random numbers produce similar outputs and you end up with a latent space that can be explored. This like the coming up with the idea I did not invent the idea of a variational autoencoder and I'm only a beginner machine learning engineer so I relied very heavily on the research from these researchers from Junsheng Pei and Benjamin Heinzer-Ling's pre-trained embeddings for for byte pairs were also a big part of this project. I should also note that I was like super inspired by Robin Sloan's sentence space project from a couple of years ago which I'd look at and I have a link to my fork of the variational autoencoder code that you can actually launch in a Jupyter notebook in your browser if you want. So once you train this model you end up with a model where you can feed a random number into the decoder and give back a line of poetry. Part of the reason that I wanted to do this was to be able to explore poetry space. Last summer Runway funded my work on this model and part of the reason that I wanted to make it so that I could use their leading space explorer in the Runway app. What you're seeing here is like a single sample from that space surrounded by samples that are from similar numbers and you can sort of move over it like Google Maps. So each box contains a line of poetry that is generated by the model from nearby points in that latent space. Once you have a model like this there's all kinds of fun stuff that you can do. One is creating interpolations between two different lines of poetry. So you encode one line of poetry and encode another line of poetry and then decode from points along the line that connects them, the line that connects them. So this is an interpolation from the first line to the last line of Robert Frost's The Road Not Taken. Taking a more direct road from the beginning to the end of this poem. So this is sort of like the result of drawing a line through a latent space of lines. It's like a meta line in a sense. When I showed this project on Mastodon, a user there suggested circular interpolations meaning pick a random hyperplane and then decode from a circle drawn in the hyperplane. And using this technique you get lovely lines of poetry that seem to return to where they started. It's just a little interface I made in P5JS to explore one of these circular interpolations. The poetry generator that I made with this technique produces poems by sampling lines from points on concentric circles that gradually increase in size in the latent space. It follows a schematic that looks like this. Each circle represents a stanza and as the circles get bigger the stanzas get longer and since each stanza is an elaboration on the last. I might read just a little bit from this one. This is a moment that is a moment full of a moment could be seen. That is made that shows a moment having been pure as a little heart was seen. That is made that shows a moment having been full of a moment still had seen her eyes making a little heart of man. The present that shows himself however that space of moment had been gone in sweet as a smile of beauty's eyes praised to my heart. The present that shows perhaps the latter of this moment having been as sweet as thou shalt have seen his eyes sweet a mile of my heart is bright only the heart the latter that shows perhaps the books of being haven't been for such a different hours had passed and all shuddering with her fair eyes of her eyes sweet a smile of my dear face seeing the word that is a moment which I like I think that's a pretty good result. All of this finally brings me back to the title of the talk computational stichography. So making that poetry variational auto encoder was fun but at this point it means so many projects with the Gutenberg poetry corpus that I'm getting kind of sick of it because it consists of poetry in the public domain specifically in the public domain in the United States. It mostly consists of 18th and 19th century white male American and British poets whose poetics by and large let's say are not especially expansive in subject matter or style. So a lot of what comes out of the variational model has a pretty monotonous diction. It all sort of sounds like a mishmash of collage and words worth like falling asleep in your high school English class. So I could just make another corpus you may say but the issue is that this is the only corpus of poetry that I know of that is both freely available and entirely unencumbered by copyright. I could scrape the web or scans of books for contemporary poetry but then I wouldn't feel like I had any real ownership in the output of the program and I'd be obligated to seek out the permission of every poet whose work was used to train the model and I don't have time for that. So the solution I'm pursuing is to find some way to introduce poetic line breaks into arbitrary text. That way I can use any kind of text to train or at least fine tune the model that I made. And if I wanted to I could train a variational autoencoder to produce lines of poetry about flowers by grabbing all of the Wikipedia pages about flowers splitting that prose into lines and then use those lines to train the model and then publish the resulting poems under the same license as Wikipedia. So I've made some progress on this and I want to show you that progress. I'm using this word stickography to describe the process of adding poetic line breaks to arbitrary text. That term comes from biblical studies where scholars use it to describe how scribes copied certain poetic or musical passages in scripture not in the regular continuous script but with graphic devices to separate poetic lines like spaces or line breaks between the verses. I'm showing here an example of the same passage from two different scrolls. One in which describe has copied the passage in continuous script and another in which the scribe has written the passage in a in a stickographic fashion with the blank spaces to denote the to note the ends of poetic lines. These are written in line and these aren't line breaks they're the lines are separated by spaces. The scroll fragment isn't from the same passage but it shows what the empty spaces looked like in the original manuscript. So as I understand it different scribes had different techniques for writing these passages stickographically. They applied a set of criteria to determine which passages should be written stickographically but came up with their own strategies for performing the stickography based on their own ideas about how the text worked phonetically, semantically, syntactically. So what I need is to be able to do what these scribes did but in an automated fashion so I can do it on you know thousands and thousands of wikipedia pages. So I trained a machine learning model to do it. I treated this as a sequence prediction task. The input to the model is sequences of tokens from the Gutenberg poetry corpus and the labels are either a zero or a one with a one indicating that there should be a line break directly following that token. My theory is that most lineation relies on the sounds of words so I used pencilate to find phonetic embeddings for each word instead of using just like numerical indexes in a vocabulary. Pencilate by the way is the graphene diphony model that I actually talked about in my bang bang con talk last year. I think that using the phonetic embeddings makes the model more accurate and it also makes it possible for the model to deal with out of vocabulary words or even like made up nonsense words which is definitely a benefit. The network architecture is a stack of bi-directional recurrent neural networks. I know that everyone has moved on to transformers or whatever but it took me like two years to understand RNNs well enough to be able to just like whip up a model like this on a whim and I am planning to make the most of that investment and I'm still working on this but my little test models accomplish the test that I that I wanted to do pretty well achieving 93 accuracy on my validation set after training on my 2013 MacBook Air overnight which isn't that. I only like to make neural network models when I can train them on my laptop. So this is a video of a live typing interface that I made with an early version of the model. You type continuous text on the left there and get back text with the model's stochography applied on the right. The model does pretty well with these two light nonsense poems. Neither of these were in the train set. I never saw purple cow I never hoped to see one but you can tell but I can tell you anyhow I'd rather see than be one. I eat my peas with honey I've done it all my life it makes the peas taste funny or keeps them on the knife. So it does pretty well actually in finding reasonable line breaks in this text. As another proof of concept this is some of the text from the Wikipedia page for Zinnia the flower Zinnia with the original continuous text there on the left and the model's stochographic output on the right. I think this is actually pretty good if you read over it. I think it's finding like you know places to break the text that sort of fall in line with your expectations about where line breaks should go in poetry. I haven't actually used this to retrain my variational model yet but that's the next step so this is still a work in progress. So to conclude what draws me to the line as a unit of language is that it's really the interface between language and the world. The line is language held taut and someone has to be doing the work of holding it taut. So what their line we're talking about is a poetic line based on rhyme and meter or a line in a spreadsheet recording a single data point or the line of code carefully crafted to elucidate or obfuscate the inner workings of a computer program. The line demonstrates the language is in Johanna Drucker's words fleshy. Line language is always situated in physical and social reality and that is my talk. Thank you. Thank you so much for that amazing talk. Wow okay so right now we have a break for 30 minutes until 12 where we're going to come back with our next session. If you're in the discord we have a few things you could do you could hang out in the virtual conference space we have a couches room and a kitchen room and you could hang out there. We have a neighborbot channel if you want to meet random people you can get matched with random people to meet. I have seen that in the general channel there is a spreadsheet party which you can join which I've looked at and it's very incredible. And if you're on on the live stream and on the discord we'll be back at 12. See you then. Hi everyone we're still on break but I just wanted to give my daughter an opportunity to say hi. She wasn't born last bang bang con so this is her first first bang bang con. Oh yeah what's that? Happy Mother's Day to her mom. Very first Mother's Day. She was just learning how to drink out of a cup so we're a little there's a lot of water going on here. Say hi okay say bye bye. Thanks Cindy. Back. Hello everyone welcome back to bang bang con 2020 virtual conference edition. I'm getting some echo and it's because I have the live stream running in another tab that's why I was getting echo and problem solved by closing the tab. Yes computers are difficult but that's why we get to have fun conferences about the surprising and emergent behaviors of computers and running your live stream at the same time that you're trying to broadcast it things like that. Thanks for coming to my bang bang con talk. So we have a couple of talks coming up in this session. The first it says here I think I forgot to change the ordinals. There we go also with the computers again. So really it's our let's see 1 2 3 4 5 6th session of bang bang con and we've got three amazing talks coming up. I think oh before I intro those I want to intro my background. My wife was running a live stream just prior to me logging on this morning that involved a lot of flowers and I said hey why don't we leave those flowers up so that everyone can enjoy them. So that's why I'm sitting in front of a beautifully decorated backdrop here. Shout out to my wife and her mom who put this together and I helped a little bit but I'm not a very good floral arranger. Hopefully I am a better conference organizer. I think Franklin is in the chat or in the live stream so if you want to pop on. Hello. Now my gain is too high. All right let's turn that down. Thank you your notification just went away but I just turned it down so hopefully that's better. Good morning Franklin. Morning yeah. Thanks for popping on to say hi. So let me see if I'm pronouncing this correctly. Your talk title is Learning Your IBC Translating Chinese Numeric Code. I'm so terrible with tones so I actually speak any any tone of language. That was great. Yeah yeah yeah thanks for the intro. We're going to roll the pre-recorded version of this but I just wanted to say hi to the or to say thanks to all the organizers. It's been a great conference and looking forward to chatting more in Discord with everybody afterward. Thank you so much. Thanks. Let's see this talk. I'm excited. Hi my name is Franklin and today I'm going to talk about translating Chinese into Morse code. You're probably familiar with Morse code. It was the way of sending messages on telegraphs in the 1800s. The original encoding scheme really only contemplated characters from the Latin alphabet so letters a through z as well as numerals zero through nine and it sent each of these as a series of electrical impulses that we represent here as dots and dashes. And as the telegraph network spread across the world it was relatively straightforward to add new encodings for different alphabetic languages and so you might have letters with diacritics that that would be slightly different. But in 1871 when Hong Kong and Shanghai were connected to the international telegraph network they had a problem. That being that Chinese is a character based language and with thousands and thousands of unique characters and so the same strategy doesn't really work. A telegraph operator working in English can memorize what a through z are and can transcribe messages as they come off the wire and that would be unfathomable to do for something like Chinese. So what ended up happening is that these first telegraph lines were built by European companies being the great northern telegraph company of Denmark and the eastern extension ANC telegraph company of the UK and since they were European companies in the 1800s they naturally chose some Europeans to come up with the code. There were a couple of different iterations. The one that we're seeing here on the right is the Chinese telegraph code of 1871 which was put together by a French customs officer based in Shanghai named Viguer and the general approach was that they took the most commonly used characters and so starting with about 5,400 of them although this grew up to about 6,800 and they mapped each of them to a four-digit code so 000 through 9999. And if we look at a concrete example if we take the character Dong which means winter if you look that up in a code book you'll see that it has the value of 0392 which then you can use the like normal international Morse code representation to turn into dots and dashes. And I built an app I have this link here I'll have it at the end as well that basically like steps through this and so you can put in a bunch of Chinese characters and it'll give you the corresponding Morse code. So this has some implications you might notice that with four digits we can actually represent 10,000 things and so we have quite a bit of extra room. These code books were arranged in a way for searchability and so given a character you could based on its characteristics find the right place in the book and then look up what what the digits were. When they arranged the book they ended up leaving a bunch of extra spaces which you can see here on on the right just in case in the future they needed sort of to add new characters that I came into Vogue without having to kind of reshuffle everything. The second implication is that this is actually fairly slow and verbose. So if we go back to the international Morse code you'll see that this is actually arranged mostly for English letter frequencies. So ease are very common and therefore very short whereas Wys and Zs are quite a bit longer and digits themselves are pretty long. If you're trying to send a single Chinese character that is represented by four zeros then you're on the hook for 20 long dashes and telegraph messages were also priced by length and so you'd be paying for for all of that. So in addition there was this kind of weird situation with plain text and secret messages. So in English telegraph operators had developed a set of kind of special words that were not commonly used to to represent longer phrases that they might come across. So if you got a message that said toothbrush what this actually meant was that your telegraph your telegram had been delayed in transmission and this was great because it was less expensive it was faster but the telegraph companies were not a huge fan because this cut into their products. So they developed a set of regulations where plain text messages were priced differently from secret ones where these sort of like meaning wasn't immediately obvious and these prices were much higher per word. So as an English speaker like you had a choice between do I want to send this message as plain text or in secret and while this originally hadn't had nothing to do with Chinese it was kind of caught in the crossfire because Chinese was intermediated by these digits it had no choice but to be priced but to be lumped in with all of the secret messages which meant that it was even more even more expensive. This problem ended up going away eventually in 1893 the government of the Chinese Republic and the Qing Dynasty negotiated a regulatory exemption and made it so that if you were sending a message with Chinese characters in a code book then it would be priced as as plain text rather than as secret which is an improvement. So here we have the telegraph book or sorry the telegraph code of 1871. It works but definitely has some drawbacks. There were a bunch of different ways that people tried to improve on this and I'm going to dive into one which is referred to as trigraphs. So a trigraph is basically just a fancy word for three alphabet letters. As we noticed before digits are pretty long in Morse code but letters are much shorter and so if we were able to map each Chinese character to a sequence of letters then that would be a lot faster and we can actually do better because we don't need nearly as many letters. We actually only need three which gives us more than enough space to kind of represent all the letters that we want. And so this is great. Going back to our example of winter the trigraph of APB is much shorter in Morse code and this isn't just a fluke if we jump back to the website we'll see that this poem originally in Morse code sorry in digits when we switch over to trigraphs is like half the length and this is true for a bunch of different things. So that seems great. There was a catch though so when this was developed originally Latin characters were not very widely understood in China and so the code books had a mapping of Latin characters to Chinese ones that were phonetically similar. So instead of A, B, C you had I, B and C and that was great because it meant that the operators could work entirely in Chinese. They could look up these characters like they were on the right. Each of those boxes has the trigraph kind of in the top left and then the digit representation in the bottom left and then the character itself on on the right. Eventually this mapping between Latin and Chinese characters became less necessary as kind of exposure to to Latin letters was more widespread and so eventually disappeared out of code books. And cool. So to give a very concrete example of we have a phrase I like eating bows which we have the conceptual equivalent in Chinese and we can see that you know the digit representation in Morse code is fairly long and with trigraphs it's actually like probably just about the the same length as the English phrase just pretty cool. So we've talked about two ways that Chinese was encoded to Morse code. Both of these had an intermediate step where they were translated first to digits or to trigraphs and this extra step is is still common. So 100 years later if you find yourself trying to type Chinese on a computer or into your your phone it's actually not that different. You might be using some type of romanization and rather than a person searching in a book for the the right output it's it's a computer but we're still kind of a step a step removed and Chinese is is weird and different when you can compare it to English or other alphabetic languages. And so that's all I have. Thank you so much. I encourage you to check out the app and play around with it. For non-Chinese speakers I have a bunch of preloaded phrases and poems that you can kind of click around with and like get a sense for for how this all worked. I highly recommend this book The Chinese Typewriter which goes into a lot more detail about both like Morse code stuff but also just all the different ways that people tried to create a typewriter for Chinese which has a lot of the same types of problems. And yeah that's about it. Thanks. Thank you Franklin. I think we're going to do a quick sound check before we get started with our next talk. So Tim I don't know if you're on. Yep. Okay should I play it? Yeah play it. Now do your intro. Could you hear that? I heard a like a like a sine wave of sorts. Annoying tone. Yeah that was exactly perfect. Okay. All right so then we're gonna just take a little bit of time to make sure everyone's ready backstage to get things prepared. Let's double check make sure everyone's ready. Okay we're good to go. So our next speaker is Tim Holman and the title of the talk is The Making of a Mosquito. Tim is an Australian developer with a love for all things weird on the web. If you don't catch him hacking away at the world's most useless code he'll be working hard on tools to get more people into generative art and feeling empowered to create. Welcome Tim. Okay hello everyone welcome to my talk Making of a Mosquito. Here are my details I would love to talk about myself more but we only have 10 minutes so we got to get through it quick. But again if you have ever seen me on the internet before or any of my talks you'll kind of know that I really want to get people into creating things and I think it's the best way to learn and teach and grow and all around have fun. So the reason that I wanted to make a mosquito in the first place was to explore the web audio API and wanted something not too crazy and not too simple to learn. The web audio API can be a little intimidating it even mentions that in the MDN documentation. So I built this beautiful sanity chart to track my sanity over time. I like the chart at this point because it shows that sanity could potentially go above one when that is truly not the case. But with every project you kind of just got to start with your building blocks and for me that is a small html page referencing a script and in that script I just have one single line to begin let context equal new window dot audio context. The audio context is how you access the web audio API.  Now to get noise from the web audio API you need a couple more things. I need an oscillator node and a gain node and don't worry I'm not going to al you here I'm going to go back and explain these but the reason that I want to do it this way is because this is how I built it and then I kind of reverse engineered backward towards making the mosquito. So if I had my code I would have the context I'd be creating an oscillator node you can see that it comes from that context that web audio context I'd be creating a gain node and then I would be connecting my gain node to the oscillator node and I'd be connecting the context destination which is the output the speakers to the gain node and then I also have to add a button in because back in the day we allowed auto playing audio and the world ruined it for everyone. So this would be my mosquito at the start and this would sound exactly like this a very annoying buzz kind of space noise my sanity immediately started to go down I immediately had to search how to reduce web audio API volume and lo and behold up comes the gain node so we figured out what that does it also mentions in the gain node never changed this value directly but I never stopped to think if I should and I decided that I will just change that value directly and it reduced the volume to half and that one that you heard just then was actually half already it was really really loud so if we looked at my mosquito now it would be an oscillator node a gain node and an audio context we know what the gain node is but what does this oscillator node do well turns out that the oscillator node is responsible for emitting a sound wave much like this sine wave that you see here and it has a few properties on it it has a frequency which is the frequency of the wave and it has a type there's a bunch of different types of waves that it can emit and of course a mosquito would be the most intimidating sounding type which is the sawtooth and then I did a little research into frequencies of mosquitoes and there are a lot of videos online upsettingly and I also kind of learned that you could just jack the frequency up to about 10 000 and make that noise that only dogs and small kids can hear do with that information what you will so I played around with my oscillator node the sawtooth again was the best the best node type for this and I set the frequency to about 750 and that gave me a mosquito that sounded a little like this very annoying I had to reduce the volume again straight away sanity is at an all-time low of 0.8 the graph is figured out now that it's not going to go above that but this mosquito I would call the base mosquito it's the yarn create mosquito the mpm init mosquito the user bin mosquito that's where it would live sanity is falling at a rapid rate at this point I've been playing with this for 30 minutes or something I listened to mosquito videos it was all kind of going crazy in my head I decided that I need to stop the mosquito noise you know at the moment I was refreshing it was really difficult to debug and the oscillator had to start and stop so I thought okay let me just plug that in to the mosquito so I can press that button I can press it again to stop but at this point I'd already closed all my tabs multiple times because I felt that I was still hearing the buzzing afterwards I decided to add a clap audio noise to give me a small brief moment of satisfaction so there we had our clap and our ending of the mosquito I decided to add in some ascii art because I was really losing the plot and then I found this when I tried to start my mosquito again it turns out you can't restart an oscillator node once you've called stop on it I have no idea why my best guess is that once you clap a mosquito completely out of existence you can't obviously clap a mosquito into existence so I had to do a bunch of different things here I had to start the oscillator node when I initially clicked the button and then otherwise I would disconnect or reconnect the gain node my sanity again took a big hit here because I knew that every time I stopped the mosquito it was secretly oscillating behind the scenes waiting for its opportunity to strike now if we think of the mosquito that I have at the moment this again here's the noise well you can hear the pain you can feel the pain the equivalent of that would really be a mosquito just kind of sitting hovering above us at exactly the same spot looking down and although that would be intimidating I kind of wanted to make the mosquito a little bit more bumbling and frustrating than the stuck mosquito in the sky and between that frequency and type option there was detune I didn't really need to read what detune was because at high school a lot of people played Wonderwall on their guitar and I had great experience with detune now my sanity is low at this point it's below the 0.6 line that I call the insane or genius line you know I've set up my citronella candles on spraying everywhere oh I'm in trouble for doing that okay but I decided to build the mosquito and the mosquito engine is what I was going to use to detune the mosquito so if I were to just set that originally up on a sine wave you can hear then that it goes up and it goes down as as with the sine wave now that mosquito would just be kind of hovering up and down which again would be intimidating but not completely what I was going for so I decided to rather than completely just keep adding to the sine counter putting it round and round I decided that 25 of the time I would move backwards and 50% of the time I would move forwards and the other remaining 25% it would just stay static which gives us this staggered mosquito engine I call mosquito engine 0.5 and if you're to listen to it you can hear that there it is much much more like a mosquito I felt that it needed a little bit more variance so I added a small pusher and puller in the middle of that this variable will just go up to 0.6 or down and it would just randomly 50% of the time go up or down that small little push looks like this and if I were to attach that to the end of my already rotating broken rotating mosquito engine I would have what I call mosquito engine 1.0 now that is a mosquito if I've ever seen one but now that I've got this mosquito engine running the detune why don't I connect it to all the pieces if I connected the gain node I can control the volume and it can go up and down again with the mosquito that is really starting to get onto the annoyance of the mosquito and I thought what else do I hate about mosquitoes aside from everything and one thing that's really difficult about mosquitoes is trying to triangulate the noise when they're in your room or they're near you know you're looking up you trying to find it you want to get it and it turns out there was one more beautiful piece in the web mosquito API called the stereo pendant node and the stereo pendant node which I'm going to connect to the mosquito engine of course controls the left or the right speaker now I'm not too sure if this will come over the stream but this was my absolute complete mosquito infuriating infuriating infuriating that is my talk thank you everybody presenting this to you of course has brought my sanity straight back up to the top thank you for the time thank you Tim before we get started with our next talk we're just going to wait a little bit and let everyone get ready but one of the things I'll talk about when it comes to organ conference organizers for bank bank con one of the difficult and yet very you know pleasurable experiences for us is the catering getting that figured out figuring out you know do we have budget to do breakfast lunch and dinner do you know do speaker dinner all these different things we try to figure out and of course with the virtual conference we didn't get to have the opportunity to do that this year but for those with access to the discord channel we have a catering channel where people are posting some of the foods they're eating and it's just like looking at these posts it's such a delight I'm just like going down this channel and I see you know fluffy souffle pancakes which I've tried to make several times and absolutely fail that I see crepes I see bagels and it's just amazing to see like what all all you different people are eating right now there's a picture of a Monte Cristo with the bite uh I guess it's just it's just fantastic I love this channel uh if people you know if you have photos of things you're eating please share them there's donuts and coffee and french press oh my god um let me see if we're getting ready for how we're looking for the next talk let's bring in Erty hey I was just uh I was just saying you know put me alive and we can we can chat about the conference hi uh shout out to dujeur bakery in brooklyn who we had contracted uh almost contracted uh unfortunately to to do our catering and then of course had to cancel so uh whenever things get back to normal uh shout out to them and I hope I hope they're all doing all right through this what I was really impressed about dujeur bakery last year is that they got up at around 4 a.m to start the food prep they made the donuts first that morning yeah it was yeah everyone right everyone uh who was here last year definitely got that that treat of like very fresh pastries so uh I hope I hope you all get to treat yourself to something today uh in kind of in in remembrance of those delicious donuts they also did they uh I'm I I'm gluten and dairy free and they like did this whole extra like set of cakes that I and I was I was super happy with that uh yeah they were they were like incredibly understanding of of diet of like everyone's dietary restrictions awesome it looks like we're ready for the next talk I'm gonna introduce our speaker awesome we'll move things okay so next up is we have Tristan Humes uh whose talk is titled using font shaping to put commas in big numbers everywhere Tristan Hume works on fast software at jane street he gets captivated by funny and cool technical ideas and develops a burning desire to implement them mostly in rust but sometimes an open type font shaping rules he writes long technical articles about things he does and he finds interesting at his blog post which you can find in the speaker description of the BangBangCon website welcome Tristan hi so I'm Tristan and my job involves a lot of staring at large tables of numbers and often I want to do something like look at this column of latency numbers in nanoseconds and pick out some larger quantity like microseconds and milliseconds and this used to involve counting a lot of digits starting from the right of the number and I found myself doing this and also the different places like my text editor and my terminal and python notebooks in my web browser and I didn't want to implement a solution for each of these places but I came up with a weird solution that would work in all of these places which is I could make a font that would insert fake commas into my numbers and I could do this using font shaping or at least I thought I could because font shaping was really powerful and I had seen all sorts of examples of really beautiful calligraphy and fancy unicode tricks of various sorts and so I have some examples here that I made by fooling around with the unicode character palette for far too long like this face here made by combining characters to arrange things on top of each other and so I thought I could do it so I looked into the open type font shaping specification which defines how all these things work and so it says that things proceed in various stages controlled by tables within the font so first the characters in the string are mapped to glyphs which are internal things used by the font and then glyph substitution rules in the g-sub table are ran on them so these replace various sequences of glyphs with various other sequences of glyphs for example to do ligatures by combining glyphs together into a specialized literature then this sequence of glyphs that's had substitution applied to it goes to the glyph positioning step and what this does is it positions glyphs on top of each other in the correct location so that your accents are for example above or below the character and at this point everything is still just internal IDs used by the font and it needs to be rendered by the UI system and in this case it needs to be translated to outlines and these outlines for each glyph are stored in a table inside the font and I was reading through the specification and I started to get a little worried because everything seems to work from the start of the string towards the end of the string and I needed to work from the end of the number towards the beginning of the number luckily at the very end I found the reverse chaining contextual single substitution rule and what this rule does is it acts kind of like a certain type of regular expression where everything is a glyph class and glyph classes are kind of like regex character classes where they match a set of glyphs that can be in a specific position so there's the single position to replace and a class of possible glyphs there and replacements for each of them as well as look ahead and backtracking classes that match various positions around that glyph and this rule type was originally designed for a type of aerobic calligraphy and it was actually meant for doing left to right shaping on that type of aerobic calligraphy but aerobic calligraphy is a right to left context so in order to left to right it has to work from the end of the string to the beginning of the string so in my left to right context that's exactly what I wanted it would work right to left so font designers don't actually use the tables directly because there's some binary format they use a language called font feature files which compiles down to the tables and here I have an example file it has glyph class definitions at the top for lowercase and uppercase vowels it says a feature that I want my rules in these are just used for applying things in different orders or disabling and enabling things and then it has my rules my first rule takes finds a lowercase character next to another lowercase character and turns it into an uppercase character the next rule takes an uppercase character next to an uppercase character and turns it into a lowercase character and because these are reverse chaining they start from the end of my string and make it so that there's no two cases next to each other and so they alternate starting from whatever was at the end and this kind of alternating capitalization is kind of like counting modulo two from the end of the string by using two different glyph sets and we want to count digits from the end of the number and we can use a very similar technique by using different glyph sets but we only have one set of number glyphs so what we need to do is create extra copies of the digit glyphs so that we can keep this state because the only type of state in this substitution is by replacing glyphs so we need as many glyph sets as we need to count glyphs and as a digression I decided that at first I didn't want to insert commas I wanted to underline alternating groups of three digits and the reason is that I wanted this to work in my terminal which is a mono space context where every character has to be the same length and inserting commas would mess up the fixed width tables and things like that so in order to do this alternating underlining I needed to count modulo six because that's the periodicity of this alternating groups of three digits pattern so I needed six different glyph sets meaning I needed to add five extra ones so this is how the first version of numbered line worked the first character on the right of the number had no rules applied to it staying as effectively glyph set zero and then the next character after it had a reverse sub-rule that matched a normal digit following a normal digit and turned it into a glyph set one digit another character over a rule matched saying that a normal digit next to a glyph set one digit gets replaced with a glyph set two digit and this chaining pattern matches uh continues over the first six digits and then the seventh digit isn't matched by any rule and so stays as a glyph set zero digit and thus if I added more digits the pattern would just repeat and now that I have all my glyphs mapped to different glyph sets I can then modify the outlines that are in the glyph table uh to change the style of some of these glyph sets so in this case I add an underline to glyph sets five four and three and this gives me my alternating underlining at least in theory now I have to actually write code for it and to do this I had heard of this thing called the power line font patcher which adds extra characters to a font which is kind of what I wanted so it was a good starting point except for the fact that the font library used crash as soon as I tried to use a reverse sub rule and the reason is that it turns out nobody uses these reverse sub rules even for the type of Arabic calligraphy they're originally intended for so they're broken in a few different places luckily I did find a font library that did work and so I've had my tables that finally had underline and I'm alternating groups of three digits making them easier to read and so I went back to my original goal and I implemented fake commas by making some glyph sets wider and inserting a comma into the outline I squished glyphs together to make this kind of visual grouping and then I inserted a little commas in between so I can have commas even in mono space contexts did other fonts things like bolding and then put them on a website for people to download and wrote a blog post and someone saw this blog post and took it even further and wrote an implementation of fizz buzz where it just adds numbered list elements and these kind of magically become the correct fizz buzz things and it does this by implementing the divisibility rules for three and five that are based on the digits of the number as font shaping rules and then basically implementing ligatures to replace those I thought this was super cool but I finally had underlines in all my places like my pandas notebooks at work and this was great now you might think this is a horrifying hack right and I don't actually think it is I think I'm kind of using font shaping for something that it's intended for or or the same kind of thing in terms of it's taking a raw representation of the text and applying stylistic rules to make it it's more easy to read and visually pleasing and that's exactly the kind of thing that font shaping is designed for it's kind of hacky in so far as I'm using it on ASCII characters that nobody expects font shaping to be applied to so various apps sometimes do performance optimizations to disable font shaping for Latin characters like sublime text by term two so I had to switch terminals to kitty to get it to work but when it does work it actually works really well and quite seamlessly including like not having the fake commas when you copy the text so I think this is actually pretty good and I think it's a cool example of what you can do when you dive into a new level of the system that you might not have looked at before and find the right level to implement things I could have done it in some kind of way involving some ANSI pattern matcher to work just in my terminal or a browser plugin but I would have had to implement multiple different things and it just wouldn't have worked very well and it actually would have been harder to do and even after the time I spent researching font shaping so I urge you to go and dive in and understand new areas of the system that you haven't seen before thanks that's all I have thank you Tristan so up next we have a break until one pm eastern time for our next session but in the meanwhile and share some things that we have going on of course there's neighbor bot which if you're have access to discord uh group you can be mapped for somebody connected somebody at the conference meet new people we have voice channels that you can jump into for uh audio conversations and we have the virtual hangout space and you can find uh links to all these things in the general channel we also have somebody created a bang bang con spreadsheet where you can share you know fun and creative stuff this is honestly one of the most interesting spreadsheets I've ever logged into uh check it out for those that are searching for a job we have a job search channel in the discord group and besides that we'll see you in just a little bit we'll take a little break or recoup we'll come back for another great session of talks thank you all for tuning in hey everyone welcome back uh since one pm eastern time and we'll be starting our next session of great talks up next we have Helen Houssandi whose talk is titled sparking musical joy at home with magnetic stripe swipe cards in tiny coat uh i just read the five sentence we have for uh Helen which is Helen is a director of open source initiatives at the fully distributed agency tenup and a lead developer for the wordpress open source software project when she's not deep in a code editor or polishing UX she likes to make complicated food and i love complicated food hang out with her family and continue performing as professional collaborative pianists please welcome Helen hi everybody hopefully you can all hear me um i think i have this set up correctly uh you let me know if you do i do have discord open over here right so any yet started um he introduced who i am um as a work person uh so that's what i do i do computer stuff by day but i have my degrees in music performance and i do still perform sometimes um also as mentioned um i also have two kids they're ages seven and three now and they're equally opinionated and into music i'm very opinionated um i mention them because they're important to this talk so here's a question you'll hear from me all the time if you work with me what's the problem being solved for in my case it's that i was spending a lot of my time being a human jukebox for my kids the older one can manage a lot of things now including voice assistants but the younger one isn't quite there yet and even if she was our house can be very noisy and 90 of people who come here are spanish speakers which gets confusing for devices set in english um i live in costarica um my husband is costarican so uh that's that's why people here speak spanish and they sort of generally did before anyway um in any case you should hear the things we have to do to get el reino infantil playing the last time i tried and i posted a video of this on twitter the last time i tried this i got the reply i couldn't find o'dradle infant eel we also have issues for the kids like some songs off of just dance where they only play edited versions but as much as i love her there is no way i'm having my small children listen to an unedited version of cardi v so they have specific things they want to listen to and i want to be able to tightly curate as well i had seen this tweet go around um whatever december 2018 and thought to myself i should make that someday so with christmas the next year 2019 approaching and two months of school break ahead because our school year here is different than it is in the us uh it was time to solve my jukebox problem so first up supplies one of the reasons i wanted to build this using a raspberry pie specifically is because i have three older ones i've gotten as speaker gifts over the years and they're just kind of hanging around doing nothing um i ordered a couple more things off amazon and had my very loyal spouse drive them back for me after one of his gigs in the us um i could probably have found most of the stuff locally but electronics can be very expensive and it takes a lot of phone calls and driving around to locate specific items here's what i used a raspberry pie it's in a little case that has wordpress stickers um magnetic stripe cards a card writer which is the bigger one a card reader the smaller one and usb powered speakers so they wouldn't have to have a separate power source i forgot to include the speakers in this photo i collected everything and i started testing components first was making sure i could get Spotify playback on a raspberry pie at all i have no idea what i'm doing after a bunch of googling i decided to do this using pie music box which was super easy to install and get set up i actually probably took longer to try to find a usb keyboard before realizing that a wireless apple keyboard works just fine with the cable plugged in and now the next part making the cards work my idea was to put spotify uris on the cards and have that trigger playback right enter spotify uri and have pie music box start the start the track usb card readers are basically keyboard emulators you swipe the card and the data is sent to the computer as they typed it super fast um i plugged in my card reader and i swiped an old main card you know just in case it was like trying to steal my data and all i got were some numbers so now i've worked in retail and i know that your name is also encoded onto the card here's what i was expecting to see a card number my name some other information oh it actually has a dash in my name i know i was expecting that the uh it's like a checksum slash encoded information about like expiration um and then those numbers are actually repeated instead this is what i would get out of my card this is real uh real swipes that i got from my card readers um and sometimes i would even just get this right so what's going on it turns out i had a faulty card reader of course uh this would become even more evident way later on after everything was done because it occasionally sends ghost output and will randomly start tracks um i went to a couple of stores nearby looking for a new card reader but i couldn't find one and nobody at those stores knew where i could even find something like that uh so i did what all coders eventually have to do i hacked around so magnetic strike cards typically have three tracks of information the first being able to store the most data and the second and third only being able to hold some numbers and it's like a uh a limited number of numbers i tried encoding Spotify URIs onto the first track but between the card reader not reading the first track and Spotify URIs being case sensitive which if you didn't notice uh my name was in all caps this was a no go my buggy card reader actually sent me down the right direction because otherwise i probably would have tried to figure something out involving the URIs because i was sort of mentally stuck on that um i decided i would encode just a number onto the second and third tracks this also did not work correctly the first time i guess maybe the amount of data is too small or something so i had to encode each number with three digits left padded with zeros as necessary right our favorite thing left pad i also decided to encode the number onto all three tracks just to be like super sure that it was there so now that this was working right i was encoding the card uh with the card reader with the number on all three tracks and then reading it with my card reader so now that this was working i knew what i had to do i needed to have each number correspond to a line in a text file but how do i make that text file and manage it with hundreds of tracks that could get very tedious very quickly well uh Spotify playlist made a lot of sense uh i can keep stuff in order you can select on copy it as a list of Spotify URIs which is actually pretty cool and then i have the same list of music to play in the car or wherever else i also just really love making playlists so this is this is a screenshot of the playlist i have a kids jukebox playlist very dry naming um i can add more songs i can add more tracks and change stuff up if like a track goes defunct right like if i get geolocked out of a track or if my kids get tired of the song which happens with children this gets copy-pasted into a text file that goes onto the raspberry pi you want to do this for bang bang one okay um so now for some code okay the the tiny code part of my talk uh take taking swipe card input and making it play a song i spent probably close to a week with a code editor open and i was just staring blankly at it like not even sure where to start i overthought myself in huge circles uh reading about people's python based setups and rfid chip versions and wondering if maybe this was all of a state because i don't know what i'm doing uh until finally one night with the holiday looming i wrote something anything and it worked on the first try i genuinely i cannot believe it this is my first run of code it's literally five lines of bash i chose bash for two reasons one i don't have to set up anything else like python in any associated libraries like for reading input and two i'm actually pretty decent with bash especially after forcing myself to myself to learn how to do do bash write bash by doing a bunch of custom things to my prompt several years ago uh this code does exactly what it says but there's a pretty cryptic bashism and i am the very liberal code commenting school so i added comments to that bashism which is called parameter expansion and the set bit this helps me and anybody else looking because of course as a director of open source i wanted to open source this again this code it worked perfectly um until i got to card number eight and then i would get an error i sat there stunned until suddenly all my ears are dealing with loose typing in php washed over me octals since i was padding with zeros when i got to zero zero eight and zero zero nine it yelled at me that that wasn't a real number so a little more said to remove those leading zeros here on line six then i wanted to add a special case card to tell the playback so my kids could stop music as necessary pie music box does come with a web UI which i sometimes use to manage things from like across the house but the idea here was to only use cards this has been extremely successful and my kids actually keep that play pause card at the back of the rack of cards so that they can always find it even the younger one has a stout she will look and get the card and pause playback i decided to use nine nine nine since i was using three digit numbers and i don't have nearly that many cards and they're really really hope that i never do and then finally my senior engineering mindset make sure you account for error cases here i want to account for not finding a number in the input as well as potentially not finding a corresponding line in the text file and that's it that's the entire script 27 lines including comments and whitespace it's tiny code okay so now i have this raspberry pi able to play Spotify tracks off of cards but it still requires me to log in and get things started so how do i make it skip manual login and start my script on its own that is how do i make it headless you don't have to worry about memorizing this i have everything written out and available to you that you can refer to at your leisure this is just to show you that's honestly really not that bad you download the file set up the automatic login and then make it run your file your script on login so everything was working so now on to the assembly of the object i went to a couple stores to see if any existing boxes jumped out of me as a good solution but didn't find anything my brother-in-law does happen to have a 3d printer but i didn't have any particular design in mine and honestly time was running out i poked around my endless supply of organizers because i'm a slightly compulsive organizer and found these in my stash some modular stacking boxes from one of my favorite places the container store and heavy duty double-sided tape seriously this tape does not add but this tape is ridiculous i've actually since acquired a properly working card reader but i can't pry off the old one so i haven't actually replaced that part it just randomly plays on the track sometimes anyway a little cable organizing later and tada a compact little jukebox ready for gifting but wait i haven't talked about the artwork yet there's stickers i love stickers now that entire that inspiration project that i talked about at the beginning also use stickers but the creator wrote an entire react app that pulled the artwork for him and everything i may be a programmer but i am not made like that so i return to something i actually know how to use which is a graphics editor so shout out to anybody else whose farmative web experiences include paint shop pro i made myself a template and then grabbed album art manually another reason why i decided to get art myself was because i didn't always want the album art that a script would grab so for instance for mother goose club songs i wanted to get a screenshot from the video for the song so that my nine two-year-old can identify the specific song she wanted because as smart as i think my kids are a two-year-old is not reading titles and artists off of a label i exported these as pdfs and headed to a nearby office supply store to have them printed out on adhesive paper but then cut to size and applied to the cards i did everything in numerical order you might have noticed i wrote a number in sharpie on the back of the card in a previous picture so that kept it easy but also i very much believe in trust but verify so i spiked every single card before applying the label and yes it includes bts so finally i presented this to my kids and show them how it worked and after just a couple of days both of them were masters of their new musical domain i put it on a little shelving unit in the area between their bedrooms and completely on their own i promise on their own they negotiated q-system between them or they light up cards on the shelf for what they want to play next they have absolutely loved their own little device my seven-year-old loves to tell people that his mom made this because she's the best programmer and it's just been all around joy watching them use a tactile device that brings them something that they love digitally i hope you've enjoyed hearing about my first foray into hardware tinkering i have everything that you need to get it going including the sticker template and files to make the card making easier available on my github where i'm just helen and here's a little demo for you brought to you by my younger child thank you thank you helen um before we get for continue on to our next talk i want to invite co-organizer lika to join me hey hey i can't stop smiling that was amazing really was fun how are things going um i'm pretty stoked i can see you know a lot of people are interacting with us on discord and twitter and i'm just getting hyped all the time because i'm reacting to a lot of things here in the background so it's amazing people are retweeting things they're posting more things people on discord and just like posting links all over and i'm learning a ton so yeah it's it's been a lot of interacting and a lot of learning and i'm i'm really happy that we're doing this virtually what what channel do you find yourself most checking or checking most frequently on discord when there is a talk going on i usually just i'm in the talk channel and i see people reacting because that's super interesting to see what people are seeing the one thing is that we as organizers see the stream a bit before everyone else so there's this 20 second delay when i'm waiting for people to see the jokes which is really fun um and other than that i mostly just sit in the spreadsheet that bang bang con you know party spreadsheet is amazing that's so awesome well it looks like uh we're ready to continue with the next talk yeah awesome yeah so we jump back let me introduce our next speaker so up next we have julia tufts whose talk is titled bang bang my interpreter shot me down uh julia is a software engineer based in brooklyn originally from halifax canada she used to study classical music and graph theory but now makes bleeps bloops and bad wordplay she enjoys imbuing abstract concept abstract concepts with tenderness please welcome julia hi hopefully you all can hear me um yeah i'm julia uh i am quietly online under the twitter handle big commitment and i wrote a song about javascript and javascript interpreters which i will play for you shortly but first a little background and contents so the inspiration for all of this was what if i made a cover of bang bang my baby shot me down but made it about the bang bang operator in javascript instead so why um a few reasons one i am committed to the full realization of terrible puns and secondly i was curious um firstly just like okay if i made this terrible pun song what would that even sound like what would the lyrics be who would the characters in this be and secondly um it just got me thinking like how does a bang bang really work in javascript like i have some idea probably we'll evaluate but really like what's going on um if i have this expression like let's some pool equal bang bang some variable i was really interested in digging into like what what is this moment that like this final bang gets applied like is it i don't know it just sounded like dramatic and i wanted to explore that more so um yeah let's just review what a bang bang is so um it's just two logical not operators together so if you have a class boolean true apply one bang you false another bang true um but it's a little more interesting in javascript land because you can apply this to anything such as numbers so the numbers will typically return true except for zero which returns false strings shout out to char so oh except the empty string returns false and even objects so the one here um but this is a part that kind of like through me because i was expecting this to return false it actually returns true and actually similarly with lists and empty lists will return true as well so it seemed that like most of the time like it's true but sometimes or it's most of them it's false but then sometimes it's true so what's that about um so it turns out that like a bang bang will return true if the input is truthy and false if it is falsely and maybe you've just heard the terms truthy and falsely for the first time certain are like did you just make that up um and no it's a real thing someone else made it up i'm just the messenger so um it's not too bad though uh so how we think about it is that there are only a set amount of falsely values there's eight in total so they're the original false three different flavors of zero the empty string null undefined and not a number and anything else will be considered truthy so now that we know a little bit about what a bang bang is how does a bang bang work how does it get evaluated interpret it um so it turns out that computers are complicated browsers are complicated a javascript is that strange but uh to answer this a little bit um so like your browser has a javascript engine and the javascript engine will run that javascript and every engine is a little bit different but um more or less they'll like press interpret the file and run and v8's case has this little side it's like it'll try to optimize it and then like there's this flow tribe that's like sometimes it like optimizes too much and it has to like de-optimize and then like but yeah it's a wild world out there um but uh this song will cover this a bit more as well um and yeah so and uh v8 is the javascript engine that chrome uses and i chose to focus on that because it's open source i can attempt to dig through the code and understand what's going on so um the song will be coming up shortly so yeah i was like okay what what's the song going to be about um and so taking all of this and like digging through the code uh i decided to focus on okay what if i am a javascript object how do i travel through this engine pipeline through this parser and interpreter and how do i get evaluated as we are so that's kind of what the song is about and i think the video will be ready to roll hey everyone welcome let me invite alicia to join me on the stream and uh thank you julia for that amazing talk did you see how awesome the live captioning was on this one this is incredible yeah there is all the sounds like whoomp and all the background it's just amazing to have that you know i'm on esl speaker so english is not my first language that helps first with me understanding everything but also as an organizer i have to like jump between things all the time and sometimes i don't catch everything during a talk and i can just look at it and scroll back it's amazing yeah i love this in script with the music of the thud thud like the capturing of the nose and mixing it with the lyrics like it's also there will be a session with mirabai today and i'm conferencing so um if you are on this card you can join it and you can see her awesome setup and how she manages to do this so fast and all the shortcuts and everything it's just mind-blowing if you've seen mirabai at previous bang bang cons uh the it is just like incredible like the lightning paste i thought it was a fast typer but i am like not anywhere near nowhere near let me see okay it looks like we are ready for our next talk awesome so up next we have matt clausen whose talk is titled repair a commodore 64 in just 64 months matt builds software for journalists and is located in newark new jersey he enjoys studying the history of computing and learning the inner workings of obsolete platforms adored by millions please welcome matt thank you so much and thank you to the bang bang con organizers for putting on this conference and for creating such a welcoming space for new speakers like me and for everyone else and uh yeah let's get to it um today i just wanted to share with you like my multi-year saga of repairing my commodore 64 that i found in the basement of the thrift store buried in the dust and dirt and just my experience of going from someone who had some basic working knowledge of playing around with electronics and someone who maybe has a little bit better of an understanding and appreciation for how these machines worked so let's talk about the commodore 64 for those who haven't used one before it might be a little unfamiliar this computer was released in 1982 it sold about 10 to 17 million units depending on who you ask and that puts it among the ranks of like the top selling computers of all time i think so when you turn this thing on you just get this screen and this is like a screen that represents a world of like infinite possibilities what you're looking at right here is an interpreter where you can just start typing in basic code like you the most common one is the 10 print go to 10 you know hello world kind of thing you can just start going now you can also plug in cartridges in the back there is a flop floppy disk attachment that you could use to load additional software and i'm sure there were many many users who never wrote a line of code or you know never created their own applications but used the software created by others and that's perfectly fine that's perfectly great but for those who wanted to learn the system a little bit better Commodore provided a programmers reference guide and this guide would basically explain to you like at a really low level how all the hardware worked and if you found that this game that you wrote or the software that you wrote in basic wasn't fast enough well their programmers reference guides would also teach you how to write assembly code and so instead of using an interpreted language you could write assembly and get that so much more performance and interact directly with the graphics hardware on the system and really like a whole generation of programmers you know grew up with this machine and it changed the course of their lives and so why do i want one of these or why did i want one of these i'm not quite sure i was a little young to have ever like used a Commodore 64 in his heyday but i knew that it existed and i especially knew that it existed by way of the demo scene which is a really fascinating internet art subculture that i just really love and a lot of folks in the demo scene also love this system so i kind of wanted one too also i had spent some time studying like the architecture of the nintendo entertainment system and learning how that worked at a really low level and that was a really rich and rewarding experience and so i kind of wanted to do the same thing with the Commodore 64 because they share this very similar processors and i thought it would be interesting to compare and contrast how engineers you know took like the same like similar building blocks but created like vastly different systems and so if you look at these old computers and you compare them to computers of modern times it's clear to see that everything has gotten smaller everything has gotten more tightly integrated and the way that we think about fixing these old computers versus fixing new computers is also drastically different on the left here you see an actual photograph of my board you can see the two chips are removed because i was debugging and trying to determine faulty chips and so i was able to do that with relative ease and on the right you see a macbook pro 16 inch model and wow that just looks so crazy i don't know if i would ever try and fix a specific component on that thing and i don't think most people do certainly apple you know has some sort of refurbishment center and i don't really know how that works but your average like mom and pop computer repair shop probably would throw their hands up at this now there are some exceptions to this rule a fellow by the name of lewis rosman based in Manhattan runs a computer repair shop and he is like a wizard he uses advanced micro soldering techniques you know under microscope and he'll debug and he'll replace like the missing components and repair traces and it's crazy but most people aren't going to do that if you look at Commodore 64 you would notice that a lot of the chips and a lot of components on there are bog standard off the shelf and it was possible for people back back then to repair their own computers they could buy these replacement chips you know there's tons of 74 000 series logic which is like a de facto standard of the of the time and there's off-the-shelf d-ram chips so if you're ram chip fried you could you know look if you're lucky it was socketed and so you just pluck it out if you're a little less lucky you probably have to wrestle with a solder sucker or if you're you know maybe a little bit more wealthy today like you can use a desoldering gun which kind of automates the process of removing chips from boards and it's really important to note that even back in the 80s Commodore provided schematics to authorize like warranty centers and also just their customers if you opened up the programmers reference guy that i mentioned earlier you could fold out you would see a followed up schematic that you could unwrap and you would see how your computer was built imagine getting one of those now with your laptop you'd be overwhelmed and i was overwhelmed looking at this schematic when i got started it's just you know an insane amount of detail and if you were you know trying to fix your old computer back in the day if you were lucky one of those standard chips failed but if you were a little more unlucky maybe one of the custom parts failed and that was kind of like your sign that you need to send this off to a authorized warranty center and you know let them use their stock of like proprietary chips to repair your system today we don't really have the luxury of buying new Commodore 6510 i think this chip number for the cpu you'd have to harvest them from some other broken machine if and we had hoped that that one works so lucky for us in the year 2020 information has been preserved and it's being shared online and i think anybody who ever tries to repair a Commodore 64 will stumble upon this individual's website which is just a goldmine of information this uh rake harrelson uh i believe was a former Commodore repair tech and he has accumulated over the years tons of documentation tons of reference material and it's super critical and super useful for someone like me who is just getting started to be able to rely on the accumulated experience of someone like him and it's just amazing to me that he has documented this so thoroughly and sharing it for free so this is all well and good and if you are experienced you can sift through this information and you know what to look for but i wasn't really experienced when i started out when i got my Commodore 64 in the thrift shop it looked like this looked like in pretty good condition but when i turned it on i was i was greeted with not a very friendly screen you might notice that this looks nothing like that friendly blue screen that we saw earlier uh i did not dare use a video clip for this presentation but if this were a video you would see these colored dots wiggling and waving across the screen and the computer was totally unusable but that didn't stop me from trying to fix it i took apart my the computer that's the thing that i kind of always do when i get old electronics i like to clean them up i like to make them presentable because i also want to use them they're not just sitting on the shelf i mean they they sometimes should sit on the shelf for a little while but every now and then i like to pick them up and use them so here's what my actual Commodore 64 looks like when you take it out of case for a beginner like this is scary what are all these chips on the board what do they mean how do they interact and i used ray's documentation and i started you know trying to get a sense for like what's what's wrong with this like what part has failed what parts have failed i was completely you know fumbling around in the dark um you can see here that ray lists all of various chips and like their failure mode what does it look like when this chip fails and all i knew was that i had a black screen with dots on it he calls them blank screens and surprise surprise there are lots of blank screens i was overloaded with information i did not know what was important what was not important but i still tried i went through chip by chip and tried to replace some things one common thing that people recommend is piggybacking ram basically just taking a replacement ram chip and sticking it on top of one that you think might be working and i did that and of course that didn't work um the sound chip is notorious for failing on the Commodore 64 it's known as the SID you might have heard of it um and sometimes when that chip fails it prevents the whole system from booting up so i plucked that out no luck there's another chip that fails a lot it's called the PLA the PLA is kind of like a switchboard operator for the system and if that chip fails then nothing will work so i bought a replacement PLA from germany by some guy and that didn't work either so i was just out of ideas i decided to set this project down and you know maybe come back to it when i could attack the problem with a little bit more rigor what i was doing was akin to you know if you had a programming project that you're working on and there was a bug i was basically just changing random things and hoping that something would work and just like that's no way to program uh that's not really a great way to diagnose a piece of hardware that's faulty so i spent some time uh working on other projects i kind of upgraded my tools a little bit i gained a better understanding for how they worked and a few years later i decided to come back to this project and i decided to take uh a more you know directed approach to figuring this out and with the first way that you the first thing that you need to do to fix something is understand it so let's take a look at the conloader 64 circuit board and kind of break it down now there's lots going on you can ignore all those little like chocolate chip cookie type things those little capacitors and the other like capacitive components on the board and we can focus on some of like the more interesting chips on the top left we have i o those are two identical chips and they are basically peripheral controllers next to them we have a series of roms they store things like the basic interpreter the kernel that basically provides low level functionality to the rest of the system specific routines a ROM chip that basically just has the character data so like the actual text that you see when you type uh the cpu that's really important we also have the cid chip that we mentioned before below that the pla the the ram over in the bottom left of course we've got a cartridge connector and the power section of the board and in the middle we have like the video section so you can kind of see that these different components of the board of the computer you know live in their own little regions and if we think about that the physical layout of the board we can kind of map that to the schematic and so now we don't need to understand the whole schematic in order to you know attack this fix we can maybe take a more directed approach and we can focus on just the information that's important to us at that time now i don't have any specific i don't have any specific like diagnostic equipment for the Commodore 64 although it does exist so i didn't necessarily know that for instance all my RAM was working and that all my ROMs you know still were returning valid data i didn't know that my PLA should be working because i had replaced it so i decided to first focus on the video circuit we knew that we were seeing this weird like color dots like what is that all about so let's zoom in on that this is the video region of the board normally this would have been covered with like an rf shield but none of us have x-ray vision and i don't have an x-ray display and i don't think you do either so like we have to take that thing off and we can see that on the circuit board now you can't see them in this picture but these chips are all labeled and so that's really useful for finding out where they exist on the circuit so u19 this is i believe a vic2 chip and so its role is to generate video signal but it needs some help from this little baby chip u31 u31 is a technically a dual voltage control oscillator its job is to take a clock signal of a certain frequency and a clock signal is kind of just like a beating drum i am not beating with any particular rhythm but use your imagination and so u31 wants to take this particular clock signal and slow it down and output that slow down signal on its pin pin six this clock signal is denoted on the schematic and all the other dot all the other clock signals that are relevant to this area of the circuit are also noted and we can use this information to our advantage thankfully i know i'm in i'm in a privileged position to be able to afford an oscilloscope which is a relatively expensive tool but these days you can actually get them for a whole lot cheaper and that's a great thing so i took my oscilloscope and i started probing around i measured the master clock signal and that looked fine and i then measured these pin six output on this u9 on this u31 pin six supposed to be 8.1818 megahertz um but it's not not on my system i had this really messy signal this is not what you want to see and i knew this chip must be bad so i bought a replacement very fortunate that these things are still for sale i didn't have to gut one from another system and so i installed it still a black screen but not a black screen with crazy dots just a solid black screen and so that got me thinking okay well the video seems to be good let's see if the rest of the system is operating now how can we do that well there are various tools that you can use like you could use a multimeter and try and see like okay what is the what are the various logic levels of these particular pins but that's not really a right tool for the job i use this thing called a logic probe i'm gonna kind of back and forth on my slides here but just bear with me this logic logic probe allows you to really easily determine if a given signal is a logical high or a logical low and if you put it on something like a ram chip on like the address lines you could see that if you're getting alternating between low and high really fast this tool will make like a little buzzy sound if it's low or high and it also give you these little indicators and so if you see activity like on the cpu bus or on like the ram you can kind of be certain that the cpu is doing something and i wasn't seeing that behavior what i was seeing though is that this signal called reset was being held low on my system now what does that mean what is reset well if you consider a computer that's turned off and then you turn it on well how do you how do you guarantee that the various components in the computer are going to be you know orchestrated properly and how can the system initialize well you need to have a ground state you need to have a known good state to progress from and so the reset signal is the the tool for the job if you can generate a reset signal when the computer turns on you say okay well for the first you know half or one second after turning on the machine send a reset signal out and all the other chips will get this reset signal and they'll know to to do whatever they need to do internally and then we can turn off that reset signal and let the magic happen it kick starts off all the rest of the booting process and on my system this was not happening every single chip had the reset signal held low so they were constantly in this state of trying to reset and that's no bueno so if we take a close look at the reset signal up the reset schematic section of the schematic we can see u20 here which is this chip on the left and we can see u8 here which is this chip on the right uh 556 indicates that this is a 556 timer chip and don't really have time to get too much into those but these chips can be used for generating all sorts of cool signals in this case the configuration of the 556 timer is set up in such a way that after a power is applied it will generate a quick blip for the reset to occur and it passes that reset signal over to u8 which is a an inverter chip which kind of takes like a thumbs up and turns the two thumbs down or a logical high to a logical low and the specifics of why that works the way it does again out of scope but basically this system was not working the way it should this reset signal was never being turned off and so i knew by my using my logic probe that the 556 end of the system was working i could see that for a little while the reset signal was being applied properly and then being turned off but if you follow the output of the 556 over to the input of the u8 with a logic probe you can see the signal and then when you look at the output you can see the signal and the output on pin 8 here was just constantly low and so that was my say that was my sign that i need to replace this chip so i replaced this chip and again i'm not going to show the video but here you can see me right after the computer is turned on we have a logical low being output so all the chips in my computer get that little signal and a brief second later it goes to high and that's kind of reversed from what you might think but if you if you just give me a little faith you can know that this fixed the system here you can see my extremely messy setup at home i have a lot of video game consoles i really like games play them when i can and i was so pleased to hear this to see this i did not hear it because unfortunately my SID chip is still dead but everything's working i wanted to talk a little bit more about preservation and nervous engineering because that's really important to keeping these old systems alive and i know i'm running a little over time so i'm just gonna whiz through this there's folks doing really amazing stuff dissolving chips in acid and imaging them and learning how they work at a super low level and we're using this knowledge to create modern replacements of those custom chips that i said had no replacements well now we have modern replacements and that's fantastic fortunately for the world at large you don't even need to have this original hardware anymore the folks at the internet archive are doing great job archiving the software and providing emulators that you can just run in your browser a project that i'm really fascinated by is the mrfpga project which uses really cool technology to reimplement uh various computers and game systems in hardware which is really cool and that's it thank you so much i just want to you know take a moment to say thank you to all the essential workers they've always been essential it's just they haven't been properly recognized for that you can find me on twitter at matt clausen thank you so much bang bang con thank you matt um so up next is we have a little we have a break until 2 30 p.m eastern time that's just 39 minutes uh there's a length of this break um before you go just connect you some more resources and remind you of what we have going on again later this afternoon we have an unconferencing session if you i'm gonna put there's things that i share right now you can find links to them in the announcements channel of the discord application uh so the unconference session if you want to connect with people about special subjects interesting subjects you want to create an unconferencing group please go check that out there's a spreadsheet for that we also have neighbor bot which you can go in to be matched with uh other attendees at the conference we have voice channels in virtual hangout spaces that you can join as open groups they can come and meet more people as well we have the bank bank con uh spreadsheet party which is just a really fun creative spreadsheet that you can share and express yourself um we have the job search channel if you want to find a job if you want to advertise the job check that out and then again all these things are going to be available in the announcements channel so you don't have to keep up with uh everything i just mentioned but check them out we'll see you back shortly hi i hope you had a great break and you're ready for our next session of talks we're going to have three amazing speakers um and the first one is ben kuhn um ben is going to talk about 89 characters of base 11 mobile networking in rural Ethiopia and before he gets started i'm just going to um tell you a bit more about ben so you know uh what he's coming with to this talk ben works on helping unbanked people in sub-Saharan Africa send and save money when he's not bashing his head against obscure network protocols he enjoys reading climbing hiking making music contra-dancing trying to improve the world and various other strange activities if you want to learn more about ben his website is linked in his bio on our website and now over to you great uh thank you so much i am really excited to be here um so uh i'm ben and i'm going to tell you about a series of increasingly silly things i've done to uh make an app work when the network is bad um this photo by the way is of uh some of my co-workers um climbing on the roof of the house that we all lived in in Ethiopia um uh it was fun it turns out the roof could barely support our weight i thought it was kind of an appropriate image um so uh first in context um i work on an app called wave which is a mobile money system so imagine you live in a village in say like the Somali region of Ethiopia and your local bank local in quotes is like three hours away so with mobile money um instead of going to like withdraw your money at that bank branch um you could withdraw at like you would find a local shopkeeper who's signed up to be a wave agent um and you can withdraw your cash at that agent um and they would give you like their like spare cash on hand and we would reimburse them later um so if you wanted to be like really stereotypical startup bro you could call it uber for bank tellers um anyway we also have this cute penguin mascot um so uh the first problem we ran into while we were trying to build this was that um a lot of our biggest users were in really rural areas um the Somali region in general is pretty um non-urbanized uh so this is my co-worker Ahmed taking his motorbike full of cash across a river to reach one of our remotest agents in east email um and the other thing about Ethiopia is that there is only a single telecom ethiotel which is a state-owned monopoly um and it's really bad so in many of these um more rural areas mobile data would be basically totally down for most of the day um and only working in like the early morning and late evening um so fortunately during those other times there was something else that did still work which was sms um so we decided to use that instead we would take all the network requests that our app was sending um and instead of sending them over mobile data like normal people um we just squeezed them down until they fit inside a text message um so a wrinkle with this is that text messages obviously aren't very big um we wanted to fit everything inside a single message because um otherwise like piecing them back together would be really hard and um we had to encode the data in like the set of 64 characters that um are actually uh like safe to be used and interpreted in text messages um so each of those that means um if a character has 64 options um it can represent like six bits per character um and uh that meant that with like the 151 characters we had we got about 113 bytes for our messages um which is not great but uh you can fit a lot in 113 bytes if you try hard enough um so I hope you remember that log base 2 equation because it will return as like the final boss um anyway uh so the other wrinkle after we like packed all our messages down was actually building this thing um so at first we thought to ourselves oh we can use twillio like we always do for this kind of thing um well it turns out that the whole like state-owned monopoly telecom thing also made ethiotel a pain to integrate with um so as you can see twillio does not let you even receive messages from ethiotel just like at all um instead of doing that we had to build our own sms integration uh from scratch so that meant we needed a computer that was connected to ethiotel's internal network um and that meant it had to be inside the country and that meant we needed our own data center uh well uh for some value of data center so this building was our actual data center um our servers are running in a little closet off to the right of the frame here um it ain't much but it's home thankfully it did have one nice thing which was a backup generator for the many times that the municipal power went down for eight plus hours of time um but uh sadly the backup generator also often broke um and at that point we would need to fail over to our backup backup system uh which is that our customers would call our employees personal cell phones and we would use slash slash commands to enter their transfer details um okay so uh that worked fine for like some value of fine for a while but soon we started having an even worse problem which is that like even during normal periods our agents couldn't do transactions fast enough because they were spending too much time staring at the loading spinner on their phones um even in the areas with relatively good network like giga the capital um so when i tested in giga i found that pings were taking something like five seconds which is like normal uh ping time uh but our apps requests were taking more like 30 seconds and that was confusing to me we were sending like somewhat more data than just uh uh the ping command but remember it all fit in a single sms so it wasn't that much data so to explain why this is going on um i need to take a detour to explain how normal mobile abscene network requests um you think of it as using the protocol https but um that's actually made of a stack of a few different protocols that operate at different layers of abstraction so the the lowest level protocol is called ip for internet protocol and that gives you only a single operation um which is do your best to send this small packet of data to this given address um but it like the packet is allowed to fail you can just sort of drop it on the floor if you feel like it um so on top of that um there's built tcp transmission controlled protocol um which gives you two things that ip doesn't one of them is reliability so um you know that the sender received the packet and the other is um undounded size so tcp lets you send a potentially infinite stream of data then um https uh runs on top of tcp but it's actually within that it's a stack of two other different protocols um so uh the lower level is tls transport layer security um which is like tcp but the stream is encrypted uh so that the data is only readable by the intended recipient and then on top of that is http um good old http which gives you the um actually lets you say like get me the document at this url and then you got a document or if you're a mobile app you can abuse that to mean things like send a payment and like the payment worked um okay so because these protocols are all stacked on top of each other like this um that introduces quite a bit of overhead so tcp requires you to send um an empty packet back and forth before uh you an empty ip packet that is back and forth before you start sending actual data um then uh on top of that tls adds an additional two back and forths uh to do things like public key exchange and cypher suite negotiation um and so it's not actually until the fourth packet round trip that you can start sending the data that you wanted to send in your http request and that's why our requests were so slow um if a single packet round trip like ping would take uh five or ten seconds um then these three like round trips that do nothing for us at the beginning of this sequence um are adding 15 to 30 seconds of overhead so can we do better than that um well yes uh these intermediate protocols are giving us a lot of stuff that we don't need um so in our case we're talking to a single server that we know ahead of time and that doesn't mean that means we don't need key exchange or cypher suite negotiation um and because we'd already packed our data down to fit inside a single sms um it also fits inside a single ip packet so we don't need the stream abstraction that tcp provides us so um in order to like uh slim down our protocol we ended up building our own based on a different lower level protocol called udp which stands for um unreliable datagram protocol so that's a very thin wrapper around ip that doesn't incur any additional round trips um unfortunately as the name might suggest uh udp sacrifi