So, hello, and welcome to Building Augur Lessons Learned. In this talk, I'd like to tell you a little bit about myself, and go quickly into what Augur is, and how it might architecturally differ from dApps you might be more used to. I'd also like to talk about what went wrong along the way. During the build and deployment process, we had issues with test net deployment. We had issues with our integration with 0x, and we had issues with the Ethereum JSON RPC interface. And finally, I'd like to talk about some of the new exciting things we're going to be releasing in Augur v2. My name is Scott Bigelow, and I got involved with Ethereum in early 2017, when I wanted to understand what a smart contract was a little better. I had some previous experience with Bitcoin back in 2013, but I just didn't get it. And I felt like the best way to understand was to pick up a tutorial and, you know, write the standard ERC-20 contract. But to do so, you needed to get on a test network. And the network that I chose at the time was Coven. This is a parody-only proof-of-authority network. Proof-of-authority means that there's a small set of validators that are eligible to mine this network. So if you want to come by any Coven-Eath, you need to go to a faucet. And one of the popular faucets at the time was this Gitter channel where you can come and type in your Coven-Eath address. And somebody will come by maybe in four hours, maybe in 36, and will send you some Coven-Eath. Now, I come from a background of software development. So my first thought was, well, we should automate this. And so I wrote this Coven Gitter bot that's monitored for these things and delivered Eath to the people who requested them in real time. And this was really my first interface with the Ethereum ecosystem. And I thought it was great. And I hadn't even gotten the smart contracts yet. This bot has been running for just over a year now and has fauceted almost 30,000 times. So if you've gotten Coven-Eath in this Gitter channel, you got it from me. But this really led me to wanting to get more involved in the Ethereum ecosystem. And I started work on Augur later that year, in 2017. Augur is a Ethereum-based decentralized prediction market protocol. This is a system where any user can come and create a market, which is really just a question that has some set of outcomes and an expiration date, some future date where this is going to be resolved. Before that resolution, users can stake Eath on what they believe that correct outcome to be. And after resolution, the market creator is responsible to come back and inform the blockchain what the real world outcome was. But of course, there's no reason to trust this market creator to give you the correct answer. And that's where Augur really comes in. Augur has created a system of economic incentives such that rep holders, the holders of the Augur rep token, are incentivized to come and dispute answers that are incorrect. And this dispute process can last a long time. This dispute process can go back and forth and back and forth. And if the system ever notices that there is a major problem, the system ends up forking. And every holder of rep needs to make a decision. Which side of this fork do I want to be on? And we believe that holders of rep token are going to be incentivized to want to be on the true fork. The fork that represents the true outcome. To accomplish this, took quite a bit of code. Augur is nearly 5,000 lines of solidity code and the base deploy is 41 contracts. And architecturally, it's a little different too. If you've used the Maker interface before, this is another depth, which is a series of smart contracts and a UI, a static UI, that can make dynamic calls into Ethereum and run an ETH call. This is a simple function call, it's just a read call to ask questions. I might ask things like, hey, how much MKR does this account have? How much dye does this account have? Now these questions come from the current state of the Ethereum network. All this data is stored in the storage of the contract space. And there's a little bit of logs here. They can tell you some things that happened in history, but mostly it is powered by asking questions about right now. So Augur does this a little differently. The UI does not communicate directly with the Ethereum node. The UI has an intermediary, which processes logs to reconstitute the state of Augur. And because Augur is a decentralized system, this isn't a server sitting in a rack, this is your box. And if you've ever seen this window before, that's why Augur is presented as an executable. Because this service that powers the UI is critical for its functionality. And so why do we do this? Why do we take the Ethereum database and turn it into another database? So we get a few features here. We get relational data. The Augur data model is fairly complex. We have lots of different things that relate to each other. And if each one of these needed to be some ETH call that led to some other ETH call, this could be dozens and dozens of queries just to look up a single page. By putting this into an index relational database, we can serve that with a simple single joined query. We also get data locality. No matter what your latency is to that Ethereum node, you are serving this data local. It's all ready for you. And users are the ones who are contributing the titles for these markets and outcomes. And whenever users are entering text, you're gonna need a good way to find that text, something that's not exactly accurate. So we have string matching and search patterns that help with this discovery. But there's another reason. I mentioned that there were 41 base contracts. But that's just where it starts. When you create a new market, that is a new contract. Plus you're gonna have two to eight outcome token contracts. You provided an initial report, that is a contract. When you provide a dispute, that is a new contract. Has a week passed, that is a contract. It's called a fee window. And so this chart from before, where we talk about the 41 contracts that make up Augur, is dwarfed by the number of contracts that have been deployed since its launch, which sits at over 5,000. Augur is also a fairly old project. We're currently sitting at block 6.6 million. The release took place around 5.9 by involvement 4.4. But if you wanna go back to that very first Augur contract deployment, the first time that somebody said, I would like to make something like Augur and deploy it to the network, need to go before block zero. This system was developed well before the Ethereum mainnet block zero was launched. On a test network that I think at the time was called POC nine. But this is actually before Solidity. Solidity 0.1.0 was released here. And so these contracts were written in Serpent. A different smart contract language that compiles down to the EVM code. So when the Reptoken launched around block 2.4 million, this token was also written in Serpent. And work got underway to develop the Augur contracts in this other language, Serpent. Unfortunately, around block 4.1 million, we asked Zeppelin to take a look and then how's this doing, how's Serpent going? We had them both look at our smart contracts and the Serpent compiler. And what they discovered was that the Serpent compiler had a critical vulnerability in it. It wasn't an issue with the smart contract code, but even the Reptoken itself that had been deployed was vulnerable to EVM code that came out of the compiler. And so the Reptoken had to get redeployed at this point. It was rewritten as a Solidity contract. And all of the Augur contracts that were already ready for audit had to get rewritten into Solidity because it was clear that the market had really gone this way and this research really justified that effort. So your smart contract is no more secure than your compiler, which is why we were proud to release yesterday a joint effort between Augur and the Ethereum Foundation for a Solidity compiler audit. And they found some great stuff. They had over 12, they had 12 issues that they considered to be high or critical. And these issues are, some of them have been addressed, some of them will be addressed, but luckily none of these created EVM code that left our contracts vulnerable, but we feel a lot better about the platform that we're leaving behind for other developers to build upon. One of the issues we ran to was a test net gas limit issue. So I mentioned we have 41 contracts and those take 50 million gas to deploy. But one of these contracts, the largest of them, takes 5.8 million, which isn't a problem for Mainnet. Mainnet has a block gas limit of 8 million. Robston, however, which was one of our main test networks for development at the time, had a block gas limit of 4.7 million. It could not contain that one transaction to deploy the largest contract, which was critical for our system. Here's a chart of the block gas limits over the last two years. You can see for large periods of time, Blue, Mainnet could host large contracts, but there was no test network where you could actually deploy these and try them out and share them with the world. So while Robston and Mainnet run the same code, there's a critical difference. Robston isn't used as much and miners look to how full a block is to slightly increase or slightly decrease the block gas limit for the next block. It's up to them, they don't have to use this as their criteria, but it's what they generally do. So what's a project to do when they want to ship to Robston? Let's just fill up the blocks. Let's get these miners to think that these blocks are really full, so they'll start voting this limit up. And so that's what we did. We wrote this smart contract whose only intention was to waste all gas that came into the transaction. Now, knowing what we know now, we could have done this a little better. If you assert false, you can actually hit an EVM opcode that doesn't waste all this time on these full nodes, but still effectively wastes gas and gives you this block.  And I started in test nets and I really wanted to help people with the faucet that I built. And so this kind of weighed pretty heavily on me. I was wondering like, why are we destroying the usability of this for days at a time for people who want to use Robston? Now I'm just kidding, somebody kept mining empty blocks and so they kept voting that gas limit down. So we had to do it ourselves. We spun up our own miners with a target gas limit that was already set to higher. So even though these blocks were mining were mostly empty, we kept voting that gas limit up. We became more than half of the network and we're able to get that gas limit up. But it really got us involved with how are we developing our software? Is test net really the right place to be doing all of this work? Here is a repository where we have created Geph and Parity in various incarnations. So this is connecting to Robston, having a local instant seal, having a local mining thing that mines blocks about every second. And this is a public repository where anybody can come and create these Docker files and images to have a consistent environment to redeploy to. So you have this local Geph that you can spin up on your local system, that's great. And then you can put contracts into it. And if anything goes wrong, you just tear it down, bring a new one up and redeploy. But for us, this redeployment actually takes a long time. It takes about 20 to 30 minutes to do an auger deploy with all the seed data that's important for our development environments. And so we took it one step further. We wrapped after the contract deployment into a new Docker image. And this became the base with which all our entire development environments work. All our developers use these images now and we version them. So whenever we have a new set of contracts to test out, we say, hey, everybody dash 12 is up and all developers pull down dash 12. And it's exactly the same for everybody. The same transaction hashes, the same addresses, the same block timestamps. So we know that we're all building on something that's the same as if it was up on Robson on something public, but it's local and it has one second block times and it has a block gas limit of eight million. We get complete control of how this thing behaves. There are new tools to do some of the same things. This is Ganache. This wasn't released by the time that we were working on this, but I think we would look into something along these lines. This is an excellent local test RPC. But no matter what you're using, invest in your Ethereum development environment. Our velocity went way up once we got this right. So we had an issue with Xerox after deployment. So I mentioned there's a 5.8 million gas contract. This contains the market functionality. If every time a user created a market, they had to deploy 5.8 million gas to be really expensive. So we deploy a delegator. A delegator is a contract that has its own contract state, has its own address, but defers all of its functionality somewhere else. The only problem, when you use this contract, you incur about a 2000 gas cost whenever you bridge this delegation. That's just the base cost of a call like that. But imagine that this isn't the market. This is the Reptoken contract. And imagine the caller isn't a person who's willing to spend 2000 extra gas. Imagine this is Xerox and the Xerox contract. And also imagine that the Xerox contract created a limit, a hard-coded limit inside of their contract that said a balance of lookup should never take more than this much gas. And they're right. That's a lot of gas to do a simple storage lookup and return that data, but not when you add in delegation. And we blew that gas limit for that one lookup. So congratulations, you've deployed. Also, you're delisted. So we had some ideas about how to solve this. We could've wrapped the Reptoken in a non-delegated contract in the same way that you can wrap Ether into an ERC-20 contract. There's no reason you can't wrap an ERC-20 into another ERC-20 that isn't delegated and is compatible. But that is a horrible user experience for the user. Luckily, we talked to Xerox about this issue. They were able to execute on their governance and they upgraded their Xerox Exchange to a V2 that has a lot of new features, but lifts this gas limit so that the Reptoken is again compatible with the Xerox Exchange. So we have these interfaces. We have ERC-20. And these things are great and it's great to follow these interfaces, but it's no substitute for testing your integrations. There's all sorts of crazy things that these contracts do that are on top of the ERC-20 interface.  So here's how different the Reptoken is. If you look on Etherscan, the Reptoken is not verified. Not because it isn't verifiable, but because Etherscan does not support verification of a contract that was deployed by another contract. And that's what the Reptoken is. And even if it was verified, this is what it looks like.  So it doesn't really instill much confidence anyway. So let's talk about JSON-RPC. So one of the nice things about deploying your software is that you get a chance to get some real user feedback. Now luckily, we received more actionable feedback than this. We got some GitHub issues. We had failed to fetch parent block. Stuff is missing. Balances are gone. I can't find anything. Things are shutting down. Sinking is a major issue. So what went wrong here? I mean, so from that prior list of Docker image we built for Gath and Parity, we took testing very seriously and we tested all of these things. We tested Infira. We tested local and remote and POA. We tested the whole thing. But in the end, this is what happened. We had an issue that was specific to Infira and specific to Mainnet. So what happened here? Well, let's talk about EastGate logs. Our system works by, when we're synchronizing the logs with the blockchain, we say, please give me the log. Please give me new blocks. It's like, great, I just found a new block, block 100. Cool. Give me the logs for this block. Here's an array of logs. Now Infira does something different on Mainnet. In fact, they do something that I would consider to be just a little bit crazy. Infira serves over 100,000 transactions per second. That's the thing that they do is different. That's the thing that I think is a little bit wild here. And to accomplish this, to accomplish this feat, they need more than one server to do it. They probably need a huge quantity of servers to do it. In fact, if I had to guess how many servers they have, this isn't a public number, I would guess they probably have like, well, let's say two. And in order to have more than one server, you're gonna need a proxy. And so let's walk through this conversation again. Hey, please give me a new block. Here's block 101. Great, can I get logs for block 101? So this system doesn't necessarily have the same perspective with the Ethereum network as the one that delivered you block 101. And so the system, this is just a standard Ethereum node, thinks to itself, well, what the hell is block 101? But the problem is that it doesn't say that.  It says empty array. So I don't think this is in Fiora's fault. This is JSON RPC, kind of not necessarily lying, but not really telling the truth either. So what solutions do we have for this? Well, we could ask for all logs and just throw everything away and then filter out the auger logs ourselves, but know that the presence of logs means that we talk to a node that had that block. But we get empty blocks all the time. Okay, but maybe we could use like the logs bloom filter and if it's empty, we know, but it's still a lot of logs. So how about we get auger logs and crypto kitties? Because we know that crypto kitties has lots of events all the time. So we're gonna get like a totally adorable cat in our JSON response, but we're gonna discard it, but use that as our indication that we talked to a proper node. And we seriously consider this, probably more than I'm willing to admit. But the problem is, is I don't want to wake up one day and write this comment as a response to a data quality issue. So a solution that would actually work, well what if every time you got a blog that was empty, it was like an empty response, you remembered that and then the next time you heard about the next block, you made a range request for 101 to 102 and then if you saw 102, you knew that you actually did hit 101. It's just, it's getting crazy, right? So what we did was, there was this EIP out there that had been out there for over a year that said, how about we query blocks by block hash? This is a more unique identifier. And more importantly, how about when it's not there, you let the person know, you don't kind of fuzz over the fact that you've never heard of that block, you deliver that to the client so they can make the right choice on their side. And if anybody heard Jack Peterson's talk last year about bounties, I think this is one of the critical success stories of bounties. Here's an EIP that had been sitting there for 15 months. After Augur experienced these issues, that EIP, we'd be able to use bounties and engaging the network to get it merged in two weeks and implement it in geth and parity in six. That is an excellent response, right? And that's a killer ecosystem. And so, but in the meantime, we had to say, okay, you got one node, you're good to go. If you have two or more nodes, whether you're in fear of it, whether you're anybody, we can't do that until this EIP gets merged. But then we got other issues. We had parity. There was a parity specific issue when people started launching their new nodes and I didn't understand this because remember, like I come from COVID, like parity is my jam. I know this works with parity. So what happened here? How does parity getting these exact same issues? A single node. Let's talk about ETH get logs. When you synchronize with the Ethereum blockchain, it goes something like this.  The parity develops something called a warp sync where you first sync, so you pick a point in the middle, pretty close to head, and you sync that part first. Then you say, hey, I'm good to go. And later you backfill from zero to that warp sync start point. But you think you're synchronized when you're right here. And if you issue a get logs for this, you get your logs back. If you issue a get logs back here, you get empty array again. And if you issue a get logs that spans this point, you get half of the logs you're expecting. And you can imagine the kind of havoc this wrecks on a system that was deployed right here. So we made sure we documented this in our readme. We tried to detect the behavior automatically. But really the solution was, we've been working with parity, and this is actually scheduled for a future release to deliver the correct information, to volunteer more information. And we've seen some similar issues on the geth JsonRPC interface. It's all about eith call. When you call a function that reverts via that eith call, you get 0x. Eith call. When you call a function on a missing contract, either because it will never exist, or because you're talking to a node that is not at the block height that has that contract deployment, you get 0x. Let's talk about eith call. When you talk to a server that is overloaded for a contract function that will succeed, you get 0x. And so we're left wondering, should I retry in 100 milliseconds, or should I never talk to you again, because this is an unrecoverable error? So what's the lesson here? I think we don't quite know what Ethereum's JsonRPC is currently for. Is this a web API, or is this a database query layer? Because these have different success metrics. This needs to be 99.99% available, but it's okay to have a little bit of caching in there. It's okay to have a little bit of fuzzy answers. This is serving a web UI. But this needs to be 100% accurate. And you need to, if you can't be confident in the answer, deliver reliable errors to the user. So I'm proposing, and I believe it is currently a web API. And to support that, here's the documentation for the JsonRPC interface. This is over 100 pages of documentation. The word error appears zero times. And I think it is not a database query layer, because if you were to pretend that the issues we've experienced appeared in the MySQL bug tracker, these would be big issues. If querying for new data returns success with zero rows, or querying for old data returns success with zero rows, this would be a big deal. But I don't really fault the JsonRPC, or parity, or geth for these implementations, because what we've built really works. We've built big things, and they work great. I haven't had a problem using the Maker interface, but if I ever did, I don't know, I would just refresh, and everything's fine. If I was in that one in a thousand case, right, I got a bad get log. But when you're synchronizing the database off of Ethereum in this way, when you lose data, you start building with missing data. Or your synchronization has paused because you don't know when or if you should be retrying. You know, and Auger's not the only system that is trying to treat the Ethereum JsonRPC like a database. The Ethereum ETL project, where Google is running daily exports of the entire Ethereum blockchain and putting it inside a big query. But this is a daily export, and there's lots of value to be had there in research and reporting and charting. You can do some really cool stuff. But this is a different product than an ETL process that tracks head closely and can serve as the API for an application that is more complex. So I believe that ETL is gonna be the future to building advanced applications like this. And so why am I up here talking about some of these specific issues? Why don't you just go file some issues? Because I think that what we're talking about here is cultural. And I think if this, we need to decide if this is the kind of thing that we wanna care about and we want to request out of these APIs and work towards achieving. So I think we need to care more about reliable data access. And there's lots of projects underway that are leading this charge. In fact, you know, the folks that in fear we've been talking to are pushing ethql. This is a new way of talking to the Ethereum blockchain. And currently it is resting on the JSON RPC interface, but the intention is to put it inside of Ethereum. And so this is our chance to really get it right and enable ourselves to build these big applications. So what's next? What's coming up in Augur V2? So currently all markets in Augur are denominated in Ether. Ether is the only token that we accept for trading on these order books. But we have decided not to add dye as the token that will be used here. We have decided to replace dye as the only token that will be traded on the Augur platform. Because we've realized that not everyone who finds use in Ethereum necessarily wants to also be long Ether. Especially not for six months or a year as long as some of these markets can take to resolve. We're also working to improve the user experience here. Downloading an application and having it sync locally as some window running on your system, like that's not really an optimal experience. That's not really the way that the web is supposed to work. And so we're working to take the same architecture and to run it inside of the browser and use some modern browser features to accomplish a lot of the same things that we need, but do it in a way that is transparent to the user and doesn't require this extra UX burden. Because I think this is gonna be a powerful pattern for larger and larger groups to implement. Because if we wanna take this chart and change the scale of it again, I wanna make sure that whatever project does that has a great platform to build on. An audited ecosystem, development environments that can support large arrays of engineering teams, integrations with other projects that are well tested and not just relying on simple interfaces. And to build on a reliable data platform. Thank you.


Good morning Hank, it's Tuesday, so I saw the Star Wars movie again this weekend, and on my fourth viewing I noticed something really troubling. By the way, this video contains no significant spoilers, unless you go to the movies primarily to see people separated from their limbs, but even then, very mild spoilers. Lots of people have complained that The Force Awakens is just a rehash of the original Star Wars movie, right? Even the creator of Star Wars, George Lucas, said they wanted to do a retro movie. I don't like that. Now I think that view of the movie oversimplifies it, but here's the thing, if it's such a rehash, where are all the severed hands? I severed my own hand by accident over here, but I still got it. Don't worry. So Hank, as you pointed out in a video more than five years ago, the first six Star Wars movies contain no fewer than 13 severed hands. Like Anakin Skywalker, despite having two total hands, manages to get one of his hands cut off three separate times. There's even a phrase in the Star Wars universe for using your lightsaber to amputate an opponent's hand, Cho-Mai. So Hank, 3750 years ago, the Babylonian King Hammurabi developed a legal code that famously uses an eye for an eye approach to legal retaliation. And the Star Wars universe is kind of similar, except it's a hand for a hand. Like it's no coincidence that over the course of the movie's Anakin Skywalker also chops off three hands. In Star Wars, the number one way to risk amputation of your own hand is to amputate someone else's hand. Well, unless you're Obi-Wan Kenobi, who by my count removes nine separate limbs from people while somehow managing to remain a tetrapod, which, yes, is the word for animals with four limbs. So anyway, when George Lucas has been asked about this, he always just says, well, that's what happens when you play with swords. Which, you know, fair enough, but there's plenty of swordplay in The Force Awakens, and yet every single hand remains attached to its body for the entire movie! It's unconscionable! Meanwhile, Marvel movies have begun chopping off hands as an homage to Star Wars, and no, I am not making that up. In fact, Marvel Studios president Kevin Faige has said that it's because of Star Wars that all the so-called Phase 2 Marvel movies feature an arm amputation. Captain America Winter Soldier definitely guardians the galaxy check. Iron Man 3, yes, and then they burn it, and Ant-Man kinda, insofar as somebody loses an arm just before losing the rest of themselves. Nobody's harshing on those movies for their nostalgia. Hank has previously noted, I really liked The Force Awakens. It is at times nostalgia, but I don't think nostalgia is all bad. In fact, I'd argue the original Star Wars was also nostalgic. It harkened back to American westerns, and to some non-existent bygone era when good was really good and evil was really evil. Really, if it hadn't been for the inexcusable failure to chop off any hands, The Force Awakens would have been a great movie. So Hank, I think some of the blowback is just due to the movie's commercial success. Most people are never comfortable loving something unironically, especially when it becomes quite popular. But I think George Lucas' feelings about The Force Awakens are probably about something deeper, and not directly related to its lack of severed hands. Basically, I think it's really hard for creators, even very successful ones, to see their work go out into the world and be loved for reasons they didn't intend. Like, I've found that one of the pleasures and terrors of writing for an audience is that the book someone reads is never quite the book you wrote. Inevitably, people bring their own experiences and expectations and desires to a story, and in that process, they change it. To use an extreme example, when I was in high school, I read The Great Gatsby, and I thought it was crap. That wasn't The Great Gatsby's fault. On the other hand, Hank, when we were little kids, we watched the movie Grease 2 over and over and over again on VHS. And by forging that mediocre movie in the smithy of our souls, we were somehow able to make it great. Now of course, that isn't to minimize the responsibility of creators to their creations. It's just to say that once you've done your best and something is out in the world, you have to let it go. Basically, Hank, to me, making stuff is like a wookie. If you try to hold on to the stuff you make with too tight a grip, it'll rip your arms right off. Hank, I'll see you on Friday.


Wooden clipper ships that carried American goods around the horn were built in towns like Essex in the state of Massachusetts. Today, wooden vessels are still preferred by many fishermen, like me, to bring in the mackerel, carved in halibut from nearby fishing woofers, and from Newfoundland's grand banks. So when a sea captain wants a ship, build a wood, exactly to his specifications, he turns to a place where an old craft has been kept alive. He turns to Essex where men build ships of wood by hand, where more than 3,000 ships have been constructed since the earliest days of settlement in North America. In this village of 1,500 people, men still choose to practice the craft of their forefathers. Ever since the year 1668, the seafaring men have come to Essex to have fine vessels built. And here I was, back in Essex at last, for my own ship. Looked just the way I'd seen it years before. Of course, there were electric lights and automobiles now, but Essex felt just the same. At last, I was heading for Story's home and office. I'd written Jonathan's story, I was coming, you see, the Story family has been building ships in Essex for over 100 years. I wanted Jonathan, his men, to build me mine. Meeting him again after 30 years was a surprise. It was so natural, like visiting with a neighbor. Plans for my ship, a trawler, had been all drawn up, a scale model had been made. It was perfect like all the Essex ships, but naturally I made a few suggestions. My ship, I decided, would be a 70-foot dragger able to pull 50 ton of fish in a drag net. And I named her the St. Rosalie. Work began up in the mold loft of Roy Burnham's barn and down in the yards. Two Story drew the patents for the ship. Roy Hodges started squaring off the timber for the keel. While I settled in at Essex to watch my ship grow. It was time for school to start and fall, but folks in Essex figure their calendar not so much by the month or season as by the building of the vessels in the shipyards. But of course, boys everywhere have their own calendar for doing things. Now, Dale Cox had ideas of his own. Yes, but so did Truand Officer Mitchell. You see, the folks in Essex know they must raise good citizens with good schooling. They carry on the shipbuilder's craft, the town's pride. Yes, Dale must learn, as Everett and Lois had learned, it takes a long time for good things to happen. Yes, it takes a long time to draft the plans of a ship. Exactly the right angle has to be drawn for every rib, for every plank, for that matter, if the ship is to be seaworthy. Old Lou knows that. He's seen more than a score of ships launched at the story yards and picked the lumber for a ship. The grain and quality of every piece of wood is keenly estimated for its particular use before one piece is cut. Almost every rib that's cut out for a vessel has its own special angle, shape, and size. Time passes quickly when there's work to be done. By the time the men at the yard started scoffing up the keel and were rolling the pieces into one, the Essex River was teeming with alewives. Just like shipbuilding, alewives are a tradition in Essex. Every year, swarms of these herring-like fish come up the river to spawn. Since they rightfully belong to everyone in Essex, privileged to fish them must be purchased from the town. The board of selectmen, Woodson, Ed Garth, Pete Winthrop, elected managers of the town's affairs choose the highest bid from those submitted. Anyone in town can make a bid. It isn't only the highest bid to selectmen consider. Some years when the fish are few, no one is permitted to take the alewife from the river. But this year, Louis Miniatis, a newcomer to town, is the lucky winner. In Essex, you see there's a time for man to take on an extra job. Might surprise a city man to see how the folks in Essex seem to have the knack in doing so many different things. Louis dated the time he had his catch barreled and ready for market as the day the keel of the St. Rosalie was turned. It wasn't the man in the yard that didn't lean through that day in building the ship who was a great event. The keel is the backbone of a vessel. From here on, you're building up. You see a ship come to life as the ribs are hauled into place. No man in the shipyard has just one job to do. Everyone must turn to a variety of tasks. Even the skilled draftsman lends a hand when blocked and tackled up with the work. When you see the ribs lined up, you can figure how important it is to have accurate designs. Then you can see the products of weeks and weeks of work. The vessel takes shape. Now you can tell how she looked, how she arrived. That day, the St. Rosalie was at the talk of Essex. We were building the ship. I was worrying it together. Story and his men were working it. And there was a feeling of plage of the work. Bystanders adding that two cents to the men high up on the ribs. Maybe it's because every man in the shipyard takes part in so much that the men of Essex have never wandered down the road apiece to big industrial cities that must ride distance. In the factories, they're building too, I guess, but I suspect no factory worker can quite experience the sense of satisfaction an Essex ship builder feels. They're worlds apart. Of course, the work at the factories has to be done, and the folks in Essex and all over the United States, for that matter, enjoy the things made in these nearby plants. But the men in Essex choose to live differently. Somehow, at Essex, it seems folks don't figure modern speed is so important. They're satisfied serving a need to us fishermen who want wooden ships. And that means work by hand. If we're cold when the planks were steamed, you see, to make the planks fit tight and shape to the hull of a vessel, you have to put them into a steaming box to make them pliable.  Jonathan's story supervised every step of the operation, made me more comfortable to know he was always on the job. The planking is the one thing between you and the bottom of the sea. The St. Rosalie was planked with hard pine below her waterline, hard oak above, and the planks were wedged hard and fast. Every piece was set in place with wooden pegs. To me, every peg meant years of working and saving. So you can understand I got a thrill watching each blow of the hammer. Besides, this vessel was going to be my home, my crew's home for weeks, months at a stretch at sea. The St. Rosalie had to be built just right. And you know, the folks in Essex understand how a skipper feels. Now it was spring, or I should say, when the Rosalie planking was nearly done, it was time for me to make my second payment after the first formal inspection. Well, inspection, as if I didn't know just how the St. Rosalie was going along. Everyone in the yard was concerned because now the men who were working on the boat would get the first payment. I've found dozens of things wrong with the work, just for the benefit of partners, you understand. I even suggested the ship would sink. I'd paid some cash down when I ordered the vessel. When work was begun, that money went for lumber, tools, plans, and a lot of other such things. Now, when the ship was half done, I was to put up more money, not just for materials, but for the men's first pay. It was at the time of my inspection of the Rosalie that the folks of Essex were meeting afternoons at the town hall, rehearsing for the annual springtime musical review. As you might expect in Essex, Sailor's got the center of the stage. One, two, one, down. It's an old tradition in Essex. It still provides a good excuse for neighbors to work together and enjoy their community. Made my second payment, it was time for celebration. A shut-up party, it's called, fancy crimmings. The St. Rosalie was getting some of the looks by which a skipper at sea would come to Noah and respect that as the vessel made at Essex. It was time for the stories to feel proud too. The reason to be pleased was his boy, Jonathan. He'd done a trim job. Every one of the odd knew how pleased I was. How proud the St. Rosalie with a home port to bluster. On the day the Rosalie got her guilt letters, in Essex, almost everybody takes part one way or another in running the town. Yes, here, there have always been enough men and women proud of their village to work for it, without pay in most cases. So planning commission met because there was some concern about enlarging town's athletic field. You see, there were some other problems beside building of my ship in Essex. Only if you asked me what to do for children when they're not in school, I'd give you an easy answer. Send them to sea. At last, a great day arrived. The St. Rosalie was all dressed up and ready to be launched. I asked Dory how he made the grease to slide the ship into the water, but he wouldn't tell me. It's a great secret with ship builders. I daresay every man and woman and child in Essex turned out for the launching. Well, almost everyone. There are only about six or eight launchings a year in Essex, so naturally, such an event is regarded as considerably important. Now the pins were being knocked away. The St. Rosalie was ready to slide into the water. Only the christening remained. On married, myself, I asked the first mate if his wife would agree to christen the ship. She agreed. There was the first entry for my ship's log. Mariah, the wife of the first mate, christened the St. Rosalie at Essex. I came for the St. Rosalie, but it seems there was another reason too. I learned it wasn't only the St. Rosalie was launched that day in Essex, and I found out not everyone went to the Rosalie's launching. Some went to the church instead. But I was proud that in tradition of Essex, the young couple, Everett and Lois, would always remember that they were married on the day the St. Rosalie was launched. And as we idled down the river to the ocean, I began to feel the ship under me. I thought to myself, if ever I give up the sea, there's a town where I'd like to live. Of course, I don't intend to give up the sea, but if I ever should, I think Essex were. I'd make my home.