You poor people. Good morning. Thanks for coming by. I'd like to talk to you today about a project I did with a friend of mine, a paperback publishing imprint called Hard Case Crime. In 2003 I was working as a creative director and writing novels at night and I had a friend named Charles Arda, he would founded an ISP called Juno and he was writing mystery stories at night. So shortly after he sold Juno to a bigger firm, I took him out for his birthday to a Japanese restaurant near Grand Central and I said you're out of a job now and you have a few dollars in your pocket. What do you want to do next? And he said I want to bring back the old gold metal paperbacks. So in case you don't know about gold metal it started with this. This is James Watts patented steam engine circa 1769. It not only kicked off the Industrial Revolution but it paved the way for the first steam locomotives by the middle of the 19th century. Rail travel was common and for the first time large numbers of people were regularly traveling long distances and they all needed something to read. So soon each railway station had a newsstand like this where you could buy a newspaper or a magazine that looked something like this. But as chromolithography gave way to offset printing at the turn of the century magazine started looking like this. It was the heyday of the old pulp magazines. They had magazines catering to all different tastes. There were Western romance adventure, war stories and crime. I have a real soft spot for this one particularly the guy on the foreground who was blasting away with a 45 with one hand and examining forensic evidence under a microscope with the other. I would say he's the complete package. Anyway it was a big market and people were looking for other ways to exploit it and eventually somebody thought what if you took popular novels and reprinted them as small books using newspaper presses and cheap newsprint. Please and sold them like magazines at newsstands rotating them regularly so they were always new and exciting. Please note the series title if you can read it. Railway library. Paperbacks themselves are not new. Books have been published in paper covers since the late Renaissance but this was a new approach. Small sizes, low prices, low production quality, high quantity and unorthodox distribution channels. In Germany you had Replementauschnitz and later in the 1930s you had Albatross and then just a few years later in the UK Penguin Books. It was launched just a bit after the highly successful Albatross imprint and oddly enough it had exactly the same page size, the same color-coded covers, the same order form included inside and even a seabird mascot centered at the bottom of the cover. Alan Lane publishing visionary. Penguin inspired pocket books in the US in 1939 who are said to have pioneered the use of revolving wire racks. This was pocket books first title and like the whole line it was hugely successful and this led to the way for a flood of books from other publishers who wanted to get in on the new paperback book. You'll notice here many of the same visual strategies developed for the pulp split stripped down typography and sometimes a bit of formal experimentation. One early US pulp magazine publisher in the US was Fawcett whose flagship title was a humor magazine named for the publisher and a type of high explosive shell and they wanted a piece of all this. In 1945 they signed a deal with New American Library to distribute their mentor and signet titles and paperback in this contract prohibited Fawcett from publishing their own paperback reprints but Fawcett had an idea for getting around this. What if you published original paperbacks books that had never been done before in hard covers? To do this in 1949 they launched Gold Medal. Gold Medal is generally credited for inventing paperback originals or PBOs and that's not strictly true but it was the first publishing imprint devoted to them exclusively and they remained a leader in the field for a quarter century. The idea behind this new kind of paperback is that it would be sold like magazines. There would be no bookseller to recommend it. There would be no reviews to get the word out and no big name authors. You'll notice that the author names here are often very small Richard Prather would be an exception. He was a star back then but Daykeen and Leonard Prinn are fairly incy and sorry the covers had to sell themselves the way magazines did and they used the same tools intense color, bold design, violence and sex. Before I go any further I should probably say a few words about paperback originals and the representation of women. These books were made by men for other men and the covers are all about the male gaze. The women in these books are the best of them are often complex and impressive. The women on the covers are not. At one point I tried to do a taxonomy of PBO cover imagery but I had to give up because it wound up looking something like this. The women were there to be decorative and to make men want to take the books out of those revolving wire racks and carry them to the cashier and I'm not going to try to defend this. But I will say that Western civilization throughout history has been a fairly racist sexist affair and that art made in that kind of a society will necessarily be encoded with different kinds of bigotry including sexual victory. If you love the art you try to work around it and we love this stuff. These covers were produced in-house by anonymous art directors with massive workloads and limited collections of type and the speed with which they were designed and their single-minded focus on selling gave them a certain vigor. They were not lukewarm. They were not careful. They were not beige. Some of them were beautifully painted and designed and some of them were terrible in interesting ways. There was a lot of formal experimentation. This is a lovely Robert McGinnis painting and we are going to be hearing from him later on. And some of them were just nuts. And some were really, do we need a bit more of that? Okay, right. And some were very stylish. We also love another Robert McGinnis. We also love the books as objects. They were laminated with acetates sealed with cellulose varnish. They had a soft satin-y sheen and the newsprint newsprint pages in the old days with yellow very, very quickly and the books wouldn't look fresh on the racks. So to counteract this and to give them more shelf appeal, they were usually tipped in brilliant colors. And with age, these colors become very rich and subtle and they have a wonderful velvety feel and a sweet smell. And the fact is, even if you're a modernist and in my professional life as a designer, I am pretty much a modernist, sometimes you need a break from this. We really loved these books. So, Charles and I started to talk about it. And a plan for the new gold metal began to take shape. We would focus on hard boiled fiction. We would publish new writers, but also the best of the old. No Hammett and Chandler or other famous hard boiled writers. We're still in print, but Day Keen and David Dodge and Wade Miller and other writers who had done a few strong books and were now forgotten. Our books would look like the old ones. We'd get brilliant painters to do painterly old school covers. We'd tip the pages in bright yellow to match the redesigned gold metal mark. We'd make them rack size and skinny like the old ones, which would let us sell them really cheap. And this would get us a lot of attention and promote impulse buying. The books would have crossover appeal. There'd be something for mystery fans, something for hipsters and something for the genre buyers at Walmart. Now, all this raised certain questions of authenticity of homage versus pastiche of appropriation. Would we be making a Disney-ish theme park facsimile of the past? We wanted them to be a bit cheesy, but would they be the wrong kind of cheesy? We figured we'd sort all that out as we went along. The point was we had an idea we loved and we thought we had a pretty good idea of how we could make it work. Then we began to look into the matter a bit more closely. In 1982, gold metal was acquired by Ballantyne Books, a division of Random House, and Random House was not interested in licensing the name. Improvements of paper in paper in the late 20th century made it unnecessary to hide yellowing edges with colored dyes. And since some of the dyes involved were toxic, very few printers offer the service anymore. The new polypropylene films in UV-cured liquid laminations were tough and plasticky and they produced a very high gloss. We tried matte lamp but that killed the colors. We found that genre buyers at Walmart are not interested in old stuff. They want things to be new and shiny. And we found that in spite of all this, you couldn't do anything that you couldn't put on the shelves at Walmart's. No paperback distributor would touch a books that might offend Walmart shoppers, some of whom were very conservative, heartland Christians. This introduced us to the concept of implied nudity. Here you see a cover that was perfectly acceptable in 1966. But because of implied nudity, Walmart would not sell it today. You see, even though we can't see the modesty blazes, breasts, those guys can. And that means it's no go. Nobody wants to buy a skinny book. People don't feel they're getting enough to read. And no one wants to sell a cheap book because booksellers are not interested in 40% of a $4 book. It's not worth their time to shelve it or inventory it. And the audience from mass paperbacks is aging and younger readers do their reading on screen. Our first publisher, Dorchester, went bankrupt in 2012. Our new publisher, Titan in the United Kingdom, basically will not do our books rack size except in very special cases. Now, we obviously did not know all this in 2003 and four, but we were learning more every day. And most of what we learned was not encouraging. We decided to go ahead anyway. So it was time to go to work. Neither was for a member who came up with the name hard case cry. But I remember we liked the yellow ribbon because it was in the tradition of the metals, crests and seals you saw for the old PBO publishers who wanted to give their cheap, shoddy products a touch of class. And we wanted to be able to say this, we call this the model box. And it appears on all of our books. The mismatched typefaces are part of the point. We mismatched a lot of typefaces just like the old guys with limited typecases and limited time. The blobby lettering and graphics were not the old photo stats, which is how you carried logos around before dot a files. A good photo stat is a beautiful sharp thing. But once you make a photo stat of the photo stat and then make stats of those, it starts to get soft and blobby and that's the look we wanted. We bled the colophon off the top edge in the upper left corner, just like the old gold metals. And I do want to say one last word about the colophon. I was doing a hard case in part to take a break from modernism, but it's hard to stop being a modernist once you've started. And the logo is built on a grid. Now, in the 1930s, when the visual language of cheap American paperbacks was being established, one of the key things the publishers wanted was to look new and exciting. Now, what was new and exciting in the 1950s, 1930s? Well, this was pretty new and exciting. The Bauhaus opened up a lot of possibilities, asymmetry, angle type and a break from the tyranny of the central axis. And they used a lot of 19th century jobbing faces, grotesques and some slab-serve types. And so did we. The other thing that was still pretty new and exciting was the talkies. The main source for PBO tropes and strategies was the pulse, but the new breed of movie poster was also an influence. You saw a greater influence, willingness to mix it up with asymmetrical and angle type and lettering and a bolder use of montage, which was another favorite mode for Bauhaus graphics. The types we chose included these new modern slab-serves that that she called like so much. And one did own, which was especially popular because it was bold and narrow enough to make a strong showing on a small cover. When it came to Bauhaus Influences, this was, of course, the big daddy, especially the bold condensed version, which was, it came very close to being the default PBO type face. There were also the modern interpretations of the old jobbing grotesques. This was a very early superfamily, when ATF was put together out of I think 23 other type foundries, they had an immense catalog of numbered grotesques, completely miscellaneous and random, and Morrisville event was tasked with organizing them into some kind of a usable and coordinated large family. Franklin Gothic was an early, was an early effort in this direction, trained Gothic was a line of type was under done along similar lines. Alternate Gothic was another such effort. And we did use one late and drink. Even though this was designed in 1994, it had the same purpose and the same spirit as the old as the old grotesque superfamilies. And it also came in a really useful range of weights and weights. Now, the interiors of the book were brutally simple. Like the originals in the old days, something as interesting as this would be an exception. This is an armed services edition soldiers have carried paperbacks and the rucksacks and the civil war and every war since and in the Second World War, the US government distributed free arms forces editions, armed services edition for soldiers. They were printed landscape in two columns, they could fit into a uniform breast pocket. And they were meant to be thrown away after one reading, but mostly the interiors were crude functional affairs, single column center axis meant to shoehorn copy into the smallest skinny as the books possible. We didn't think it would be nice if we picked a handsome face for headlines and chapter numbers. And I think you'll agree this is a pretty good one. We also needed body copy. So the obvious choice would have been Times Roman, which then as now is was omnipresent after its introduction into the US market in 1942. The problem was that we had the opposite problem. The old guys we had to pad out old 50,000 word quickies into a market of a size. So while space saving times would have been pretty, it would have been an obvious choice. We needed something more expansive. And we settled on Caledonia tracked out a bit to make it look more like metal. It added 12% to the lengths of our books. And a lot of the time we needed that. So here's how the title page is turned out. And here's a text page old paperback originals did not usually use N slugs, but they sometimes did. And we had fun designing this one and using it. And here's some of the back ads. I'm kind of proud of the fact that we made the URL look just a little bit period. The spines were easy. The old spines were often most often plain white with red and black type. And that's what we did. This also had the advantage of standing out on the shelf since these days, most spines are colored. As far as the back covers are concerned, the main thing in the old days was speed. You just couldn't spend much time on them. And often you didn't have much to say because the PBOs weren't blurred. Our approach was to set up a simple one approach was to set up a simple standard text only template with a simple border and use it for everything. The other approach was to just knock something out at random for each title and not worry too much about it. And we kind of like the sound of that. That's the way we decided to go. One thing we did consider was color. Back in the old days, if you wanted to use composed colors, every CMYK tint that you added was a separate a separate zinc block from the engraver and that cost money. You didn't want to spend money. So you focused on simple colors that need one or two blocks and still have a lot of impact. And these are the colors you saw everywhere. So we didn't limit ourselves to this, but we focused on these. And here's well, one of our covers that covers turned out. It's got a beautiful painting there by Michael Kelch. And here's another. This was actually Mickey Spillane's last novel unfinished at his death. We convinced the estate to let us finish it and publish it. We had one of our best authors do it. And that is a beautiful Robert McGinnis painting. Once again, more about him later on the front. Even though most back covers were kind of nothing in the old PB does, PBO days, there was one notable exception. The old Dell map backs who did beautiful diagrams and illustrations of the settings of the novels on the back. And we were very happy when we got a chance to do one of our own. And that illustration is done by a wonderful map maker and illustrator we found called Susan Hunt Yule. Okay, very often when people try to emulate hard boiled style, you get something like this. And that's how you know the designer has no idea what a PBO looks like. The Indiana Jones logo was based on old pulp magazine, name plates like these. And by the way, if you think women were poorly treated in the old pulp magazines, you should see the treatment people of color got. This kind of work with outlines, drop shadows and gradients generally was way too labor intensive for a single PBO title. You could only afford to do this if you were going to amortize your investment over the course of a few years. So we did not use this sort of thing for cover treatments. kinds of cover treatments we looked at with ease. This is the most common kind of title treatment. And it was also the most basic was a bold, condensed songs. The cover on the right was a common format for series books like the ones for Shell Scott and Mike Shane. And we will be getting back to it later. So this approach was simple, safe and generally effective. And it usually meant someone was in a hurry. Another approach, the second commonest approach was a slap surf. And I would say this had a similar rationale. Also, a lot of art directors would attempt script because scripts are sort of dynamic. But this is difficult because, you know, they weren't sending these jobs out to Tommy Thompson. And most in house art directors, then is now couldn't letter very well. So you'll notice the one on the right, which looks a bit nicer is actually tight. This one, the one on the right. It looks a little nicer, but it could almost be announcing a spring sale at Bon Whitteler. So this at any rate was another characteristic style. And the thing that's always struck me about it is that it seems filled with a bizarre kind of enthusiasm. An extremely popular style was this sort of distorted off kilter approach, which I think was pretty attracted to most art directors because it didn't seem to take much skill. But it's still gave you a kind of hip, jazzy look. And the message here was that the book on offer was pretty wild, maybe with a whiff of marijuana. Then there was the horror movie look and stencil, which was trendy back then too. But in those days, stencil had a much more earnest feel. And it signal that the book was going to deal with a very important societal problem. Calligraphic titles or rather calligraphy inflected titles were reserved for books with some pretension to quality. It helped if they had historical settings or were in some way exotic to American readers. And lastly, a bouncy did own suggested that this book was hot stuff. When it came time to commission the cover illustrations, we discovered one more fact of life. There's a smaller supply of skilled realistic painters these days. The rise of photography and Photoshop have made it harder for illustrators to survive. And art schools these days are not so focused on life drawing and painting. So our first thought thought was, let's begin by talking to some of the great old guys. In most cases, this was not practical of the guys that were still alive. When we launched our case, James the body had lost his sight and stopped painting. And his good friend Stanley melts off has also basically retired. We talked to Robert McGuire, who was wonderful at a convention. And he very sadly said, you wouldn't want me these days. I'm just not that good anymore. And we couldn't get through the Mitchell hooks. That left one guy. He hadn't painted a paperback cover in a couple of decades. But he was possibly the finest painter of them all. He had been called the king of paperbacks. And he was the last man standing. Most people know him for painting this gal. And this guy. We were very happy when he agreed to be part of our case. I found a painter called Greg Manchester. He's an illustrator with a closet full of gold and silver society illustrator's medals and we could not come close to affording his rate. But he liked what we were doing. And he liked the fact that we pretty much got gave him carte blanche. And Glenn Orbeck was basically a comic book cover artist, but he seemed to have the right spirit. And in the end, he became more than anyone else the visual mainstay of the line. He did over two dozen covers for us and many of them are among the strongest ones we've done. And I wish we could say we were still working with him. But earlier this year, he succumbed to cancer at the age of 52. He was a great artist, a complete pro and a lovely man to work with. And we will miss him. We could not have done the line as it is. One spark of brightness in all this is that his wife Laurel Blackman is also a very fine painter. And at the very end, when he was too sick to fulfill his last commission, she stepped in. And she did it for him. I can't show it to you now, because it hasn't been released yet. It's gorgeous. And we she's working on the second hard case commission now. And we hope for a very long partnership with her. So here's a few of the books themselves. I want to talk briefly about fade to blonde, not just because I wrote it, but because there's a little bit of a story behind it. I am or was a novelist, but I didn't get into this to write a novel. I got into this as someone interested in the design. But when I was working out the cover issues, I needed I needed a title and a tagline and, you know, a name to use just for sample covers.  She'll have it. I'm not going to do the Molotys Falcon. So I just made this up fade to blonde.  And I made up a Pope writer. Thank you. Thank you. It's still in print. It's still in print. And I made up a Pope writer named Forrest DeVoe Jr. Anyhow, I was working on this one night. I thought, this kind of interesting. I wonder how it would be if somebody actually wrote a book with this title and this tagline. So I opened up word. And I began making a few notes at eight o'clock the next morning, I had a first chapter, and it was time to go to my day job. And so that's how I got started writing fade to blonde, which one of us got first chain this award. It was a lot of fun. I did it at a 2000 day word, a day clip like the old guys, not letting my stop self stop and revise. That was really nice. Okay, here's a book by Samuel Fuller with psycho jazzy lettering. Here's some murderific lettering for novel called false negative. Here, if you remember the old Mike Shane series look, we always wanted to do one of those. And we finally got the opportunity to do that for our own Max Allen Collins, who has published God knows how many books with us at this point. And we reprinted some of the early ones in a series like this with an old Robert McGinnis illustration. Ellison, there were a number of the old science fiction guys who used to write whatever people would pay them to write. And this is an early street punk, you know, novel from the 50s with a beautiful plan or the cover. This is rejected lettering for our second Stephen King original novel. The sales force said, what the hell is that? And I said, this novel has a carnival theme. And this is old show card style lettering. And I said, Stephen King never has brush lettering. I like that. And I said, Well, maybe now he can he said, how will people know it's the same Stephen King? Go argue with a publishing sales person. Okay, some modified horror risk for Madison Smart Bell, some classy calligraphy for Gore Vidal, who also when he was young and struggling, did a pot boiler, a crime novel with an exotic locale and another beautiful Glen Orbit painting. And this was actually James M. Keynes last novel also unpublished. And we worked for years to let the estate to we work for the resume on the estate before they let us clean it up a bit and publish it. Because it was very far gone at that point. And it just wasn't very good. We thought there was good stuff in it, though. And it was James M. Keynes. So we cleaned it up a bit and shoot it out the door. In order to do this one, the editor sent me pics of mid century cocktail bar signage. And that was that was the reference I used. We never got to revive gold metal. But we did get to revive some of their books. Branded Woman, another great Glen Orbit painting, one of our favorites. Stop this man, Robert McGinnis. Another McGinnis for the girl with a long green heart. She had a grifter's soul. A touch of death, Charles Williams, lovely puff, chalk, chalk, chalk pile cover. And this is interesting kill now pay later. Robert Terrell is Robert Kyle's real name. And these are two Robert McGinnis paintings for the same title 50 years apart. Michael Crichton, when he was in he paid his way through med school by writing one thriller a year under an assumed name. And we got his estate to let us have those. This one's called binary. Another Glen Orbit. Clunder the Sun with a beautiful bomb. And say it with bullets by the guy who this is serious stuff. We get denied by the guy who created the pink panther. This is a comic novel and we thought it would be okay if we made the cover a little extra camp and baby more, which aside from the title, what do you need? You also have another Bob McGinnis here. And I should point out that a few of these, including this one, were designed by Steve Cooley. I took a hiatus from the line in the for a few years because I was kind of maxed out on time. And Steve Cooley of Cooley Design Lab stepped in and did some beautiful covers like this in style that the house style that we had set up. So we have today published over 100 books. And we have published some authors I never thought we'd get anywhere near along with Stephen King and Gore Vidal. We have Watergate conspirator conspirator E Howard Hunt, who believe it or not, actually started out as a very good hard boiled crime novelist, Robert Block, Lawrence Block, James M. Cain, Pete Hamill, Michael Crichton, Mickey's Blame, Cornell Wolrich. And like I said, there were a few science fiction guys who started out doing a bit of crime to pay the bills, Roger Zalazny, Robert Silverbill, and AC Doyle, whom you may know best is Arthur Conan Doyle. Our original novels of one awards, including two shameless awards and an Edgar and people have said some very kind things about them. This came from an actual paper letter that Mickey's Blame wrote us shortly before his death. Those questions of authenticity and appropriation has still not been resolved. All I can say for sure is that we really, really wanted to do these books. And we did them. And people seem to like them. Some people really, really like them.


I suppose we suspected that there was a problem but then over the last year or two reports at a state and national level have really brought that problem into focus. In New South Wales there are around 680,000 small businesses employing about half of the workforce and each of those employers, particularly in the high-risk industries, can be complying with up to 75 different regulations each and they're spending about five hours every week purely on compliance. For small businesses who have less time, less resources, complying with that stuff takes just that much more. One of their reports suggested if you could reduce the amount of time that a small business needs to spend on compliance by half an hour in each week you'd be saving the New South Wales economy $800 million every year. One of the worrying things for us was that work health and safety was touted as probably the worst offender in terms of the number of regulations and what you need to do to comply with that. I mean with very good reason but nevertheless it was exacerbating the problem so we knew there was room for improvement. Earlier this year we introduced a new website, better design, better content, better accessibility and with continuous improvement we know we're making good strides in that space. But beyond the static web content we wanted to look at how we could equip people with information to enable them to do the right thing. Now we knew from some previous segmentation that you can really broadly put people into one of two groups. There's the group who are not massively engaged with health and safety at the moment. Sure if you make it easy for them they're going to do the right things but they're pretty conscious of their competing priorities. Then you've got the group that are really switched on already, really engaged with health and safety. They know what they need to do and they want to stay abreast of the latest news. So for that group we asked a thousand employers in New South Wales what information they'd like to receive on health and safety and also on workers compensation. We gave them 30 different article headings, tested them across five sets and covered everything from news and research, practical tips to case studies, thought leadership, technical information and the results themselves were extremely clear. What people wanted was simple content, engagingly written and really task oriented. So some of the headings that popped for them were workplace safety and the law, 10 things every small business should know, the two-minute drill, identify health and safety risks in your business and five signs your workers are stressed out and what to do about it. So really practical, really task-based and you can tell just from the heading if you were to spend two minutes reading this article you're going to get something useful from it. We also tested their visual and design preferences and from those things we developed an e-news letter called the safe work wrap. Early signs are all positive, open rates over 40% all the indicators suggest that it works and we're doing some interesting experimentation in that space using some A-B testing. One other really interesting finding from that research was that the majority of respondents said if we could provide them with some clear information on health and safety requirements they would save 30 minutes or more every week. Here's a quote from one of them, as a small business owner I want to get straight to the point and not read 100 pages when someone else can condense it into a framework for me. Now just going back to what I said earlier if you could save small business on average 30 minutes each week on compliance you'd be saving the New South Wales economy 800 million dollars, 800 million dollars every year. So we figured we should do something about this. We set out to develop some concepts some content models for how to deliver the legislation in a really user-friendly way. We tested these we did some quoll research this time. We interviewed 34 small to medium-sized business owners through a mixture of face-to-face and phone interviews and we tested a variety of formats from excerpts from codes of practice to existing guidance material to these concept models that we'd also developed and we've refined that model with them because the results from this group again were extremely clear. What they wanted was number one tell me why this is an issue what's the personal relevance for me number two what does the law actually say and here they wanted some really clear line of sight to the legislation itself and thirdly how can I comply what do you want me to do make it really practical and they said if we gave this to them then they'd feel that compliance would become possible and this has been a worry for us if the requirements aren't all that clear you've got one of two options right you either spend a hell of a lot of resources finding out what you should be doing and small businesses don't have that luxury or you can be knowingly non-compliant and that is not a great result it's not a great result for us as the regulator and it's certainly not a great result for small businesses who be putting themselves and others at risk of getting hurt they also said that this would give them reassurance that they were doing the right thing they would know they were complying and they would know that what they were investing is was providing value and they won't be wasting their time on stuff that wasn't and for those reasons they said they were more likely to take action and getting them to take action to make their workplaces safer is really what this is all about so this spawned a big body of work for us we've broken down the regulation into about 25 to 30 topics and we're developing content to deliver against a model that they've asked for we're also going to be introducing a layer of industry relevance because we know from some other research that small businesses in particular will relate much more to information if it feels like it's for them or at least tailored for their industry and we're looking to deliver all of this in a really interactive way and we're looking at some exciting opportunities for doing that later this year now taking legislative requirements that everybody needs to comply with and delivering them in the way that everyone can understand just seems logical right it seems simple and so the working title that we've got for this body of work is simple safety it just seems to make sense we're aiming to save people time we're aiming to save the money and through this process we're aiming to save people's lives so for those who aren't massively up with their requirements for health and safety at the moment for those in the dark we should be bringing the light sometime later this year for the other group who are really switched on already who are super engaged and i'm assuming that if you've watched this video through to this point you're probably in that camp go to safeworkwrap.com and see for yourself what small business ones


OK, good morning, everyone. It's almost about time, so let's get started. So in this session, we are going to be talking about rolling upgrades. So we will be briefly go through what is a rolling upgrade and the current upstream development status in each project. And then we will show our test results of our OpenStack deployment, both in VMs and containers. So my name is Lu Jingying. I'm currently working as a software engineer for Fujitsu. And this is my colleague, Hugh. Yeah, I'm software engineer from Fujitsu, Vietnam. So if you have any questions or suggestions regarding our presentation, you are very much welcome to reach us through either the email or ping us on our IRC channels. So let's get started. Before we go through our test results, we will briefly explain what is a rolling upgrade. So in the technical community's definition of OpenStack Foundation, we are trying to do a rolling upgrade. For example, we have a project, and we have some services inside this project. And if we can upgrade one of the service inside this project, bring it to another version. And then if we can do this to all of the services inside this project one by one, we call this a rolling upgrade. And in the TCS definition, we do not strictly require that we do this kind of shutdown and restart one by one. So we do allow some of the services are being shut down together and bring up at the same time. So in this case, if we can have a HAC setup of this project, so while we are trying to upgrade this project, we can be doing this from one copy to another one. So from the user's perspective, we can try to be minimize the service downtime while we are trying to upgrade this project. So why do we want rolling upgrades? If we do not have rolling upgrades, we have two other options. The first one is we just shut down all the service and bring them up together, and we call this a code upgrade. So in code upgrades case, we will be experiencing a large amount of service downtime because while we are shutting down and restart all of this stuff, this project has no ability to respond to any API requests. And also we can build another identical OpenStack cloud, which are running in the new release. So we're just migrating all the OpenStack resources from the old release of this OpenStack to a new release of OpenStack. And we call it blue-green and secondary upgrade model. But in this case, we will be suffering from really high hardware cost because we are building an identical OpenStack cloud only for upgrading purposes. And we also want to emphasize that what rolling upgrades are not. So this is from the OpenStack TC's definition. As I briefly mentioned earlier, in rolling upgrades, we do allow some services and bring up together at the same time. So this means we cannot guarantee zero downtime upgrades while we are doing rolling upgrades. And even in the definition of zero downtime upgrades, we cannot guarantee that the users are not affected by this upgrade happening in the backend because our API requests may be experiencing some delay of response. And if we want to ensure that the users are not affected by this upgrade, not at all, we call it a zero impact upgrade. So bear in mind that in the OpenStack world, we have three tiers of upgrades, rolling upgrades, zero downtime upgrade, and also zero impact upgrade. So why don't we just have rolling upgrades by nature?  The reason is why we are doing the rolling upgrade will end up in a case where we have both old codes and new codes running at the same time. And if this happens while in the OpenStack's world as a distributed system, this communication between services inside a project is being done in an RPC manner. So while we have new codes running up, how can we guarantee that it can have also working RPC communication with the old codes? And also normally we upgrade the database first. Then how can we guarantee that the old codes can still be accessing our new database schema without any problems? So in order to address these two kinds of incompatibilities, different projects came up with different functionalities that they want to use to solve these incompatibilities. So for the RPC layer, we may have some RPC versioning so that we can control even new version of RPC codes, can communicate with old version of RPC codes. And also if we have any payloads that are transferring between this RPC layer, we can use all slow version objects to address these payloads. And also for the database side, we definitely need that mechanism that both old codes and new codes can be accessing the database. So for some projects, they are using database triggers to guarantee this access. And also while we are doing the database schema or data migration, we do not want this migration to be affecting our running codes. So we want to have an online kind of manner, an online manner of schema and data migration. So this is the current upstream project status with respect to the rolling upgrades implementation. As we can see, most of the core projects in OpenStack has already support rolling upgrades feature. And the only left one is neutral and the community is still actively trying to implement and hopefully we can get it in the P-cycle. And for other projects like Ironic and Heat, which are also broadly deployed in the public clouds, the community are also working hard to have it. And for some other projects such as Sparky and Designate, the community has not have any plans to support it, but we are trying to push it in the upstream. And starts from here, we will be explaining how we're trying to do the test of rolling upgrades in our two deployments. So in our VM deployed OpenStack, we have a very typical three controller nodes running in active active mode. And then we have three compute nodes, two network nodes and three storage nodes. And the active active running controllers are running behind a load balancer and a virtual IP. And in our container deployed OpenStack, the architecture of OpenStack itself are basically the same, but we are using an open stack official project called COLA to do the container deployment. So we have two other nodes specifically for COLA's purpose that are provisioning node and Docker private registry node. And with respect to our tests, we are only been testing these five projects, which are Keystone, Glance, Nova, Neutron and Cinder. And the upgrade order, we followed the recommendation from the OpenStack official upgrade guide. And we are trying to do the upgrade from Newton release to stable Orcata release. And then how do we try to imitate that the actual things happen in the production cloud? So we continuously send some asynchronous CRUD API requests and for some of the create requests, which takes a long time. For example, booting a VM, we create a amount of VMs beforehand only for the deletion's purposes. And also, while we are trying to upgrade one of the controllers running behind a load balancer, and there may be a time that this controller cannot respond to any API requests, which is expected during our upgrades case. And in order to avoid that the load balancer continuously sends this API request to this controller, we try to drain this upgrading node from the load balancer while it is being upgraded.  And our estimation of the service downtime. So with respect to the performance, at this moment we are mainly focusing on the API service downtime for now. So we may have multiple slots of API downtime. So we record all of the response we received from the API request we sent to the cluster. We just simply add up all the multiple slots of downtime we are encountering during all these API requests. And the general steps to upgrade each project are basically the same. The very first step is we need to expand the database schema. And then for some projects, if they use database triggered kind of database access, we need to set up the database triggers before we have any running new release of codes. And then in the control plane, we just do this upgrade one by one in a rolling manner. And for some projects, if we need to contract the database schema, it has to be wait until we have finished the upgrade of the whole controllers first. And then if this project has any data plane services, we do this rolling upgrade after we contracted the database schema. And all these steps can be automatically through some Ansible tools or batch scripts. And while we are trying to do this rolling upgrades in VMs, it's pretty straightforward. We install the five projects, Newton release distro. And in beforehand, we prepare the Orcata release local repo because we do not want to be downloading all these new codes while we are doing the upgrading. And then we just deploy an Orcata release step. Oh, and a special trick here is for the projects that we need to do the database migration, we deployed another Orcata release step stack only for the purpose of trying to do the database migration for this OpenStack VM cluster. And then you can just go online for each project. They have a very well documented documents showing you how we can do this rolling upgrades in each project. And now I will hand over to my colleague. He will explain us about our container tests. And for rolling upgrade doing in the container environment, so we come up with a COLA project is at the official Big Ten project. And your package on the OpenStack service and its dependency into the container base and then your Ansible Kubernetes to deploy the full stack OpenStack cluster. And while we're testing the rolling upgrade in the container environment, we miss some trouble with the COLA Ansible doing rolling upgrade. We come with the first option is we use the native COLA Ansible upgrade in common. And because the Ansible, we are using Ansible Engine, Ansible Zero Line Engine, and Ansible Engine remove all container with the same name and then bring up the new container with the same name. So there could be a little bit downtime and there are not the same way as we do native rolling upgrade. For example, in Keystone, you need to fence down the first node and then doing upgrade in the DB layer or RPC layer. And then after that, we restart on the service in the first node and do the same step in the next node. So with the COLA Ansible upgrade command, we cannot do this, so we eliminate this first option. The second option here, we try to deploy, for example, we try to deploy the new turn release and then go into each container you made and do some command-line APT update or APT upgrade. But all the COLA container images are restricted with the root permission, so we cannot do some kind of this way. And the last option here, we try to deploy the new turn release and then put the octa-release into the ignose and stop manually the new turn release and start manually the octa-release. So this comes with two problems. The first thing is if we manually start the container, so we may lost some environment sexting or some configuration from the COLA automation tool. And the second thing here is that from new turn to octa, so COLA deprecated the load container from the hacker to FluentD. So this option also cannot achieve the rolling upgrade in container. So we finally, we found some workaround to try to do the rolling upgrade in the container environment. So the first step here is firstly, we deploy the octa-release with COLA and then we stop and rename the five project octa-release container. For example, we remove, rename the, from the novel API container to the novel API octa-container. And then we deploy the new turn release, open stack with COLA. Yeah, the first step here, this is the second step we rename and then the last step, we deploy the new turn release, open stack with COLA. This and that, we have the bulk container versions of open stack service running on the same node and then we can start manually and do not lose the config or environment sexting. So the general step for rolling upgrade in each project is for each service in each node, we stop the new turn container and then we start the octa-container immediately and we do some special trick to do the database migration is that we deploy another DevStack environment and reconfig the DevStack DB to the COLA cluster for ensuring the online data migration or trigger can come. There are some nodes here for Docker run and Docker start.  It can create the union read-review, malvolume set configuration and start the container. While Docker start, only start the container. So the first three steps of the Docker run is very fast compared with the container start time so you can eliminate or evaluate the three steps while the Docker start only depend on the specific service running in container. So we start the open stack service in Docker and then we need to wait for the service to come up. So here, my colleagues, we can come some results from the already upgrade into both environment. Okay, so this is our test results with respect to the API service downtime in each project. So as you can see for projects such as Keystone, Glonx, Cinder and Neutron, we can almost reach zero downtime upgrades. But in Keystone's case, while we did, like, dozens of times of tests, but we still occasionally encounter some DB.log quest DB.log failures. So we cannot guarantee like zero downtime upgrades. We still have one or two of the API failures. And with respect to Cinder's case, because currently Cinder volume still has to be running in active passive HA mode. So while we are trying to bring up the passive node and shut down the active node, there might be a slight window that the API, the Cinder volume cannot answer to the API requests. So we met some failures in the container's case while we're trying to upgrade the Cinder. So some explanation of why we encountered so much downtime in Nova is that for now, the Nova's rolling upgrades will still require a full shutdown and restart of controller playing in Nova. So now, for example, we are running an old release, maybe any release of Nova, and while trying to upgrade it, we need to bring down all the controller playing service, upgrade the database, reconfigure the configuration files, and then bring up the new releases. But the good news is while we are doing this in the controller playing, the running Nova instances are not affected by this procedure. So at the moment where we are trying to upgrade the controller playing, we are only restricted from launching new instances or resizing or deleting the existing Nova instances. And the rolling upgrades in Nova only happens while we are trying to upgrade the Nova compute. So for these running compute nodes, if you are trying to upgrade them to the new release, you can just live migrate the running instances on one of the compute nodes to another one, and you do the upgrade of this Nova compute. So the rolling upgrade here is the, remember the RPC incompatibilities we mentioned earlier? So this RPC incompatibilities happens between the new version of controller playing and the old version of the compute nodes. And also because we are doing the live migration between Nova compute nodes, this RPC incompatibility also happens between the old Nova compute nodes and new Nova compute nodes. And for Neutron servers case, since Okata release, we can have multiple version of Neutron server running at the same time. But actually the development of Oslo version object hasn't been finished yet. The community is still trying very hard to get it done in P-cycle. And the reason we can support this from the old cycle is that we banned all the contract scripts so that previously all the Neutron server downtime happens while we are trying to do the contract database migration. And now since all these kind of contract scripts are banned from the upstream, we will be no longer encountering any contract cost Neutron server downtime. And also as I briefly mentioned earlier, sender volume now only supports active passive HA mode, although the community is working hard to get us to the really active active HA mode. But we are not there yet. So we have some options we can do to try to minimize the downtime while we are bringing up the passive node and shutting down the active node. The first one is we can configure both active and passive sender volume to be listening to the same queue. So that while we shut down the active one, the passive one once it is really, it can read from the same queue seamlessly. Or we can increase the timeouts between sender API and sender volume. So while we're doing the upgrade of sender volume in the backend, the sender API will hold all the API requests until it hits the API timeouts. And if we can finish the sender upgrade before the API hits the timeouts, we will not see any API failures either. And so the downtime difference between the VM deployed containers, VM deployed open stack and container deployed open stack is that in the VM's case, we need to upgrade and unpack all the packages and then restart the services while we are upgrading one project. While in containers case, the only time we need to wait is just the restart time of each services. And normally people believe that the restart time of containers has already been much quicker than the VM's restart time, so it's reasonable that we are gaining some less service downtime if we deployed our open stack in container environment. And also for these four projects, even though we did not see any failed, we didn't see that much failed of API requests, but it is reasonable to assume that if you are in an intense environment, because we are bringing down one of the, one copies of these running services, so it is, now we only have two thirds of the services answering to the API requests, so if you are in an intense environment, it is reasonable to expect that your API requests will be experiencing some delay of response. So some conclusions here that for these four projects, now we can upgrade them with almost the lower downtime and which is a good news. And for the Novus case, if you are deploying them in the containers, you can expect less service downtime compared to VM's. So some remaining tasks that we want to achieve in rolling upgrades we definitely want to see rolling upgrades feature in all the projects that we want to use. So now we are actively pushing the community to implement them in Barbican and designate. And in the future, maybe we will turn to Trove and Melina or Sahala. And also because we want to use COLA to deploy our container environments, so we would like to see COLA can support the native rolling upgrades which have already been provided by these projects. So we have some ongoing blueprints we are taking care of and once they are fully merged, we can just use the COLA and Sable upgrade to do the rolling upgrades in containers. And some future work is rolling upgrades are cool, but for sure we would like to see totally zero downtime upgrades. And in order to do this, we have two options. The first one is just like the rolling upgrades case, we just implement this feature in each project. And this can just directly give us rolling upgrades, no matter what your deployment is. And the second is while we are trying to do the zero downtime in containers, we have another session explained our current approach which can give us a zero downtime open stack cluster upgrade and the room is just next to this one and right after this session. So if you are interested in, you are very welcome to join us. And also zero downtime give us some delayed response of API requests. So we would also like to see that how can we just have no impact to our users while we are doing the upgrade in the backend. So this would be our second future work. And also for all these three different kinds of upgrades we are discussing here, they are all talking about adjacent release upgrades, which means if you want to have a P-release of open stack, you have to be at least in O-cycle first. You cannot jump from N-cycle to P-cycle directly, but this can be painful for some public cloud providers because they have to do all this bunch of upgrades case every six months. So if possible, we would also like to see the skip release upgrades, but this can be painful because due to the current deprecation rules in the open stack community, most features are deprecated only within two cycles. So if we want to have skip release upgrade, we need to address that issue first. And we made our upgrade scripts and our testbed information public. So if you are interested in follow our procedure of testing rolling upgrades in these two deployments, you are welcome to pull this GitHub repo. And we also would like to thank the communities for helping us while we are trying to do these upgrades. And if you have any questions, I think they will be very happy to answer your questions on IRC. And also a special thanks to our colleagues who help us a lot while we are trying to implement these tests. Okay, so now we are ready to take some questions. Hopefully we can answer your questions. If anyone has a question, okay, like that gentleman over there. Thank you for sharing with us a very useful direction. We need it in six years. Everyone needed to discuss about seeing this upgrade, but in my mind, practically, first thing we would like to make sure for upgrading is that even with non-zero downtime, let's say, okay, I'm not even expecting zero downtime. Even one day, even a week after really upgrading and it's really working as before. That's the kind of first goal, but even that goal has not been easy. Because mainly, even next release, they change the configuration parameters. So how did you deal with that when there's a significant number of configuration parameters that they've created or newly introduced? And not only upgrading source code itself, you have to find matching configuration file per project, right? Yes, yes. So in our tests, before we upgrade it to the new codes, we prepare the next release configuration file in place. And then after the upgrade to the new codes, it can use the new configuration file. But yeah, I understand your concern. So eventually we will need to restart the new version of codes in order to bring those new configurations really into practical use. Yeah, so eventually we still need to restart the new codes even though we have upgraded them. So are you planning to, with new release all the time, if you can provide, right? Every project can provide, okay, see here, if you have this kind of parameter, then because of the new change, you have to use some different way that people can follow, right? Yeah, so before you upgrade that part of the codes, you need to bring the new configuration files in place first. So there are no way to manage the new train of the configuration. And they submit, there are two sessions about how we can implement the configuration management in an OpenStack project. And there are some tries with the ETCD version three, but there are no final consensus. So OpenStack 4.0 is recommending us to upgrade it only from the X version to the X plus one, not keep really the upgrade because of the application or removed configuration. Yeah, hi. Thank you very much for what you're doing here. This is huge from a new comma to this space. You know, our fear has always been if we start up something like this and we can't do the zero time, I mean, zero downtime upgrade, that just things that are just, you know, prevent us from just even coming to this space. So seeing this, what you're doing here is really outstanding. I just also have a question too. Obviously, looking at from the VM perspective and from the container perspective, you're saying that with the container, there's less downtime. So if someone's new like myself, starting out on this path, would you recommend going the container path versus the VM just for that purpose? Well, I think if you are deciding on like which deployment you want, you cannot only seeing it from your upgrades case because this is only one of the factors that will influence your decision. You also need to see like what kind of workloads you will be running in your OpenStack cloud. And maybe in your case, the container may not be a super choice for you, but with only respect of the rolling upgrades perspective, so we think we would prefer containers, but that's only for rolling upgrades case. Absolutely, that's what I'm trying to figure out just because we're definitely, if you're new to this space, you're not gonna get fancy right away, so thank you. No problem, so if no one has any questions, we can close here. Oh, sorry, I've got one more. Hi, morning. Thank you for sharing this interesting experience. So I'm just wondering about one more piece. So you talked about rolling up updates, and maybe you have some experience with rolling back and if something goes wrong. Yes, so this is really, how can I address it, painful or, so I mean like you're asking if we made some problems while we're doing the rolling upgrades, how can we roll back to the previous version? So with the perspective of database, the OpenStack community has banned the downgrade of database, so if you want to bring back the old database, you need to have a backup of the database, and if something bad happens, you can restore that database part. But for the codes part, we don't have a reasonable or now satisfying approach to do the rollback of this kind of downgradation. So this is also something we need to address in the future. But for now, we don't have a satisfying solution for that. Sorry. Thank you. No problem. Thank you very much for your time. I hope you enjoyed the session. Thank you. Thank you.


Blue Beetle. Weeping down upon the underworld, a smash ganglin comes the mysterious, all-powerful character who is a problem to the police, but a crusader for law. In reality, Dan Garrett a rookie patrolman, loved by everyone, but suspected by none of being the Blue Beetle. As the Blue Beetle, he hides behind a strained mask and a suit of impenetrable blue chain armor, flexible as silk, but stronger than steel. Today's episode of the Blue Beetle is the second part of a story entitled Crime Incorporated. In the first part of this story, crime overlords of various cities have gathered in York City to organize crime on a nationwide basis. Patrolman Dan Garrett, through a clever disguise, has convinced Frankie Capetto, the ringleader of the gang, that he would be an asset to the crime ring. Hired by Capetto, he has been given a Blue Beetle costume in order to report to a man named Slick. Meanwhile, underworld characters masquerading in Blue Beetle costumes have committed several crimes throughout the city. The First National Bank has been robbed, Commissioner Donnelly's daughter has been kidnapped, and the mayor's office has been bombed. As our story opens, a meeting of crime incorporated is in session. Well, boys, the first part of our plan is underway. The city officials are scared. Our next move is to terrorize the citizens themselves. How? The guy I hired last night, Strangler Van Norton, he calls himself, says he's a wrestler. He's husky enough to be one. Yeah, but he ain't fooling me. He's got brains. What are you going to use him for? Hold ups. Can he shoot? I don't know and I don't care. He won't use a gun. But how do you expect it? He'll just come up and back of a guy in the dark, give him the old hug-a-mugger with his arm around his neck, rob him and let him drop. That's swell. No noise in that one. That's right. Once he gets started, the citizens will demand a crime cleanup. Well, what about the real blue beetle? Well, I figure he'll fall into a net as soon as he hears about all these fake blue beetles. Once we get him out of the way, we're all set. We'll organize this town and move on to other big cities. Now, each of you specialists will hedge your own racket, but the take will be pooled. Every so often, the syndicate will declare a dividend. That's a swell idea. Great. I hope it works. Yeah. All the blue beetles are here. What about Strangler Van Norden? He's in the other office. All the others and send in Van Norden. Okay. Well, listen, Frankie. Yeah. No one but a name of Van Norden never worked for B.J. You sure? Positive. Okay. Send in this Strangler and we'll unmask him. What's the lowdown? This wrestler's a fake.  I don't know. But we'll soon find out. Here's the Strangler. Oh, come in, Van Norden. You met some of these gentlemen before? Yes, I have. Say, you look swell in your blue beetle costume. Thanks. It's a very good fit, too. By the way, just what sort of work did you do for the late B.J.? Well, I... Well, I sever mine. I won't pry into your past. I'll let somebody else do that. Yeah, Frankie. Come in and bring the other blue beetles with you. Okay. Mr. Strangler Van Norden, I want you to meet B.J.'s former chief of staff. Slick Thomas. He says you never work for B.J. Oh, come in, boys. Hey, Slick. Yeah, Frankie? Strip the mask from the Strangler's face. Okay. Grab him, boys. Oh, you don't. There's one blue beetle left. Two blue beetles left. An offer. Look how well I plug them. Wait a minute. Wait a minute. Not here. Slick. Yeah, Frankie? Unmask him. Sure. There. Well, I'll be... Are you knowing? Sure. Who is he? A copper. Dan Garrett. Dan Garrett. A copper. Hey, what do you say, Capetal? Let me block him.  We don't want any shooting around here. What do I do with them, Frankie? Examine his custom. Looks like one of ours. How about the midget antenna in his helmet? Just a minute. It's a fake. It ain't connected with anything. Search him. You find anything? Nothing. No magic ray or flashlight or master keys? Nothing, Frankie. Then he isn't a real blue beetle.  No, not so dumb. What do I do with him? Throw some water on him. We'll take him out to Hangman's Island where we've got the girl. Hey, it's a cops. Hey, Frankie, I should have stayed on guard. Hey, how do we get out of here? We'll blast our way out. Hey, quick. This way, everybody. Through this panel. There's a secret stairway to the roof. Okay. Here we go. But first, here's one foot of copper on the floor there. His next costume will be a white robe and a harp. There you are, Danny. All done in step. Thanks, Doc. A little more to the right and that thug's bullet would have put an end to the career of patrolman Dan Garrett. And the blue beetle. Why didn't you wear the real blue beetle costume and mask and take along the magic ray and other things? I was out after information. I figured there was no sense in the blue beetle's nipping and until I knew more about the plans of Crime Incorporated. And what did you find out? Well, from what I heard as I was regaining consciousness after being hit on the head with a chair, Mary Donnelly's being held captive out of Hangman's Island. When are you going out there? Right away. But you're wounded.  It's only a flesh wound. You're going out to the real blue beetle this time, I hope. You bet. And I'd like to take along that electric ray pistol if it's ready. It's already, Danny. Ah, fine, Doc. As soon as I get into my blue beetle costume, I'm off to Hangman's Island. Tonight the blue beetle will sting as well as nip. So, Miss Donnelly, you refuse to sign this ransom note to your father? I certainly do. And I'm afraid he'll never see you again. You can't frighten me. You'd never dare harm me. My father would... You've been reading the wrong kind of literature, Miss Donnelly. What do you mean? You've been reading old-fashioned romances where the brave and the true survive and the wicked perish. My father will have every policeman on the force out searching for me. But you'll never be found alive. You let me out of here. Let me out of here. I'll scratch your eyes out. Easy, easier. I'll have you tied up again. I'll be sorry for this. Why don't you sign this note, huh? Your father can raise $50,000. And as soon as we have the money, you'll be free. I'll never sign it. Never, never. And I'm afraid you'll die of starvation in this cell unless your father's men get too close on our heels. In which case I'll set fire to the powder magazine in the room next to yours and blow this whole place to kingdom come. Hey, what's that noise? The Blue Beetle. And he's going to nip. Peter, Peter, Blue Beetle, don't try to hijack me.  Not this, Blue Beetle. Rick, Joe, Rod.  I'll take them all on. What's the matter? One of the Blue Beetles has gone haywire. He's holding up the bed over there. Stand back, all of you, or I'll shoot. What are you, Blue Beetles, there? You're wearing chain armor? Rush him. Okay, go on. Don't forget him, Will. What kind of a gun is that? How much do they shoot in blanks on him? Now I'll shoot, Rocky, and it won't be blank.  Peter, give me the key to Miss Donnelly's cell. Try and get it. Here, here, take the key, take the key. Take all of them. Now, all of you, get over against the wall. Are you all right, Miss Donnelly? Just about. Thank heavens you came when you did.  Blow you up? Yes. You see, the powder magazine is in that room next to this cell here. That's an idea. Here, take the keys and see if one of them will open that door. All right. It's unlocked. Now, slip the bolt. Now, open the door. And what are you going to do? Blow up this hideout. Quiet, quiet.  I'll crack open this keg of gunpowder. There we are. Now, Miss Donnelly, go up those stairs there as fast as you can. What about you, Blue Beetle? I'll follow you leaving a trail of gunpowder as I go. If any of these crooks move, I'll set fire to the trail and blow them all off the map. Here we go. Hey, Frankie, Frankie, the money. It's upstairs. One more shot can't do no harm. Hey, look out. The Blue Beetle's set fire to the gunpowder trail. Come on. Let's get out of here. Come on. Well, Danny, the Blue Beetle certainly wiped out one rat nest. Yes, and a lot of fake Blue Beetles with it. I meant to give them an even chance with the law when we got outside. But when I gave Rocky another burn with a ray gun, it ignited the powder trail. And you and Miss Donnelly were lucky to escape with your lives. Yes. As soon as I got her back to the mainland, I sent her home in a taxi. Well, you'd better turn in. Yes, I guess I will, Doc. After this adventure, I'll be seeing Blue Beetles in my sleep. And so the real Blue Beetle cleaned up a lot of fake Blue Beetles and smashed a nationwide crime ring before it had a chance to incorporate. The moral of this story is that no matter what the odds, good will triumph over evil. Might is not right, but right is might. Further adventures of the Blue Beetle will be presented in the next episode of the Blue Beetle. The Blue Beetle is a copyrighted box feature appearing in Mystery Men Comics magazine and the Blue Beetle magazine. The Blue Beetle is on the air twice a week on this same station. Consult the broadcast schedule in your local newspapers. And don't forget to listen in to The Blue Beetle.


So I had a question asked about realistically how much do people actually work while in PA school? So I wanted to answer that because I made that video like a long time ago about somebody working in PA school. So if that's something that you're interested in or you feel like to actually like pay for PA school and make it through, you're gonna need to work those videos for you. So let's get into the video right now.  Is it down and welcome back to my channel. So like I said, had this question asked just a few days ago about how realistically is it for people like realistically, how many hours do they work while in PA school? And before I get into that, I just wanna hit a few things first.  My daughter wasn't feeling well, my friend's not feeling well, like the school has a little outbreak. So if you have kids, be careful, wash your hands, keep them safe. That was my first little thing. Secondly, if you didn't see my short, we have a film coming out. So be sure to head on over to Fandango or the perfect movie. So be sure to head on over to Fandango or the perfect Christmas movie this week so that you can buy your tickets for the perfect Christmas, which is a movie that my husband did and both of us produced. And it's a family friendly Christmas movie and it's coming out this Friday and it will be in Regal theaters and some other theaters. You just have to go to our website to like type in your zip code so you can see the closest theater to you. I know it's so exciting. I don't usually like talk about things like this with you guys, like most of the time, you know, I try to keep it PA related but this is exciting stuff. So I wanted to share that with you. Now, how realistically or how many hours do people actually work in PA school? So the ones that I knew worked about like 20 or so hours a week, okay? 20, 24, like they kept it just at the PRN mark. So I don't know if that is like realistic for you. For me, I guess maybe if I didn't have any kids 20 hours a week would have been fine because I probably, you know, spent greater than that, like taking care of my kids, cooking, all, doing all of those things. And so like if I only had myself to take care of, then yeah, maybe 20 hours a week spent doing other things would be realistic. But that is generally speaking like what the students that I knew that were in PA school worked. Now I did have a conversation with somebody else. She was going to PA school before me and she had already graduated and became a PA, but she worked like 40 hours on the weekends. And that was because she had a family that she had to take care of and she was like the sole provider. So she couldn't afford to give up her job as a nurse to then like go to PA school and not have any income at all. So just looking at your own dynamic in terms of how your makeup is, like if you have a family, you have to take up, take care of you if you really cannot like realistically stop working, you're gonna have to like kind of source it out and see exactly like what PA school's rigor is for you, like how much time you have to dedicate and devote to it. And maybe during that time period, take like some PTO so that you can get like a true assessment of just going to PA school without working.  But that is what she did. And that's what some of the other students did. So I think it's really just a matter of what works best for you. So you're just gonna have to kind of source it out. But realistically, I would say probably like 20 hours just to keep your part time like PRN status. All right, if you have any other questions for me leave them in the comment section below. Don't forget to like and subscribe and don't forget to go to theperfectchristmasmovie.com to buy your tickets for the perfect Christmas. Thank you guys so much for watching and supporting and follow me on this journey. I will talk to you guys next time. Bye.


Hi you guys! Do I have a fun little tutorial project for us today? As some of you may know, I am going to be doing some traveling this summer. So excited about that. One of the trips I'm going to be taking is down the east coast of Florida.  Free Spirit just came out with this brand new collection called Treasure Island and it is seriously so adorable. As soon as I saw it, I knew that I was going to be wanting to make something for my upcoming trip to Florida. I mean the sand dollars, the parrots, the butterflies, the coral reefs, all of the sea creatures, all of the sea florals are just screaming Florida. The colors are super bright and super vibrant and I am so excited to make something with the sand dollar fabric. And I had the fun idea to make some like travel accessories and the first one that I'm going to be working on today is a curling iron or a flat iron holder. So the tutorial I'm going to be showing you today is actually based off of instructions that are available through Sew It Yourself. If you're not familiar, S-I-Y Sew It Yourself is a free resource for all of us. They have tons, like dozens and dozens of free projects for us to make. A lot of home decor, a lot of fun little craft projects that they want like this one that I'm making today and they're all completely free for us to use. So I've taken the curling iron holder pattern and instructions that they have on their website and today I'm going to be making a video tutorial to compliment that. So without further ado, let's head to the cutting table where I can walk you through the materials that we're going to need and the steps to make our curling iron holder. Okay, so for this project you need your main fabric. You need batting and you need some kind of insulating fabric that is going to prevent the curling iron from actually burning anything. This is ironing board cover so I know that it's going to be able to absorb the heat. If it's good enough for an iron, it's good enough for a curling iron and then you're also going to need some binding fabric. This is going to go around the perimeter of our curling iron holder so it will be seen from the right side. The first step of the instructions tutorial is to quilt your main fabric to your batting like so but because I'm using the ironing board cover that already has a quilting pattern on it, instead of marking all my lines and drawing on my fabric and doing all that measuring, I'm just going to use, I'm going to cheat and use the grid lines that are already on the insulation. So what we need to do is this goes right side down and then your batting and your fabric go on top of that like so. This square is roughly 14 inches by 11 inches. It does not have to be exact because we're going to end up cutting this down and doing curved corners anyways. But you just want to give yourself a little bit, a few extra inches on each side in the event that there is any shrinkage whenever we go to our sewing machines and mimic all of these grid lines here to get our quilting on the right side of our fabric. So I don't know about you but every single time I quilt something it just infinitely makes it look so much cuter. I just love it so much. So some tips for quilting though. If you have a walking foot, be sure to put that on your sewing machine that will help keep all the layers kind of moving through the machine at the same time. And then I also alternated the direction in which the fabric went through the machine. So I started in the middle. I did one big long stitch here and then I turned my fabric and then put that side through the machine this way and then this way and then this way and then went back to the other side and did the same thing and then on this diagonal again did one long straight stitch then put it through the machine this way then this way and again that just kind of like you know the fabric is going to shift then the next stitch is going to make it shift the other way. I also did a nice big basting stitch. This is not something that's going to like keep your whole project together so having a longer stitch I think is prettier and also you know really makes that the thread like really kind of stand out in some of these parts where it's contrasting. So we have that here. Our next step is to cut this down to 12 by 9. So easy enough we are going to take our clear ruler. We're going to measure this down to 12 by 9 and then you're going to take either a bowl or cup or any kind of rounded tool that you have and you're going to make your rounded corners and you're going to have four rounded corners on a long skinny rectangular piece. Well and that is the meat of our ironing curling iron holder. So now we need to bind this thing and then we're going to fold it in half and stitch around this whole thing and make like a little pocket. It's really so simple very straightforward. This project would be perfect for like bridesmaids gifts or like a family reunion type party anything that you have going on like that. So I just need to cut a good length of this binding fabric obviously feel free to buy the purchase binding from the fabric store but it's also so fun just to kind of make your own. Super simple to do no big deal and it's a lot more affordable especially if you're going to be cranking out a whole bunch of these. So I have all my raw edges lined up here. I am going to be making a one inch binding. All right now you are going to go to your machine and you are going to apply this binding around this entire edge sewing it right sides together at a quarter of an inch seam allowance and turning the whole thing to the inside like so. Okay could she be any cuter? I don't think so. All right we are at our last step here and for reference I ended up using the 44 inch length of the binding fabric and that was just enough to get all the way around mine. I had this much fabric left over. Another tip for the curves if you're having trouble getting your curves to lay nice and flat cut your ironing pad holder or curling iron holder cut the rounded edges more fat. So instead of like a narrow curve cut a fatter curve and it'll be easier for you to get around. But our last step is to fold this little guy in half like so you're going to want to grab some pins because we're going to be sewing through all the layers. We want to take the folded edge put that on a straight line and then we're going to measure three inches down from this point here one two three and then our stitch is going to start right here. You can also use binding clips if that is easier for you but start roughly three inches down and you're going to sew all the way down here either stitch in the ditch or stitch along this binding stitch that you just made all the way around the bottom of our holder and that's what's going to create the little pouch for you to put your curling iron in and it is just that easy for you to sew a little pouch to hold your curling iron or your straightener or any hot hair tool that you have while you are traveling. Didn't she turn out so cute? I could really see you adorning this a good jillion different ways whether you have a cricket and you want to do like monograms or you want to do like the year and the place that you're traveling to you could even do instead of bias binding or instead of binding on the edge you can even do some kind of trim like if you wanted to do pom-poms would be super cute if you've got like a teenager-y type of thing coming up. The possibilities are truly endless with this super simple project. Be sure to check out the description box. I will have a link to this fabric collection. Again all of the fabrics in the collection, the Treasure Island collection are stunning. The colors are super super bright. I also have the link for you to SIY Sew It Yourself to the entire list of their free projects for you to download and then this specific project is also linked in the description box as well. So I hope that you will check it out and that you will make a curling iron holder for yourself. Go make one of these. They're so cute. I will see you all very soon. Bye!