I'm Rachel Curtis. This is my colleague, Laura Davis. We are both digital project specialists at the Library of Congress. And though our titles are the same, we do different work. So we're going to kind of go through our workflows, how we use open source tools in our daily work. So like I said, I'm Rachel Curtis. I am the project coordinator for the American Archive of Public Broadcasting. So I manage all of the recorded sound and moving image files that comes to the library through the American Archive of Public Broadcasting. And it's very similar to the community-based acquisition process for the library, and it's not really something the library had undertaken previous to this project start. So I'll give an overview of what I've done to handle the digital material coming into our section, how I manage it, and some of the challenges and open source tools that I found useful for my project. So my first thing is I'm going to sort of start at the beginning before the library had hired either Laura and myself and sort of how the library was handling digital material. And it was kind of a wild west. So hard drives that came in either through copyright deposit or just since the library content from gifts were usually shelved, sometimes catalogued as items themselves and just set aside for later processing. The files that came in were stored in a server space, we call it embargo, but it is very similar to basically a digital closet. And items are difficult to pull down from that. It's a storage space basically. There was no one person on staff dedicated to doing born digital ingest. Those kind of different people as needed. Files were ingested on an as needed basis. There was no coordination between staff, there was no workflow documentation, and there was a pretty big digital backlog. As larger batches of files started to arrive with contractual obligations like the American Archive, it became very obvious that dedicated staff members were needed to manage this content, process them in a timely manner and create documentation. And then when I was hired about three years ago, there was already plans in motion to hire two more digital project archivists, Laura and then Keith Paramore, who works with Recorded Sound Material. So there were two projects, sort of three things that sort of was the catalyst for this. The lovely picture of how we were receiving hard drives, so there was an increasing backlog of material, which is more of Laura's purview, so she's going to get into that. There was the History Makers Project and the American Archive. So the History Makers Project, I mentioned this here because a lot of the processes put in place for ingesting that content were duplicated for the initial ingestion of the American Archive files. There were contractual obligations, so files had to be processed in a timely manner. Workflows had to be developed for our systems, which our internal library systems are really years toward ingesting material digitized in our labs from library content that already had a database record for them. History Makers and American Archive are files coming from without our institution and just as digital files. So they don't have any, you have to create the metadata records for them and sort of plan an ingest path for them. A lot of work when I started was falling on our video lab supervisor and it wasn't something he could be decked with, he had other responsibilities he had to come. So I was hired to, Lauren Sarnson was my predecessor and then I came on after she left to sort of take up this project. So the American Archive with Public Broadcasting should start a lot of you here have heard of. It was a Corporation of Public Broadcasting funded project and that later WGBH and the Library of Congress took on. So it initially funded a two year position at the library to coordinate the arrival of the 70,000 initial preservation files. Lauren's work was focused on metadata mapping, pulling records down from WGBH's archival management system into our Mavis database, putting in place technical specifications with the input of James Sider. And then when I started there were already grants in place for the American Masters Digitization Project, News Hour and the NET cataloging project. And then when I came on I was really tasked with dealing with some of the outline issues with that initial ingestion of preservation files and then dealing with the new grant projects that were going to be coming in. So I briefly touched on the libraries where the American Archive is really on long term preservation. We do provide onsite access to the material digitized through this. We have joint policy decisions and governance with WGBH for this project. And now that we finished our NET cataloging project and have a better handle on that material, we're doing in-house digitization and sending files to the American Archive for access online, which is great. And so when I started it was quickly obvious that, well, there was no documentation of any of the workflows, so that was one of my first tasks. And I also recognize that some of our in-house proprietary software was not sufficient to just handle this project. So I started advocating for more open source tools to be accepted by the library officially for us to work through these challenges. So some of my challenges when I initially started were these large digitization grants that were in place, except for NET, that was a cataloging project. So anything digitized, usually from analog material that the institution wants to add to the American Archive, usually needs grant funding. So American Masters, recently Riverside Church received a clear grant. P-Body, PBS NewsHour, and these are large projects. These are anywhere from 4,000 to, in NewsHour, 8,000 assets being digitized in files that need to be managed as they come into the archive. And these projects take a lot of time from managing just the administrative things to, it's about two years, and then to ingest the actual preservation files. In addition to that, a lot of stuff is coming from multiple sources, both from small institutions, small local institutions to larger national institutions, all of which staff has a varying degree of either inventories of what they have, technical expertise, no one institution is the same. So it's kind of a mixed bay as we work with people to give them as much help, because we really want their stuff and to work with them to give their preservation files. We also get a variety of file formats. So this is just kind of an example of based on projects that we've already adjusted and that are in the works, just a whole range of stuff. The libraries, our preservation standard is the JPEG 2000, but we also get uncompressed files, QuickTime, NPT, ProLez, NPT files, ABI, DMX, HD, just a whole range, especially when it comes to born digital content, we're getting, what they have is what we get. So that is also another challenge. And then making the case for open source tools, the library for a variety of reasons, and there is a long bureaucratic process too, to make the case that it should accept open source tools, despite maybe some of the security risks, or just make the library more comfortable in making these official tools that we can work with. So some things I used to make the case are we're able to manage projects with more agility, more collaboratively across multiple institutions, keep track of project progress, especially when it involves multiple institutions, managing multiple file formats, and managing inconsistent metadata. So I'm going to go through a couple of case studies, namely the News Network project, which was a grant funding project, which we had, because it was a grant, we had a little more control over what files were that we were going to receive, how we were receiving them, and the timeline. And then also, my second case study is working with smaller institutions, where it's a little bit more of a mixed bag, and it's a little more time-intensive in some cases to keep track of all the stuff that's coming in from the smaller institutions. So for News Hour, this, like I said, this was a project to digitize analog content. WN, let's see, WIDA and, let's see, NARA held the tapes from News Hour that spawned a whole range of file formats. And the preservation files came to the Library of Congress. Access files went to WGBH, and also preservation files also went to News Hour. So this was a clear-funded grant that ran over two and a half years back in KCF. So we've recently just finished, we're just wrapping up this project, which has been very successful, but really kind of was my first project that I took on when I started to make the case that I really needed some open-source tools. So like I said, over 8,000 takes were digitized, a whole range of formats stored in three different locations, and then the library agreed to digitize about 570 outstanding tapes from this project. There were some inventory issues, so WGBH hired a contractor to perform an item-level inventory. We got any duplicates, and that stuff was really necessary because as I said, our system, we needed to navigate it in place in order to accept any sort of files into our system. And my first task was to get us to accept open-refine because I really needed that to normalize the data. Otherwise it was going to be a lot harder for me to create batch records. So then as I'm going through and planning with my team to ingest these files, we really built on the lessons that were learned during the initial CPB-funded digitization of 40,000 hours. Initially, file delivery had been LTO tape. We decided to shift to hard drive. That was easier to manage because I didn't have access to our LTO tape drives. But I didn't have access to computers that would allow me to offload hard drives. We needed to do metadata cleanup and normalization, which open-refine was really necessary to that step. So the metadata ingestion, we needed to do a bunch of batch updates to our database. We had initially been using XSLT for the initial CPB project. We switched to Python, which is a little easier for me to wrap my head around. And actually the scripts I found were more manageable to do. And that just allows me to do batch updates into our system very easily. We refined the ingest process. So I collaborated with the staff at the library to set up automated workflows. And that's going to be a theme both Laura and I are going to touch on to automate as much of this as possible. Because that just makes it far more efficient. And to take a lot of the manual work out of it. And then we also added a quality control component. We requested a pilot batch from the vendor to confirm the quality and then we had monthly delivery. The initial ingestion of 40,000 hours arrived as one big batch to the library in LTO tapes with no mechanism really in place to say, I'm sorry, we can't read this file. There's something wrong with it. We need a new copy. There wasn't really that process in place. So with NewsHour, we really needed that in there. So we could go back to the vendor and say, please re-digitize this tape or re-wrap this file. There was something wrong with us. Wrong with it, delivery and copy. And then we used Google Docs to track issues across all three institutions. WGBH, NewsHour and ourselves, both in the preservation and proxy files to track any issues. WGBH was also tracking missing dates as well for episodes that were missing. So the library received baked items. Again, this is something we can request this from vendors, but we very rarely get files and bakes from individual donors. We got a preservation file, a QC tools report. Our own software we use for QC is called Baton. That report was added to the ingest package. We got SRT files with closed captioning when it was available and then checksums as well. We had a whole. The only manual part of this process was really some of the metadata and checking the files that failed QC. Those had to be manually checked, really kind of tested the limits of our automated QC. And we found that most errors, there were a few we had to send back to the vendor, but most of them were not actually the errors that the software was flagging. So those just had to be manually put back into the automated process. But otherwise, I really relied on a staff that was, I was becoming more familiar with Python, but there was other staff that were even more familiar with it, so we were getting this automated process set up. And then there was just the last 570 tapes that were digitizing that goes through our normal workflows as part of this project. So here's kind of my open source toolkit for this project. MediaInfo was part of our automated scripts to check the files and sort of troubleshoot any errors that might occur. There are actually very few of them. Open refined to deal with the metadata. Confluence we used internally to all my documentation is stored on there. So if for some reason I was hit by a bus the next day, someone would be able to take over my project and continue on the ingestion. Python was again key for our automation process and Google Docs for keeping track of issues that occurred across the whole project. So that was kind of the news hour project. So in contrast to that, we also get more digital donations from individual stations and producers. And like I said, sometimes this is a bit more time intensive. We can't put really any demands on what we receive. When we station contact us or we contact them, we ask them what they have, what sort of collections are there. But again, what they have is what we get. And this is just sort of a screenshot of a modified, an example of what a Trello page that we sort of used to manage these looks like. Because we have several elements we kind of need to keep track of for this. So like I said, for file acquisition from individual donors and stations, we do some information gathering. So what do they have? Are their materials analog or digital? If they're analog, they're probably going to need to apply for a grant. What file formats or storage media do they have? What metadata or some inventory do they have? And we need to track all the information because to accept this collection, we need to have it approved by both the Library of Congress and WGBH. There's a D of GIF, sort of a legal process that's in place. If they are getting a grant, we're often helping them with their grant application status, which Casey has spent much time helping stations do. And then when the material arrives, when it's complete, any issues that pop up. So by this Trello, sort of, there's some color coding to track where the D of GIF is, where the grant application is, when it's sort of coming, in what fiscal year are we expected to receive it because we do have a limit of 25,000 hours to accept for each year into the American archive. So Trello has been really helpful for this. I did try to do this on an Excel spreadsheet. That did not work. This is much easier to track since you can move the boards and parts around to where they need to be. And then we also do use Google Docs, especially when giving. Stations and donors, sort of metadata templates and legal information and grant application aids. That's really been helpful. The other challenge with these is that these ingests can't be easily automated because they're often a variety of file formats. There's no checksums. Again, the files aren't baked, which isn't necessarily a problem, but sometimes I get unexpected file formats and unexpected storage device, inconsistent metadata. So there's a lot more manual tweaking that needs to happen here that I can't just, I like to use our project where what I could do was offload the files into a watch folder and then the process just picked up through the different scripts to get them adjusted into our system. This one just needs a little more hand-holding. So SSM Paid, Media Info, and OpenRefine are really the things I use to manage these sort of individual collections. And there's sort of an example, just a graphic visual for what I'm using, but yeah, all of us in Laura will go into this a little more, but sort of analyzing what open-source tools are available and what ones we could get, the libraries approval to use, has really for both of us, but it's just an ongoing process to make the case. So I'm going to turn it over. She and I share the same title, but we do have different responsibilities. There's some overlaps. I just want to, and this is a quick way to show you where my responsibilities, what they include, is I do basic processing for gift and copyright collections for born digital collections, not digitized. And part of the goal of my collection is to eliminate the backlog or the contents of embargo or digital closet. I love the digital closet term. And that includes creating latest records, creating derivatives, and then adjusting the content. In addition, I'm responsible for minimizing the, you know, it's great to take it out, but we don't want a whole bunch more to come in. So kind of keeping up with what's coming in, and also minimizing the backlog, and automate as much as possible. I also support the addition of content to our website, such as the National Screening Room and selections from the National Film Registry. We also have the Geographer Zone of Film, which was a collaboration with the Geography and Maps Division and the For-It Home Movies, which was a collaboration with the Managed Script Division. And with that, I compile all of our content, the movie and image files, the still images, and the metadata, and compile all of that, and then send it to the team up in DC. We're based in Culpepper, Virginia, to send it to the teams in DC to add it to our website. I also manage the creation of our ITER, we do register ITER values, and I also manage handle creation as well. So here's some of the selected collections. I don't know how many of you have the bin of drives. That's an example of some of the drives that I have. I came in and there were just boxes of drives, DCPs. Even the receiver for your wireless mouse, did you know that they have hard drives that size? They have little flash drives that are that size, and those are received in copyright, and then we do have some things that are received, gift collections that are also on hard drives. Also, we're good video games, but we're not going to talk about that today. Everyone loves to talk about video games, some include. So across all of this, this turn is accurate about three weeks ago, and at that point I still had, I think it was about 30 drives left to look at, since then I got about 10 more. So talking about 117, notice is not on moving image, because we did get a company documentation with some of these, and font files, that's great. So if we're just going to move image files, this is us, 8 format. This is a big red area. I shouldn't point, you can see it. This is, we have a preservation copy of the Vanderbilt Television News Archive, so I can blame them for that. Thank you guys. So this is what we have, and these are the things that I'm working on. So, we talked about workflows, and one of the things that's really important, especially when you're dealing with collections that can contain one title, you know, one digital object and many digital objects that are all related, is setting project goals. And we do have my colleague Keith Paramore in Recorded Sound, and I set up a whole series where we do have project documentation. We start with a project charter, then we have a project plan, and then we document as we go forward, so that we have records of what we do. And this has been really helpful in terms of starting a project so everyone knows where we're going, what's happening, what changes may come across, and especially good for communicating with stakeholders and supervision. So for example, this one is really simple. We have administrative information, the project background, what it is about the collection, details, and then the project requirements, which are the goals. And there's some common goals to most of the projects, create a Memphis Record, create a derivative, ingest. And some of these other projects have additional components as well. Once the goals have been established and improved, then we can look at available resources in order to complete the project. And those include, obviously, the skill sets that are available, because that's going to determine the tools you can use. Rachel was talking earlier about, in the previous session, about using the XSLT. Well, that's not going to work for me unless I go out and learn more about XSLT because I've done very little with the XSLT. So learning your skill sets so you can apply the appropriate tools and minimize your learning curve, although we do always want continual growth. So here are some of the tools that we use. Not all of them are open source. I did put a couple in oxygen, but you can just substitute XML editor of your choice. Confluence. You can, from the way we use confidence, which I'll explain a little later, easily substitute a Google Doc. And we'll talk about a lot of these during this session. We're not going to talk about all of them, but I just want to tell you that this is kind of our core toolkit right now, and we're always looking to add more. So today I'll be talking about three case studies. And they'll illustrate the planning process, including the identification of tools. There's going to be some nice charts about that. And then discuss the processes and procedures. And then we'll discuss the successes and challenges. And then we'll answer them for questions for both Rachel and I. So the first one is Saturday Night Live. So Saturday Night Live is a preservation project for us. We receive, every week or so, we receive files from NBC with Saturday Night Live. We're current up to at least right before Thanksgiving. I haven't looked at the same one the last one we got is. But when I came, part of the digital closet was all of the Saturday Night Live episodes from 1975 to the time when I started in January of 2017. So by the time I got oriented and marked the tools and everything, we're up to over 800 episodes, many of which also have an accompanying documentation with scripts and case sheets. And the newer ones, I think the past three or four seasons, also included additional videos, moving image files that are not only the broadcast version, they're the repeat, the dress rehearsal, and the syndication versions. So there is a lot of content here to manage. And this is the first project that I worked on at the institution. So the first step was to establish the workflow. You can see a lot of similarities during the workflow because, again, basically, create an eggs record, create the derivative in just one. So with this one, we don't get any metadata with the materials from SNL. They basically just send us the files, and they'll have the host name in there. So for the 1975, you know, that big chunk of material, I went out and went to Wikipedia and other resources in order to get the episode information for the broadcast date, the guest host, the musical guest, and the cast. Then we're also assigning, we also pulled ITER values from using the API with ITER. Are you all familiar with ITER? It's a DOI, the entertainment identification registry, the identifier registry, excuse me. So we received the content. Creating the access derivative, that was pretty easy. The metadata took a while to do. We went through various iterations of a Mavis record. For this, I worked with my supervisor to create a template, basically, using the metadata off of the CSV that I created. You know, hindsight, I should use a database, but we'll get to that with this learning, if you're always learning. And then once we have the template, then I could go ahead and batch create. I did not do 800 at a time. I never do 800 at a time. That things happen. It could be the most perfect thing in the world and you'll have one thing, even though I'm in there. And I don't say this from my own experience, I say this from the experience of others that I have heard. Usually we do one, for all of our new augmenting processes, we do one, then we'll do two. And then we may do five, and then we'll start going into like 100, which I was doing five seasons at a time. So here are the tools I used for this. I think everyone's familiar with it. Just about all of these, except for Postman, is an API tool where you can write your query and then submit it for that. And then PyCharm, it's a little PC about Python. It's a Python editor. I find it a little more helpful, especially in error-resulting, than in idle. So here's an example of the PyCharm screen with, I put in an error, so you get the yellow boxes, it'll tell you when you get the red with the fatal errors over there. So again, that's really helpful when you're creating new scripts. So going about it. And in DLC, you'll see that a lot. I use that for troubleshooting, in case we do get some files that are truncated or have some damage, because we don't receive tracks like this with these files. So if they're damaged, I will go through and use DLC just to see if I get it to play. And sometimes I'll pull it back, try to record that from the barcode again, try to play it again, and then it goes online with the things that don't play, then we have to go back to our own demo about it. So here's the workflow with the tools. Yeah, we use a lot of pipeline. This is not a fully automated process. So all of our automated processes are demo pipeline. The open refine and the osman down here are under the episode level metadata. The open refine we use when pulling back the list from barcode, because it's a rich text file. So I do that part and then kind of create a more manageable CSP using open refine. Epiphythetic and media info. So case standard number two, we have the US Senate floor recordings. We built off of the processes that we did in the Saturday Night Live. And the key with this one is project goals. Current backlog, I should have the dates there, and I did not. And that was 2015 to January 30th this year. And then current receipts, which is January 30th, are posted this year to the present. And we've got content from 2007 to 2015 coming at some point. We're creating, and for all of these, we create documentation, and again, automated as much as possible. And then we're always looking to apply processes from previous projects into new projects. So again, we have this. The key difference between this and the two key differences between this and, okay, three, sorry. Between this and Saturday Night Live is that one of the Senate gets this metadata and that has made all the difference in the world. Two, we, and because of that, we've been able to fully automate this process. So we receive the files, they're transferred from the Senate to us from the Senate recording studio and they're placed in a watch holder. MySQL database picks it up. Inventory is the comment, or the Python program picks it up. Inventory is the content in the database. And then the derivative creation starts and then the Mavis record is populated with the ingest package. So it's all one thing and then it just goes in ingest. The last part, the handle registration, that is not automated. That's the one part of this process that is not automated. And that's due to the Library of Congress handle tool that we use. I can automate that. I can do batches. But generally, I just come in every day and I have identifications when we get content from the Senate. And I look and see when they're done and once it's done, I build and create a handle so that's just part of my daily activity. So that's an activity thing. Here's a sample of another thing that we get and specifically the fields that we use. So we're only using five fields from a much larger XML document. Is it? And then here's an example of the Mavis record. It's not robust in a typical cataloging way but what we do is we do reference the Senate record, which will have more detail about what happened that day. We have the URL because they do have a URL pattern. And then that's where I also put the handle in. So the handle is created as well. And the handle is created as part of that whole ingest process. The handle is created when the object has been in the database. Now for that Senate, it's not like we're getting one file per day. We don't get one file per day. One moving image file. We get one moving image file per hour that the Senate is in session on the floor. So we get one moving image file. We get as many as I think the most we've gotten is 24. And then you get an XML file for each of those NXF files. So two files per hour. So that's what this is doing. So the fact that we're able to automate it because we do get it every day is a really big efficiency for us. So again, go through the process here. These are the tools that we use. We've talked about all of those. We get an open refinement used for the backlog to look at the things in the bar graph and pull that out. And then here's the chart. So even with the batch document with the text file that I used for the handles, I actually used a Python. I used a Python firmware to create that when I was going through that 2015 to 2015 to 2018 content. All of this, for this process, automation was really a primary goal because we knew that we were going to get this 2005 to 2015 content. And we do estimate that that's going to be at least 14,000 files, evenly distributed between the XML and the NXF files. The third case study is the National Screening Room. We launched the National Screening Room earlier this year. Some of the processes for this were based off of the selections from the National Film Registry, which was launched last year. Here I'm not generating to make this record. This is a little different, so we should keep yours a little bit. And so our curator selects the content. He identifies the files that we're going to use. Some of our colleagues, they edit the files and do speed correction and do whatever magic it is that they do. They drop it into a large folder where a Python script will pick it up and add the bumpers from the Library of Congress in Washington, D.C. I've heard that many times. And so that's where that comes in, okay? With the still image files, there's a... I'll show you the program I'm using for those, is we use FNPEG to do that with a Python, it's Python and FNPEG. For the still image files, I need to create a JPEG at the original size. And then I need to create a GIF at 500 pixels wide. What we found, though, is that the quality of the GIF is not as good as the JPEG when we're doing that. So I use FNPEG as the little red cap that I'm going to say on the next slide to derive the GIF from those JPEGs. Then the metadata. Our catalogers review it. And our metadata can come from two sources, either Mavis, at which point I have to pull the Mavis records out. Mavis exported as an XML. We're on a Python script to create a file to be imported into the DM tool, which is an alternate source to implement the data into our website. Our website was built to repurpose ILS records. So the fact that not all of our material has ILS records, we have to have this additional methodology to have our metadata and our content included on the website. Then I gather everything up, submit the files and the metadata for the DM tool. I have to submit it through the DM tool. If it's an ILS record, the catalogers edit two fields, add or edit two fields, and then I alert our team in DC. So here's a little red cap that we use for the still images. We use trial and error to manage the content. We have over 300 titles, or almost 300 titles on the screen right now. And we're trying to add titles every month. So there's a lot of content to manage through all of these different parts. Here's the trial and error board. So this is part of the trial and error board. All of these have already been put online. So we have every step in the trial and error board, and we just move the cards across. But with trial and error, we're really taking advantage of some of the things like we're looking at rights information. So some of the things are only streaming. It's in the screening room. It's only streaming. It's in the screening room. It's on the film registry. So we do have indications for that. Green is the screen room. I think yellow is the phone registry. Red is streaming only. And then the black, the dark color is replacement files. We're also working to, for those files that were digitized 20 years ago when there was a time segment, we're trying to get those replaced with better quality scams. So here's the workflow for this. Again, for this Trullo, it would be useful more because Python isn't quite as critical for this process. But this is our workflow for this. So some of the challenges. We always have challenges when we do this. So Saturday Night Live already mentioned missing and damaged content. So there's something we don't have that we're going to go back and ask about that. For the Senate, we have, we just started getting close caption content along with the, for every hour. So that's an X amount. So we're having to retool the entire automated process right now. The screen room is the process for replacing older moving image files. Again, there's a lot of those segmented or early scans. It's a lot harder than I thought it would be. But I'm not familiar with our systems. So it's been a learning tool for me. I've only been at the Library of Congress for just over two years. So, but we are working out processes with our colleagues in the U.C. And then the repurposing of ILS metadata for the screening room has presented a sentence set of challenges. For example, we recently put up a whole bunch of newsreels from the All American News. And the way that the titling was set up in the website that we call the system P1, the way the titling was set up in P1 is it did not include the field 245 subfield N, which in serials will give you the date, which will also give you the date in the newsreels. So instead of having the newsreel being from a specific date, we had 23 files that said All American News with no other indication to distinguish one file from another just by looking at the title, in the title displayed. So we worked with our colleagues in D.C. and they assessed what type of impact it would have on all of the content in P1 across all of the divisions of the Library of Congress. They determined that it didn't seem like it was going to do too much damage, so we were able to lobby and give that change made and that will be applied at the next update of the National Screening Room in December. And for all collections, it's technical issues resulting in the non-receive of files. This is an IT issue, so anytime we have any changes in our system, anytime any of our partners have changes in their system, we always have to be careful that we're still receiving the files that we're supposed to. Firewalls can really get in the way sometimes. I know that's what they're designed to do, but sometimes you don't want them to be that effective. Finally, and this is the part sometimes I have a hard time with is you do the project draft of an evaluation because you know, I'm like, okay, good, I have a lot on my plate.   And then you move on to the next thing. And this is the step that's probably the most important is going through doing the project draft but doing your final project report and doing the evaluation. What went really well? What could them better? You know, what, if you could do it again, what would you do? For me, SNL, I wouldn't have used a CSD to manage my metadata. I would have built up my SQL database. Okay, fair enough. But again, that's my first project. So probably a CSD was okay for my first one. That's fine. And then what elements can be reused from this project for other projects? Again, no use in reinventing the wheel on this one. You know, for every collection that we do, we are creating a derivative on most all of them. So we just reuse that same script. We use the same script to move our ingest packages because we only have to, we only put like three ingest packages in our watch folder with the content at one time. So we have a mover, it's called a mover, a mover script that will make sure that that number is kept at three or below. So when we get to the M, we can make sure everything runs so I can go home and I have to babysit all of those, all those ingest boxes whenever. And then you're finally done. So thank you very much. Are there any questions? You want to go home? You want to help? Yes, ma'am. I was wondering if you could elaborate a little bit more about why you need to like using CSV. Oh, it's not that I didn't like it. I like CSV a lot. But considering the volume of information I have for the Saturday Night Live project with over 800 episodes to do it one time and then cast information, my spreadsheet was going into, you know, it went A through Z, and B, A to like B, N. I mean, it was huge. It was absolutely huge. Because once you start putting in, you know, here's my preservation file, here's my access file, here's my original filing, here are all these things that we're documenting. It just kept going and going and going. And a database would have been a much more efficient way in order to do that. A show that's been on the air. Don't they ever have to access old episodes? And did they do that before they met you and how they had that? How did they do that themselves? So SNL recently launched an app where you can search by cast member and by episode. They have an amazing wealth of metadata that hopefully when we have our next conversations with them that we'll be able to tap into it as well. That they will be able to share that with us. When they send over the digital files. So yeah, they have, you should check it out. You have to sign up for it, but really it is amazing. The granularity that you can get to with that. There's one in the way. Yeah, hi, thank you. That's really good. Walking into the tools and everything. In my work too, I'm always trying to automate as much as possible so you know, get more done. So I wonder if you could talk to parts that you can't automate or that you know that you can't and that you won't be able to. So one of those is the handle tool that I already mentioned because the way that we create handles at the Library of Congress is we have a specific tool that we use and that's based on submitting a text file. I haven't investigated too much on how much automation we can do but considering the way it's structured I don't think there's a way that we're going to be able to do that. The other automation and the difference between Saturday Night Live and the Senate is the amount of metadata. If we had the granularity of metadata that's in the app, we could fully automate the Senate Saturday Night Live. If we had the host, the date, the musical guest, and the cast, we could automate Saturday Night Live as well. So really so far what it looks like is our barriers for these two collections are the two that we have with regular accruals as we just need to be able to have those key pieces of information and the metadata has really proven to be in these two scenarios, that's the big difference right there. That is the game changer. And for those who were in the session yesterday I mentioned, well-formed, persistent, consistently well-formed metadata makes all the difference. For my projects it's very similar. Metadata is inconsistent so there needs to be a sort of manual analysis of what I get that can't really be automated but we kind of build off each other's processes. I'm hoping to adapt what you've done with SNL for a project that's going to be starting soon for the American Archive. And then the other component, with the QC component I have with some of my stuff, it's still, when it gets kicked out of the system because the QC software recognizes something's gone wrong, that's still a manual process for me to push it back in or to go back to the vendor to say that something's gone wrong. So there's that added process. And I should say that one thing is sadder than that while I'm doing this, I'm trying to automate as much as possible even now because I get two files at a time maybe four every week. And I'm trying to start, I'm starting to add those automated portions now in expectation of moving towards that fully automated process. I should put a little bit of clarity on these tools. Did you teach yourself? Do you have an ongoing training at L and T or do you want to go to school? Sure, so it's kind of a mix. Some of it is self taught. A lot of it actually is finding the people I work with who know more about it than I do and so can kind of, yeah, so can help us out with, I can do some basic scripting, but for some of the complicated processes we have a colleague who, he's worked with our internal systems and he's familiar with Python and kind of goes the ins and outs but I've also bought the reference books. I've taken a couple of online classes through library juice to kind of learn more about SSLT, Python, some of the more technical things that I'm not entirely familiar with but yeah, it's sort of a mix of self taught, finding those who know more and then reaching out to someone who knows more than I do about it and yeah, teaching myself, taking a class. It's sort of a mix of everything because I found in library school I didn't learn any of this so it was really a self kind of finding resource thing. Tell me because they didn't offer any of this in library school because none of them existed and I hate the fact that I can say that but really the teaching yourself like I taught myself this and I just played with it and I had a lot of calls to IR because they wanted their password encrypted in a certain way and I'm like, okay, how do I, you know, so I run around with people in the building and you might know more and Keith has been an amazing resource for both of us but yeah, I'm totally, you know, no real formal framework. You know what we need to know on the job. I like the project management worksheet thing or you know, for each project it's a good idea that helping you allocate, you know, figure out how many resources you need to tackle certain projects. Do you find that you have more projects that you can tackle and then what do you do about that? The answer is yes, there are more projects that you tackle and the way we negotiate that is prioritization. So I work with my supervisor and our colleagues to prioritize the projects so everything has, we have kind of a, when I first started we went around and assessed everything that was waiting in a burger and then put a high, medium, low assignment to it and then we're working through that. So I don't necessarily make the determination and there's some things that come up that it's a, oh, we need to do this now so everything else gets put aside and then we tackle that. And then with the American Archive it's really a discussion with WGBH and that's why we have the 25,000 hour limit to make sure we're not taxing all of our resources and there's projects that are more intensive than others and certainly doing the grant and application management is one of those things but yeah, just sort of, because I kind of mentioned when the donor contacts us or we contact them we kind of just get an idea of what they have and that helps inform our decision of can we accept this collection, how much work do both sides need to put into to bring it into the archive but yeah, sort of assigning priorities to things, yeah. And one thing for me is that because I also, for some projects I need cataloging resources as well and my supervisor also supervises the catalogers so we can balance those projects according to the priorities for me and for the priorities for the cataloger and really kind of balance that out.


There was a story that came out about a week and a half ago. I didn't put too much thought into it, but I have to tell you, things make a lot of sense when we take a look at what's going on in Hong Kong and China as they're moving rapidly into Bitcoin digital assets. The story that I'm talking about is this one. This was on June 15th and it states, Hong Kong regulators push HSBC, Standard Charter, and the Bank of China to embrace crypto clients. Now HSBC is the largest bank in Hong Kong. When I first read the title, I just skipped over. I didn't think too much of it. But when we get into the next article, you'll see why I'm talking about this piece. So here's what this section had. Hong Kong's banking regulator, the Hong Kong Monetary Authority or HKMA is applying pressure on HSBC, Standard Charter, and Bank of China. You have to understand something. This is very important. And that is that Hong Kong is different than mainland China. Mainland China is roughly 1.2 billion people in Hong Kong. It's around 7.8 million somewhere around there. China is not Hong Kong. It is one country, but two systems. When we talk about these things, don't get it confused that everybody of these 1.2 billion people are going to automatically get exposed to Bitcoin. That's not the case yet. So at a meeting in May, the HKMA questioned the banks and their reluctance to onboard crypto exchanges as clients. And a letter dated April 27, seen by the Financial Times, the HKMA stated that due diligence on potential customers should not create undue burdens, particularly for those setting up an office in Hong Kong to look for the opportunities here. And basically what they're saying is, stop dragging your feet just green like these people. Let's pull everybody in while America is falling flat in its face with this guy named Gary Genzer who can't approve anything. On the same date, in a blog post signed by Arthur Yen, the watchdog's deputy chief executive, it states he expects that regulated virtual asset service providers or VASPs will be able to successfully apply for a bank account through a reasonable process. That is refreshing. A regulator who says if you apply, it's going to happen. Just follow these directions. So that was the story that I just kind of skimmed over. And this is what happened just today, June 26. HSBC rolls out crypto services in Hong Kong. And this is a Bitcoin and Ethereum ETF. Don't get excited yet. I'll explain. Hong Kong and Shanghai Banking Corporation, HSBC, Biggest Bank in Hong Kong were probably introduced its first local crypto services. CSOP, Bitcoin Futures ETF. CSOP, Ethereum Futures ETF. And Samsung Bitcoin Futures Active ETF. I want to stress that a futures ETF is dramatically different from a spot ETF. This is just about the underlying asset. They don't have to buy Bitcoin itself. It's really just paper Bitcoin. On a spot ETF, they have to buy the underlying asset. So I know everybody gets really excited about an ETF, but you really shouldn't because it's really just paper Bitcoin. That's what they did here in the United States in 2017 by the CBOE, which essentially crashed Bitcoin for that rally. Anyhow, so this is actually positive. I think just because it's a futures ETF, it's a step in the right direction and it will give those people exposure. And of course, we'll blockchain is one that originally reported it. So fantastic work. But I will remind you at HSBC, it's amazing how these things turn around. You see this was with Jamie Dimon. We see this was with like Deutsche Bank, a lot of different people. And HSBC is no different. This was dated 7th of the number 2022. HSBC UK issues crypto warning. Pretty much says that you should look out because it is a scam. So it's amazing. And just, well, geez, under a year, they have come out with a futures contract. So again, ETFs are all the rage right now. Let's see if it gets approved. This was a report from CoinDesk. Bitcoin futures attract biggest bets since tariffs collapse. Open interest has surged to 11 billion. It's highest level in a year. Also, we've seen a lot of inflows. Digital asset inflows highest in a year after BlackRock's spot Bitcoin ETF. So everybody's getting ramped up. And maybe it happens. Maybe it doesn't. You already know my opinion on this. I don't think that BlackRock will be approved. But hey, I've been wrong before. But it doesn't matter because the cat's out of the bag.  So we talk a lot about Bitcoin. We talk a little bit about Ethereum. But we should really take a look at what these other countries are doing with other altcoins and how they're integrating them into the system. And you have to remember that pay attention to the crypto projects that are building in the bear because they will crush it in the bull. Near Foundation partners with Alibaba Cloud to accelerate Web3 growth in Asia. Here's what we have. Partnership grants Near Foundation access to Alibaba Cloud's developer ecosystem across Asia, Middle East, and a bid to attract more developers. So if you don't know, Alibaba is a massive company and is located in China. Developers that want to launch new near validers will be able to do so with Alibaba's Cloud's plug-and-play infrastructure. This comes as Alibaba recently announced the hiring of its new chairman, Joseph Tsai, who has been an active Web3 investor and has signed multiple deals with crypto-related entities. Near Foundation Alibaba Cloud partnership is an important one as we continue to support Web3 developers to explore opportunities. It's also significant for developers and validators in the Asian markets as they can leverage Alibaba Cloud's comprehensive infrastructure in Asia. And when I read these types of stories, it gets me excited because I know we're going in the right direction. They want these companies to come over there. They want to work with them. They want to build and everything is great. However, there are some things that are stuck in the back of my mind. And you got to understand, Chinese companies are a little bit different than companies in a lot of different places. So the founder of Alibaba is Jack Ma. And Jack Ma, he had an issue with a lot of things that were going on in China and he became very outspoken. And when that happened, this was just a year, year and a half ago, he seemed to disappear. And it's not just Jack Ma. There's a bunch of different Chinese billionaires that keep vanishing. So I know like we applaud these things. We want mass adoption. We think it's great. But remember, China works on a whole new set of rules and what maybe you and I are used to. Anyhow, let me know what you think about that in the comment section. And then also to talk about more or different parts of what is happening in the altcoin space, Algorand. Algorand partners with DWF Labs in a $50 million deal. And pay attention to the ones that are building the bear. This partnership aims to help support and grow development. Funding, DeFi, art and music, gaming, oracles and bridges and infrastructure. And just so you know, before we go on, I am super biased on this channel. I don't usually talk about things I don't own. So I talk a lot about Bitcoin and Ethereum. And you better believe I own near and you better believe I own Algorand because I'm talking about it. So don't be disillusioned and think that this is nice. I'm just talking about these random things. No, it's because I own it. So Algorand, again, looking pretty good. I thought about this part where it says DeFi, art and music, gaming, oracles and bridges and I thought to myself because I invested into it.  And if you ever dealt with Bitcoin during the bull runs, you know those costs are actually very high as well. So I took a look at Algorand and on its website itself it says the transaction fee is .0001. Wouldn't that be nice if you could do that on the Ethereum blockchain? Well, you can't. So that's why Algorand, it says current activity transaction per second, only eight. Remember, they're not using too much. So hopefully that'll be a little better. But then to dig into it, you take a look at the fees. It states on the website, but this is from developer.algorand.org. Fees are calculated based on the size of the transaction and the user can choose to augment a fee to help prioritize acceptance and you can pay higher to get things faster into a block when network traffic is high and blocks are consistently full. But it did stay here right here, right here, right here. .001 Algos. So I don't think it's that much different. I mean, we're talking a little bit, but it's still way cheaper than what it is. And just so you know, I like Algorand. I like what they're doing. Sue McCauley, he's one of the forefathers of cryptography. And it's not just me that is investing in Algorand. There's a guy named Anthony Scaramucci. When I interviewed him, we were talking about different investor mindset. He let loose that his company had invested $250 million into Algorand. So they believe they did their due diligence and they think it's going to be a heck of a winner. But I will tell you this.  Anthony Scaramucci also does due diligence in his whole organization and they invest it into FTX. So take it with a grain of salt. Anyhow, let me just think about that in the comments section. And then to finish up, I just want to make two mentions here. One is that there's a security tip for you. I found this out today. And I said in the tweet, I said, I know some of you have two factor authentication. And some of you think that 2FA is just the text message. Like if you want to log in somewhere, you get a text message from somebody, from the company, you're from Ledger or whatever else it is or the exchange and it'll allow you to get into the app or the device. And I said that's no good. You really should have a Google Authenticator app. You don't know what I'm talking about. There's a great website. It's 100% free. It's called DanTeaches Crypto. There's a bunch of different things in there. And I talk about how to secure yourself. But the thing is I found this out today, which is on my Google account, I could get authorized in three different ways.  But the second one was 2FA via text message. And the third one was the Google Authenticator app, which generates a number every 30 seconds. And I could get into my account one of those three ways. And I didn't know that. I thought it was all of them for some reason. If you have another account where you're not for sure, like am I using a Google Authenticator app where it generates those numbers every 30 seconds, or it's just text message, get out of the text message one because they can do a thing called a SIM swap and they can take your accounts and they can get into it and do all those things. But this was actually news for me.  So just be aware of that. And then lastly, if you haven't got an email from Voyager, this meaning that you can now start to take out roughly 35.7% of what you had on the platform. And this is from the app itself. And it'll tell you what you can take out, which is pretty great. And then you have to, unfortunately, you have to go through some disclaimers, click on that, and then of course continue again. And just so you know, whatever you take out, you can send it to, if you have Bitcoin, you can send it to any Bitcoin address. It doesn't have to be an exchange. It doesn't have to be a storage or a hot store, whatever you want to do, but it's just to tap the pace to a Bitcoin address. So just make sure you know that. And that is available for up to 35.7%. Some things I've noted that people have told me, they said that sometimes it doesn't work because their app is outdated. So they have to either update the app or like me, I had to delete it and then reinstall it on my phone and everything just worked just fine. So that is that. So I mean, I guess take it, take the win, right? And out of all these things, at least we got one of these guys to give us some of our funds back. But I will tell you, it is a learning experience and all these guys are very slick. They're either incompetent or they're just thieves, and I'm glad this is going to be behind us. That's it for today. So look, like today's video, put a thumbs up, consider subscribing. Everything to talk about is time-sensitive. I don't care who you subscribe to, just make sure you get your news from somebody because this is not a sudden forget it. That's it. Thanks so much. I do appreciate you and I'll see you on the next one.


Svingen är här för att tala om Svällsjöck. Det är en nobbygd system för. Designning parametrik objekt, och jag vill starta varje sjunga short movie. Det första parten är med mig, och det är särskilt bra. Bara Nikitron. Det är det mängderlöpare behind produkt. Och jag vill lätt stressa att vi är inte animatörer. Vi är arkitekt. Det är just för att pläga runt med det här parameters av det. Jag har inte sett alldeles. Nice videos. everybody been showing is a bit. Shameful to show off this. Some of the pictures are very nice and some of the things are not so nice, but it's all about potential and not actually what we have done yet so much. These are the few manufactured objects like a table, a bar disk and this three to the things. So my name is Lin Sving. I'm a Swedish architect. I'm not a developer and if you would look at this red shock source code, I think it will after a while become sadly clear that it's not a professional quality Python atom, but it's a working atom and it can do some very nice things. I started in early 2014. I did these two nodes that you can see down to the left first. A simple thing to get a series of integral floating values and add the seed setting to the random to make it actually useful in the design context, because before it was continuously updating and so I started the whole thing by scratching and itch. And I just continued and it became a hobby of mine and I have to say I'm not using. This is any professional capacity. I haven't done anything professional with it, but some other people have been doing things and I will show some things. The development team is listed here. Alexander and Nikita started the whole thing. Augustin joined them and made some nodes and I joined them and I took over like some core parts and I continued developing. Seffi, the Algar McArdle, has been doing a lot of interesting work and Konstantin has done some very nice blender integration in certain parts. So it's a node based system to create geometry. We manipulate mesh geometry. We don't really work with curves. You can manipulate geometry by creating vector mathematics, simple mathematics and you can form this relationship between the vectors and vertices. You can create mesh geometry. So it's like it's an optimistic project and it's continuously growing. Right now the add-on download size is 800 kilobit, the zip file from GitHub. It's an incredibly large add-on for being in blender, I think. But it's also because it's a very, by default, it's a very modular system. Every node is a module and all the modules are basically independent, but can be reused in different ways. A visual programming language. The node group is evaluated as a directed ice-clicked graph, like all blender node groups are by default, which limits what people want to do. For example, we cannot do recursion, iteration and certain things. I have some work to do with abstractions for these things, but I haven't done anything with it. So why are architects interested in parametric design? This is excerpts from the parametric manifesto by Patrick Schumart, who is one of the main characters at Sahadid Architects. You might agree or disagree with what he says. I personally dislike him and lots of the work they do, but I think it's very interesting, the relationship between the computer and the tool and how you create things. What we can do is very deeply affected by how we do it. The tools available to control what we can do and what we can realize. So if you look at this example, you can basically see the mesh grids being built into the rebuilding. It's very easy to see, like NARV flow and other things, realizing the physical form. There's a very derelect relationship between the software and what it does and the aesthetics in the building in this case. So parametric designs in the Svanschok, this is by Delga McCardle. Sefi, his thinking term of causes, symptoms resolve automatically. If you think it can be done simpler, it probably can start. I heard a lot here from people today and the previous days that they had looked at the add-on, but very few people have felt like how to get started. And there's a serious problem with this system. So the work flows of this is that you have to realize it's a tool set. It's not a complete solution. There's no completeness in it. It's just you have to build everything by yourself. You give a use in tools and you can choose to use this as something simple. Just you want to place some geometry in a very specific metamatic way. You can do that and you can use that later as a mesh. You can do something powerful and create a complete building. It's extensible in a different way. The basic concept is you position mesh data and organize meshes. Currently there are 128 nodes. It's written in 100% Python using the PyNodes interface. We're doing a documentation. About 70% of the nodes are documented. You can see an example in the bottom part of the picture. The Blender artist thread has reached 80 pages and almost 130,000 views as of today. So it has generated enormous interest in this also why I'm here talking about it. So what can you do with this? You can play with geometry. This is a fish example by Delger McArdle. I choose it because Frank Geary once said, a fish is architecture but a cow is not. Another simple example is you can manipulate the vertex weights of a mesh which can be very useful in this. A very simple example we use the dot product that we remapped to a different range based on the vertex normal in this mesh. You can do with different addons or different things. This is a generic tool set that you can do many such things with and you can continue doing this. You can use it for simple things like this in many different ways. We have an extensible scripting system. We have two different script nodes where you can input your own simple code to generate things. In this case there is extremely simple generating a grid. I won't go through the code. It's very simple for anybody that knows Python and you don't have to deal with these things. It's to show that if you want to do your own node or you want to generate your own geometry you can very simply input this and parameterize it and you get the results. This is a very simple flower petal done with sinus curves. It's not hard to model this in other ways. This is a very simple demonstration of how to create variation. Everything in Svertock is a list. If you input a list of numbers into the number of petals here you get a series of different flowers and you can position them in this case. We use a plane as a generator to generate a series of points to distribute the flowers in space. Or you can do something extremely advanced as Philippe Guimiot, Dormon architecture.com. This is a set up of 350 nodes. I can say that this clearly highlights some problem of the current approach. We don't have node groups, we lack many abstractions. There's not a lot of unneeded repetition here. But many of the we have different techniques in ways now to simplify this. My intention was to show how simple this layout could look compared to this but I haven't had time to disassemble it and reassemble it in a good way this weekend. I've been to busy drinking beer. I want to show some simple files. You can simply animate this fish in a very basic way and it's not a useful animation perhaps in the sense that you can manipulate it in a good way. As we said we're not animators but you can very simply parameterize and show the parameters. I won't really go through this example right now because it takes some time and it's not really perhaps whatever. I will do a very simple basic demo a little bit later just to show this. This is a simple amount of nodes basically for doing a very simple thing and you get something like this. And this is the complex example the building shown before. It's a totally parametric object. There's no external geometry. Everything is generated from numbers in different steps and solidified offset and many different operations are applied to it. And here you have the settings of the node tree that we exposed in the user interface. So if for example this is the floor height parameter it's now before 3.1. Let's set it to 1.5 so we have a midget building and it updates this very quickly and the updates are not quick enough to perhaps to drag these numbers on my computer perhaps this one is quicker but every parameter that you can design here you can animate or you can do other things with it. Of course this I want to show you this node graph. Total craziness. I mean this is an unacceptable amount of complexity perhaps with signing a building if you look at it firstly but it's all made in like it's very repetitive the operations applied and if we had node groups it could not be done much simpler and I won't say that this building perhaps is like how you should design a building or anything like this but it's a possible way to design a building or design geometry use these controls and it's a horrible, horrible mess in many ways this model and one of the problems we have that is every change you do in a setup like this takes a lot of time and if you have used it recently basically until this week there's a new version available when I solve some of these problems and I have time now to change fix things but there are for every change when he did this he have to wait like one second for the node tree to update every time I don't know how he could do it like this would be too annoying to me but it resolves now finally and we also expose a series of settings like you can lock it from being animated we can show to display it in the OpenGL preview or you can bake it if you want as a mesh object so this is a very simple object I want to show you some basic techniques Sveljschok is based on that we have nodes to produce data and we have output nodes that you can use to visualize the data or produce a functional mesh object that works with almost all modifiers I think the skin modifier is the only one that doesn't really work so in this case we have taken four circles lifted them and created a cylinder and by changing the parameters here of the width the diameter of the circle we can do whatever we want with it and we position the circles by creating a list of numbers that we input into a vector we get a series of vectors that are locations for matrix and we display the vertices using a matrix operation and from this simple one we can then interpolate another surface that this is using cubic interpolation and you have the same four control curves that you can then choose what will happen with the object and there you have the mesh object instead it's a proper blender mesh object that you can then continue to work on and keep modifiable and we can construct many different types of geometry and you can choose this won't be anything this won't slow anything down until you change the parameter when it has to recalculate of course if you edit the mesh object and you recalculate it you provide your original your edits but it's unavoidable in this kind of context and here we have a very simple I mean this is extremely in this case simple setup it should be even simpler of course we have a series of interpolation objects being created from the vertices that we change the direction of the lists and a lot of things and the main concept of course is that everything is a list of lists and constraints numbers so you should be able to modify create a number of objects in the same sequence all the time this is somewhat working sometimes and sometimes not working depending on the age of the specific node so what we need to do we need a better structure it was started as a hack, a very optimistic hack to get things done and we got things done but it has grown perhaps too much for its own good and the number of code and the number so it's becoming harder to try to change it and as I said earlier we are not developers especially not me but I have some ideas about how it should be working and we are very far from that but even today you can do very very powerful things with it I think and standing here at the conference and looking at what all the other people are doing I feel because I'm a blender I'm a user of blender, I don't know so much so I feel that many of the things that I've shown here can be done of course in better and like more standard ways so we also need, we need carve objects, we need text objects we need real, to work with real objects and similar things to do better of learning integration of course even today you can already achieve this by using these scripts you plug into the system for example you could do a dancing node layout which is kind of useless but you can do it another point I would like to raise is that there are now like a series of different node systems that kind of could work together but they don't and I think that for the future it would be very nice if you could unify these systems perhaps not our system is not supposed to be the base we might even be like a hindrance in some ways but I really think that there is a great potential in a node based design where you can manipulate the parametric objects and the parameters of what you want to design and keep it flexible so I want to point you to these two web pages the Github and the Nikkejons Russian thread where you can find a series of lessons in Russians that are actually quite usable with Google Translate if you don't speak Russian yet and there's also the Blender Artistress we are always welcome to ask questions and come with input or any requests or anything of course we can't promise anything for more technical issues of course we prefer to use the Github issue system and everybody is welcome to participate or look at code and we welcome any help with either design or any help basically let's stop there En amazing example of the power of our Python APR so I'm still wondering if Campbell is not here the guy is calling probably but did you meet with him? No I haven't spoken with Campbell he could look at this he really likes these kind of systems and he could probably advise you on some speed up or he might get inspired in optimizing some parts of the API especially the parts where you have to handle lots of points or mesh data which is possible to make much faster any quick questions from the audience we have a few minutes we have a lot of time because there will be a half hour break so anyone has a question about this Adam you are an architect can I answering my own question I can make my own Python node right? Yes of course Is there also a script node that I don't have to make it in? Yes and everything is animatable 3 questions sorry so we have these script nodes here we can load let's take an for a very advanced example in a way We could have done a PyNode workshop for we should do more workshops anywhere in the microphone because it's little Python script little pieces of code and they convert to nodes and then you have a visible programming language for your pleasure so everybody can make little script convert it to a node and then you start noodling around that's how it works mind blown so I'll try to do that I'm very impressed I wasn't here since the beginning where can I get this? You can get it at the GitHub address or you can get it at the GitHub I should mention also that the version I'm showing today is the experimental version available on GitHub if you want we have a different series and this is the version I was showing today that has some node groups that are kind of working and other new features I have a question for me this thing is just wonderful and I think it was really missing in Blender and I'm teaching in the School of Architecture and they start to teach another software I will not give the name but I think you know which one and is there a stable branch that we could use already to teach it or are we I would say that I try to keep it as stable as possible but we are a couple of developers and not everybody has a shared same idea about like stable and how everything should work and I think that partly it should be redone and that will probably break some things but I think you can use it it's working in a lot of ways and we cannot develop it without people using it and trying to find new use cases so somebody has to take a leap and give us feedback and we can improve it but I would say honestly that it's not comparable to the other software but you can do amazing things The other software is not used completely but today it's full presentation and I think here we have enough for teaching what is parametric design and this is what is very important for me Thank you One more, Madame Claude I think it's happening We have a short break Thank you for your talk In 13 minutes we are going to see Monica with her Bio Blender project


You didn't stop by that thing, but he's you did it Question of the day when the babies roll over comment below. We don't frankly remember Seems early. He's three months seems early Wow, wow Michael, what are you doing rolling over three months? He's actually Heading toward four months, but still I'm pretty impressed Okay, pretty easy to cut toenails when you're missing five of them taking your advice everyone, especially I think Dave Said keep them trimmed keep them clean. There's not much else you can do so that's what I'm doing Thanks for the comments a couple days ago for the question of the day and Basically, I'm only concerned about my big toe because only half Sorry for the details only half of the toenail is black and blue I wish the whole thing was black and blue so it just all fall off at the same time. Anyway, I digress. I digress Okay, we're getting a lot done It's a Saturday and just get a lot done and the neighbor's air conditioning just turned on and so it's probably really loud right now Love you guys. Whoo finishing off the tequila from the birthday a couple ten days ago No, two weeks ago. Basically this bottle only has like a finger left in it And I'm sick of looking at it in the outdoor fridge and we need the space Frankly for my water bottles for the runs That I'm doing the big mountain run is ginger ale and tequila a thing. I don't know One of the best in the game is good as any of Louis submarine just one of those days Haji-paji living life trying to figure it out. I was going to paint this evening and guess what I'm still gonna do it I'm gonna do it. I'm gonna do it Dave. I'm gonna do it In the rain. Yes, Joseph in the rain. I'm going to paint no matter what we're gonna It basically I'm watching the forecast watching the radar the Doppler radar And it's past. It's now east of us. So I'm gonna persevere and just go I just gonna keep going But now we gotta wait mama That it that is not our car I Love you Yeah, yeah, not our car yet. She says.  You are smart high-five Seth is staying in he he gets it. He's staying in Seth's not getting wet. Oh is he sad? Oh? He said oh a party. No, no, where's my house? Where's my too much excitement for two days looking at YouTube He's like I rolled over today. Did I have to also go to a party? Hello, hello. Oh the rain stopped the rain stopped. Uh, that's a good thing. I'm gonna paint tonight I'm gonna paint tonight finally so folks. I Occasionally lose my temper. I know may not come across in the vlog that often But I do I'm a human being and I always want to be transparent and honest with you guys about everything and so This chair behind me from Ikea basically About four months ago. I Was awoken in the morning on a Saturday and I did not want to be awoken in the morning on a Saturday By the children. Love you boys And so I picked up the chair and I slammed it on the ground and it's Ikea and listen We love Ikea. We appreciate Ikea. However, it's not meant to be slammed on the ground this type of chair And so it snapped and I continued to sit on the chair for about two months until finally it gave way So basically listen, I lose my temper. That's the transparency in today's vlog and the tip of the day Big piggybacking on that transparency is that if you glue your chair And you don't have a clamp that's big enough to clamp the chair together after you glue it Don't be afraid to use a weight like this guy right here to clamp to bring it all together Great pitching coach and now pitching coach of that 5.56 Jeff Weaver 5.46 Hedekirabu Fourth of July is coming up. You can always count on Uncle Troy to provide a fresh flag for the 4th of July Thank You Uncle Troy always giving me a good nice beautiful American flag to hang. Come on. Let's hang it up Thank You Uncle Troy. Thank you


The International Committee of the Red Cross ICRC is training over 150 journalists in Nigeria and safety teams for dealing with life-threatening conditions during their daily assignments. The organisation which operates in 100 countries is focusing on the enough ears in providing humanitarian support to those affected by armed conflicts. The training will include basic knowledge and first aid application to support journalists during coverage of events in Abuja. Daniel Edobo further explains the benefits of the training and why it is necessary to everyone. We believe that conflict reporters pass through a lot of challenges that have to do with getting injured, sometimes getting killed. There are so many platforms where you see journalists, the number of journalists that are killed within a year. So we deemed it very necessary to give our own support in this regard, to be able to train journalists on phase eight so that they would be able to take care of themselves and take care of their colleagues in the field, in the course of operations if something happens and they are injured or need some free hospital support to go to the hospital. It's important that we all learn first aid, the fact that first aid save lives cannot be overemphasised. We need to learn these skills, not just journalists, every household needs to learn first aid because sometimes at some point in our daily work, at some point in our lives, we need first aid, not to save our loved ones or our colleagues or ourselves. And this skill we have learned as a takeaway for the participants, we want them to teach their colleagues and families because it may just be them that have the knowledge that may be a casualty or a victim so that they will just be able to assist them. The participants at the training commanded the facilitators for the positive impact the training has added to their knowledge. It is a very, very nice training for me personally because I have benefited a lot. Before now I have not been exposed or have the knowledge of all that was taught but after the training I think I am very good at what we are taught. We learn a lot of things, we learn on how to place people in recovery position. Participants, everybody I meet was very nice and jovial. We were just like a family and as a team. We have a joyful training so far, so good. I think nothing was so bad. And CPR, I learned how to place people in a COVID position. I learned what to do to stop a bleeding, burns, scars and lots more and I was issued a certificate at the end of it all. It was an amazing experience and I am so glad to be a part of it.


So we have to go to the cake right now so we can sing Happy Birthday. This is Martin's 80th birthday, and so we'd like to wish him a happy birthday.  Love, love, love you. So, love the candles. Thank you very much for coming on to this very special event. You know, I'm very surprised. And I think this is a great opportunity to meet those friends and to talk about lots of old times, including old times, of course, when you are 18. But I think this is great. Thank you very much for being here and for your help in work, which some of you I have very good collaborations for many years. So I hope this will still go on and be doing more fruitful in the future than it was in the past. Thank you very much.


What's up guys my name is Michael and welcome to my YouTube channel today We're gonna do another leco challenge called converted sorted array to binary search tree So you're given an array where elements are sorted in ascending order and you have to convert it to a height balanced binary search tree So height balanced binary search tree is where the sub trees are never differed by more than one So guys, um, I know I haven't been uploading a video lately for a lot But I'm gonna get back to it. I'm getting back into the grind of uploading daily videos So now every day I'm gonna upload at least at least one video every day. But yeah Yeah, I'm not gonna upload like at least 10 I'd like 10 in one day, but I'm gonna upload one at least one leco challenge every day So Back to this question converting a sorted array to binary search tree So how do we convert a sorted array to binary search tree well one way to do this is to every time you We would take the middle of our list So we go through it. We have a list of values negative ten negative three zero five nine One way you could do this is to take the middle value of your list and have that be the root Okay, so in this case negative ten negative three zero five nine This is now we're gonna have the root as Zero, okay, and that would be a root and then our left the left and right would be The left and right would be the next middle of the left the left side so The left and right sub parents would be actually the middle values of the left the left array which is whatever on the left side of this side and Right array, whatever the side on this side So I'm not gonna create their tree like this But I'm gonna create the tree some some somewhat similar, but it's not gonna be it because they say they just want a following balanced binary search tree, so What I'm gonna do is here you have negative ten negative three zero five nine, right? so here Let's see negative ten negative three Zero five nine So if you want what I'm gonna do is I'm gonna take the middle value. So this is gonna be my middle Zero and I'm gonna make this the root of my tree my new tree. That's supposed to be a node. Okay. Let me redraw that So this is gonna be my new tree value zero and then I'm gonna have a sub sub values on the left and right and The left tree the left left root is gonna actually be the middle of Whatever is on the left side the rest of the list on this side So my root of the left side is gonna be the middle value of whatever is less on this side and then my right side Would be the middle value of whatever list on the right side so in this case the Middle value so zero one zero one two zero one zero plus one is gonna be Zero plus one is gonna be Let's see Minus one so yeah, okay, so zero plus one is gonna be one divided by two would be one So that this negative three would be my root Of my lower value Yeah, that's what it would be and then the same thing would be on the right side five and then nine so Zero, so this is the the index starts at zero one two three four, right? I'll show you the indexes. I'm gonna choose them in the middle value of my right my right side, which in this case is Five five nine whatever on the right side, so if I want a middle value I'm gonna take the indexes and add them up and divide by two so three plus four is Seven divided by two what seven over two. Let's see. Let's quickly do the math Three yeah, it would be three because we round downward and What's three's index zero one two three three the index of three is five so that will be my next tree my next root and then On the left side, I'll do the same thing again, and I'll pick the next value Which would be the root of that value and you'll get negative ten here If you keep going and do like that you get negative ten and then you'll get nine Basically you would get nine and then you have like no values on the side, but yeah, that's how I would do this so You I would just pick the middle value The left side and then whatever value middle value on the right side to be my root because that would that would make more sense Okay, so how do we code this? We're gonna create a method called construct that takes in We have a method called construct This is gonna take in the the start value and then the end value which is gonna be our the starting and for our Constructing our Tree that we're making okay, then what we're gonna do is This start and end this construct method would actually return a tree node and That's gonna be the tree node of That we're gonna return We also have to pass in the whoops my fault My insert was on okay, we have to pass in the Numbers that we're passing in because we need to construct it so what we what are we gonna do? Well, if start is greater than end We're just gonna return null Because what we're gonna do is we're gonna create a We're gonna recursively call this method on the left side of middle and the right side of middle And we're gonna get the middle middle index every time so Starts greater than and we're gonna return all Otherwise we're gonna get the middle index, so I'm gonna do middle Middle index is gonna start plus end and then I'm gonna divide by two Okay, so now what I'm gonna do is I need to create a new node that has the Value at nums at start so I'm gonna create a new node country node a New node and it's gonna have the value of nums at middle index Okay Then I'm gonna do new nodes left child is going to It's left child is gonna equal to I'm gonna recursively call this method on the left side We're gonna be point at whatever's the new node has returned on the left side And then I'm gonna do the same thing on the right side And I'm gonna return new node. Okay, so now what is gonna be on the left side? Left side is gonna still be nums except now we're gonna Our starting values and ending values are gonna be different. So our starting value is going to be Zero to middle minus one because that's gonna be the left side of our tree my bed So that's gonna be remember the middle zero Now we're gonna create recursively call on the left side and that that left side is going to be Whatever our middle value our middle index Right the middle next subtract one and I'll be this and end value this So this is gonna be yeah my bed middle index and Then this is gonna be Middle index plus one to And this has to be the size of the list Yeah, okay, it would have to be the size of the list Okay, then what I'm gonna do is I'm gonna just going to return construct and pass in the nums Start will be zero and Would be nums dot size Okay submit the code Compiler, what is the problem? Let's see? Base pointer has non pointer type tree node. Oh, okay. I actually have to create a new tree node with this Okay, um So what I'm gonna do is I have to use a new keyword Equals a new tree node It's gonna be a pointer to a node and Then they'll they'll fix that because then I'll get left side. We're actually better if I just use the dot operator It makes more sense But I think this is fine. Anyway runtime error. What is the problem stack overflow? Okay, let's see Let's see what the issue. Okay, so I Actually meant to do size minus one because that's actually the ending of where you're gonna do This is should just be end Because here we're constructing our a using size minus one Because a construct is gonna the end value is gonna be the length of the vector subtract by one Right because it can't be the size of the vector right here here The ending is it's not always going to be the length of the Array our ending is actually going to be recursively called so In this case this ending is actually going to be Here and in the middle value from zero to middle minus one right here. It's actually gonna be zero to Middle plus one put to the end right and could end could be whatever and could be changing So that's why it's not it's not always gonna be num size minus one All right, we'll submit that and then see What occurs wrong answer, okay, let's see. What's the issue? Do do do do wrong answer negative three. Okay, so we're getting a lot of nulls. We're getting a lot of nulls. Um, let's see Okay, so the problem that we had before is that it's not always gonna be zero to middle index minus one it's actually just gonna change based on start because Start could it's your your partitioning your recursively calling on the left side and the right side and that could change based on Whatever value that's starting. So that's why this this value should be start and this value should be end. Yeah Okay, I'll submit it and then this should be working Yeah, okay, and that got accepted. So that's how you do it. He would convert a story right to a binary search tree Let's see other people's recursive Solution, let's see this guy's Let's see how they do it Okay, so that's basically what we did Instead they return zero. That's that's funny Yeah, okay, that doesn't What this okay, so yeah, that's basically what we did They checked if the size equals zero then they do know okay Let's see. There has to be another one. Oh, this one's faster 94 percent. Let's see what he did. Oh Wow, okay, so he actually did a Okay, okay So what he did was he created? If start is equal to end Start plus and plus one over two Okay, he gave the he created the same list If it's not he were sets root as and then he adds the tree Okay, so he does the same thing start Index minus one start plus one and Okay, so that's literally the same thing as what we did. Uh, what is I think this is like read This code is just like we changing the Tree to make sure it's balanced. That's what this code is doing. But yeah That's how you would do this solution rate com subscribe. I'll check you guys later on next video. Peace


Okay, folks, Larry Pesavendro, mid-afternoon update. My guest today will be Mike Moore at 1.30 this afternoon, specialist in crude oil heating oil and gasoline and many other things. But those are his specialties and we'll be chatting with him at 1.30. We'll also be watching for a possible appearance by Joe DiNapoli on Friday. But right now, I would give it 60-40 on the downside, in other words, he might not make it. We'll try to find out here today, but we'll know tomorrow no matter what. But we'll try to get him on. Today's a very, very strong day in the market. We've been down for quite a few days now, especially these last four or five days. Last four days have been basically straight down. We have major targets yesterday, if you remember, in the S&P 500. And then we also hit it in the Dow Jones Industrial Average right on the money. So that's why I think we're getting this nice little rally today. I'm half-forward. It's going to proceed. We'll have to wait and see, but we're following it very, very closely. One of the things I'm thinking of doing, folks, is doing a two-hour special on how to trade a bear market. I've got some tools other than ABCD that work really nicely in really strong trending markets. I'm thinking of doing that a two-hour show for maybe like $127, something like that. And we'll see if we can get a few folks who might want to watch it. I've got a few things in the old quiver that helps the arrows go straight through the air. Sometimes they work. Sometimes they don't. But bear markets are different than bull markets, and most people are bullish and don't know how to trade a bear market. So we'll pay attention to that. Speaking of bear markets, we just had a big move down in gold, folks. We were down actually $90 from the high that we made at 1985 when we were on the air here on the 15th of September, just 14 days ago. And we've dropped $90, believe it or not. And we've hit that level. We were waiting to see the 786 retracement came in at night, it's 1875. The low today is 1874.50. We bought it 1881, put our stop at below, right below 74. Whether it works or not, I don't know. But that's a big completed pattern that we're going to be discussing as we start looking at some of these other charts when we come on to the main show here that we'll start here in a couple of minutes. So those are the main things sold of every day and an attitude of gratitude. And please stay with us more to come.


All right, let's go in the truck. All right, let's go, Addy. All right, you get in there, okay? Addy, we gotta get you on the other side, come on. So, we're daily vlogging now, and we're doing it. So, that's pretty fun. Okay, we're not actually daily vlogging. We're just gonna vlog our family life here a little bit more, because people are asking for it. Oh, yeah, it's in here. It's actually right back here. That's lovely, Sarah. Hi, Sarah. That's Addy. I was just gonna get the stroller. Yeah, let me get the stroller just in case. Hi, Addy. Hi, buddy. Say hi. Hi. Yup. Also, look at these amazing trees, isn't that really good? They're pretty, aren't they? These are like dogwood trees, I don't know. Dogwood? Can we get a picture? It's like you and me. The trees. All right, let's go, kiddos. We got the stroller in the car. Right, Addy? Addy is, Addy, you're good, Addy. Look, Easter. Oh, Sarah, tell us what we're doing here. Easter egg hunt. Addy line's first one. Do you want your basket? Yeah, okay. Oh. Yup, Addy, there it is. Nice. Addy, how do you like the eggs? She just woke up. Look, there's some right there. We gotta put it in the basket. Okay, put them in. Addy just looks pitiful. How was it, Ethan? How'd it go? Did you find eggs? So this is Katie, you guys seen Katie. We're on the hunt for more eggs in his backyard. Ethan. Yeah, we have a full one. Ethan, did you find another one? Are you finding more eggs? Good job. That's amazing. Catch them all. Like Pokemon. Can I see it? Wow, look at you. You got a whole collection. So Sarah, how's it going? So much fun. Are you finding all the eggs? I am. That is a Easter egg hunt. Can I help you, ma'am? You didn't see my hair. I'll just turn this way. Easter eggs. Easter's. That's Easter's. Wait, what? I had diarrhea since Easter's. What's Emily from? Nacho Libre. Look at Addy. Look at her. Oh, two guys. Oh, no. Oh, she's eating. She's eating something. She's got something. Oh, wow, she's got chocolate. Oh, no. Yeah. Hi, Kobe. So, hi, Kobe. We should go on the grass. Come here, Kobe. Well, it's, is it Tuesday? It's, it's Tuesday and we decided to, it's more gonna be like a compilation of what we've done like this week when this vlog comes out, more so than like a day in the life. Cause it's just pretty much a day in the life. This is what we do. Come on. You gotta get it. Hi, Kobe. Let's play with the ball first. Ready? Yeah. Ball? Hi. Throw it. Kobe's got a, Kobe, go. Hi, Kobe. Hi, buddy. Let's go run. Let's go run. Where's the ball? Addy. Yay. I'm hanging out on the sun today, so.