All right, welcome to the June 28th areas working group call 2023. A number of things on the agenda slightly out of order but I'll get that in when we get into the topics. We'll start with the marketing group update and then I've got a pile of RFCs PR so that I'd like to merge if we can get to that and then we'll see what else we can get to. This is a Linux Foundation meeting and a hyper leisure meeting so the antitrust policy of the Linux Foundation is in effect, as is the hyper leisure code of conduct. It is not the 21st of June so we've got to edit that. Here we go. Please add your name to the list of attendees I put it the link to this in chat, so feel free to add your name. If anyone would like to introduce themselves or make announcements the mic is now open for anyone to take that. Feel free to step up for introductions for those new to the community that want to introduce themselves and talk about what you're doing in this space we welcome that. All right. This announcement we're going to talk about in a bit I believe so I'll defer that until the discussion topics. If that's okay. Release statuses. In Python RC one of release 082 has been released. There is probably one more PR to merge into that so we're holding off on a final 082 release. We do have an issue with GitHub actions related to a move to Python 3.9. Basically we're in a situation where we've got a couple of integration tests that were fine on 3637. And we're fine on 38 and 39 locally but but hang on GitHub actions 3.8 and 3.9 without any feedback at this point so we're a couple of people that have been looking at are at a loss we're stuck. Anyone can help out on that we really really appreciate it so let me know if you're able to help out or willing to help out. We really need to get that one resolved. We'll take some other actions but we need some new eyes on that one which is just a weird one and very very frustrating. Any other announcements about frameworks or any other releases release updates and so on. A couple of things to note Aries Asian test harness failed last night to run I don't think any of the tests ran or very few so I'll be investigating that in a bit this morning. So heads up on that. Aries mediator service. We'll talk a bit about this but we really would be super useful to have this updated with socket doc so that would be a nice project if people are interested in and are looking for something to do. We have something noteworthy Steven. What's that. We, when you were asking about framework releases we did integrate if J040. By fold last week so that's a huge win. Thanks to all the hard work from our friends at animal. It seems to be working great. We've got one thing that we forgot to fix ourselves but yeah it's working great. This of course got all the new shared components in it with we're not using in the SDK anymore. We're using all the rest libraries like ask so they ask car to interact with the ledgers and the wallet and non creds RS and all those other shared components were using as well, which worked great. That's awesome. Yeah, that was the fact that it ran, you know, even with small issues that it ran on a first go was amazing so nice work on that. Thanks Jason. All right. Discussion topics. Helen Alex, do you guys want to take over the screen and talk about what you're doing or just talk about what you're doing. We can just talk about what we're doing that'd be fine. So the first official meeting of the Aries marketing committee took place yesterday. This is, I think we're going to meet just once monthly to talk about updating all things Aries branding messaging, etc. So please, you know, starting off please encourage anybody on your team who might be interested in this effort to join us. The, it is on the groups IO like calendar reminders or whatever so folks should should get them all the information's there. We also have a wiki page. Just if you click backwards a couple times you'll see it so anyways it's, it's in the Aries, you know, a branch of the Aries wiki. So all the information's there to join please, folks. We will be inviting some speakers to come and clarify a few topics for us. We're looking at sort of how to explain Aries externally to non people that are like not really familiar with the project or self sovereign identity or what a credential is like kind of you know how to, how to explain things at a very kind of, you know, basic fundamental based way, and then also how to explain to those developers kind of who are in the open source community who are more familiar with it and that kind of thing so kind of both sides because I think that there's some you know we've sort of decided that there's some some things that one side would want and not the other and vice versa. So Alex, I'll turn over to Alex here in a minute to talk about the questionnaire but we would love to get some feedback from the community on this it shouldn't just be coming from Alex and myself and but so we're hoping to get some some feedback in a simple questionnaire so Alex do you want to explain what that process is and what the goals are with that effort. Thank you. I think you said most of it very well so we're going to put the questionnaire out if you see anything resembling a questionnaire or survey come your way. Please take the hopefully two and a half minutes it'll take to complete right there and then be much appreciated. What we've achieved is to is to flesh out understanding of the things that you guys know in your daily interaction by areas. So when a person needs areas comes along where do you point them. And what are the most important things to be telling other people about areas right now. And maybe what are the challenges people are facing when they hear about it or what's the resistance you're facing you're hearing about it. Sounds like a lot of things we're going to sit down some great targeted questions probably a ranking question but a free text space you can just give us your thoughts and get the core proposition of areas together because it's still very clear that there's clarity about the lack of clarity that we that we need to get a solid core understanding of what the areas proposition is right now. I know that's quite broad, and then we can specialize it in depending on the target audience for someone looking at this it's this if it's for exact as this if it's for a developer on board into to leverage it or it's this. So yeah, short version is, I think he'll, he'll think it'll be next week timescale sub the chain maybe see a questionnaire survey across your path please. He'll be very quick to complete and we'd really appreciate your input. And I would just say one of those questions that will be on there.  So start or start getting those together organizing those, because I think those will be really valuable that just as an ongoing effort to always kind of be updating hey we just had this cool event I spoke it. And this is the record whatever like, you know, we just just keep kind of ongoing. Yeah, bulletin board of sorts with resources, public facing resources so start getting those links together but we'll be organizing and collecting those in that as well so if you want to be featured on kind of the main areas page with that with your information. That's another kind of added benefit to filling out the questionnaire. Excellent. So one of those in the community that's for sure, having them organized would be very useful. Yeah, like, there we realize there's so much content that's already made by everybody on this call that let's you know let's maximize it and maximize visibility on it. Absolutely. Excellent. All right. Any questions comments for Alex or Helen. All right. Okay, next topic is areas are FC PRs. This is something we do regularly that haven't done in a while so there's a long list of PRs in the in the community to be reviewed. Have I got it right okay you people can see when I scan back and forth across my screens. Yes, between the tickets and the PowerPoint yep. Yeah, good. Thank you. Sorry about that. I just wanted to make sure zoom was working the way I thought. Let me do one thing before I can share this and change anything to an editor copy link. And I'll put this in chat, as well in case anyone wants to go through it, and I'll add it to the really quickly should done this before sorry about that but I want to get it into the notes. So we don't lose it. Okay. All right, so we're going to play let's merge I'm going to keep in this view because I've got links and things to show 789 at clarify so I'm going to what I'm going to do is just list these off and then go through them. What I'd like to do is get to some sort of resolution on these. And ideally we either merge them close them or identify questions to go back. So definitely need your feedback and and input as we do this. So looking forward to that is what I'm looking for so 789 is adding a clarification to a to the thread RFC about the handling of an empty thread decorator so basically this is a clarification. As noted, some frameworks may send over a till the thread empty, which causes other frameworks possibly to crash to to throw an exception, or even to reject a message when a when an empty thread decorator comes in. So simple messages don't do that don't send those over if you're going to have an empty one just don't even send it but if, if someone does happen to send one over. The framework should be defensive and they're handling of it shouldn't reject the message simply ignore the decorator. So that's the advice in that. So if I come over here to look at the change. Simple clarification saying it's not recommended may send an empty thread areas agents receiving a message with an empty thread must gracefully handle such a message so that's the clarification. Any comments or any questions about merging this. We've had some conversation on this couple of suggestions of maybe to make if you're going to put a thread in and you have to put the did the thread ID in. But we don't want to make that a require as noted so we think all of the three of the frameworks now handle this at least a VCX and go we're not sure of their status. We're not going to be merging this one right I'm going to go ahead and merge this one I think this is probably the only one will do on the fly because the rest will require updates, but we have at least accomplish that. Probably should have put in the notes that we what we did. Oops. Okay. I'll try to keep track of the status as we go. Each five is another one that I put in based on activities in the in in the day to day work going on in this case with bifold the bifold team. Why is the well known goal codes Aries rel and Aries rel build again a clarification. Relation rel is a relationship in this case to the Aries rel and Aries rel build indicate that the goal code is to build a relationship. I think in the, in the item to say these should not be used when just establishing a did calm relationship but that actually happens to be exactly what we needed for. The change is just to remove that restriction and since sometimes that is all you need so let me get to that change 785. Let me see the where I've got things placed I can't see it so I'll just tweak that to say 785 so I stay on the oh I wasn't even helpful to do that but anyway. Basically a minor change that says this should not be confused with building a did calm channel. And, and in fact, it could be used for a did calm connection itself is not the relationship but would be used to carry out interactions to facilitate their relationship. And so, basically this is saying this may be accomplished by establishing updating or deleting a dead calm message connection that provides a secure communication channel, basically saying it's okay to use this goal code for that purpose. It's not specifically banned. Any questions or concerns with that. Right. No comments. Are we okay to merge that one. Any objections, no objections. Okay, I'll mark that as to merge. Then I wanted to throw this. Basically these the goal codes is and and other events are needed to deal with user experiences user experience. So the this one came about because of activities on the BC wallet. The goal codes to fine tune the UX so that an event, an event occurs that triggers some behavior. In this case, lacking a goal code on the connections mechanism. The BC wallet is now assuming that after a connection is established something else will happen, either an offer of a credential or perhaps a presentation request. So this this issue came up because there was no other action other than we just wanted to create a connection, but there was no events no logic in there for the wallet to use in in making that user experience distinction. A we need goal codes to help out in the user experience so we need to define new goal codes and use them in places where they're enabled. Also brings especially the concept of done in an action so that we know when we're done this is one sticking point that we've seen and we asked people to think about as they are developing wallets and things which is, once you get to an interaction with another party, and you get to the end of it how do you know you're at the end of it or, or that the other party is going to continue on with some other staff. And so coming up with the concept of done is something that the BC wallet team has been thinking about and trying to figure out how to accomplish, particularly with goal codes. Any feedback on that would be of interest and any other experience in developing and tuning user experiences would be good. Okay, leave a moment for anyone to comment and we'll jump to 784 to show that one while we go. This is an interesting one. Basically, as you can see what it does is there's a table in the revocation notification v2 that says hey here's the type of revocation notification it is that the type of credential and examples of the format. It adds in addition to Indian on credits adds just plain and on credits, and uses an example of a did as the, as the mechanism, the identifier for the credential held by the, by the holder to say, oh this credential this specific credential has been, has been deleted, or has been revoked. Sorry. So that's the purpose of RFC 721. It's hard to tell whether this is a clarification or a minor update I would lean towards this being a clarification. I hope that this is still in the proposed this 721 is still in the proposed status and the actual current revocation notification is 0183 which is a bit problematic but that's for another day. This kind of implies if you're adding another revocation format, it implies that you're going to have a different handler for it, you're going to, you're going to get this message and it's going to say a revocation format and you're going to handle it with one way with an indie and on credits and another way with an on credits. A bit of a clarification, I think it's mainly a clarification that those will will not appear since it's still in proposed. It doesn't really matter if this is a clarification or a minor update. Any comments on this one. Anyone run into this or, or have an issue. Any concerns about merging this I think this is a pretty straightforward one it's a doesn't really matter since it's still in the proposed state whether it's a clarification or minor update any concerns with merging this. No concerns with merging this but a comment on the subject at least that I think we should basically just treat this like the did spec has did method registries when there's a change to the method registry it doesn't change the did course. I think this is a similar change. So I think we're, I think we should be good with making these sorts of modifications without needing to grab the specification for the protocol. Thank you. Thanks. Okay. All right. On to the next one, which is 780. Let me jump to that one. So that I'm ready. Okay, we have an entire brand new. RFC. With this one. I think this one is pretty straightforward we have had a few people review it basically this says. When you are issuing a credential and particularly for images and perhaps other data types in those attributes that you're issuing the recommendation is to use a data URL. Those unfamiliar. That's a, that's an image. That's not what I wanted to do. Just trying to click that button. So a data URL looks like this, which is you've got an attribute called a photo and then the data URL looks like this. So that is a hard coded. You've got the the mind type, an optional semi colon and base 64 to say that the data is base 64 encoded a comma and then the actual data. And so this particular data URL is that image that I you saw slightly earlier on the screen so this is a itf RFC 2397. And I think that the, there's been a few people read through the RFC others are welcome but I think it's ready to merge. Interestingly, another use of it is JSON can actually just put in as the data type is JSON and either base 64 or not. But that would help in places where for example, in a non credits where you want to have an array. You cannot have an array in an on credits. So, not, not a valid. Not you can't do that because of how the non credits signature works so you can't have an array of data elements but you could have a single attribute that is the array. So it would give an indicator for a holder to say oh by the way this is is JSON data and they might have a chance at being able to display it in a reasonable way rather than just dumping a bunch of JSON on the screen. So lots of flexibility in there. There was a discussion about whether this is an Aries level or should be a credential formats issue at the at the specification level of credential formats like a non credits or the W3 CVC format. So those looks like I had it. Since those do not deal with pretty much don't deal with the data I don't think it's appropriate personally that that those go in the non credits you can put whatever you want. Those an on credit W3C doesn't care what kind of data or the format of the data you put in the attribute. It's really up to you. It just makes things a whole lot more flexible and easier to use easier for things like holders to be able to display data if a data URL is used, particularly for that photo use case I think that's one that's going to come up a lot and needs to be there. Any comments other than Colton's comments in the chat about this one. Anyone want to say anything about it. Any objections to this being merged. And that could be you just want to be able to read it before it gets merged to make sure that the RFC is complete. I have to put these in capitals. Okay, we're going to go with that. 768 is the proposed legacy peer did method. So I'll jump over to here. I'm not going to be as easy to do this one but let's see if I can just get rid of that and 768 and see what the changes are so this is was proposed as a way to deal with unqualified. Unqualified dids using, sorry. Yeah, unqualified dids that are being used in peer to peer relationships basically in did calm. We since have updated and should this should not emerge the latest solution is we would adopt in peer to and did peer three. And transform unqualified dids which is what is talked about in this method into did peer to and then subsequently into did peer three so encode the did doc that was shared in the end as the unqualified did and then transition encode that as it did peer to and then subsequently encode that into into did peer three. Since we already have the did doc so I think this one is to be canceled. Any objections to canceling this one. Somebody make a comment just for the fun of it. Not even that comment. Perfect. This is 55. This is a very large one, one that I think is now ready to merge one that I created. So, and I've done a few presentations on this one. So, the background for people who have not heard but I've talked about it lots is OCA is a mech OCA for areas uses and the overlays capture architecture specification OCA in a way that allows us to make credentials beautiful by adding multilingual support and issue or branding to the display of credentials, which is particularly useful in holders in wallets and things so this has been incubating for a very long time and we've been making minor adjustments as we've gone along. I've given updates presentations on what updates have been made as for things we've learned while implementing it now I'll talk down the bottom here. Since the last time I presented this, and we talked about it only two changes have been made the addition of a display of a stacked view in the style guide itself so that was assumed but not actually displayed in the style guide so that is now there. And we've added a data element called watermark to the meta overlay and that is used in particular for when showing a non production credential, which for developers happens a lot. Putting a watermark on so the difference can a difference can be seen between a, a production and non production credential in a wallet. Excuse me that's gone on is areas by full now has an MPM model that is published on on. Published on the MPM repo called hyper ledger slash areas OCA that implements the pulling in of a. OCA bundle for giving credential and, and grabbing the data and using that data for the display. And there's also a repository created in BC gov called areas OCA bundles, which implements a way to publish OCA bundles anyone can submit a PR of an OCA bundle with appropriate data associated with it that data gets humanly verified gets checked out and as long as the, you know, the rules are followed that it is actually an OCA bundle, and the bundle points to existing identifiers. It will get merged and anyone can use this as they wish. So, with all that, are we able to merge this, any objections to merging the OCA for areas and the seven, and it comes along with a RFC 756 OCA for areas style guide. So both RFCs are included in this PR. I'm not hearing objections. Anyone want to comment on this whole idea, the lunacy of it. I'm going to mark it. Great. And I think we're we're also thinking about trying to make this work for the web. So we had a conversation at our last by full user group meeting, because we want to make like this this will this package will work in in bifold so react native but we also want to see about adding support for the web so that if you're making something like traction or your own thing. You can have the same look and feel that you would get in the wallet so you can I understand how it's going to look when you release your credential. If you've got any interest in that feel free to find us in the bifold channel a Keith would be champion championing that endeavor. I'm just going to show off the OCA Explorer but I don't have it handy, but but it is an example in this repository. We have a published get have pages that displays it. Okay. That is to be merged which makes for an easy next one. That is 740. I did. So 740 is an update of the read me of the overlays capture architecture RFC 013. Basically, on that one, let me jump to that one. Well, that doesn't look right. Or is it sorry about this. There it is. This. So, there is a RFC overlays. RFC 13 is called overlays. And basically what this RFP or RFC was is a definition, an early specification of OCA. Since then the specification has been moved into the human classes foundation has its own spec has its own repository has its own working group. So the fact that we've got an early version of the OCA spec in areas does not make a whole lot of sense. So as part of the previous one. So this is 755 and with agreement of Paul is that I basically have updated RFC 13 to say retire. And so I, there's no need to merge this PR. And we can just close it since since the update does not make sense the specification is published elsewhere. Any objections to merging this and retiring this. I believe that is the last one of the easy ones for me. Hey, I got a question on that last one Steven, what does it mean to be moving it into the human classes foundation. So OCA this one was written in 2019 this RFC 13 as you can tell it's very, very new. Since then, although this was there as a as a as an RFC that was proposed it basically was merged as a proposed but never updated or never moved to a higher status. Paul Knowles and collaborators then took the entire specification and just opened a new specification at human classes foundation. And that's where the, the OCA spec resides and we've got pointers to that in the OCA for areas. We are in this one and we've got in the, in the where we retire 13, we also point to this work has been moved to that organization in that repository. That makes sense. Cool. The next one is a new RFC that adds push notifications as Expo protocol. So there is, this is mobile specific push notifications between a mediator and mobile wall say hey I've got a message for you. There's some administrative work if we do want to merge this one. And there was a question raised by by Sam that has not been answered. This is 745. Other than that, I am not qualified to comment on this one and and so would suggest anyone with that expertise take a look at this and add a comment on this. And this does supplement 699 and 734 which are other forms of push notification. FCM and APN S again I'm not don't know enough about it so I can't comment on this can anyone has anyone looked at this. This is ready to merge. It looks complete to me. And so I'd be happy to merge it as a proposed. I haven't actually read this but just a thought that jumps to my head and probably maybe not correct but for an RFC like this is technology specific like oh yeah. Expo versus Azure versus Firebase versus yeah I just wonder if there should be a not I don't know if it can be but more more agnostic or it's okay to have this technology specific RFC so it's really open. Yep. Has has anyone implemented notifications, sort of but not in this, not using this protocol. What, what do you do. We were just adding a metadata property to the did calm messages that would say if it was read or not and then checking that was kind of it was not a very good. Oh, I see to me yeah. This is more specifically for push notifications for a mobile app. As opposed to the messaging. So, for instance when the mobile app is not connected to the mediator directly and the mediator can push some form of notification to the mobile app. Yeah, while the app is offline. Yeah. Okay. I think the recommendation here would be to see if we can get a session at an upcoming meeting to to discuss notifications and where we are with this and have somebody who knows what they're talking about lead that discussion so we'll see what we can do about that. Yeah, and the comment about the different types of protocols with like Firebase etc makes me wonder if I don't know if there is but if there should be a more generalized method for, or like RFC for push notifications that allows for the different mechanisms out there to be implemented. So this is a matter of interest how come there's four, or there's three. I would have expected to see. So there's Expo, FCM and, and that other one. APNS. How come three, when there's only two mobile operating systems. Well expose their own. That's not. Okay. That's what I'm saying that's part of, that's probably react native that's an Expo native solution. Okay, it does it cover both OSes. I believe so as does as does Firebase and maybe Expo calls it all the ones I'm not sure I mean Firebase people are so Azure is universal.  I've never really looked at Expo implementation. Most of the world uses Firebase. You can also use the, you can also use the natives like if you just wanted to push notifications rather than you could just do that as well. Yeah. Okay. Okay, thank you. Yeah, let's do this one as well I want to get to the end of this we've done almost enough. We're almost to the end. 744 is adding a timing capability to the mediator anyone with mediator experience that's dug into this basically this is saying how long and recipient is willing to wait to receive messages as they are arriving so the generic protocol says give me M messages that are Q, you know whatever messages are queued up to a maximum of M. And then the mediator would say oh I've got some so I'll send those across what this adds is, if there's less than M then I'm willing to wait and milliseconds for more messages before, before sending. So if you've got none in the queue, I'm still going to wait, you know, 3000 milliseconds to see if more arrive, because as a holder I'm expecting more to be coming, or if there's 10 in the queue, and I request 20, then wait some seconds to see if more messages arrive. I don't know the utility of this. Since it's an optional one and there is an implementation and it was found to be useful. Does anyone else know if this is a useful feature. Just from my perspective this seems more applicable to when the agent is connected over HTTP rather than say web sockets. Yeah, basically informing the mediator. How long is my connection time out. Yeah, so it's a more graceful way of doing it rather than just having it cut. Okay, and the client's basically saying please give me a bigger batch if possible. Yeah, yeah, might help at minimum to have have that. Okay, I'm going to skip past a couple of these. The rest of these so we've got a few more to do. Perhaps next week just to finalize these other PR is a very old I'm going to try to get them just closed because I just think they're too old to look at. And last thing I want to do is a pitch for a proper RFC website so this is something that could go into the marketing. So, Alex might be useful here. I spent some time doing the acupy.org website, basically using M docs material as a default. My process was just to create a new repo add welcome content for the repo and then a script that copies all the content in the acupy was every and defile into an appropriate structure on the new static website. And then just a manual process to update that website nice to have with GitHub actions. So, repositories would be something like welcome and administrative page to say you know the life cycle of RFCs and how to submit an RFC and what the process is. But then this is the really important a IP one would actually have a list on a clickable website not a GitHub repository with a bunch of read me's but you could actually go through and see all of the IP one and in a nice published form, as well as, you know a column of other RFCs that are, you know, indifferent in proposed states. So I think this would be a much better way for people to be greeted by areas RFC and be a much more comfortable way for them to read them so I throw that out there I've got it on my to do list if anyone wants to help out or work on that with me. Let me know I'm more likely to get it done if someone's pushing me on it so I'll throw that out there. And with that will end this part of the discussion with just a couple of minutes left because I wanted to hit a couple of other things. Any final comments from anyone on that section. Awesome. Okay, mediators. We did have the presentation last week lots of good stuff in there in DCO has open source socket doc awesome work on that which I think this is a total game changer and how to construct scalable mediators. So I think this is really great work. So mediator basically turns it from being the meat, the mediator from having to worry about WebSock it's to simply having to worry about sending and receiving HTTP messages so much easier to make completely scalable so the next will be, as I mentioned Aries mediator service AMS be updated to use socket doc I think that's the way to go. So, we'll see how that goes if people are doing that work or or just what becomes available for that but I did want to highlight that in DCO has open source that repository that we talked about last week. I wanted to address it or to use it. Last topic I wanted to go over and Alex, Andrea, I'm glad you're here. There's like to highlight two things that have been commented in the deep peer spec. Issues were raised. So one of them was area errors in the example so a clarification is needed. The specification is about hex and code of shot 256. And attempting to do that in implementing did peer three in areas BCX this issue was raised. So really good if we could take a look at that. As well. Daniel Bloom posted notice of a couple of inconsistencies in did peer three that I think should be resolved. Daniel merged this a little quickly. We explained why but I didn't think we had quite enough review so it's not surprising to me and I think it's not a problem for us to do that but basically the simple is removing the dots because the other methods don't have a dot in in between the three of the method number and the rest of the did and then use multi base multi Kodak hash. So we know exactly what hash algorithm is being used and hash algorithm and base encoding of the data. So again I think those are two wise changes to put in. Alex you're here or Daniel Bloom you're here. If one of you could put those in I was going to do it but I really don't know enough about the multi base and multi Kodak hash to explain it. I did notice that there are multiple shot 256 algorithms and so I think we have to be more specific of which one and I wonder if that's related to the other issue that was raised. I'm going to throw that out to others. I see Daniel is not on the call anymore. He was on earlier but so he raised at it Alex thank you for taking a look at that that'd be awesome. I'd really like to get that as clean as possible and keep that updated. I would note that one of the PRs that was recently merged in the did peer spec was to add was to remove the big red text warning text. So that has been removed from the specification. There used to be some warning text saying yeah should you really be using this and we really think this has the did peer spec and the did peer method has a lot of valid use. And so we did not like the fact that there was this kind of ugly red text where no replacement was particularly available at the time so really good to see this that update made. And with that, that covers all I wanted to go over. I'm exciting one once more I went through this added a couple of issues Timo posted a migration doc for unqualified dids, moving them to qualify dids. Again strongly encourage people to read through that. Take a look at it at comments via issues into about the document. I found it a very useful one and and thank you. And with that I will stop sharing and give anyone a last second to respond. Make any comments, and then we'll wrap the meeting up. Excellent. Have a great day. Great rest of your day depending on where you are. Take care folks back. Thanks everybody.


Penguin Random House Audio presents Scenes From My Life, A Memoir, by Michael K. Williams with John Sternfeld. Read For You, by Dionne Graham. This book is a work of memoir. It is a true story based on Michael K. Williams' best and honest recollections of the events of his life. The following individuals portrayed in the book were given assumed names in the interest of privacy and discretion. Barry, Bill, Daniel, Darlene, Dina, Joni, Kay, Mr. S., and Walter. Co-authors note. We all are works in progress. Everybody is a work in progress. Michael Kenneth Williams. Michael K. Williams, Mike, to those who knew him, died on September 6, 2021, a few weeks before we were due to turn in this manuscript. He and I had been working for two and a half years on various drafts and iterations of this book, which began as one thing and gradually evolved into another. After his passing, the book was completed off of the extensive interviews he gave me. Mike was open to sharing his personal story and experiences, because he thought his journey, his openness itself, would offer solace to others. He was adamant that the book he wanted to write shouldn't be about self-glorification but was instead an honest chronicle of what he'd been through and how it informed the man he became. He wouldn't want to hide the fact of what killed him, his addiction, and he spoke openly about the daily struggle the disease was for him and for so many others. There are references in this book to how we thought he was one false move from having it all slip away about the fragility of life, and how it all could be snatched away at any moment. These are all directly from him. This is how Mike felt and how he spoke. Early on in the process, he finished telling me an embarrassing story connected to his drug use, and I asked if he was okay with putting it in the book. Yes, definitely. He said, I don't think I have the liberty of leaving that out. The liberty of leaving that out. It told me so much about it, how he felt his struggles had to be about something more than just himself, how he suffered, and how it was not in his nature to deprive anyone who might benefit from the sharing of that suffering. He knew how pain kept inside, multiplies, and how pain shared subtracts. This idea would come up again and again in our work. Sometimes my phone would ring and it would be Mike in a frenzy, that deep and raspy baritone excited about some news piece he saw on television. Sample complete. Ready to continue?


Hello and welcome to the session. Let us discuss the following question. Question is, state the reason whether the following function is inverse. G is a function from set 5, 6, 7, 8 to set 1, 2, 3, 4 when G is equal to set of ordered pairs of 5, 4, 6, 3, 7, 4, A and 8, 2. Let us start the solution now. We know G is a function from set 5, 6, 7, 8, 1, 2, 3, 4 with G is equal to set of ordered pairs of 3, 7, 4, 8, we know the function is invertible only if it is 1, 1 and odd 2 function. Now we know G5 is equal to 4, G6 is equal to 3, G7 is equal to 4 and G8 is equal to 2. So we can say the elements 5 and 7 of the set 5, 6, 7, 8 have the same image 4 in the set 1, 2, 3, 4. So it is a minimum function. So we can write G5 is equal to G7 is equal to 4. Therefore G is a minimum function since it is not a 1, 1 function so it is not invertible. This implies G does not have its inverse so our required answer is no. G does not have its inverse since the minimum function. This completes the session. Hope you understood the session. Goodbye.


What Lowe's is saying about, hey, we're buying material for the next two to three years because all the permits that are issued today, short of a war or an economic crisis happening overnight, we're still going to need these houses. There's such a shortage in houses a year from now. We still won't be caught up. Welcome to the Smarter Building Materials Marketing Podcast, helping you find better ways to grow leads, sales, and outperform your competition. All right, everybody. Welcome to Smarter Building Materials Marketing, where we believe your online presence should be your best salesperson. I am Zach Williams, alongside my co-host, Beth Popnikov. Today, we're talking to a builder on today's show, having an honest conversation about the state of the industry, what manufacturers are doing, that's working, what's not working, and then, in the backdrop of that, what's happening with labor in a way that you can, as a manufacturer, be helping. We are really excited to welcome Caleb Williams to the show. He is the president of TrueCraft Builders, Inc., and Caleb, you are a generational builder, right? Your dad is a builder also? Yeah, and when you calculate the amount of people we have in our company and the years that my dad put into it, my dad started the company 27 years ago, but he actually worked with another contractor as a lead site super for years before that. And so, he started the company, and I want to say about 10 to 12 years ago, I started running the office. I was actually involved in a work injury, and I had a back surgery, and so, it actually put me in the office, and about the same time, my dad started doing missionary work, and it kind of forced my hand to start working more and more in the office, and I took over running the company about 10 years ago, and we're still here today, so. You haven't looked back. Yeah, exactly, exactly. My dad put the foundation in for where our company is, and I have five brothers that work with me, and we're a family-owned business, and a lot of people say that. You have five brothers? I do, yes. You have five brothers. Do you have any sisters? Five brothers? I do, one.  She's the oldest, and yes, she's a treasure to our family, but she doesn't work with us. The rest of us work together, and so there's six of us boys total, and we all work together, and that's the best one thing that everybody says. How in the world do you work with family? And not only do we work together, but we are best friends. We cook out on the weekends together. We hang out, ride motorcycles. Our family is a close-knit family, and it has worked well for our company. That's really cool. Jinks. So, Caleb, tell us a little bit about your business. I mean, I was really interested to chat with you because I was looking at your website, and you guys do both residential, you do commercial. You've done some really big projects as well. Just give us a high-level overview of your business, the type of work you guys do, and the size of your tour. Sure, you're exactly right. We do both residential and commercial. We started off as a more residential company when my dad started it. We was kind of a service company, but a lot of his clients were the NASCAR community, and with Lowe's being in our hometown here, it was a natural fit. We started doing small outfits and things like that, and it turned into more and more commercial. And when you start that field, it's very hard to do both at the same time. And if anybody's, if you've ever done it before, you understand because the trades people that work in commercial typically don't like to do residential. And so, we've actually found a niche where we do a little bit of both. A lot of our residential clientele are previous customers that we've done commercial outfits for or a commercial building. And so, we create a relationship. And I'm all about having a relationship with clients that's a long-term relationship, that it's not just a one time you build a house and you never see them again, but it's a continued relationship, and that's what we strobe for. And it's worked out really well. Yeah, let's check on your side. You guys have, gosh, you guys, when you work with Lowe's as well? That's right, yeah. So, we have been working for Lowe's for about six years, about seven years ago, we did an outfit for one of their advertisement companies. And I got a call one day out of the blue. A guy from the facilities there asked, was we interested in doing a small project there at the Lowe's headquarters? And it was able to, we got our foot in the door with just a small project and we haven't left. And we've been really fortunate, like I said, that was six years ago. And we've renovated close to, I want to say about 800,000 square feet of space there at Lowe's at different times over the last six years. And we've been just, yes, it's been an incredible, incredible run. We're really blessed to have some great partners there at Lowe's. And not only that, but also getting to meet architects, designers and people that are lead trades people in the industry and people that you want to associate yourself with. So, we've been blessed in every one of those areas to be able to partner with the likes of Gensler and different companies and Lowe's 3P. And when you look back at your project portfolio and you see the names that some of the people that we've worked for, we've been really blessed. So, I know what our listeners want to know is what is it, what does it like to be a builder right now? What is it like to be a builder right now? Where it's like the uncertainty level, like if there was a meter, like I think it's probably greater than ever, like what's going to happen in the market.  You're saying that there's more permits out today than ever before. Talk me through what you're hearing from other builders you know as well as yourself. Like what are you guys dealing with today? What are some of the biggest problems you're dealing with? What's the market feel like to you all? Yeah, you're 100% right, Zach. It is so challenging to be a contractor today. And I know anything we do in the world, you really, since the pandemic started, it has made it harder. It's challenging. But as a contractor, I was actually walking the site this morning with our Lowe's Project Manager. And we were talking, it seems like it's almost a daily occurrence to get an email or get a call saying, hey, you're still didn't arrive or, you know, product that you ordered 12 weeks ago, there's a delay because of shipping. And that's a daily occurrence and it has made it extremely challenging. But at the same time, I've never seen growth like we're seeing it today.  And she was telling me that they have more permits issued on a daily basis in the last several weeks than they've ever had ever. And so when you look at that, it's really hard to judge where we're going with this and when the pullback will be because we always know that there will be a slowdown at some point. But it even has made it more challenging because the demand for materials, the demand for labor. And so it's a constant, you go home at the end of the day with, you've earned your pay for that day. Let's just say that. Is there anything you're doing, Caleb, that is helping you feel comfortable in this level of unpredictability? I know you mentioned, just as we were talking before the show that the market's never been hotter, but there's things that you're seeing that are happening that tell you this isn't sustainable, material shortage, labor shortage, cost of materials. I'm sure you're seeing other things as well. Is there anything you're doing to help you prepare for the future when things either cool down or something shifts? Yeah, that's a great point, Beth. One of the things that my dad found when he started the company is we never wanted to go into it. And I've seen too many times, I've seen it in 2007, 2008 where companies will go out there and start developments. They would start 10 houses and really get themselves upside down. And we've always tried to operate on a cash basis where we've never overobligated ourselves. So while our growth has been really fast in the last six years, we've also tried to manage it and never tried to overextend ourselves in a way that's going to affect us. But at the same time, you have to grow. As a business owner, you want to see the growth and you want to hire more employees. And so there is definitely a balancing act of watching the market, knowing where the market is today. And just like you said, planning ahead so that you don't get yourselves in trouble when things do slow down. One of the benefits that I feel like that we've had that's really huge in the industry, we've been able to work with some of the architects and what that has allowed us to do is to learn to plan 12, 18, 24 weeks and even 12 months in advance. A lot of the projects we're doing, if you're starting a project today, a large commercial project, you know it's going to take you six to eight months to design and permitting. And so you're planning months and years in advance. And so that in the residential side, I've taken that same concept and used that in the residential side. So when you're planning ahead and you've got a project starting, you're really starting to look at indicators way down the road. And obviously having a client like Lowe's and been able to rub shoulders with a lot of their people that really have key indicators in the market. And you can be in a casual conversation but bring up, hey, what are you seeing in the market? What are you guys planning for a year from now or two years from now? What are you seeing that's going to happen with the industry, you know, based on what we're dealing with now? And so that has really benefited us. Can you share anything? That's exactly what I was going to say. I had one of the Lowe's purchasing agents tell me when there were such shortages with plywood and things like that, they had to go out and guarantee, hey, we're going to buy materials from you guys for the next two years.  And I think we're seeing that and when you go to Lowe's or Home Depot or wherever you buy your local materials, there's shortages on stud, there's shortages on plywood, there's shortages on everything. And I've thought about it a lot of times, you know, either builders, I think it's something that's happening where there's a shortage of labor. Contractors are buying more today than they have ever bought before because it's a shortage of houses. And then you've got delays because of COVID, you've got protocols that have been put in place because of whether you're in a state that's really cracked down and will limit what you can do and how many people you can work. You have those combination of things and it's really, it's just drove the market crazy. And I think if you're not planning ahead today for a year from now, materials and things like that, you're getting behind. So they're hedging their bets, if I'm hearing correctly here, Caleb, they're hedging their bets saying, hey, we don't know what the market's going to look like, but if we want to survive today, we're having to buy material and just hope, I hope it's the wrong word here, but like expect that demand is going to continue. And like we're hearing that from other even contractors too, like I'm having to overbuy product in order to make sure my doors stay open. You know, we're, that's right, we're having, that's right, we're having to overbuy your, I told the guys, I said, we'll use the material somewhere. If you have a leftover store, we'll use it somewhere because it's so hard to get, whether it's plywood, sheetrock, whatever the material is. So we know we're going to use it somewhere. But the other thing, you know, no one, what they're saying as far as material and the other thing, they're not making any more land, you know, the land that's here is what you have.  It's probably not going to be there in two or three years. So if you see land today and you want to start a development and you have the backing to do that, you've got to go ahead, even though it might be two years from now, you've got to buy that land today. And so knowing what Lowe's is saying about, hey, we're buying material for the next two to three years because all the permits that are issued today, short of a war, or an economic crisis happening overnight, we're still going to need these houses. There's such a shortage in houses a year from now. We still won't be caught up. So by looking at that, you can say, hey, I'm going to go ahead and purchase this land because I know even if I'm not ready for another two years, I've got this land in place. I've got this development planned so that I can move forward at that time. And if you're not thinking that far ahead now, you're going to be behind. The days of going out and just buying a lot here and there, I think are there in the past, you've got to be planning for the future. And you can't wait for just months at a time. I can remember three years ago, if you wanted to go a lot, you'd row down the road and you see a lot of sign, you call about it and you get it under contract. And that's not the way it is today. So we talked about materials. We talked about land. What are you doing about labor? Or what do you think should be done about labor? Because that's not all on you, Caleb. You don't have to solve that. That's not a pain point for you, is it Caleb? Caleb is fine. Yeah. Yeah, we definitely, somebody needs to solve it. I think we had all subscribed to figuring out that solution. But you're 100% right. We've had several positions open here at our company. And we're needing to hire a project manager. We're needing to hire two site superintendents, a full-time estimator. And the problems we're seeing today, there's such a shortage. You're going to pay a premium in wages to get those people. And you want people to be happy. So I think the key thing, when you find them, you've got to pay them to keep them, offer them benefits so that they will stay and create a work environment that they love to be in. I've said for a long time that a contractor is only as good as the people he surrounds himself with. If you have trades people that do shoddy work, you're not going to stay in business. And so I think one of the things that needs to happen is we've got to bring, I grew up working with my dad. I grew up after school going and helping him. I started with just cleaning up the job sites. And I worked with my dad and another carpenter named Mark Brant. They really showed me how to build. And as a trim carpenter, and one of the things that I did right out of high school, I built a structure for a piano. And so I've always prided myself in being a trim carpenter. And I find today, you can find help, but they're not really qualified help. People aren't teaching today that old crap, the old way to do things, that something that's going to last. And a lot of the mentality today is I'll get it done, I'll get paid, I'll move on. Because it's probably going to get remodeled, you know, when the next family moves in, it's going to get remodeled anyway, so not that big of a deal. And that's the wrong mindset to have. I think there's still quite a few quality trades people and builders out there, but there are fewer and fewer far between. And so I feel like that we've got to bring that old, whether it be in the high school, we've got to bring back the trades people and learn a crap from a young age so that when they step into it, they're not doing so much on the job training and cost and won the general contractor dollars, but also cost them a lot of heartache for the homeowner, whether it be delays or things like that. And them not getting the product that they were hoping for. Kale, I want to show you something here. I found this chart from, and I'll make sure we link to this for our listeners here, but I found this chart from NAHB. What they did was they surveyed 18 to 25 year olds. And they said, Hey, what are your career plans? And then what they did was the 26% of the people that said, Hey, I'm undecided at this point. They asked them point blank, and this is what I love. They weren't like, Hey, what do you want in order to get into construction? They just asked them like, how much money do you need to make in order for us to convince you to get into construction? Like there's a need. We have opportunity. How much money do you want? And what's really interesting for listeners, we'll make sure we link to this on the show notes, 7% said $40,000, 7% said $50,000, 22% said $75,000, 21% said $100,000. And then the best part is that 43% of the vast majority of people said not interested at any price. You cannot get me into construction. Isn't that interesting, Caleb? Does that surprise you at all? It really does it. And, you know, I think there is a group that, with the dollar amount, the look of the dollar amount, they say, I'm going to need this much to make what I want to do in life, whether whatever it be. But I think the other part of that is, society has created this mindset that you need to go to college to make a lot of money or to make a good living to be successful. You got to have so many degrees and that hard work, because let's face it, construction work can be stressful. It can be labor intensive. And so we've created a mindset that says, hey, you've got to make a living as easy as possible and make as much money as you can. And by the way, you need to go to college and get four to eight years degree. So construction is probably not the way to go. And so we've created that mindset. And so there's fewer and fewer people willing to jump in the industry. I had my accountant tell me this probably a year ago. He told me, he said, 95% of my clients are business owners that have to do with the construction industry, whether they own a mechanical business, plumbing, electrical, general contractor, graders, whatever it is. He said, in 90% of them make more than I do as an accountant. He said, and at least 60% didn't even do one year in college. And so that tells you what the industry has changed so much. I know when I was growing up, and I'm just going to be brutally honest here, the majority of people that I worked with on the construction sites were either had a drug addiction, they had so much alcohol problems and things like that. And I remember even on Friday afternoon, seeing people crack open beers on the job, you know, but that's been a long time ago. That's 30 years in the career. And today is so much more professional. I think we're starting to see the mindset change a little bit about the construction industry, but we definitely have a long way to go. And I think we've got to start bringing up younger people to fill those roles because so many people that I know and that I've worked with my entire career, they're retiring and they're, you know, either they're going to sell their business or they'll just retire and nobody will pick up that mantle. And I think we've got to start bringing young people in. And I would like to start, you know, doing apprenticeships in the summer because really you need to get people right out of high school and start bringing them in and letting them sample and start paying them a good enough wage up front so they want to stay in it. Because I think, you know, too often we try to pay the least amount we can and you're not going to attract anybody by just giving them hard work and then not willing to pay them anything. I think that's a really important piece that you're mentioning, Caleb, that doesn't get brought up with addressing the labor shortage. Tons of people have talked about reintroducing, you know, trade schools or trade certifications in high school, which is a great idea. Absolutely destigmatizing the idea of physical or manual labor, not being as sought after or not being as worthy of a career as an office job or as worthy of a career as a four year degree. I completely agree with that. But I think it's really important to address, like if we want to destigmatize it, a great way to do that is throw money at it. It's really hard to say that about a person making 100 K a year. Like, you know, if you're making 150 K a year, that's a totally different scenario on what you're doing. I know, yeah, I just think that's a really, really great point.  And most people that I know, they're actually trades people in the field, you know, they're making 60, 75, 100,000. And business owners, if you can have a, you know, just a van, a work van and two guys helping you and you can be making upwards of $120,000, $150,000 a year. And I think that has to be, it has to be brought out that, hey, you can make a good living without going to, and nothing wrong with going to college. But I think so many people today go out and they, they'll spend four to six years going to college and they'll, they'll go home with $80,000 in debt and, and then start their career making $60,000 a year, where you can go in the construction industry. You know, it's backwards in four or five years, you can be making 60 or $70,000 a year and you don't have that debt. And I think that, that's another thing that has to be brought up in the day and age when we're talking about, you know, canceling college debt and all these things. Hey, there's other alternatives out there to make a good living than to go on and, and, you know, getting behind in life by taking on this massive debt. That's great. Caleb, man, thank you so much for coming to the show.  What's the one piece of advice that you would give them? You know, I would say get out, get out on the field and talk with, talk with the contractors. I remember about six years ago, maybe, maybe five years ago, we were doing a project with Stanley, and it was when Stanley was selling all their tape measures to Lowe's, and they actually reached out to our team and they said, hey, can we come out in the field, follow you guys around, and let's talk about our products and what we can do different. And I think that's a great mindset. Get out and talk to the builders, get out on the sites and see what you, what, if anything needs to be changed, what you're doing wrong, and also what you're doing right. One thing I think that I find interesting with, with contractors and the entire construction industry, we are pretty brand loyal. So if you, if you have somebody that's used a Stanley tape measure, for instance, they're probably going to use that Stanley tape measure for their career. And that's whether they're buying materials, you know, plumbing fixture, whatever it is. When you find something you like and it works well for you, you're going to continue to buy that. And so what I would say is get out on the field and talk to those contractors, talk to those tradespeople and have that one-on-one conversation. Hey, it'd be open to change and what we can do different. I think, I think that would, that would help. Awesome. Kayla Mann, thank you so much again for coming on the show. If someone's to connect with you, what's the best way for them to do that? Our websites, truecrabbuildersint.com, we can reach out to us that way or you can call our office and we'll be glad to collaborate any way possible. That's great. Kayla Mann, this is awesome. Appreciate your time. And for our listeners, if you enjoyed this content, make sure you go to venvio.com slash podcast to subscribe and get more. Until next time, I'm Zach Williams alongside Beth Hopney Galov. Thanks, everybody.


Hello and welcome. Johannes Bechberger will tell us about open source Java profilers. He likes baking, cooking, riding the bike and open source developing. I wish you much enjoyment. It's the wrong title, I just found out. It's a single thing I missed. Yes, I'm talking about unleashed the power of systems that break. So yes, without further ado, please turn down the lights. So originally I wanted to talk to you about this topic. Nightmares of a profile developer. Johannes Bechberger's Nightmarish Tales on Java Profiling APIs. But it turned out I didn't want to be that gloomy, so we started with an introduction example. You all probably know that the situation when you're like a students union or somewhere else, where they ask you aren't you a Java developer? And so like, yeah, I can do some Java and can you do also some Web development? And I'm like, yeah, that's how it happened to me in the Night of Sciences, when my this year that they asked me, hey, couldn't we have a web app where people could just answer questions? And I was like, cool, I could do this. It's just so people can answer like how questions like how all this the universe during the Night of Sciences, which is like a little bit like the Gulasch program, but fairly smaller where like professors can tell people about their topics in short talks. And we wanted to be JDPR compliant, so we didn't want to use any large platform like Google Forms because yeah, it's Nightmare. And so I just wrote a simple application. It has a client. It has an admin interface. It has also server and I thought like, hey, the server could just store every information in a JSON file. So how it works is that the client asks the server regularly, hey, do you have a question? Yes, cool. I presented to the user. If not, then no. And I thought, yeah, using a database is totally overkill because we've said premature optimization is the root of all evil. The problem was the event came and any guesses what happened with this tiny little application and 90 users coming to the website. Any guesses? Yes, it fell, of course. It failed miserably and you know, probably the feeling when you're on a laptop at an event hacking down, restarting the server and trying to hot-fig things and it doesn't work and it breaks and you're like, no, what have I done the last 10 years? What have I learned in my studies? Because I missed the whole code of Donikov and many other people also do. He wrote, we should forget about the small efficiencies in about 97% of the time. Premature optimization is the root of all evil and he further told us, yet we should not pass up our critical opportunities in the 3%. A good programmer will not be allowed into complacency by such reasoning. He will be wise to let carefully add the critical code but only after that code has been identified. So how do we do this? Profiles to rescue. When I ask people on the street and ask them, hey, they ask me like, what are you doing at work? I'm like, I develop profiles and they're like, I'm profiles. People like Sherlock Holmes and who will look into crime scenes or like Detective Heinrich for the people that grew up in Karlsruhe and remember this, this folk from the university film course who looks into such mysterious things that we are all the AKK cops are at. And so essentially what they look, they look at crime scenes and look how to solve them. And then they say, Jean, that's what we're doing here too with profiles. So consider on the top a pseudocode version of my application and below a flamethrower that builds up. And then it helps you to know how much a method is, how many times a method is called or how much time it takes compared to the caller methods. So for example, here the main method takes all the time, of course, because it's the main method. With no surprise it has a server loop because it's a question server. And the server loop calls a method handle question request because it works by just looping over all requests that it gets in, handles the question request and as client requests are much more likely than admin requests, most of the time is like dominated by this method. But of course there are also other methods and they take some time. And then how does the question request work? It's just asking, is the current question enabled? If yes, emit the current question. If not, emit something else and then wall you. And any guesses what both these methods are taking most of the time in computing or doing? Any guesses here? Yes? What? Yeah, essentially they parse JSON and that was great. That's good revelation and to quote Mario Fusco, I love frame graphs.  And so replacing all the presentation with a proper database in this case SQLite, because overall using something like MariaDB would be really overengineering, but SQLite is nice. It's small. You can use it easily and it worked. And the promise, I developed this application during Corona. So for Corona best event and now Corona is over. So I can never use this application. But again, maybe some other pandemic comes around. So what do you see here? Profiling is a good part of your toolbox of methodologies. It's like debugging, it's like testing and so on. So you should just put it in and then when you have a performance problem, you can log, hey, I know some profiling. Look at that. So who I am, I'm Johannes Bechberg. I'm sometimes talking about profilers there in London. I work at the sub machine team. We have stickers for later if you want one. It's a team at SAP with around 15 people where we work on OpenJDK and I work specifically on profiles, work on profile fronts, and profiles themselves and also profiling APIs. And that's one of the reasons why I'm standing in front of you because and why I wanted to initially name this talk tales of Johannes Bechberg's tales of a profile API developer, because I know a lot of profiling APIs. I worked on them. So what is profile actually? And I think the question had been asked in the beginning. And to quote the new hacker sectionary, it tells us that a profile, so the thing that we obtain by a profile is essentially a list of methods with the runtime so we can tune away the hotspots in it. And that's quite cool. We have, of course, different profiles and they have advantages and disadvantages. And that's the first lesson of, hey, why you shouldn't trust a profile, for example, an instrumenting profile. How an instrumenting profile essentially works is that it inserts instructions automatically at runtime in your code. For example, here we have our server loop. And it works by inserting at the beginning, hey, we're starting with the server loop method at the end. We're exiting this method. And that's quite nice. It works properly. It's easy to understand. The main problem is jits don't like it. And inline doesn't like it. Essentially, you're not measuring the system that you actually want to measure about the different system. So people came up with a different idea. It's sampling profiles. So the idea is here that we only sample what the application is doing. For example, we ask the JVN regularly, hey, what's this application doing around? And that's pretty cool. It's approximate, but it's usually fine because we only care about the hotspots in the application. And we don't disturb your application because we're not actually modifying any code here. And of course, before I can tell you the bad parts of profiling, I want to tell you what the profiler is. And for this, it's most important to really know how to write a profiler because in the beginning I also thought profiles are rocket science. They are these large, complicated systems. And then with some preconditions, I wrote a blog about it here. And it got on hack and use. And people liked it. So I'm giving talks on this. So essentially how my profile works, it's a Java agent. You attach it at runtime at the application. And it consists of a main class, of course. And this starts the profiler. And this profiler as a sampling profiler just asks the JVM, hey, what are you doing? Like the stack traces. And then sleeps in the samples and sleeps and does it in a row. And then we store it in some kind of store class. And so to start with the main method, the main class, what we essentially do is that we attach it at the start of the application run via minus Java agent. And that's quite cool. And then we can pass some arguments. The question is now, how do we create such an entry method? Because we could, of course, we have different main methods. Like we have the main method here. That's for normal applications. Java is a bit weird because this is also valid Java because you see, and in JK21, which is upcoming, you can also write this, but this really doesn't cut it. So what we have here are two different methods because we can attach a profiler at the beginning, like as you see above, but we can also attach it later. So we can only profile an application like after 10 minutes because we think all the setup code doesn't matter anyway. And for when we attach it later, the method agent minus call, when we attach it to the beginning, the method pre-main is called. And so the main class is pretty simple. It just starts a new profiler. It has to create a new thread for this profiler. And then it starts it. What's really important, please give your threads names. And please give everything that can be given a name like class law. Let's give it a name. It makes debugging so much easier. And yeah, then that's the profiler. That's pretty simple. The only thing that you have to care about is like, yeah, we want to print a profiler at the end using the edgehawntone hook. And then the profiling is just this. We asked the JVM for all stack traces. We store them and yay, that's a profiler. And then what is in these stack traces? What are contained?  That's essentially a method call. It tells us where in the method we're currently in, like the line number, where this class, the method is defined in lives in which file, and also method name, and also class name. And so we then store it in a store. And that's also not too hard. We have a store with some methods. But essentially the most important part is here, how do we come from all these stack traces that we collected here? For example, these are when we overlaid over the flamecraft when we collected them. How do we get to this flamecraft? Because a flamecraft is a pretty simple visualization. So we just pick one out. So we know that at some point in the application there was a call from main, then server loop, then handle, then current question, and parse JSON. So this is what JVM is currently executing. So we take this and we think, hey, we can just make a tree data structure because computer scientists love trace. So what we do is we start with a main node here on the bottom. We say, oh, we found this main node once. That's cool. We also know that the main method calls the server loop. Cool, because it has, currently. We also know that it's the handle question request, and so on. And that's the first one. Then we get another one because, again, we sample and we again hit this. We add it to and then we record, hey, I saw the main method. Yay. Now we call it twice. And I saw that again server loop was called and so on. So we mark it down. And now we have a different one that's missing all the parse JSON on top. So essentially, we're doing the same, but now parse JSON doesn't get a three because we didn't call it. More interesting here, we're having a new branch. And this just means we're adding a new branch here, there too. And the cool thing is we can turn this quite easily into flame crafts. And so we know we have at most four time slots. And then main method takes four time slots, server loop and the question request and current question takes just three time loops, time, time, units, and this question enables just one and parse JSON above. So that's essentially what flame crafts are.  And that's the cool thing. Profiles are pretty simple. You can write your own. I even talked at Milan with someone who said, after I gave this talk, hey, I want to use this in production. Like I want to use this in our system because we can't attach a profile. But I can essentially copy your code into my application. And we can run it. I don't know. It's a prototype, but you can play around with it. But that's of course not the reality. In reality, we have different sampling profiles. Because yes, we have instrumentation based profiles, but they are rubbish, as I told you before. They're not profiling the thing that you want to profile, your application. So essentially what we have here, we have external sampling profiles and built-in. Built-in are directly built into your JVM. And then we have, on the one hand, Visual VM and NetBeans, which is like the older ones. And then we have, then, Sun decided to create a for to analyze in 1992, which is essentially a profiling tool for many languages in the Sun ecosystem then. And then they added a library. Then they added a library function called async-get-call-trace in 2002 at the end. And we moved this three months later. And of course, this is the API that everyone depends on. Especially async-profiler. And they tell you later what this is a bad idea. And there's, of course, built-in ones called jkflash-recorder, which works similar ish to async-profiler and they share some code internal ways of working. And essentially all the modern profilers that you can buy are a mixture of JVR and async-profiler. So how do you obtain a profile? Bear with me, I'm coming to the juicy bit soon. Just so you know how to use it, you come at least away with something from a talk. You can just attach it. It's just a JVR agent. It's a native agent. So you can just tell the JVM, like, minus agent path, please start it and please also print a line graph on it. That's OK. With JDK5 recorder, that's not that they're similar. It's built-in. So we can just tell the JVM, hey, please do it. But should you trust them? And that's probably the reason why you came here. And I think no. So it's a nightmare. I stay down at the abyss and I'm here, one of the survivors to tell you. So it's really horrible. It's like, don't trust your profilers. They are just tools. Don't trust them because for once we have save points and modern profilers don't suffer it, but some of them do like visual VM. So problem is some profilers, only profil adds save points. A save point means that at different parts of a program's execution, a threat asks, hey, should I go into a save point?  We have less concurrency. The JVM can do things like garbage collection more properly and that's a cool thing and it's really safe and everything. The main problem is it just happens at some points in the program and they aren't that many, especially if you have large loops inline and such. It can take quite a while to get there, so that's not a trait. To have a visualization, I take here the sub machine logo and every time it gets bold and dark, it checks for a save point. So essentially what you're doing here is you're using save points and want to like profile. We're telling the JVM like, hey, please, please, please, stop when we're ready and then we wait and at some point in the future it comes to a point. But then when we're going asynchronously, what we eventually want is we're asking JVM just stop and it stops and it's quite fine. It's even more important when we have multiple threats and some of the applications that we typically write have multiple threats. Because then when we only want to get a set of threats of threat one, we have to wait till all threats are at the save point, which is quite nasty. And when we're fully asynchronous, we can just tell the threat one to stop and the other threat doesn't care. And now there are some different APIs, but one of the most used APIs that doesn't have to save point bias is asynchronous. In a nutshell, it's essentially you have your stack on your operating system. It has some Java methods on it, some C++ method on it, and it essentially turns and essentially reads your stack and returns to the wall use. But any guesses how many tests are there in the OpenJDK for this API? Any guesses, any numbers? It's well used. It's used by large companies and yes, it's kind of not really in there, but it's still, you can access it with DL Sim, but it's well used. Any guesses, any numbers? No, that's too pessimistic. That's too optimistic who wouldn't test their application so much. Yes, one, that's a good one. Yes, who? Yes. And is it a good one? Yes, it isn't a good one. Thanks. That's all pre and most pre and audience that I had today. It's my only talk today now. So essentially what this test does and in a nutshell, it asks in the main method, hey, if dollar check, if exclamation mark check, then please, then please throw and it doesn't work, and it fails. So essentially it boils down to the check method and of course, what did the check method test for? Only that the check method is called, but not that all the methods are called. Not that all the above methods are called. So take profiles for grain of salt and for example, in this example that I show you, it's not really complete because the JVM also uses a JT-REG framework which uses some reflection. So the actual test that we were running before is like this. So the main method gets access to a method test via reflection and then causes method test. The method test essentially calls the Java loop method that just loops around. And what we wanted to see in the flame graph will be hey, on the bottom there's the main method then some reflection stuff and then the test method and then the travel loop method. Any guesses what we actually got before I fix the bug? No, that's not right. But essentially what we got is just the first three frames and that's horrible, but our test in the OpenJK before I fix it just tested for the first frame up there like for the Java loop method and this is horrible because this bug was in for three years because it came in because there were some changes in another part of the JVM that caused this change to happen. That's not great and the problem is like your applications that you write daily they aren't tested as well as you want them to be and that's the same with profiles. But surely the people from IBM are doing it better so any guesses how many tests OpenJ9 has for the implementation? No, zero, of course because it couldn't be better, couldn't it? But yeah, that's nice and IBM people implemented but they're currently contemplating just removing it because there are other nightmare things about it. And there's the synchronous version so at safe points and there the answer to is ready. Yes, that's correct. So you see, profiling APIs are pretty, pretty good tested and of course as I told you before like safe one buys is bad. That's a problem profiles are really just software. Don't treat them differently. Don't treat all the tools you get differently. Don't treat compiles differently. Don't treat IDs differently. They are just simple software. So what should be better like tests should be better? Performance and accuracy could be better because there are some problems in this area. They could be safer. They shouldn't cause segmentation falls and there are currently some problems especially because like what I think Coltray's currently returns are J-Method IDs and they are pretty bad because there is no way to see whether a J-Method ID is still valid because when a glass, glasses can be unloaded in Java isn't there anymore and you access it then you can crash and that's not that great. And it's not designed for safety in mind. So once I started working last year at the OpenJK team at SAP and I wanted to just wrap the whole IPI of Async at Coltray's in our old third crash protection so it protects us against any crashes. And I then wrote it in a podcast and then I got an answer like hey, from someone called Markus Krönenhorn who is quite big in this community I should add that the crash protection mechanism was mainly put in place as a result of having to deliver J-F-R from J-Rocket into hotspot under the deadline upholding feature parity. The stack-working code was in barely bad shape back then. And I would add the bad shape didn't change through the ages and David Holmes, another person active in the community, also wrote like Async at Coltray's is a legacy mechanism that was created for one single unsupported purpose and it's like used by everyone and that's cool, isn't it? So you know the term test-driven development? Ta-da! I've even wrote it down so you can remember it if you haven't heard about it but dislike test-driven development rather like is RDD it's called rage-driven development so I'm so enraged by all this bad quality so yes there are some constraints but I'm enraged by the missing test so I want to crash all these bugs all these moths and really, really crash them down and so it's rage-driven development I'm currently working on a new profiling API called Async get-sectories that has tests, a lot of them and that is probably specified that's supported because I really dislike the current set of the ecosystem and so I want to come from this place where I'm just looking down into the business and being like oh that's high to this where I'm like can look at all the nice architecture of this building that's actually a tower that I visited last weekend it's a 40 meters high tower built only out of wood and some metal nearby the syrish airport if you have the chance to come there test-go dates, a really, really cool tower, it's cool to use it here in the presentation so how can we test of course we could do some unit tests but automated tests are better because I think like when you're doing unit tests you're always missing things so what I introduced was the concept of a laid oracle so essentially at the bottom we have to have some truth and our truth is we have instrumentation we create an instrumentation-based profile because this is like round truth the Java application really executed these lines and then we know because we're like inserting it as shown before and then we know we can use this to verify that getStackTrace is correct and then we can use this to ask whether us and getStackTrace is correct as a safe one because getStackTrace is only valid as a safe one and then we can on top just check whether us and getCalledRace is correct I implemented this and found some interesting cases it turns out getStackTrace and us and getStackTrace are pretty safe and pretty accurate just that any change in the Open2DK that we're doing might change this and might even prick the API so my API might not work on your own system and that's pretty terrible so then there's also the approach that I found a few bugs with is fuzzing through when considering the memory space I just picked two random memory traces in the application and then I called us and get us and getStackTrace with it so the idea is I'm just throwing random data to it and hope that it doesn't crash and it works I run it a long time and it didn't crash and that's cool so I'm really testing it so it doesn't crash so people can use it as a application and to deal with these kinds of issues I invented an experimentation technique that is stole sneakily from life sciences so essentially what we're doing is a profiling loop so in the beginning we have our mental model for example in my application in the beginning I have the mental model okay this is the application I know some information on it so I know how it works kind of and then I make the hypothesis in my case the application is slow because I'm passing a lot of JSON that might be the case and then I adapt the model as so long until I can really express my hypothesis and then I go further to the evolution when I have a working hypothesis and I evaluate I check whether it's really the passing JSON for example I use a profile and it's all what it is and if I notice I can go back to the hypothesis refine it or even go back to the model if I learned something new about the model so in the end I want you to know that profiles are great people doing lots of core work in this area but just don't trust them too much so they just suffer use them when ready but accept that they might sometimes have bugs and if you find them please write to your fellow open dedicated developer at the bug report and it's just open source many of these tools are essentially the underlying APIs are too so I'm a part of Northern Twitter I'm part of Northern GitHub you can find me, you can read every two weeks a blog post on these topics where I go down rabbit hole and so my team is at sweet sub machine so that was my talk and thanks for being here thank you Rach do you have any questions yes in terms of the open JDK are there problems with other languages because like for example is this tech traced or the flame graphs still as readable as if just profiling Java as compared to Kotlin or Scala or whatever yes with Kotlin yes it's as readable because Kotlin is a tiny gem across Java essentially with Scala of course you have a lot of things that Scala has to Scala has to go through a lot of hoops to get to the language that it is and so the bytecode is more complicated but profiler developers that work like on the profiling UIs can mitigate this they can extract usually all the information if they probably support Scala it's far harder of course because the mapping from bytecode to Scala code is far harder in Scala than in Java because Scala introduces lots of new functions, lots of new frames but I think it's still feasible but in the end profiling Scala code isn't really a point I thought that Scala was only used for performance in matter otherwise you use all low level Java code other questions none thank you really much if someone wants some stickers stickers stickers so have a nice day and have a nice week please another round of applause