If you want to get the most out of e-cam live's latest screen share green screen feature I have to say that one slowly because it's such a tongue twister Stick around because I got not one not two not three or four, but five videos for you and this is the first one Hello, welcome to take one tech. My name is Alec and in this video I'm going to be focused really on one primary feature of Well, it was actually the new beta. So I did a video about this earlier today the e-cam live version 3.9 Beta 4 but as far as I understand we are pretty close to this actually being the final release now So it will come to the sort of mainstream e-cam any day now now the video that I made about it I did feature some rather silly I suppose graphics in there But I did have a few people asking me exactly how I've done those now That's the purpose of these these five videos and it's to cover those because I did use some of these techniques in making that video But it's also just because I've been waiting for this feature to come For so long and I've asked about it a number of times and so have other people and now it's finally here And I'm just excited to actually get stuck in and start using this because I can see that this is going to Actually really make my workflow a hell of a lot easier with some of the stuff that I do with e-cam Not actually so much on this channel, although you can bet I will be implementing some of these little features as well Into the channel, but it's more for when I'm doing things for presentations and stuff like that and then also for creating course material and so on So it is all related to their green screen feature and it's the way that you can add now green screen or remove rather remove green screen from Screen sharing overlays. So that means that you can share Applications like keynote or like PowerPoint and then do some pretty cool things with them And so one thing that I did is a little example earlier was if I bring up the right window hang on Was something like this so That looks quite simple really it's bringing a little bit transparent type over the top and then it's zooming out and Disappearing over the top of me But the thing that's different about that than how I would have had to do that before is that that is actually just all done live win in keynote The other thing that's very different about it is it's all been done just on the fly in e-cam live There is no need for any separate software or any importing or exporting or all that nonsense before if I wanted to do that You could always make overlays in Video overlays in keynote with transparent backgrounds So then you could use things like have moving images and things like that on the screen The tricky one about that that I've just shown you is It's actually the the sort of holes if you like the words the letters that are that are transparent Those are moving on the screen and that's something that you couldn't technically do in keynote to have the the transparency Move with the letters if that makes sense you can do the animation of the letters no problem You can make the background transparent no problem But what you couldn't do is sort of subtract the letter from the bottom the background as it was actually animating so that is something that I previously had to do by Making the animation Exporting it and then bring it into another program to remove the the green part of the the little movie file And then import that into e-cam live And then if I wanted to make any changes It's back to the drawing board and start all over again Whereas now the way that I can do it is I can just do it sort of on the fly in PowerPoint Oh, sorry in in keynote, but you can do it in PowerPoint as well But the other thing that you can then do is you can just go and edit it So that last one said I really love the new beta or something like that Well, if I wanted to change that then literally I can just go and make it and change the letters to say something else such as Green screen screen share so you can see what I mean It's the letters themselves that are transparent and then this whole thing falls away at the front as well And it's really well implemented in my view in there in e-cam live So that is what these five videos are about and I'm going to show you how to use it So I'm going to come into demo mode just for a moment and show you basically here. I am I've got my green screen down below Or rather my green presentation because that's all it is It's just a keynote presentation Running in a window and then I've got screen sharing running in my e-cam live as an overlay Which was another feature that was added in in the beta 2 I think or maybe beta 1 I can't remember which there were so many good things in all of my car remember which came in which But in any case that is what is happening. So let me show you how to basically Construct this. I'm just going to come into this other Scene here for a moment. Let me just quickly get rid of a couple of these I did use this scene earlier for another demo. So I'll just delete those. So now basically all I've got in this scene And in fact, I tell you what let me just back up even further Let me just back up even further in case you weren't aware of these extra features I'll just tell you quickly. So what they've added into the the whole beta program is for this this sort of version so far beta One two three and four They've added a few things which basically changed the way that I will be building out scenes and things like that in e-cam live The first thing is Let me just show you what's in this scene. I'll just take my camera out for a moment So basically in the background I've just got a little animated loop running and I actually got this from pexels the website p exe ls I'll leave a link in the description. I also did a video all about pexels actually So I'll leave a link to that up in the top corner as well and then That allows you to basically just add a little moving image in the background And so we can add our camera over on the top of it. So there we go I've just added my camera as a camera overlay nothing new there We could always do that but what they did add with the camera overlays is they allowed you to basically come in and edit it So if I come into my camera overlay, let me just hide this other one for one second There we go. That's gone So we've got this little pencil here where you can actually adjust the settings And so we could always adjust the settings before here with the shape and so on by the way in the beta They did also add this shape the squircle slightly rounded corners, but it's got a sort of Multiple radius corners a little bit of a complicated shape, but it's just got a quite a pleasing look to it Let me just come back to the original one for the moment But what they added was corner radius So it allows us to add on a radius to the corners to get the UI out of the way So you can see what's happening and in fact come out of screen sharing for a moment out of demo mode So you can see how I'm basically just adding roundness to the corners The other thing that they allowed you to do was basically toggle which corners you want the radius on so you can see We've got this little sort of four-way grid here So we can just toggle those and so here you can see we've got square at the top Round at this bottom corner round at this corner and square at this bottom corner So you just toggle on which ones you want them on this applies to to image overlays and also to The screen sharing overlays rather and to the camera overlays and to text overlays not to image overlays So what they added in this Latest beta as well was this border So now we can just add a border and as you can see the borders getting thicker as I move the slider and as I move the slider back We can select it does actually have a readout I'm not sure if you can read that on the screen, but it tells me how many pixels. So there we go That is six pixels You can add on however big you want the the border to be around the outside Now what that's done is it's almost Negated the need for a lot of the sort of graphic overlays that certainly I used to make for my things before Where I had sort of like an image with a transparent hole in it And then that would go over the top of my video So that it looks like I was basically like this in a border So now it still looks like I'm in a border whereas in in fact I'm actually over the top of the background and it's just got a sort of white border around it Now that can cause or you might think it doesn't really But you might think that it could cause a few issues with the way that Certainly the way I used to do things before so when I had things fly into the scene for example They would fly in sort of behind the layer the top layer, which is the border So it appears that they actually come into the scene Whereas now there everything is just sort of quite flat. You've literally just got the border You've got the sorry the background and then you've got your camera over the top So you wouldn't think that you'd be able to do something like this Where if I just actually activate my screen sharing that what might help bite into My green screen overlay. I'll try that one more time So something like this that overlay has just flown in from behind the border It appeared to come from behind the border and then it falls out over the front of the border See what I mean? It's like it you would think that it's either got to be one or the other But if you watch that again, it comes in from the side and appears that it's coming from behind the sort of colored background over here So it looks like it's coming from behind that but then when I exit from the scene It looks like it's falling forward over the top of it. So There's a few little tricks that you can use to sort of get around that to or to give that effect And that is the sort of things that I'm going to be talking about here But first of all or in these videos rather, but first of all, how do you actually add the screen sharing overlay? Well, let me show you I'll come back into my demo mode This was added in in the beta as well So in one of the one or two, I can't remember which So we've now got this screen sharing overlay. So you simply add this in And then it's actually defaulted to The round corners. So I'll come in here. I'll just take the border off and the round corners for now So we've got a square. So this is just as you might expect with a Screen sharing overlay, but what you can do here is come into the pencil and I'm going to change it to my keynote So now it's showing just this green keynote slide So now I could add this up into the top corner and drag it across So we've now got basically a canvas all over my scene that I can use to do whatever I want on And if I just come into the little pencil icon again When you're in screen sharing, you've also got this green screen key here. So if I click that one Then basically the green disappears But anything else that appears on that screen would then come, you know into the scene So that is basically how I did that last thing Uh, which was something like this wherever it's gone It's disappeared now. Hang on. I'll get it back. There we go that one So you can see that that is basically just appearing on the screen here And then it's just appearing on here minus the green is as simple as that So for the benefit of this first video, what I thought to do is that's how to get the actual scene in I thought I'd go through the actual setup steps that you're going to need to do with Keynote and also with if you want to use powerpoint And then a few of the settings that you just need to get right from the outset And then we'll just do a little simple animation and then in the next videos I'll get into doing something a little bit more complex So first of all, I'm going to come out of my screen sharing and as soon as I do this or rather out of my presentation You'll see what's happening because now that I've exited the presentation in keynote You can see that I've still got keynote selected as the what I'm sharing the screen of And so now you can see all of the interface except anything that's green and anything that is green down here Is basically see through up here. So it's as simple as that So let me just hide this again. We'll just take this one out And what I'm going to do is I'm going to start with a completely new keynote slide One second. There we go. So we'll just come to a new keynote slide I'm just going to delete this text off here. We don't need that to start with And the first thing that I always do with keynote is get my sort of master slide setup Now in this case, we actually just want to set it as a green slide. So When you've got your new file open in keynote If you come up to the top corner and click on format And then anywhere in the slide if you've got nothing selected, which you won't because it'll be a blank presentation Just come to this one down here that says edit slide layout Now this is where you can basically create a whole series of templates that you might want to use for your presentation So in the future, I'll make a video to show you like various different templates You might want to have set up for different types of use cases for using keynote with With ecam live for presentations, but for the moment, I'm actually just going to delete them all because all we need is just one So I'm just going to start with one. We don't need all of this stuff We don't need that one as well. Let me just choose that. So now we've just basically got a blank slide Now the default for it is black But what you might want to do here or what you do want to do here is if you come over to the little color wheel and then you click on the color wheel And then what you want to do is select a quite a specific color of green not too specific It's within a range. I suppose but I'll leave a link to or rather I'll leave the hex code in the description because there is a specific color that is Generally used for green screens, although it will work with a range of colors. But anyway This is the the hex code courtesy of doc rock my source for a lot of technical information Is a mine of information So that's the the color that I'm using for this So I've set that as my background basically for my master slide. So if I come down to the bottom here and click done There we go. We've got a a blank slide This is now an actual slide and if I was to click add slide up here To create more slides then we can just pick that one and it would just always have that green background So we've just set that as a base because all that we're going to be doing in here is create using it for creating these These animations or overlays or whatever you want to call them that are going to be bought into ecamm live Now I should say that this is a little bit of a niche case I'm not suggesting that everybody for all of your you know live streams and ecamm live shows and things like that Would actually use this because it is more of a sort of presentation tool You don't necessarily want to have this running all of the time while you're just doing your general You know youtube live stream or something like that. However, there are specific use cases Where it can come in really handy and that's the sort of things that i'm going to be talking about in this video series as well so now that we've got our basically our blank green slides it's just a case of Thinking what we want to put on there. So it can be as simple as In fact, let me just come and get a an image because one of the questions that I got asked was How I had done something in particular which was Let me just come over here a minute Okay, I just paused there for a moment while I got to the right folder So one of the things that I got asked was in the little video that I did There was four cameras that just came dropping down and with a little bit of a silly animation to be honest So the way that I did that was I got a picture of a camera and I'll just drop it into the scene here So this is a one of the four cameras that is now compatible over usb or one of the new Compatibility cameras that is Compatible over usb with ecamm live and this is something that I mentioned in my previous video So I bought the image in now. There is actually an instant alpha tool built right into PowerPoint so here I've got this tool and obviously I want to get rid of the white around the outside So if I come over to the format again These are basically over on this side. This is where you've got all the different panels For doing various different functions and editing and things like that. So Over on the format and then if I come down to the image Click on image because I've got an image selected These menus by the way are contextual so it'll depend what you've got selected as to what shows up here But because I have got a picture selected Then I've got this little image menu and you can see here that I've got Instant mask which means that I could mask certain parts of the image and crop out certain things Or there is instant alpha So if I click on instant alpha and then move my mouse over the thing You'll see that my 10 if this shows up with pro mouse actually. Oh, yeah, you can see it I've got this little box that appears And this is going to be very easy to key out because it's all white So it's all a uniform color with nice sharp edges But basically you just click anywhere on the image and then what you can see is actually Let me just come back and show you this again. I click on here You can see this area around here Around the the top here. It didn't actually key out So if I click and hold you can see that perhaps see that there are some little Little bits that are not quite tight up to the edges So basically as you're clicking and holding you just drag down and it will just really tighten this image up So that'll give you a nice crisp edge now I'm going to release my mouse And that looks like it's got all of those little bits and so then I'm going to click done And there we go. We've got a nice little Transparent image of the camera and then it's really just a case of adding in some sort of animation So if I put this one over here, uh, and then let's just for the sake of it Add one over this side as well And then what I'll do is I'll just come to the little animate panel Which is where we can add animations. I've done quite a few videos actually about keynotes and how you can use it for animation So I'm just I won't go too much into this But basically with animation For in keynote terms You've basically got three ways to categorize the animation You've got build in which means that something's starting out of this out of the picture And then it's going to appear in some way or other You've got actions which are the thing is in the scene and then it's going to do something on the screen But not actually leave and then build out is the way that the item is going to disappear So for example, uh the build in let's just say we want these to build in So I could come to build in I've got them both selected and I'm going to add an effect There are lots of different effects you can add, but let's just add a drop So you can just go down and you can preview some of these so you can do things like this Uh, or you can do things like this one just appearing But let's just do drop because that's what I did on the first video. So those are just basically dropping down from the top So I'm going to click on that you can change change things like the duration of the the motion and so on So let me just do that And then also once you've got more than one thing and you've got things that are animating What you're going to want to do is a couple of things really First of all, you've got this build order down at the bottom And that's basically the order of all these animations. So each of these little stages of animation is called a build And the other thing that you might want to do is come over to here And then you've got this little menu up on the top left, which is the view menu And in here, we've got the navigator open. So the navigator is where you see all your different slides But you've also got this one, which is show object list Now I personally find it very useful to have this one open Especially when you start getting complex slides built up Because it allows you to see all the different objects that you've got on your screen So here we've got two identical items and they're both called the same because it's the same file name Well, if I click on that one, I can see that that is the one on the left. So let's just call that one Fuji left So I can rename that there we go. Let me select it So I'm going to rename that and then from now on I will be able to see exactly which one is which And then I'll come into here and I'll call this one Fuji, right? There we go And the benefit of doing that is whatever you call these objects in here is also you'll notice it's changed them in here So it makes it very easy to understand Which one we're looking at Now when it comes to the build order, there are a few different things that you can do in here Uh, so notably it is the the start time. So when is this action going to be triggered? Uh, so the default is usually on click. That means that when you click the mouse Or maybe a little presentation clicker or something like that It is going to advance to that action. Uh, so That is the sort of default behavior bearing in mind that this is presentation software ultimately It's not intended necessarily to be used for Uh, you know movie Motion graphics and things like that. Uh, but we can sort of get away with quite a lot with it So I want these to drop at the same time. So rather than be happening on click I want the first one to happen on click so that I know when it's going to happen But for the second one, uh, what I'd actually like to happen is I want that to happen at the same time So I'm going to click on this start and then say with build one So you can see the builds. I've all got numbers. So here's number one and here's number two So now those are both going to happen at the same time and if I come to the preview here It's going to show you they both drop at the same time So that is a build in action. Uh, you can also do An action during the uh, that uh, well the thing is on the the uh, the scene rather So let's do something with these now that they're actually in position So what I can do here is I can add an effect and let's say we want to uh, move this one So we can move it that way So I could actually have it move back up to this corner, perhaps like that So that's showing you the starting position and then this line is showing you where it is going to go to You can also by the way, you can grab this one in the middle and you could also make it go like around like that So that is the path that it is going to move to go to get to where you want it to go Uh, you can all do all sorts of different paths as soon as you start creating a path You can make it go whichever way you want. So let's uh, leave that one like that Meanwhile, let's have this one, uh start doing something else. So you could have it maybe doing another action Maybe you might want it to rotate something like that. So let's have that one rotate by 360 degrees Or what's one rotation might be a better way of saying that Uh, so that's going to rotate while this one's moving. This is all totally silly and ridiculous I'm just showing you the sorts of things that you can do with it. Let's have those two things happen together Um, then we want to them to build out as well So let's have this one have an effect and let's say that that one is going to burst into flames And then let's say that this other one over here is going to go by a different way So let's have a look at what we can do with this one. Uh, this one. We're just going to have that one as a Uh, let's have a little look crumble. Let's try that one. There we go like that So we've got those effects now and I'm just going to have the two build outs happen at exactly the same time as well So I'll have those happen together Uh, and now I'll just run that as it is. So if I just play this one now And in fact, let me just play it in window. I'll come back to what exactly playing window is in a moment But what you'll see is now these should appear on the screen with a bit of luck So now if I advance the slide, there you go. The two cameras have moved in And in fact, let me go full screen for this so you get the full benefit So the two cameras have moved in and now if I advance to the next part, which was the move part You can see that that one spins while that one moves over to the other side of the screen And then let's try the next one where one of them is going to crumble and the other one is going to burst into flames So that was all just done in a matter of moments in keynotes And yes, it did look pretty ridiculous, didn't it? So I wouldn't necessarily recommend you do professional presentations in that manner It's just to give you a bit of an example So let me come out of here now because the one part that I didn't show you in there Is although I mentioned it in my previous video, you may not have seen that one Is how you actually get keynote to play in a separate window rather than the full screen Because the default in keynote is to play in full screen Well, the way that you do that is you come over to here And where it says play at the top You've got play slideshow, which is the default and that's what normally happens when you press the play key But you can also go to play slideshow in window and when you click on that one That's when it comes into this window form rather than a full screen You can add this to the toolbar as well So you can come to the toolbar up at the top and click right click and click customize And then you'll have this play in window button that you can add to the top So then when you are ready to play your slideshow You just get ready like that and there it is So let me just come out to there for a minute go back to that slide Try it again And then you'll see what's happening as I actually play the slideshow It is going to appear on the screen. So there we go. We've got the cameras moving down Now they're moving over the screen and then now they're disappearing out So that is basically an introduction to how you can use the green screen function to make all sorts of animations and things like that That you can create in keynote and whatever you can imagine in keynote Actually appear in the ecum live itself So what i'm going to do is i'm going to leave it there for this one This is just a sort of slight introduction I should I say to the process because what we haven't covered there is sort of more technical things that you can do And how you can use masking and things like that to make things look a lot better and a little bit more professional Than that little effort of a demonstration So I will be making these videos sort of consecutively So over the next, you know a few days, I won't be sitting down and making it right now. I have a little break But over the next few days, I'll just release these five videos Which is how you can get the most out of this feature If you found this useful then do let me know in the comments and in the meantime I'll leave a link to some other ecum videos over on the bottom right. So have a great day and I'll see you soon


I help people get jobs. So to start with, the word selenium is so closely associated with test automation. So the moment someone says selenium, the next thing that comes to mind is test automation. So at Indeed, we try to think of selenium a bit differently to help people get jobs, more specifically on smartphones. Let's see how. So before we do that, let me tell a bit about smartphones. We all have seen the rise, the advent, and I guess we are all immersed in the era of smartphones. This statement is not an exaggeration. There is literally an app for everything these days. Now at Indeed, we have an app for job search and we were trying to understand the penetration of mobiles in the job search. So this was the graph we gathered for the last one year and if you look at the percentage of traffic coming from the mobile, like percentage of traffic of people searching for jobs, the traffic from mobiles is like 2% higher than desktops. Right now it's around 52%, whereas the job search happening from the desktop is around 48% and it's not just searching for jobs. People are also actively applying for jobs. So these are some rough numbers we were trying to get over the last year. So around 200,000 applications are being submitted through mobile phones and around 51% of the total applications being submitted on Indeed are from mobile. So all of these might be a bit of surprise because we think job is a serious thing. Like why would someone apply for a job from a mobile, right? But mostly in the United States where the jobs like truck drivers, people working in restaurants and people doing part-time jobs, right? So there is a huge flux of jobs. People change jobs very often and it is not an expectation that all of them will sit in front of a computer and search for a job, right? These days with mobile phones, smart phones and 4G, people are inclined to search for jobs on mobile. So that's the reason why we are seeing such a very high numbers for searching and applying for jobs on mobile phone. Now let's look at the way people used to apply for jobs. Like even I apply for job like this. I sit in front of a computer, I open up an employer website and I open up the career section, I browse through the jobs, I select a job and then I start the job application. But the scene is changing. People want to apply for jobs from mobile phones. Well, that's an option always. Now what's the problem with this, right? So there are two typical problems with employer career portals. The first one is they're not mobile friendly and the second one is they're slow. These employer portals, like this is a real example of a job description page if you see it on a mobile phone. So the problem here is these employer websites were designed keeping desktop in mind. They were not designed keeping mobile in mind and I'll make it even worse. So this is a gigantic form that people are supposed to fill in when they apply for a job on mobile phone and the experience is atrocious if you try to apply for it. So coupling to this, there is this other factor that these employer websites are kind of ancient and they take very long time to load. I have not picked up a 99 percentile latency or something to make it look worse. The mean latency of an employer page, career's portal page is typically around five to eight seconds. So these are the two problems that job seekers face when they try to apply for jobs on mobile phone. So indeed's idea was to fix this problem. They wanted to design a mobile friendly website. That's the core problem that they wanted to solve. Now to translate this problem into a software requirement, so indeed wanted to fill this magical box called mobile which had to generate a simple, fast, easy to use mobile friendly website from a heavy, clunky, slow employer website. Now there were two options for us. The first natural option that comes into mind is use an API. APIs are fast, well maintained, they're versioned, right? But APIs are also costly, right? And there is a bit of learning curve when you have to learn an API in its entirety. So since we wanted to start it as a prototype, we didn't choose the API option. Instead we chose the second option, scraping, right? So scraping, yes. On its face, it sounds, it's slow. Scraping may be flaky. Scraping may lead to un-maintainable code. Yes, there are all these costs associated with scraping. But at the same time, scraping is quick to start with. So we decided to go ahead with scraping. So now the problem statement is a bit more refined. We need to develop a system that generates a mobile-friendly website out of the employer website and use scraping as our methodology, right? Now, before we do that, we need to be aware of certain responsibilities that a typical careers portal generally fulfills. I'll go to each of one. So the first one is about jobs. So these careers portals have numerous number of jobs. We have to make sure all of those jobs are also present in our new website. The second thing is when you apply for a job on the careers portal, you'll be taken through a series of pages where you'll be asked about your personal details, your education information, your previous work information, and certifications, if at all you have. So you'll be taken through a list of pages and each page will have certain set of questions. So you are supposed to answer all of these to completely submit the application. So you should basically emulate this behavior as well. The second thing is any website that does a serious job has to generally tends to authenticate people before they do something. So employer websites are also not an exception. So the question is, do we want to set up a new authentication mechanism? We said no because authentication, we didn't want users to remember one more password. So we try to leverage on authentication mechanism of existing employer website. Now when we say that we leverage on existing system, we should act kind of like a proxy between the job seeker and the employer website. When you say you're proxying authentication, privacy concerns come into picture. There is this big three-letter word PII which makes people scary, right? So the third thing is applications. So this employer website captures a lot of information about the candidate. We should make sure we capture all the information about the candidate as well. And again, since we are gathering some information about job seekers, we should respect their privacy. So that's a factor we need to take into account. And you also, right now, you are also providing a solution that claims to simplify the life of job seekers. Now in case you as a solution fail, you should provide job seekers to fall back and use the existing solution. So falling back to an existing mechanism was another design criteria that we should be sure that we are emulating. So the problem statement now gets a bit even more clear. Generate a new mobile-friendly website out of an employer website. Let's use scraping and you have certain responsibilities to also be emulated. Now with this, let's start, let's start building it together, right? So we are saying that we want to develop a mobile-friendly website, which means we want a web application, okay? Let's start with a web application, right? We've added a web app. Now web app is empty, right? Now the job seeker comes to the web app, finds nothing, he can't do anything. So what's the next thing job seeker wants to do on a web app? He has to find some jobs there, which means we need a database to store jobs. So let's add jobs, a database for jobs. Now, again, this database is empty. The web app is empty. We need to fill in some jobs. So we need a system that can get all the jobs present in the employer website and put it in the jobs database. So let's designate a service for it. Let's call it job service. Now, the role of job service is to get the jobs present in the employer website and put it in the jobs database. For this, we use Selenium. So this Selenium code basically opens up the employer website, types the job URL, extracts all the metadata about the job and the entire job information we talked about and puts the job in the database. I'll give an example of the output of this Selenium code. So for a specific job called installation technician, we store the URL of the job, we store which the current page it is, we store the actual question that we're trying to extract called first name and we also try to extract what type of widget it is. But in this case, it's a text box and we try to identify this particular text box. We say that the locator is first name, like yeah, we say the locator type is x path and the locator is first name. Now, why do we need to gather information about the widget type and the information about locator types? So this information has to be shown on the web app, which means web app has to understand what type of a widget that it has to render for this field called first name. So web app sees the widget type to be text box and it renders a text box and with a label called first name. So we are good here, right? We extracted all the information from the jobs, like from the employer website and put that in jobs database and now the job seeker is able to browse through the jobs. Now he selects a given job and wants to apply. Now when he wants to apply, just like the employer website, we also have to perform the authentication, right? So which means let's designate a service called Auth service for it. Now the role of Auth service is to take the username and password from the job seeker and validate it against the employer website. Here again, we use Selenium to do this. So this Selenium code basically opens up the employer website, clicks the login button, takes the username and password that the job seeker gave and tries to go through the authentication page. So either the user will be successfully authenticated or the authentication fades. So the Auth service conveys this information to web app and web app returns this information to the job seeker, right? So the moment a job seeker successfully authenticates, now he will be taken through a series of pages where he's answering questions like, what is your first name? What is your last name? What is your previous work experience? So he fills all of these pages and in the end the application is completed. Now to save the completed application, we need a database. So let's add a database, right? Now we have a bunch of applications sitting in the database and these applications have to be written to the employer website, right? Let's designate a service to do this. And by the way, the moment the web app is putting an application into the database, it fills the answer that the job seeker gave for the first name question. So the web app's job is to show all these questions to the job seeker and collect his answers and save it in the database. Now, yeah, as I said, we have to write all of these applications back to the employer website and the service that is supposed to do this is apply service. Again, apply service uses Selenium to do this job. So this way, we have kind of developed a parallel website for an employer website. We kind of take the entire job information, application information from the job seeker and we rewrite to the employer website. So now the question is how does apply service try to write to the employer website? It has, if you see this, it has the entire information to load that particular job on the employer website and to specifically answer the first name question. So this is how the apply service goes through every question belonging to that application trying to answer every answer on the employer website. Yeah, so if we do all of what we talked so far, perfectly or near perfectly, that's how the site will look like. We call these sites as mobilized sites named after the product. So the site that was looking like this, like we'll develop a parallel website that looks like that. And you remember the gigantic form I showed you that will translate into nice cute little forms like this. Now, so we have solved this problem for one employer. Now what do we do to solve this problem for many employers? Obviously we don't want one solution for each employer. We want a single solution for all the employers, right? So this is the system that we have developed so far. Now we need to extend this to multiple employers. When you say multiple employers, there will be multiple blocks of employer websites there. Now just the natural process of extending this model is for every employer website you want to implement, add one more chunk of Selenium code, right? So that will change the design to this where each block S here is a Selenium code, right? Now this is where a bit of software engineering comes into picture. When you do a same thing, similarly many number of times, right? Then there will be a need and there will be an opportunity for you to extract out that part that does not change across all these. Like these Selenium codes will obviously be sharing a lot of Selenium related utilities and a lot of general page navigation patterns, right? So we try to extract out that part that is common across all of these Selenium codes and we call it back-end framework, okay? Now still there are certain parts that cannot be, it cannot be obviously 100% generalized. There'll still be information about each client that you have to maintain. For example, the locator for the X path for the first name question for an employer one will definitely be different for employer two, right? So that's the part that is still unique for each employer. So all of those parts go into these blocks called C, which we mean configuration. So you have a framework and for each client you're trying to implement, you will have a specific configuration for the client. Now we do the same thing for back-end framework and we do the same thing for, do the same thing for app service and we do the same thing for apply service. Now this is how we extended mobile for many employer sites. I am not supposed to reveal statistics. Yes, the CCC part is manual for, it depends on the website. Like for example, a lot of these employer websites share a common back-end called applicant tracking system. So if four or five clients use the same applicant tracking system, the locators of each client will be identical. They'll have some standard prefix and they'll be an unique identifier. So we do it manually for one client but we try to understand the pattern and for all the subsequent clients, it's like split second, right? So, right, this is how we extended for many employer sites and I can't share the numbers but this model really works. Okay, so some of the challenges I faced, like we faced as a part of this problem, I'm only highlighting some and neither am I going to explain the solutions for these challenges but I'm trying to give you a glimpse of how complex it can get. The first thing is staying in sync with the employer website, which means, so these employer sites, it's not that you extracted jobs from the employer site, you build a client and you go home, right? These employer sites keep changing. There are more jobs that are being added. Existing jobs can be removed. The employer might decide to add a new page in the application process for a given client or there may be new questions added for the same job. So we need to understand, A, when this change has happened, B, what's the exact change and C, how soon can we get that change into our system and into our database. So this is a constant process. You need to have a separate pipeline for this to automate all of these. So that's the complexity in staying in sync. The second thing is we have written Selenium code for everything, for authentication, for writing the application to the employer website and for scraping jobs. Now what if the employer has put a capture on the signup page? How do you do? That's a challenge. And the third interesting thing is about dependent questions. If you have applied for a job on a careers portal, you would have come across a question like, do you know anyone working in this company? It'll have a radio button yes or no and the moment you say yes, you'll be popped, a new question pops up saying, enter the employee ID. So the automated script has to understand, the automated script written in Selenium has to understand that by selecting different options for a given question, your DOM may change and it has to be trained to understand this dependency. So not only just understanding the dependency, it also has to save this dependency in a specific format that your web application also understands. So your web application also has to again, ask the same question and if the job seeker answered yes for that question, it has to show the second question, right? So this is dependent questions and it gets even complex when the page flows are different. And let's say if a user answers, if user selects a particular answer for a question in page four, suddenly page five appears, right? So your Selenium code has to understand that this is a possibility and B it has to be trained to understand that this is a possibility and store this again page level dependencies also. There are other challenges like, how do we test our implementations? Because we don't operate on staging environments of clients, we directly run our application against original employer website, which means we cannot spam, like create a test applications exhaustively to verify our implementation. So that's a challenge. And some of the current work that we are doing is we want to reduce the drop off rate. Like for example, if you've seen the slides, it's, let me go back, right? So let's say if this particular personal information form itself, the user feels that it's long enough to answer, then he'll drop off obviously, right? So we want to reduce these sorts of dropping off behavior from users. That's a challenge we're trying to solve. And the other thing is reducing the amount of time we take to implement a particular employer. In the beginning, when we started this, we used to take 30 minutes for implementing an employer, sorry, 30 days for implementing an employer. And for certain kinds of employers, we were able to reduce it to 10 minutes. But there are still certain complex employers where employer websites where we take around like 15 days, 20 days to complete the implementation. So reducing the implementation is a problem that we still have not solved perfectly and we're trying to solve it. That's pretty much it from my side. And since, if you see, there is a lot of selenium happening here, right? And all of this requires you to create browsers, which means we need a browser service. So see, we'll be talking about the browser service that we have set up in the next part of the conversation. I work as a software engineer for Indeed. If you see the mobile target set, so we have three components. So job service, web app, and apply service. So all of the three components have one thing in common. They all need a browser to perform their action. So at the scale at which we are operating, we need hundreds of concurrent browsers at a point of time and thousands of browsers per day. So for two designs such a service, we came up with a list of requirements. So first thing, it has to be a HTTP browser service and then it has to be, it has to support the returning sessions, which means for example, if you take the scenario of CAPTCHA, where we need to send the information of the CAPTCHA image to user and user takes the user answers it and we take that user answer and submit it into the employer website to check whether the CAPTCHA is correct or not. So we might need a session which can extend across multiple processes and we should be able to purge expired sessions. So every session will have expired time and our browser service should be able to kill those, kill such sessions and there should be no single point of failure in our service. And we need state and health APIs to reflect the state of the current browser service and like any highly scalable system, we need logging and real-time monitoring to maintain it in a better way and browsers tend to release memory. So we wanted to make our browser service resilient to memory leaks and it has to be horizontally scalable. So with these set of requirements, so the immediate option that we had was the native Selenium Grid, I mean, which is the open source Selenium Grid that we have available on the internet. So it has, so I'll talk about the architecture of the native Selenium Grid first. So it has a hub and it talks to nodes, nodes are the places where we execute the, where we run the browsers and client code talks to hub. So hub is a kind of single point of contact for all the nodes. So it looks good, but it comes with its own set of limitations. So what are those limitations? So hub is a single point of failure. So if it fails, we lose all the access to the nodes and it's not trivial to add logging and monitoring, which is a important requirement for us and we can't restart node gracefully. So let's say for some reason, if some node is running low on memory, we can't restart it because some users might have some active sessions. So it's a very critical need for us to drain all the active sessions and then restart the node. So this is not available in the native Selenium Grid. So what we did was like we came up with our own implementation of Selenium Grid, which is mobile Selenium Grid. So where we have a load balancer and behind the load balancer, we have set of grid servers. So client code talks to grid servers like ask CLB and ELB will delegate that request to grid servers and grid servers will in turn select the grid nodes based on the availability of the browsers and we have something called as grid scheduler, which does some periodic actions. So I'll talk about the each individual component in detail now. So the first component is a grid server. So this is a web server bundled with JDK and it has a state and health endpoints. So which reflects the state of the browser service and we have a node endpoint, like whenever there is a node, it registers itself with the grid server. So for that, we needed a node endpoint and for example, if we need a browser session, Chrome browser session, we have a session endpoint where we send a request to grid server asking that I need a Chrome session and the grid server will understand it and delegate that request to the appropriate node. So what grid server does is like, it maintains an inventory of nodes and sessions, like whenever there is a request for a session, it stores it in the MongoDB. So every grid server is a stateless, like even if one grid server goes down, so it doesn't impact the sessions which are running on the nodes. So the important difference from the native selenium grid is like, here, grid server is not the single point of contact. Once we create a browser session, we pass that information to client code and client code will talk to node directly. So in the grid node, so grid node is a common piece of component from the native selenium grid and it runs a selenium server and which creates the browser based on the grid server request. And so the problem with grid nodes is like, we don't have a proper monitoring mechanism to monitor the health of the node. So for that, we wrote some set of shell scripts which monitor the node health and if some metric is going low, like if the available memory or available sessions is going low, then we restart that node and we face this problem of unresponsive browsers. So when we create a session, there are some cases like where browsers can go unresponsive. In that case, it won't respond to our selenium APIs. So the only other option that we had was to kill those unresponsive browsers. And the third component in our architecture is grid scheduler. What it does is basically pings the nodes periodically to check whether they are active or not and if something is not responding, then it just removes that from the fleet of the nodes. And it checks for expired browser sessions. So if some browser has crossed the expiry time, it'll just try to kill it. So we host our grid in AWS. So right now in production, we have five grid servers and 25 grid nodes. And per grid server, we use C3.Lodge, which is two virtual CPU units and then 3.5 gigs of memory. And per grid nodes, so grid nodes are usually the heavy lifting machines which run the browser. So we needed a high config machine there. So we have 15.75 gigs of memory there. And this is the comparative analysis of the native selenium grid. So if you see the top three hard beat returning sessions and the purge expired sessions, these three things are common in the native selenium grid and mobile selenium grid. So in the mobile selenium grid, so we added state and health APIs. So there is no single point of failure, which means even if a grid server goes down, nothing will happen. So, and we since we wrote the grid server by ourselves, so we added logging and monitoring. And whenever there is, we detect a memory leak, we just put the node in the maintenance mode so that won't affect the existing active sessions. And we have a graceful node restart. So we train the active browsers in a node and then restart the particular node. And in our case, so grid servers and grid nodes are independently scalable. So in case of native selenium grid, we can't extend a hub. So in this case, we can extend both hub and node. So what are the, like no system is perfect. So these are the current challenges that we are facing the rogue sessions. So sometimes browsers don't respond as expected. So though this is a small percentage of browsers, this is still a concern for us. And we face this problem of gateway timeouts. So sometimes if there is a huge load on nodes, session creation takes, if it takes more than one minute, then we respond with a timeout error. So this is still a challenge for us. And these are the grid traffic statistics. Like in a minute, we handle up to 625 requests per minute and the maximum traffic per day is 89,000 requests. And in the last 30 days, we handled 2 million. And in the last one year, it's 8.7 million. So the future work. So we wanted to implement a browser pool. So where we have a pool of browsers and when out there is a browser request, we allocate a browser from that pool. So that, I mean, if there is any possibility of failure in the creation of a browser, this will avoid that. Autoscaling Selenium Grid. Like right now we anticipate the traffic and we make sure like there is sufficient capacity in our grid, but it's in case of any unusual spike. So auto scaling will help us. So this is one of the future action items that we have right now. And this is indeed engineering blog and talks. So where it shares it's the technologies and systems. So if you're interested, please go through this. Yeah, thank you. Yeah, so yes, that's a cooler option that that would work out very well. But in general, these employer websites are, they do not invest so much in this part. So setting up a web service and adding the API is necessary for us to get all the jobs or submit an application. Yes, it's an option, but most companies do it when they want to do it, not when we ask. Okay, yeah. So for you, like when you're running Selenium Grid, some of the nodes die, right? How do you identify those nodes when it dies? So basically, we try to open a socket to that particular question and if the socket creation fails, then we treat that node is not responding, then we remove it from the free one. Okay.  So that's the current, like that's the future work. So that's not in production yet. So we don't know how it will work. So that's still in still in recent phases. So I mean, that's one thing that we feel like it may address some of our current problems, which very well, but yeah, right now we don't have any stats. So we hope to mitigate some of our problems. So the current model is when some, some client asks for a session, we start creating the session then and there. So this process usually takes somewhere between like five seconds to 35 seconds. And if it fails, it fails very late. Like it fails after a minute. So we want to like, and also we need to understand that, for example, if the job seeker was trying to sign in, we have to create the browser then, right? Which means all the time we take in creating this browser, the job seeker is sitting there waiting for his authentication to go through. So we wanted to reduce this time. And hence we wanted to have browsers readily available. So when the job seeker comes to authentication page, we take a browser and he fills the answers there. We immediately, so there is no overhead of creating the browser then and there. It's kind of like we maintain a cache of sessions. So that way we try to reduce around two to three seconds of job seeker's time, which is quite precious on a mobile. Actually, your example is very good. You are using certainly not for testing for other purposes. Right, correct. Yeah. Now we also use like similar to our production server. Some of the like main trees. And usually the first option we had also was like, using some crawling libraries, but we also want to write the information back, not just read. So that gets a bit trickier if you want to use crawling libraries. We also use Phantom.js as a part of this whole thing. It's not just only Chrome's and Firefox's. We found Phantom.js offering us good performance as well. The amount of time you take to boot up browsers is like quite small with Phantom.js. Come back to Selenium and Chrome where you have to set up a new process all together on the OS. Phantom.js, you use Unix everywhere? We use Unix everywhere, yeah. When we say it's great, we check for memory. So if it goes with the available memory is lower than some x percent of the total, like some threshold. So we keep that node in maintenance mode. So memory is one such thing and number of sessions. Like if the number of sessions available on a particular node is less than particular threshold, it goes into maintenance mode. So right now, the criteria are the low memory and low available sessions. What do you mean by node circulate exactly? So actually we, since every node by itself is a Selenium server and Selenium server exposes this URL like WDE slash sessions, we actually query that API from the maintenance script which is written in shell, we parse that JSON response. There is no guessing here. Since we know the capacity with which we registered this node with the server and when we query this API, we get the current number of sessions being served by the Selenium server. Then we just do like if we are crossing around 85% of the active availability of that node, then we think that it's a good time to restart the node. The second point that you mentioned is, why don't we add it as a part of the servlet in the node? You can add it as a servlet in your Selenium server, right? The problem with that is if we added that, we would not have needed grid server at all, right? So the question is it was easy for us to, like the Selenium servers keep changing. You get a new version of jar every time, right? So every time there is a new version of jar, you have to basically change your code or verify your code, but instead, it was easy for us to actually query the output of the Selenium server through HTTP API. Yes, exactly. The grid server's role is just a bookkeeper of monitoring the absolute capacity that all your nodes can offer and their current capacity and it tries to identify which is the best candidate node that can serve a new session. It's exactly what you said. What we noticed was like, even though there are a few browsers running on a particular node, so like let's say we are using a 16 gate machine. So even though the number of browsers running on a particular node is not very high, we have seen instances like the memory is going down. So I mean, when we say resilient, I mean, we wanted a system where memory leak should not create a crash of node. So whenever there is a memory leak, we detect it and we stop any new traffic to that particular node and all the current sessions will still be able to continue to run. So also when you say memory leaks, you're basically talking about memory being bloated at the node level, right? Right, right. So I have seen mostly that is basically due to, let's say either your node is basically running for a really long time without being killed or within a short span of time, it basically starts supporting a lot of browser requests, especially if it is Firefox. It tends to, I mean, I have seen that it basically tends to eat up a lot of memory. Right. Okay, yeah. Thank you. At the current rate, every node at two-hour traffic, every node goes to a memory hungry position in around three to four days. So a node gets restarted every four days and yeah, it's primarily due to your browser sleeping memory. Any other questions? So in your presentation, you have mentioned about a single point of failure and how in your project you have avoided that. So in that you just mentioned about redirecting the test directly to the node instead of the grid. That's what I'm sorry, instead of? So you just mentioned about redirecting the text that is the test directly to the node instead of the grid once the session is created. Is that what you are doing? So yeah, so what we basically do is in the traditional hub model, the hub creates a session on the node and as far as client code is concerned, hub is a source of truth. So entire communication happens through the hub and to the respective node. But in our model, what we do is the hub's role is to create a session and just return the handle of that session to the client code and the client code can now talk to the node directly. So even if one of the grid server died due to some crashing, it's the actual communication between your client code and the node so it's not affected. So that means you have done changes to the client code. That means remote web drive or things like that. Yes, correct. We establish a remote web driver client on our client code. You have implemented your custom remote web drive. Exactly, you're right. How do you know that the browser is unresponsive? And good question. Some rule is if it doesn't respond to our Selenium request, we try to kill sessions. Like let's say, I mean, if the browser is unresponsive, we don't have a way to know that. We don't have a way like, okay, at this point of time, browser has become unresponsive. So usually we detect it during the time of killing browsers post expiring. So we generally give a driver.fit call. So if it doesn't respond for three calls, like right now the logic is if it fails to respond within three calls, we browse the session. So since we exactly know the session ID and what node URL this session was created on, we use remote web driver to create a driver object out of this. And the moment we were not able to create a driver object, we think there is something wrong with that particular node, sorry, particular session. And we tried this three or four times. And if we understand that we were not able to successfully connect back to that session using remote web driver, we understand that the particular session is like screwed up. We call it rogue sessions in our term and we basically don't do anything about it. And when this node goes into this regular maintenance mode, like restarting after three to four days, that particular process is totally purged. So we kind of keep those lying there or fund till the node goes into the maintenance. And to be honest, we were not really able to identify what causes such unresponsive sessions yet. Right, so there are two ways to do this. One was the question was how do you know that a new job was added to the employer website? So the way which we have been doing so far is we run a periodic process where there'll be a worker machine that kind of tries to scrape all the jobs from the employer website every four hours. And we try to compare it with our snapshot of jobs. We identify the delta. And now, like if a particular job was no longer with the employer, we remove it from our database. If there was a new job, then we try to go and fetch information about that job. So this is one model versus the second model that we're trying now actively is, Indeed also does job search, right? Indeed has a huge cluster of machines constantly scouting the entire internet and trying to identify new job for every employer. So the model we're trying to do is, Indeed was already doing job search. Then there is no point in mobile system also doing the same job search and identifying the delta. So the contract that right now we're trying to establish between these two systems is, whenever the Indeed's job search engine identifies a new job in a particular employer, it pushes a notification to mobile system to update mobile's database. So we're trying to get to a model where both these two systems do not do the same job and we get notified about these job updates. As in the same job being updated? Yeah, that's a good question. So for this, what we do is we kind of do a periodic job reload of the entire jobs in the client. So we kind of forget all the jobs that we have fetched a week back and we scrape all of them again and we try to fetch all the information about the job, the pages in the job, the questions in the job. So that is how we get to know, like we try to keep ourselves up to date with the employer website. And I said that as a challenge because there is still a possibility that during this one week period after which the employer has made a change to the job, we are still unaware of that change. So that's still a challenge and we need to figure out a solution that does not, that is correct and yet that does not affect the performance of the original employer's site as well. Because we could do the entire job reload every day also, that's an option. But we don't want to spam the original employer website so often. Any other questions? Great, thanks guys.


Today I have the privilege of speaking with Mike Druhan from Medics Health. How are you today Mike? Great Trace Mike, I know you're trying to make a real difference in the world Can you tell those out there at investor intel land who may not be familiar with Medics Health? What your objective is please? Yeah So our objective is to be able to get dermatological service into a patient within 72 hours and right now the wait times to get into a dermatologist can be Anywhere from six months depending on which geographic location you're in to two years People are dying because they can't get service. It's ridiculous. We can read we can do that same service in 72 hours So Mike you told me once that there was a real discrepancy between the numbers in Quebec around skin cancer testing Do you remember that conversation and can you tell our audience a little bit more about this? Yeah, so what we looked at was the general population in Canada and the percentage of population of Quebec So we're back for an example 21% of the Canadian population But it was a much higher incidence of skin cancer deaths in in Quebec a little bit of that's behavioral with the people in Quebec Just terms outside and sun and pools and that sort of thing in the southern tip of Quebec being you know Southern Florida, but the other side of it is just the access to the healthcare around dermatology So we've been what's one of those things not at me a bit Tracy So we're happy to say we're really starting to open up in our Quebec market now And so it'll be easier for people to get screened and therefore early detection and a lot a lot You know a lot lower the risk of dying of skin cancer. We're pretty excited about the Quebec market So Mike, um, it's my understanding that as a med tech company you had a new kind of systems integration Process that took a little bit of time to get set up for the sales processes. Would that be a fair point? Yeah, I think it's a fair point And I think we're we're now at the point where we understand What our customer looks like how to go in and get a quick turnaround in terms of the deployment of the technology Yeah, we're there now we're there now I think I think you've got a handle on it based on the series of news releases But I think it may take some talking through for investors out there to appreciate and reward you properly for this So let's start with the most recent one I thought the one on the derm secure screening services with medicure clinic looked a very impressive President looked like a systems integration deal for me. Can you talk to us about this deal? Right, so that's one of the legs of the sort of one of the legs of the stool that we have And that's a physician driven one. So that is just purely the physician saying I represent 5000 patients This is an underserved community and we can't get them in for dermatological services in a timely fashion Medics fits the bill perfectly the other side that's interesting about it Is there's a pharmacy attached to that which allows them not only to do that for you know melanomas But other skin conditions as well that are inflammatory skin conditions that typically use Topicals and pharmacology where they can get rate in and get the right amount of tech or the right Dosage in the right pharmacology to deal with things like eczema Psoriasis rosacea that sort of thing So that's a really unique model for us and we're looking to replicate that across the country as well Pretty excited about that one Mike, I think what I'm seeing as well with COVID-19 our entire medical infrastructure has changed changing So you've also got a deal with empower clinics with rexall which looks fascinating to me Can you give us kind of the highlights of that particular partnership? Yeah, we've been really fortunate with that Steve McCauley and the group over at empower They have this real vision of putting clinics over the large rexall Pharmacies and going into serve underserved areas. It really is a really unique model So if for example in the long branch area where we've just done the big opening one down there That represents a 10,000 patient area that has certain services that are underserved That that you would hope you know you could get access to and we're the ones that give them instant access to dermatology And that's a really great win-win for both organizations So we're really happy with that partnership and as they continue to build out their clinics We'll be in there with them as one of their select services So that's been a great partnership and that one of course has national ramifications for us Well, Mike love seeing the deal flow. You've got your distribution distribution in place Please join us monthly so we can get an update and let's see medics achieve effective market valuation Thank you, Mike so much for medics health. Once again. Thanks, Tracy


The Section 8a volleyball championship paired two familiar foes against each other as park Rabbids and Rozo met in a rematch of last year's championship game in which the Rams came away with the win and the state birth. Game played in Fauston, lots of fans to cheer on both teams. Park Rabbids won the first set in the second, Panthers leading 18-7, Camry Carlson with the kill finding wide open spaces. They take the second 25-13 in the third. Park Rabbids storms out to attend a three-lead Riley Migerson with the thunderous attack. Rozo made things interesting. Katie Barovic with the pancake to keep it alive. Her sister Casey ends the point with the kill, part of a 9-5 run. But the Panthers were just too much all day on the match point. Rozo goes long on the attack. It's out of bounds. Park Rabbids takes it 25-21. They win the section title. Park Rabbids is going to state for the first time in program history, able to make that history and avenge last year's loss. It's kind of cool how we got to kind of redeem ourselves in the same atmosphere and just to have our hard work off season and stuff. I don't know, it just felt so good to be able to come back and redeem ourselves. It's exciting for our community because it's something that we've never happened before. So just to have everyone here at community just is super exciting. The players that played before us for sure won that victory for us tonight and it just feels so good to actually leave this gym this year, feeling way better than we did last year. So it's just I'm so excited for them. They deserved it. They played so well tonight against a really good Rozo team. And Park Rabbids will take on third seeded Marshall Thursday in St. Paul. If you enjoyed this segment of Lakeland News, please consider making a tax-deductible contribution to Lakeland PBS.