Hi everybody and welcome to a very exciting career foundry event this evening where I'm joined by Alex or better known as Alex the analyst Who is obviously an analyst himself, but YouTube star with his channel Yeah, Alex the analyst and we know that we've got a lot of people joining this evening So if you just want to pop on the right hand side where you're joining from maybe also why you're interested in learning data analytics and Any questions just pop them in the in the chat on the right hand side I'll be doing the quick introduction and I'll be handing the floor over to Alex But before we kick off this evening, I just like to very briefly introduce career foundry so as some of you may know we are one of the older players in the industry and We teach people in-demand skills from scratch We do data analytics, but we've also got other programs UX design UI design web development Digital marketing and we've just released product management too, but this evening is all about data I can see lots of people are joining from Portsmouth Boston, New York London. Hi everybody We've also got people watching on YouTube and LinkedIn A couple of house rules before I pass over to Alex We will be doing a live Q&A at the end So whilst Alex is talking and you may have any questions about I don't know data analytics or the industry Or salaries or anything which Alex is going to present just drop them in the Q&A Also, if you're watching on YouTube and LinkedIn, I'll reappear at the end and I will answer those questions Sorry, I forgot to introduce myself. My name is William and I'm the communications and events lead here at career foundry Um, I don't really want to steal um Alex's thunder because I know there's a really great presentation here So I'll just turn back to the first slide Alex, um, I think I'll just pass it over to you I'll disappear into the background and then when you've finished up just to call me back into the room and We can jump right into that Q&A All right, that sounds good Awesome. Thanks, Alex. Thank you. Um Hey everybody, if you don't know who I am I'm Alex freeberg Um And I'm a data analytics manager Let me actually go to the slide where I have some of my information so I can look at it Here we go so You know, I started off my career very at a very small nonprofit I was just a data collection specialist analyst over the past five years since I started I've worked my way up to a data analytics manager. I've had a very fast career path And a lot of the things that we're going to be talking about today are things that Um, you know, I'm hoping you cannot make those same mistakes I thought this was a really great topic because I don't often talk about all the mistakes I've made. I try to like Not talk about those talk about the things I did, right? Um, but you know, we can also learn from the mistakes I've made But I work at a very large company a fortune 10 company. Um, I specialize in sequel python cloud applications. I use um, a lot of power bi Microsoft azure a lot of sequel And some other software as well And then at the very bottom, I have a youtube channel. If you've never heard of my youtube channel, go check it out It's not bad. It's it's for data analysts. I get a lot of good stuff So i'm gonna jump right into it um My background is very untraditional I I have a degree in recreational therapy. It's It's probably something nobody here has heard of I'm guessing Um, but it was going towards the path of like occupational therapy physical therapy Like I was going to go to masters and go get my masters in that Um, but I moved to dallas for this internship in recreational therapy at this behavioral health hospital And it was a really interesting time because I was away from everything that I'd ever known. I'd never lived in dallas And I met this girl And I was like, I gotta stay and so I like scratched all my plans to go back and live with my parents to get my master's degree To become an occupational therapist and I just like started looking for any jobs That I could possibly get and I got one at the nonprofit that my wife was working at Well, she's my girlfriend at the time. So I'm really glad that panned out But she was my girlfriend at the time and uh, she got me a job as like basically a resident advocate I was just working with You know people who had been abused and I was I was helping them Live and get jobs and do paperwork and I would drive them places. That's what I did Um, so they had an opening at this company for a data collection specialist and analyst position and it was Not great pay It was nothing. I really had a lot of knowledge on Um, but they really liked me there just as a person and so I applied they gave me a shot and I and I got it and it was mostly only using excel So, you know, it wasn't super tech heavy, but it was very data focused because we had to submit the data for the grants Well at the very end of that job I Realized or I came to learn about something called sequel I instantly fell in love with it because I was just like this is a great skill to have I really should learn this And that's kind of where everything uh snowballed And after linds sequel the very first job I got was one of what I would consider my first like Big boy job or my first like real job data analyst job where I was working at the small healthcare analytics company Which was a fantastic company to to work for Um, so that's that's me. That's where I come from again very untraditional I had to learn the hard way most of the time About what not to do and so Let me go to the next one. This is our first one We have eight of them and I'm gonna share A lot of stories that I don't think I've ever shared before with My youtube channel or in linkedin or anywhere because again, I don't like talking too much about the mistakes But that's 100 what we're gonna be talking about today So hopefully we can all laugh at alex's mistakes together But the very first one. I think this is one of the first very first mistakes I ever made it was I I learned the basics and then I was like, okay, I don't need those anymore I'm gonna move on to like the important stuff the good stuff And you know as I've matured in my career I've quickly come to realize that 90% of you the job that you actually do is working with those basics the ones that you want to skip over The ones you think are super simple and you're like, I just want to get to the advanced stuff The super simple stuff is typically the stuff that is like really the meat and potatoes of what you do At least that's what I found and I've talked to lots of other analysts and you know, they say the same thing um I would I do mentorships and oftentimes people will come and they'll talk to me and they'll they'll show me their portfolio Or they'll show me um, you know some work that they've been working on And it'll be some of the most advanced stuff like really like they're doing some machine learning even they're doing You know some really advanced python Store procedures and data architecture and databases, which none of that is bad None of it is inherently a bad thing to learn But the problem with it was I was like, hey, what are you trying to showcase? What are you trying to do when you're actually in the job? You're never going to use this like 99 percent of the time you will never use You know these types of things that you're learning And so I really I highly suggest doing that and that was something that I messed up early on I I started learning, you know, really advanced sequel and stuff like that on my own before I really understood the basics And when I got into my job that healthcare analytics job that I was talking to you about A lot of these mistakes happened in that job They got like the worst version of me because they got me like right out of like learning And so I made most of my mistakes then And so that is just it's a big one that I've seen across the board not just with myself with a lot of people Um, I typically will recommend people starting with sequel excel and tableau or power bi if you want to learn power bi That is completely up to you But these three are like I consider like the trifecta of like beginner analytics you can do almost 95 percent With just these tools Different companies will have different softwares, of course And you're gonna need to pick those up But if you're looking for a job if you're just getting into a job you can do so much with just these three I I you know people always ask me do I need to learn python right away? I'm like, you know, python is amazing I love python. I'm a huge huge huge python fan, but I don't think you need to learn it like the very first skill that's that's kind of Bit more advanced, uh, in my opinion. So that is the first mistake that I would say I do have My notes over here, so don't mind me. Uh, if I have to glance over here, but Let's keep going. Let's go on a mistake number two, which is not asking questions now I want to say I was not a proud person But I was and I wanted to give the impression that I knew what I was doing in my job So, you know, again, let's go back to that healthcare analytics company. I mean I just I wanted to give off the impression that I knew what I was talking about that I knew what I was doing And they made a good hire, right?  It's only like 50 people. So oftentimes I would go to my boss who was the director of I can't remember his exact title, but he was a director level and his boss was like the senior VP so he was like really high up there in this really small company and They would ask me questions and he'd ask me to do things in 99 percent of the time Especially when I first started like right off the bat. I'd be like, I got it. No problem Like I got that and I go back to my desk and I'd be like Okay, what on earth was he talking about? I need to like research this and I would research it for Like some of these really easy things that he would ask me to do would take me an entire day of researching understanding what he actually asked Maybe going to my boss and kind of like subtly bringing it up. It's like, hey, remember when he said he wanted this um, yeah, so I'm gonna go do this right and he's like, yeah Because I didn't just ask him follow-up questions Which in retrospect would have made me look like I knew more about what I was doing But again, that's why there's a mistake is because I feel like when you get into a job You have this perception like everybody knows everything Everybody knows what they're talking about. Everybody already knows all that stuff that you don't know Typically, that's not how it is typically, you know, most people are in that same boat as you they are just You know, they have questions themselves and you should ask questions um, so You know The first bullet point says not asking questions slows down your growth Which means if I had just asked questions right away I would have learned a lot faster in that actual job Like I would have learned a lot more about what they wanted what they were looking for the the skills that they wanted um, better communication skills like I just really I was just I was a bit too proud. I just wanted to give off this impression that I knew what I was doing When it was evident that I did not I think Me not asking questions actually Did not know what I had the opposite effect as what I was going for um, so Oh, let me see real quick excuse me, uh This when I glance over here, I laugh because there's this one specific story That really makes me cringe when I think about it and I hope you get a laugh out of it too. It was um one of the first projects that I had And my boss brings me into his office.  He had a full workload. I was his only direct report So he had he was doing like a ton of hands-on work himself So he didn't really have time to like do a bunch of stuff He'd come in and check on me every so often like once or twice a day And I'd be like, yep, I'm working on it. I got it. I would bring it back to him and he's like Alex you you realize what I asked you to do, right? I was like, yes, you wanted this type of report He's like, no, that's not what I needed like at all. Um, I actually needed this and It was really, um I want to say sad. I was I was like, I am so sorry. I was like, I messed up I'm really really sorry. I should have like asked follow-up questions Um, so again, it just wasted like two three days of that week where I could have been doing actual work When what he wanted it ended up only taking about an hour or two once I actually really knew what he needed It just takes a little bit of it took a little bit of maturity and me to start realizing that I needed to ask those questions Uh, let's go on to the next one Number three is not learning the business context now when I say business context that usually talks about, you know, why Are we building this? Why do you need this dashboard? Why do you need this report? Why do you need this data cleaned? Um, you know, why do you need this? It's just asking that why oftentimes, um Especially because you know, I'll hire people into my team or um, I'll I'll mentor somebody oftentimes I see a lot of people will kind of Just not ask it goes back to that second question, right? Are you asking the right questions? uh business context is super important because Sometimes they're not gonna know Exactly what they need to know So they'll you'll say, you know, I want this report. I want this dashboard. That's the client coming to you And I'll say, okay explain me why you need that they'll start explaining things to me And I'm like these these these things aren't adding up Um, and this happens more often than you think especially in the job that I'm in now It happens more often than you think they're like, oh, well, this is why we need them like Well, if you're trying to get this if you're wanting this outcome You actually need to be looking at this data or you need to be building this type of report And understanding that business context of what they actually want with the end goal of it is is Extremely useful. Um, so you can ask questions like why are we building this and how will this help the stakeholders who use this? Because sometimes you're building things and they're going to hand that off to their You know team or their boss or something um, and they're trying to you know Display some type of information. So like with my work I work as kind of the internal. I don't work on client facing in my current job I work with everybody internal. So I work with all these different business teams within it And they'll come to me. They're like, hey, my boss wants to report that shows this information I'm like, great, you know, that's that's really good. Like why do they want that? What are they trying to get out of it? What are they trying to measure? What metrics or kpis? Are they tracking? Or are you guys tracking to make sure that he has that? And as we start kind of going down that rabbit hole as I'm asking these these questions about why and how Typically it brings up some like issue. I'm like, well, you know, we don't you don't actually need that. So learning the business context Has been Really important and that comes there's a later point that I'm going to get to I think it's like number six or seven You know, it really comes to just understanding how your team works What your team does and that comes with experience and so You know, you just kind of got to get into the job and learn it Don't try to do what I did, which is be too proud not ask questions Not try to understand the full scope of the project You want to understand it from beginning to end how you're going to be building it and and why it's being used all right mistake number four, uh, this one to me might maybe Uh If we look at the entire life cycle of a data analyst, this is one of the least Sexy parts of the job And something that I don't think gets enough love in my opinion because I like data cleaning But mistake number four is not cleaning the data or not cleaning it properly so Data in almost every aspect is going to be dirty You know, if you go on Kaggle or you go on google data sets or you you know go to my channel and you download my github and download these data sets Oftentimes they can seem very very perfect. It's like it's made exactly for what you need it for that is It is a very big misconception that you're ever going to get that in a real job Most of the time you're getting like just horrible. I mean it is shocking how bad this data is Um, and there have been times where I've like I've spent weeks Um, like two or three weeks Just cleaning data, um to prepare this data for future uses and it is brutal But if I didn't do it it would just lead to a Copious amount of mistakes, um down the line And so when I first started I didn't really know what data cleaning was that was That was something that was really foreign to me because I just learned the basics of sequel excel Little bit of tableau So data cleaning I didn't even understand what it meant um, but it's It's basically just the way that data is collected the way that data is stored in a database often is not ready to be exactly Used for what you need it for This second one clean data may still be needs may still need to be changed depending on what you're using the data for Even if you have perfect data, I mean the people you're working with are doing everything perfectly on their side They're giving you the data. It looks pristine Even then you are still going to have to transform that data like Almost every time Based on their end goal or their end product. So if they want a specific dashboard if they want a specific report You are going to need to transform that data and clean that data to make it even more useful and more Um Specific to what they want. So clean data is even if you get perfectly clean data It's not always clean But one thing I promise you Is that if you are the person that does that work you are going to be like Your people's favorite person because nobody most people don't want to do it. I worked on a team We had several analysts And my boss would always ask who wants to clean the data and I just became known as the data cleaning guy Like that was just my thing. I would work with the data engineers I'd work with the database developers and I just I really liked it But everyone else would be like really thankful and like really appreciative that I was cleaning this data um Because they didn't want to and so, you know, I feel like my skills really advance because the data cleaning process can be quite intensive Um, I my skills grew People liked that I was doing that work for them and that they didn't have to do it So there was a there was a lot of benefits to it. It just took me a while to understand That I needed to clean data and what that really was All right, mistake number five is rushing to build now This is one that I think That I I initially started doing because I was just again very I wanted to show that I was doing a good job that I was getting them a product Quickly like I wanted to just get to the end product and be like, hey, I delivered it in a day They're like, how on earth did how did you do that? This is supposed to take like a week Well, it was because I was just really rushing to get out a product I was rushing to get out the deliverables that they wanted me to give them and You know As I've been more and more in this job I I realized that it's a much deeper well And a much more difficult task to actually get that end product than it looks like Um, you know, most people if you're using dashboards, like if you're at your job right now and you use a dashboard you use something If you're working on a on a larger scale There was a huge team working on the back end to do all these different things just so you can see this one dashboard It's a large process and so You know, most people want to jump into like what I would consider the fun parts the fun part Which is actually creating the visualizations making it look nice giving the deliverable handing it off and like signing off on it And you're like, this is fantastic. It's done But that's not where like 80 of the work is it's getting the data from the source Which is already a process in and of itself cleaning that data transforming it moving it from production environment or a development environment or testing environments to production environments Um, there was a lot a lot a lot of steps So rushing into build is kind of one of those things that I just had to I you know, I got I just had to learn it Because I kept making this mistake um for probably the first like month or two of my job. I was just like, all right I'm gonna go in. I'm gonna do it It just wasn't smart to do um and With that oftentimes when I finish those projects really quickly Nine times out of 10 It was not done correctly. It it either the the numbers were really off and something that would happen Again, it's embarrassing to say but something that would happen often is I go to my boss. I'm like, hey Here's what you wanted. It looks great. You know all the all all the data was there and he looks at it. He's like Just knowing what I know I know that this is wrong and it just immediately like you're just like, oh gosh What do you know like what do you know that I don't know and he's like well He's like they I know for a fact They only you know build a million dollars worth of something and over here you have like 1.8 million. I don't know where those numbers came from. I don't know how you got that I just know what's wrong So I go back and then I do the whole process and I do it correctly I'd have to clean the data make sure I was pulling from the right hospitals when I was working at that health care analytics company Um, and then I would get the right number and bring it back to him and he's like this is what I was looking for So just you know slowing down Is is really really important Um, so this slide right here and I'm gonna zoom in. I don't I hope it doesn't disrupt what you guys are doing I'm gonna zoom in on on my screen This is the life cycle of a data analyst and data analysis project. I just found this online I google I found this and I was like, hey, I really like this. I want to show this to you guys because this kind of backs up my points of doing things in stages So Let me see because I think there's this thing where I can draw on here. Here I go All right, this may look terrible, but we're gonna do it anyways. Let's see how it turns out this step Oh, that looks great. This step step six This is the step that I typically was rushing into And I would just kind of jump to I'm like, okay. I have the data. Boom. I'm gonna go Uh and go build it and that's what this part is. So this is actually like, you know completing the story Making these recommendations. I wanted to get to this step as quickly as possible without having to do all the other steps So really quickly you can see that step one oops step one Is understanding the business issue that is that understanding the business context that we talked about that is one of the first thing You need to do and you have to understand Why we need it and you know define these objectives and actually gather some of that information That's the asking those questions Then you need to collect the data And then the very next thing you have to do is cleanse that data that cleansing and formatting And joining all this like right here when it says gather data from multiple sources You have to join that Information let me take that off because you can't see but cleansing it that is a huge huge huge step That needs to be done properly. Otherwise, you're going to get make mistakes this next one step number four It really depends on what you're building. So whether it's You know the step four isn't always done it depending on the project because sometimes You know, you're going to skip skip steps But this one, you know, whether it's creating models developing different methodologies And then step five and six, which is you're finalizing that project, right? You're you're now at the final stages and that's that's where I kept wanting to rush into Ignore all those circles. I just put those on there, but we'll go on to the next one Next one is not learning domain knowledge. Now, this is on a few different levels And so I'm going to give you a few different examples of why domain knowledge is important now You don't have to learn no domain knowledge. I actually know people who are kind of like generalists They're consultants. They work in healthcare finance different different industries and they never really specialize anything And then that's totally fine. That is a okay but At least for me personally and this is you know, this is my presentation. So I can say whatever I want From my perspective it is it really hinders you in the long run by being a generalist let me give me an example and this is This was a an exact conversation that I had with one of my mentees that I was working with he um was coming in from a totally different field and He was like You know, I he got his first job as a data analyst at a nice company. I was really proud of him because it's not his background Um, he got his first job. He was doing great. He was still my mentee a year later because he was with me for a while He just wanted to you know, keep up a year later. He got got a new job And then he got another contract and all these things And he asked me he's like he's like I feel like I want to go and take this other position And at that point he had done it for almost two years. I've worked with him for a long time. He had been a data analyst for almost two years And I was like, you know, you're getting to a point where you need to make a decision Like you need to choose and he and he he didn't understand Why I said that the reason is is let's say you were a data analyst for five years You're five years of experience.  You're not really focusing on one Um, you have broad industry knowledge, which isn't a bad thing again But I said if you really want to get to the next level if you really want to take your your Salary to the next step to make like a hundred thousand or more. I was like, you I really recommend you learning domain knowledge so The other side to that is he has five years and he doesn't specialize Or he does specialize. Well, if he does specialize Those companies are going to be very attracted his resume Him as a person because he specialized and they are they are going to pay really well for that five years experience in that industry Whereas if he just has that general knowledge, you know, they're going to be like, okay That's great. But we need someone to be able to specialize There is a whole You know consulting freelancing where you can be a generalist I have found that it is better to be a specialist than to be a generalist, especially in in data um So let me see if I missed anything in these uh, these questions really quick. So I was just I was I was on a roll um But actually this goes back. This is what I was talking about on number three, which is Knowing business context This domain knowledge is directly correlated to the business context and understanding that It's hard to know what to ask if you don't understand that industry well And so sometimes you can kind of probe and understand it a little bit But really understanding it at a deeper level you really need to understand that domain So for me it was understanding healthcare So I had worked on the healthcare side like the client facing not in analytics at all like working in hospitals working at, um uh what was that called a um rehabilitation center like different places like I knew what it was like to work in healthcare. I I knew how tough it was understanding the data side of it was Really utilizing that that information that I had but understanding the data side of it and so I was able to ask a lot better questions and I was able to Um understand where they were coming from because I would talk to doctors like directly and the doctors would be working with me And they they're like, oh, I would say. Yeah, you know It sounds like this and they're like, oh, you must have worked in a hospital for and I'm like, yeah, I have Like I did lots of internships and I've done this stuff on your side So I know what it looks like working in an EHR or stuff like that That stuff is invaluable. Um, and that's what really projected me uh on a very Quick pace in my career. That's why I'm a uh an analytics manager in five years rather than a lot of people It takes longer I think it's just because I specialized in one area and became really really good at it That's you know, that to me has been a huge huge thing in my career. So I don't want you to make the mistake or You just generalize too much for most people. That's not going to be a good a good thing to do All right mistake number seven. This one is a little bit more specific. Um This one is documentation Now when I say documentation, I think most people out there watching are not happy that I'm saying this and that's okay but documentation is one of those Hidden things that you just have to do and nobody no not many people talk about it. I probably should at some point like on my youtube channel, but um documentation is one of those things that You really really really should do it. Like it's There are so many consequences to not doing it and so many upsides to doing it. It's kind of hard to Make the case for the opposite For example Someone is going to come to you at some point in your career and they're going to be like, hey john or hey alex Remember that project you did six months ago. Well, could you go and find those source files? Could you go and show me where that data is? Can you go and tell me how you built out that dashboard? And you do not want to be the person who's like, I don't know any of that Let me go look it up and then you can't find it because you didn't document properly um this one to me is One of those ones that people don't like to hear It's it's not fun. It's not For my some people some of you guys are like the alex. I love documenting. That's my life. That's like my love language I'm like, hey go for it. That's not me. I I didn't like it So I didn't do it and inevitably that came back to might be on several occasions where they're like Hey, do you remember that project from three weeks ago a month ago? Do you have the emails for that you have? um You know the the sign off for that you have um where those where the data's coming from all these different things And I just didn't keep track of it like I that was not anything I cared about anything I was thinking about it just what it was a honestly it was a mistake. It was mistake number seven um, but genuinely it it was something that I learned especially when I got to my current company I've been working here for almost four years um They really drilled into me the importance of documentation and it saved me on so many occasions that now I'm kind of like a big I'm a big believer in it and so you know if you're in a job where you don't do that I would recommend doing it because somebody is really going to benefit from that probably you in the future so One other thing on this is like imagine now just imagine You get sick you have to be out for like a week or two weeks and you have to hand off your projects to the next person the question you should ask yourself is Is that person who you're handing your work off to are they gonna be able to pick up where you left off based off your documentation? If the answer is no You probably should be documenting better That's kind of like the my barometer if they cannot pick up my work um If i'm gone for a week two weeks a month or i'm taking a huge vacation for like two months If they can't pick up and understand what i've done and where they're going what they need to do I've failed it's not good documentation. Um, there's also and that's kind of broader documentation There's also documenting within your code which is similar but different right this is more like When you're writing your code if you're writing in python if you're doing pie spark if you're writing in r You can make comments or if you're writing sequel you can write comments And explain how these things are connected. Where's the state is coming from highly recommend that as well super super important All right on to the last one. I think i'm running over on time. I apologize. Um, I'm supposed to go 30 minutes, but we'll keep going. Um, this last one is uh Many people don't know so the mistake is isolating yourself and many people don't know this. I've I've worked on this I've worked on myself a lot. Um, I I would I would definitely say i'm much more Introverted. Um, once I get to know you I can be outgoing typical introvert, but I don't um, I Again, I this is kind of awkward to say I don't make friends super easily Like I just i'm not a talker.  Like sometimes I prefer it. I just it's not something that I really Thrive in I don't thrive in like making personal connections with people which um I don't know it feels weird to say now because I've gotten a lot better at it But especially at the beginning of my career. I was just I was a loner. I just like to do things myself. Um, and so One thing that I've noticed especially as I've come out of that that come out of That introversion or however you want to call that I tried to make myself enforce myself to be more personable and more outgoing As I've done that I found that my job has gotten easier Um, it's easier to ask questions because now I'm like, it's just you know, it's a buddy at work I can go ask them a question. It's no big deal. Whereas before I was like, I feel like I was like a nuisance It's like I was bothering them. Um, I just didn't have any connection with them. It was just like a work Mate not a work like friend or anything like that So it took longer to get things done because I would try to just do stuff myself Um, and if you've ever worked on a team in in the data world like you have to rely on each other for certain things Like I'm not going to go and try to do or find things that the database developer or data engineer does because that's going to Take me hours where it could take them literally like a minute. Um, and so it makes your job harder being by yourself And I also think it's just a big psychological thing to Have somebody at work or have someone at work that you can talk with Um, again, I I didn't do that probably for the first three four months of my first job as a data analyst And it was just rough. It wasn't fantastic, but I I learned a lot about myself I was like I learned that If I want to get my job done better and I want to connect and be and actually be like invested in this job I need to connect with people so actually talking with them connecting with them making like work friends That's important Um, and another huge side effect the last one is you're going to get promoted faster That uh, and just like make connections faster and get promoted for different jobs faster. That is I would say You know, a lot of these things have contributed to how I've gotten to where I am But one of the biggest things is I've made Really strong connections with the people that I've worked with Um, the very company the company that I'm at three years ago. I was a junior data analyst um And now three years later after starting a junior data analyst position I may I'm an analytics manager and so, you know A lot of that came from being very personable being very Accountable and making these relationships and connections because whenever there was up for a promotion or different things, you know They were like, oh alex is a great fit for this. He has the personality for it. He has the skills for it We want alex like that's what can come from Um, um making these connections and not just being a loner, right? It's just Um, it's hard and so, you know, I'm fully remote now And I still have to make an effort to reach out and connect with people because It's it's so much easier to isolate yourself when you're remote I've been remote for two and a half years now ever since uh, march of 2020 when you know, the pandemic really hit the u.s Uh, it's I really have to make a more tangible effort because I used to just be able to walk over to my My buddies like desk or their office or whatever and just talk to them Now I have to like reach out and and you know schedule time on our calendar to just chat and make that connection. So Um, that is all of the mistakes. Uh, again, I have made all of them at some point or another um It it and I've seen it just a lot in people I've worked with or people who ask me questions A lot of these are mistakes that not just I make although I have made all of them myself I promise you the mistakes that I see a lot and so hopefully, you know, just listening to this and Seeing it and kind of hearing my perspective on things you can be like, oh, I'm making that mistake Literally right now like I'm not documenting. I'm isolating myself I'm trying to do way too advanced things that could have a much simpler thing could be done with it Um, hopefully somebody out there, you know benefited from You know learning about all the mistakes that I've made in my career, which this is only the the tip of the iceberg, but Let's go on to the next one. So now We're going on to our very last segment. I'm supposed to tell you when I'm done. I'm done Uh, so I think we're gonna get into the actual, um the actual questions now, which I love answering questions So I I can't wait to hear what you guys, uh, ask me Alex, thank you very much for the, um, presentation I also find it amazing that there seems to be a lot of people in the audience who love data cleaning Um, but I think um, I don't have my chat open so I can't see that but Guys my all my data cleaners out there.  You guys really are I think we there's a lot of people in the audience who love data cleaning too. So just watch this space Um, but I thought the presentation was really really good Not only for people who maybe just started out in the field, but also for those considering jumping into Into the world of data analytics. I'm going to take questions. We're going to do a mixture between big marker YouTube and LinkedIn And um, there was a good question up here on big marker I think I'll start with some intro questions and apologies if I pronounce anyone's name wrong. Um, hi Jim is asking Good morning, Alex. Um, how easy or hard is it to switch to a career in data analytics? um That's a question that I think is should and Needs to be asked more because the landscape is changing. Um, I'm going to try to keep this brief. I know we have lots of questions. Um The landscape has changed. I started five years ago. It's much different now. Um, because of remote work The access to available jobs has just broadened that horizon like analytics and data in general is just It's it's not going anywhere anytime soon. There's so many people trying to get into it But actually breaking in. Um, it's very possible for switching careers. I personally I like I said, I do some mentorships I have helped many people from the just the most random backgrounds get jobs Uh pharmacist teacher a lawyer Those are just a few of the ones completely random backgrounds Um, it's very possible. You really just need to understand what you're what you're good at understand what your domain knowledge is in try to Reformat your previous experience to make it more analytics focused if you can really make a good portfolio projects Make a great resume. All those things are It definitely makes it possible. I think almost anybody can do it. It just You have to put in a lot of work to make it happen Great answer. Great answer. And I think Simon is actually following up with that question Just about how long will it take to secure a job in data analytics? So if you're starting out from scratch Yeah, so It's very it depends on how much time you can invest. Um, let's say it's a full-time job. You can invest all your time You know, I've talked on my channel a lot about exactly the career path exact resources that you can take I you know, I just talked about a career founder on my channel as well There's so many resources out there if you hunker down and you really focus it on it like it's your full-time job I think it's very possible to do it within about four to five months Most people it's going to take longer, right? Um, it just depends on how quickly you learn But I would say on the short end about four to five months Most people, you know, you have you have to make money.  Like I like that's how I did it. I just learned it at night after my my full-time job I did it over the course of like six to seven months But I studied I started studying like just every single night very consistently and I Had of I got lucky and I chose sequel I just really hone in on sequel and that turned out to be like the best skill to learn So it can range it could be as early as four months or it could take some people a year year and a half Again, it just depends on how much time you can invest And when you were working Alex were you putting all that work into a portfolio? How was how did you present what you were learning? Uh, well when I first started out, I didn't know what I was doing So I just like I put some my portfolio project was a bunch of sequel scripts on a word document that I sent in with my resume Uh, that's how I that's what I do when I first started. I don't recommend doing that anymore Um, I recommend doing a project. So as you're learning the skill as you learn sequel You know, try to find sequel projects try to find projects in python or tablo or RBI and put them somewhere put them in a on a website Something like github pages or creating a website using wix, which you can create a free website You can host and that can be your Portfolio project or that can be your portfolio and you can put all your projects in there And then you know you have a link in your resume or on your linked in or wherever and you can point people to that And they can go and be like wow, this is really good sequel code. It's well written. This is really good python project um, and yeah, it's it's That's how I would do it at least Fantastic, I'm gonna switch to youtube because I know we've got a lot of people watching over on youtube also on alex's channel Um, I usually is asking an interesting questions for a data analyst Or is being a data analyst a better career option for a person from a non technical background? um That's a really great question because um, I I personally think it is I don't I don't I don't want to stir any waters. I think it's one of the better ones I'll say that I don't I don't know if it's the best although. I think it's the best If I if you look at it on the spectrum data analysis business analysis or or different type of analytic work is Is a lot of understanding business operations the business side of things even data analytics The more technical you get when you go into data science data engineering data architecture Those are very technical much more technical than a data analytics position I think data analytics is one of the best Places to start to get into data or even, you know business analytics I think is one of the best and low probably the lower barrier of entry for most people because again data scientists All these other things they've taken most people have a degree in it or um have a master's degree or you know, just Really good on the technical side which can take a long time to to get good at um, so I think it's a very good career path for people who are kind of starting out because It's a low barrier of entry, but it's a high high high skill cap Like I would consider myself pretty much an expert at sequel really good at python Like i'm really good at these skills, but even I have places to go and learn with them So there's a low barrier of entry, but there's a high high skill cap where you have a lot of room for growth Which it just makes it a really unique Kind of place to start And another question on youtube, let me just scroll up here to the top I think somebody was asking about how important is is building a portfolio Whilst you're you know learning or being a data analyst I suppose this is kind of the next stage once you've put it all together How important is that portfolio? Maybe if you use a little bit difficult, but from your experience with interacting with other people What's your experience of portfolio as an industry? I'm a huge believer in it, but what is the importance? I don't think It is the most important I think your resume is the number one thing that's going to get you a job or get you into an interview Your personality and how you speak and your your Just knowledge on it is the other piece those those two pieces are the most important Your portfolio projects are for two things one. It can help you get an interview It definitely can if somebody clicks on it They you know you even in an email say hey check out my portfolio projects on these skills like specify it to them They can go and check it out and verify that you've worked on these and you know what you're doing The second thing is is one that I think is even more important If you do not have any experience and you go into a job interview They're going to ask you how have you you seek when you're going to be like well You know I I learned it on data camp. I learned it on youtube I followed Alex's tutorials whatever that doesn't give you any credibility What does give you credibility is if you say Actually because they say how have you sequel actually? I just built out the product I built out a project. I pulled the data in via this I use these types of queries and you can point to that project as experience that is How I see it most of the time how it should be used It can help you get an interview, but it can it can more help you land that job in the interview than anything else I think that also touches on another question asked on big market. How how can I get my first work experience? You know if you if you studied and you've got your portfolio ready together You know, how can you get that first kind of like real-world project about land that first kind of gig? yeah projects are Probably the easier one because you can do that all yourself right you can make a project You can build it. You can use that as experience. I I recommend people do that I tell people to do that You're not you're not putting it on your resume as the job. You're just putting it under Personal projects or portfolio projects you can point to And say you've used the skill. You know how to do it Probably a much more difficult thing to do is try to do something like freelancing Try to do something like and this is something I did and it actually did work But I it's rare Is reaching out to local non-profits reaching out to local small businesses and asking them asking them If you can help them in you know in this area It actually worked for me because I was working with this small animal shelter And I reached out to them and I just did some like basic visualizations with their data and put it on their website They put it on their website and I could point to that and be like, hey I consulted with this company. I did it for free if I consulted with this company I worked on it. Here's the tools I used and and you can go and see the see the work A little more difficult projects are a lot more Independent you don't really need anybody else to do it That's also that's also what I've heard from career foundry students as well I'm doing online events is also people have been doing your x-design or your design and they've asked friends Or you know, some family might have the You know kind of I don't know like a wildlife group or something and they need a little bit more help with their website And it's a good kind of like entry level way to get that kind of real world experience and show that you've done a project For sure I feel I feel that we because we've got so many questions or you know We've obviously got such a fan base from the data cleaners. I feel that we should take a few questions That go down that direction. Say it is asking how do you make sure that data is clean? Yeah, that's um, where do you stop? That's the question because um, oftentimes And there are certain things that you're trying to achieve And it definitely comes down back to that business context understanding what the data needs to be used for To give you like a super simple example You know say you're doing something with countries and the countries there's there's one that's spelled united states There's one that's spelled united states. There's one that's america. Those are all the same thing But they're spelled differently or said differently. Those need to be normalized, right? So you go and you normalize all that data and it's now it's all just united states Well, is that as far as you need to go? Sometimes yes, sometimes no sometimes you need to break it out Um and break those things out even more and do you know categorize it on a higher level Maybe you need to add data north. Uh, you know, we need to categorize it by um a continent It it could be endless and it really comes out of business context But honestly typically when i'm cleaning data, you can never get it 100 clean If we can get it to 90 to 95 exactly what we're looking for Most of the time that is plenty It you can keep going it is possible to just keep digging in and keep cleaning But at what point does it become less? Valuable like is your what you're working on going to actually return that 1% extra that you're going to get to be a cleaner Is it worth it? um sometimes no, so it really is it it's very subjective, um, but Kind of just know it when you know it that helps The deep clean um at this point I would also just like to shamelessly plug the career foundry blog We've got some great articles and we've also got some specific articles on data cleaning Um, and we've got the in-house editors who have put this together and there's a lot of data analytics content there Um, also on um alex's channel. You've also found some great videos. Um alex the analyst On youtube I'm going to take one more question from the data to cleaners because they were out in such strong forces evening Denise is asking what is your favorite resource to learn how to clean data? um You know, I learned it 100 on the job. I I learned what it was on the job I was taught how to do it on the job. I never until I was I understood it better. I never understood it was Now that that now that there's more resources out there. There's a few good places um I I have found that there's a like if you google data cleaning course or tutorial. There's some out there um, I have one on my channel and I'm going to plug it just because I think it's really good I have a I have a A portfolio project of cleaning data and it's super simple and easy to do. That's a good place to start Then you can branch out and try to find more advanced ones Um, but tutorials and projects where they actually show you how to clean data. That's what I would be looking for um or You know techniques on how to do it because you you'll find a lot of articles about kind of like What data cleaning is and why it's important? But somebody's showing you how to do it. It can be a little harder to find definitely Um, I would just also say with the career foundry program that you do get a dual mentorship model So we do offer with a mentor and a tutor to help guide you through. Um, so you're not there on your own Um, Alex, you've got a lot of fans out there. Claudia is a very big fan and she loves your videos Um, I'm watching them regularly Um, and I think cloud is also asking a very interesting question looking more at companies So when looking for a data analyst role How to know if the company is a good one to work at? Um, cloud is based in london if that makes any, um Just a just for context That's a wonderful question. Um I'll give you a bit of my history when I first got my first job I didn't I had never thought about that your way ahead of where I was when I first started Um, wasn't even I just wanted a job. I just wanted a anything The company I worked at was when it was a great company. I learned a ton, but it was a very small company. Um, and Looking back I learned so much there, but that wasn't what I was looking for because I got three kids I needed something more stable So you need What I do now is I literally make a list of what I am looking for. What is the number one thing? And on my list right now is not money. The number one thing on my list is flexibility I if if I get a job offer and it's like $200,000 and you know, they want me to move and do something like no I'm not doing it. I I can't work. You know 10 hour days. I just I have a family I can't prioritizing what you want is really good. But where should you research that? I like glastor. Glastor has a good um barometer for reviews and stuff like that I would also go on linkedin linkedin has good stuff like that Um where you can go and look at the company review it see what people are saying about it I would also search it on google There there was a company that I got it It was like a loan company or something like that and they wanted to me to be a senior data analyst or something And I was considering it and I looked him up online and they had like a rating of a 2.4 Everyone was like the work life balance is terrible. The pay is just okay. The benefits are garbage and I was like, whoa No, thank you. So doing your due diligence knowing what you find important That's you know, that's where I would start and then, you know, you can see what you can compromise on Are you willing to compromise on the healthcare package? Are you willing to compromise on being remote versus in look uh in person? You know, but I like lists. That's what I do Um also from live events that I've known also with our career specialists Remember that if you get to the stage of an interview too That's also a great chance for you to kind of scope the company out. So it's not just the focus is not just on you You know, it's also your chance to work out. Is this a company that I actually want to work for? What's the career? What's the interview process like? So don't just um see interviews as this kind of like focus just on you. It's a kind of getting to know on both sides Um far had is asking I don't have a degree. Can I still become a data analyst? Um far had currently works in sales That's a great question. I'm going to give you somebody I mentored About a year and a half ago. I started mentoring with he's my biggest success story of all my mentees He was a warehouse worker and he now works at um paypal as a data analyst He got his first job as a contract job at something random use that experience and leveraged it To get this larger job and now he's been at paypal for about a year That he is my successor. I still he's still one of my mentees. Uh, he He doesn't need me anymore, but we just like to chat it is 100 possible. Um, and you can go And you can see lots of other stories like that online of people switching careers and not having degrees um In my opinion, it's more of a mental It's more of a mental block than anything because When you see that other people with degrees are applying It usually discourages you and it stops you the number one thing that Helped this person get to where he was at was he literally was reaching out to recruiters He had like six or seven recruiters. He was reaching out to weekly. He would he would schedule calls to them weekly. He was constantly Updating his portfolio.  It just takes a lot of work Especially when you don't have a degree It is a small barrier of some companies want a minimum of that degree But you're you need to be out there to convince them and you need to believe that you can do it That to me is one of the biggest Mental barriers. It's not it's not as big of a deal anymore.  That was five years ago Now it's really not a huge issue as long as you have the skills You have the the passion for it and like the drive for it. Those factors are much more important Definitely, that's great advice Skipping across again to youtube We've got another big fan. Um called gave the data analyst Watch this space alex. I feel that um competition is on the horizon. Um, hello alex. I am a big fan. Um What would be better freelancing as a data analyst or an actual data analyst job? um, hey I personally would probably just say the data analyst job. It's hard. I mean It happens and you can do it. It's just I've heard it's really hard to do freelancing Especially if you have no experience, um, you kind of have to have a history Of work in order to do well freelancing That's my that's what I've heard. I've never done it myself when you get a job as a data analyst You're going to be taught Here's what you need to do. Here's how we do our processes. You learn a lot on the job. Uh a lot of Getting a entry-level job is that's the hardest part because once you get that you get experience on the job And you can leverage that experience to get other jobs Um, that's why it's so great. It's just hard to get that first job But once you get it, it's so much easier to get other jobs. I feel like freelancing is a little bit More difficult to actually make happen and be successful at it. That's just again I haven't done freelancing, but that's just what I've heard and what I've seen Definitely, I think that will be the advice of career foundry and career advisors too um The career foundry, um full program the data analytics analytics program The kind of the end goal is to land you either a junior or a mid to junior level Job, uh, which ties in quite nicely with um, kiran's question. What type of projects, um Will I be working on as a data analyst in a junior role? Sure, um At a junior role, you know, you're expected to know a little bit You're expected to know the technical skills, uh, some of the basics of the technical skills But they're more often than not you're not expected to really know The industry knowledge super well So typically they're gonna give you um, they're gonna Yeah, it depends on what kind of data analyst role, but if you're doing um Stuff where you're doing a lot of visualizations a lot of reports and dashboards You're on that side of the data analysis spectrum, you know, they'll hand off to you Projects typically they're gonna give you like one or two projects at a time Now as you get to mid-level senior, you're gonna have six seven eight projects at a time and you'll juggle a lot more But you know their goal is to get you to that place So they're gonna hand off small dashboards that they need built And hopefully they'll work with you to understand where the data comes from comes from and all that stuff I wouldn't be expecting to do anything too advanced You're probably not going to be building or helping build or design any data pipelines Probably not going to be doing a ton of the Really in-depth data cleaning because again that can be really technical You're probably doing some of the easier things and they'll grow you and hopefully Train you to get to that place definitely, um I think that's great advice and also, you know, you kind of on board new skills If you're working in a great company you on board new skills as you go and you should be delegated more responsibility So you grow in the role I think a lot of people fear that they have to go into a new position and know absolutely everything Which is which is not the case, especially in the junior kind of roles Um, Ben says thanks so much for the presentation Um, Ben is very good technically and has a good business understanding But his most important problem is communication Um, would you say that this is the most important skill or soft skill as a data analyst? Well, it's probably one of the more important soft skills. Um, and you know I think it's I think it's worth working on and investing in Um, and really the only the best way because again, that was something that I semi struggled with at the beginning Now I'm I'm especially through my youtube stuff. I become a much better communicator But when I was first starting out, I I did struggle with that as well So what I did is what I got a lot of interviews No, nobody wanted to hire me at first, but I got a lot of interviews I use those interviews to really practice my communication skills. Um, and I would ask for feedback Um, most those interviews I was like, they're not going to hire me. So let me just you know, do my best and um, you know really try to Again, I point I point back to this and I know most people Don't believe me, but I I was not a good communicator. I wasn't very outgoing or personable for a long time I um, I had to get into my own head before an interview I would talk to myself and I would kind of like be like, okay You know remember to smile remember to be outgoing remember to nod and like Do these things because I would usually just like Do this You know, it takes practice communication takes practice.  So I mean, yeah communication is super important Um, you need to be able to communicate your findings what you're asking for documentation You need to be able to communicate. Well, I it just it takes practice and you can you can develop it You just have to be really intentional which um, that's something I really had to be intentional about You know, probably five years ago or so And for alex if you want to keep up to date with what's happening in the data analytics industry in terms of Like podcasts and resources, you know, what are your kind of like main places that you go to to find out information? Uh, youtube a lot believe it or not. I I stay up to date with a lot of the other youtubers.  Kenji Tina Hwang. I you know, they're all great resources my channel as well I also do read a lot of um I'll like go on medium and um all about data science is that what it's called I can That's like a a blog Um, there are some places that I go, but I I do a lot. I do a lot of youtube believe it or not Where that's kind of where I keep up to date on a lot of different stuff happening and I like twitter. I've gotten into twitter lately So twitter and linkedin are great places to keep up to date if you follow the right people And um, if I follow the right pages and stuff, you know, there are there are really great free resources out there To stay up to date know what people are looking for and and what technologies to focus on Have you ever been recognized on the street walking down? Let's see if you happen the first time like two or three days ago Really?  I'm and they're like and they whisper to their mom and they're like he's famous I was like whoa, whoa, whoa That that is not that is not true. It was the very first time it ever happened. It was very it felt very weird Um, but you know, they they were really nice. And so yeah, that was the that's the only time it's ever happened though Um, I am mindful of the clock Just a couple more questions just to wrap this up. Denise is asking What came as the biggest surprise to you when you started your career as a data analyst? Something that was really shocking to me Was how little people not how little that sounds bad How much how I felt like everyone else knew everything when in fact Not everybody knew as much as I thought they did My perception was that everybody was like they knew these skills off the top of their head They could do them they knew the business information What I found out is everybody research is almost everything. Um, that is 100 true Like I have a I have a developer on my team We just started using power bi Along with this other service that we've been like a legacy system We started using power bi and I was like, hey, listen, I know you're not gonna know this and she's like every Couple days she'll be like, hey, I just looked up how to do this and now I know how to do this I was like that's awesome My expectation has completely changed because I'm like You don't know what this is. Why would I expect you to know that was the biggest shock like the huge I was like I thought everyone just knew everything Nobody knows anything. Um, they know how to research. Well, they understand the basics They know how to get things done, but a lot of people they they research things they go on google They find better ideas than what they had they use those resources to do their job better. That was a huge shock to me Definitely, that's super interesting And also, um, as a data analyst, you can go down many different avenues So there's like the whole kind of like spectrum of different ways that you can take the career Um, in your opinion, what are some of the kind of the most interesting avenues that you can go down?   You can stay in analytics for a long long long time Or you can switch careers and there's some really closely knit ones that as you get into the field as you start working as a data analyst You'll start working with these people you can go data scientist Which is um more technical. Um pays well I think it's a really good place for a lot of people. I almost went that route and I was like, I don't want to do that One that I personally love and that I really was close to doing before I got this last job was going down the data engineering path or database developer path my sequel skills are just Really aligned with that and I really like that type of work of building those pipelines data ingestion etl processes I love that stuff and In my last role as a data analyst on my team I was working with the data engineers and database developers every single day So I really started to understand their work and I was like I could do that and I really like that kind of work. Um, so those are some um You can you can also go so like data analysts is kind of a General term there's more specific ones. You can go and you can do be a financial analyst marketing analyst business analyst I just had someone who was a data analyst who switched to business analytics because they was more aligned with what they like to do and wanted to do Because they liked working with people On the client facing side and then translating those requirements and and you know doing some of that some of that work So, yeah, there's a lot of different options. Whether you just want to stay in that or branch out Definitely, it's a very broad career and a very exciting career and alex. Thank you so much for presenting this evening I thought it was a great presentation. Thank you for fielding all the questions I know that we had a lot of questions that we couldn't take this evening But thank you for everybody for participating and being so engaged What I would say is that alex has done a fantastic video Which I think was released a couple of weeks ago, which looks at the career boundary Intro course, but also the full data analytics program Definitely watch that out. Check out alex's channel alex the analysts. It's a great channel lots of content And if you're there, you might as well go over to the career boundary youtube channel because you've also got lots of videos about data analytics Um, briefly towards the end. I'll just and say that currently we do have a special offer that we're currently offering 20% off Our full data analytics program If you click the little sticky if you're in big marker on the right hand corner You can get a link to that. We'll also be posting that on youtube and linked in too And all you have to do to secure that is book a call with one of our lovely program advisors and Our program advisors will go through the application with you Um, all super friendly and if you've got any questions about the curriculum or maybe Payment plans or maybe even jobs in your locality feel free to ask our program advisors that and they will get back to you alex, I really enjoyed that Thank you so much. Um, also, thank you so much for doing all the stuff that you do on youtube thank you for the audience this evening and Would you like to say any last words alex about data cleaning maybe? Now one thing I will say I know a lot of people in the in the chatter You know beginners are trying to get into the field, you know I just want to say that I hope that you can push past that initial the initial toughness of getting into this career. It's tough Um, it really is especially if you don't come from that background I just want to tell you like A lot of his mentality if you just keep going and you push yourself and you really Can believe that you can do it and you push yourself to do it you can make it happen Um, I truly believe that and so I hope that you know, someone out there Is a little bit more motivated and a little bit more Um at ease with getting into this career, you know after this session and thank you so much to career foundry for putting this on It was really great It's awesome. Thank you so much alex and uh, I wish well, what time is it with you at the moment? It's noon It's noon. Okay A late lunch then I wish you a good late lunch and um, it's evening here in Berlin and for everyone watching Thank you so much And I'll just quickly say that we've also got some great data analytics events coming up We've got an intro intro to data analytics event coming up with dr. Humira where we go through a data data set from start to finish And um, if you're interested in SD, um, I'll be back tomorrow with craig hats and looking at dbxd So yeah, thanks alex and uh, enjoy the rest of your afternoon and see everybody soon


E o que se enxinou a José Manuel Ortega, coisitando epícolas e píatónas. Bonas tardes e quesos por vir. O DISTAL é para comentar os títulos que nós temos no Paito Eco Sistema para obtenir información para servidores que coisam información na internet. O DISTAL é para o meu espícaro de España onde eu tenho outras presentaciones relacionadas a escrapar, securidad mobile e etc. Estas são os pontos principales, eu vou fazer uma introducción para o Python testing, o que são os métodos que nós temos, nós comentamos os módulos principales, como Chokes, Requests, Beautiful Shoe e Shun para obtener información para os servidores publicos. Láder, eu vou mencionar como extractar metadata para documentos e imagens e finalmente, módulos más avançados, como por escaning, e como conectar com a escena de vulnerabilidades. E finalmente, eu vou mostrar um pouco de consenso, onde eu tenho integrado todos os módulos na Python testing tool. O Python is a little introduction, Python is very useful for making rapid prototypes and proof of concepts and many of the tools that we have for testing security in database and web applications are made with Python. The main advantage that we have is that there are very good documentation in internet for all these tools that we will comment. Well, for example, one or two of the main tools that we have for testing application, for example, SQL map for testing security injection, vulnerabilities and social engineering tool kit, these tools are made with Python. Well, we have another tools like Sparta. Sparta is a poor scanning that use Python in a map. Python in a map is another tool that I will comment after for checking the ports that are open in a specific domain, server or application. Basicamente, with this tool we can check the service that are open and make a process, launch a brute force process over a specific service and so on. Another tool that we have that are interesting for analyzing is the Harvested, is a tool for obtaining information, pulling information from what this tool is, you can use for obtaining information about the domains, e-mail accounts and domains for a specific user role or server. Another very new tool is the Web Application Attack and Only Framework. This is another tool made with Python, this is very useful for auditing web security and detecting vulnerabilities, SQL injection, cross scripting and so on. Another tools that we have, for example, SCAPI for analyzing network packets and for example if we want to detecting some attacks, SQL injection, you can use these tools. Another tool that we have is FIMA for example for detecting remote file inclusion vulnerabilities. All these tools that I have commented are made in pattern and well. I will show the main modules that we can use for developing our tools for testing the security of the servers and web applications. Well, the first proof of concept that we can see is showcase, with showcase module basically what we want to do is checking the basically port scan. We connect this method, we can check in a specific port if the port is open, filter or close, in a sequence. This is the most simple program that we can write, basically what we do is a question to the user, the API of the server and starting port number and an ending port number and with a for loop simple we check if the port is open, closed or filter. The difference in between, for example, in the close and filter is the filter, the port is filter where is blocked by a firewall, for example. Showcase allows also obtain information, resolving the API address from the domain and vice versa. If the methods get host by address and get host my name, we can obtain this information. With the showcase module also we can check, we can obtain the banner servers. The banner servers is information related with the name of the version of the server, the word server, for example. In a easy way we can check, for example, this is an script where we pass as parameter the API address and the port and retours information about the server. In this case we see that retours that the domain, the EuroPython 2016 has an inject server. Well, another of the modules, very now for all Python developers, is request, request is a very useful module for making, for testing web service, API res and so on. And basically what we can do for testing the security outside, for example, we can check, for example, the headers of the request and the response in a easy way. Accessing to the headers property and iterating over the item dictionary, we can obtain this information. For example, if we check over the EuroPython site, we see that obtain this information, the inject server and also we see another headers like cookies, the user agent and so on. Another interesting feature that we can make with request is, for example, it works behind a proxy for making request, we can use the proxy dictionary where we indicate the HTTP or HTTPS proxy and in a easy way we can check the connection behind a proxy with request. And another interesting feature is with request we can authenticate, we have a server that supports basic or dyes authentication, we can check this information, we can check the security of this server with the methods, this HTTP dyes and the basic. Well, another of the tools that we have bring now for web scrapping, for example, it's a beautiful soup. Beautiful soup is basically a parser, it's a parser HTML. Basically what it does is extracting information from a specific tags that we use request for obtain the page and with method final we recover the information, in this case we recover the links. A more advanced program where we extract the internal links, internal links basically consists in finding all links that begin with a slash and for external links we try to find all links that start with HTTP or HTTPS that no contain the current word. In this example we can see that we extract the external and internal links, the external links go to an external page, I know this in the same domain that is not in the domain that we are testing and internal links go to pages in the internal domain. Another interesting feature for example is we want to extract image and PDF documents, we can use a specific parser, a beautiful soup has two or three parsers, we have seen the LXML parser and in this case we are using the HTML parser where we are using sparse presions for extracting the element that we want to extract in this case in image and PDFs. Well, another interesting tool that we have in Biodecosystem being very useful is Scrappy, Scrappy is for developing, we can use for developing our web spiders, web crawlers and well it's very useful for obtaining the information for web service and URLs. This tool is a tool for making asynchronous calls and follow the event driving development paradigm for making these calls in asynchronous mode. Well, for stunting with Shodan, Shodan basicaly is a useful tool for obtaining pooling information that are available on the internet. Shodan what is that is obtaining the banner of the servers, operating systems, the versions, the server styles and so on. And basicaly what is provides a developer API for in this case for Python developers and in a nice way you can connect with this service through a Shodan API key that provides when you register in the site you get an API key and with this API key you can search we can make the same search, the same search that we can do in the web you can do in the Python API. In this case we can see that we look an information for a specific host and we obtain the ports that are open, retus the banner of the server, the information of the services that are open in this port and so on. Also where we search for a specific host we obtain information about the host name, the ports that are open and the service that are available in each port. The same this information that we see in the web also we can access with Python in a nice way but the information for accessing is a little treaty that returs in a dictionary but it's not a K-value directed K-value it returs the information in a dictionary but certain positions have a vector or a write and you have to play with accessing the information. Finally we can obtain the same information that we have seen in the Shodan web for a specific host we can obtain this information from Python API. Another easy module that we have in Python is WillWidth WillWidth is a very easy module that only have one method, parts, and obtain the information about the frameworks that the website is using and the web servers that are using the website. Well, for Metadata, nice is Metadata basicaly what we can do with Python direct in a easy way is extract information for Metadata from PDFs with PyPDF2 module and basicaly WillWidth is very easy with the pay where we can, we have to fix create a PDF file, created object and with the get document info we obtain the Metadata or the PDF the same we can do with image with a pill.exif tax module and in a easy way we can do, we can obtain, we can decode the tax that are available in an image for example we can obtain the GPS info the data of the image where, when, how is the resolution of the image and so on. Well, another a little advanced future that we can do is poor scanning poor scanning is very no for the NMAP tool that we can see in multiplat for operating system and so on. With Python NMAP we can launch the NMAP from Python basicaly what we have is two models synchronous and asynchronous for a synchronous model we have to launch the poor scanner well we can see the NMAP the object NMAP with poor scanner and we use the scan method where we pass as parameter the IP address and the port the port listing that we want to scan in this IP this is an example where we define a class NMAP scanner we initialize the NMAP scanner with calling the port scanner of the NMAP and inside the NMAP scan we go checking the port and we call in the scan method an internally scan method called the NMAP command that it install in the operating system well we can see information about this is a call sample where we pass as parameter the target and the port listing and we can see that the NMAP is a code with the full parameters in the port specific and in the IP and we see on retours if it's open and also we can access to the specific port is open we can access the version of the service and so on the other model that we have with NMAP is asynchronous model allows launch and scan in a lot of ports simultaneously and we define a callback function for when a specific scan in a port is finished call this function for an additional assignment in a specific port for example in this case we launch in a specific target in the 21 port the 50p port and we check firstly we have to check if it's open and if it's open we can launch a specific process for detecting vulnerabilities in this port for example in this case we are testing the 50p port launching and and scripts for checking anonymous FTP login checking another type or vulnerabilities if the service is vulnerable to a specific backdoor for example ordered or specific versions to this service for checking these vulnerabilities when we install NMAP NMAP comes by default with NMAP scripts that are in the mass script folder in the instance when you install this tool and have a lot of scripts for checking vulnerabilities in a specific service like FTP, HTTP and MySQL for example basically what provides these scripts are routines for finding potential vulnerabilities in a given target and then the idea first we have to check if the port is open and if the port is open we can launch the specific script for this service in this case we are checking that if the MySQL port is open and if it's open we are launching scripts like MySQL audit MySQL boot MySQL databases these scripts provide more information about the service and checking for example the names if the database is open provides no security or you can see the user without authentication these kinds of things you can see with launching the scripts well for example with Sonan we can check the FTP login anonymous basically with this search we can check all servers and machines that allow this this type of login with all anonymous is the login anonymous we don't need to to provide user and password for accessing the FTP server for checking this in Python we have the FTP module that in a nice way we can check if a specific server allows anonymous login well for checking websites we have another tools called P with booth provides resources for checking websites basically with predictable URLs that is to say we have a list of URLs of URLs and each URL each resource well we will see with an example for example we can access in this example we obtain predictable URLs for a specific future for example for logging in a website we get predictable URLs like can mean logging default web page administrator and so on and we can do is testing for each predictable URL testing this this predictable URL over the domain we are testing and in a nice way we see that we can check we make a request over the domain for each predictable URL to see if we can access or not many times there are URLs that we don't see they are not public but after testing after doing this type of testing we see that there are URLs that are filtered or are not protected for example and we can access and navigate and discover other vulnerabilities in the site well the Herblade Book is another book that we can testing with in a nice way with Python Herblade is a vulnerability in a specific open SSL version in servers and well this this book was discovered in in 2014 and well is a little old but with this they are we check this page Filippoio Herblade we see that there are servers many servers that are also that are vulnerable to this work basically for testing if a specific server has this book basically what we have to do we can use the socket module and set a specific request a specific package and if the server responds with a specific payload Herblade this the server will be vulnerable to this book in this case we can see an example where we launch testing over a specific machine with 100 tools that this machine is vulnerable to this book well mod advanced tool that we can use for integrating with Python for example with Metasploit Metasploit is a very useful tool for sending exploiting vulnerabilities in the servers in the websites and so on and Metasploit can be integrated it has a module called Python MC RPC for making calls to Metasploit server from Python basically what we can do is we have to do is start the server the service of Metasploit throw a plugin and internally Metasploit functions like modules for checking for testing for exploiting vulnerabilities in this case with Metasploit api call we can see that with calls in a format specific call mcgpec pack we can a specific module in this case we are using the mcgpec login for launching for testing this exploiting in from Python well and next post nexus and open bus are security security analyzers for vulnerabilities and also we can integrate these tools from Python for example is really easy to integrate from Python for example we have a server with next post where we have vulnerabilities and reports and so on we can connect with this server from Python to the beautiful soup we can access to this information the information that retours the server is in a format and in a easy way with beautiful soup we can iterate over the vulnerabilities and sites that are deployed in the next in the next post server I will show now a pen testing tool that this is a quick provost concept that has integrated all modules that I have mentioned basically what we can do with pen testing tool is for example with a specific machine a specific port we can check in if the port is open or not in this case we are checking the 21 ctp port to see the information that retours in this case we can see that the port is open and obtain we can obtain information about the version the specific version of the ctp the name and so on like the ctp port is open we can check if the ctp server allows anonimo login and then if we go to the 13 option we see that retours that the port 21 is open and question about if you like to connect with anonimo user we question yes we see that loading successful retours 230 the version of the ctp the connection is ok and shows the directory alistin directory of the of the server more things that we can do if obtain the headers of the server in this case we obtain that we have a php run to 5.10 if run in anonimo with the version 2.2 0.8 the version of the php and so on all this is the headers info more things that we can do for example check if the ctp server has the buffer overflow this is another vulnerability that has a lot of servers and we can check with this option first we check if the port is open and we send a client request to see the server is vulnerability we connect to this to this ip in the port 444 and we check that the server is vulnerable to this to this book basicaly all this testing are over a virtual machine that they have here in local this is a virtual machine that has a distribution with a lot of vulnerabilities many many ports open and a lot of vulnerabilities with vtp, php, apache server and so on this is called a metasplotable Linux if I don't if I remember more things that we can do is check domains and obtain metadata for example if the server has information with mails, hosts other servers or URLs that are exposed in the server we can check this information in this case we see that we are testing the options method that retours the server and we can obtain the e-mails that are public in the server and also we can check for example if the if the server is public we can check the sodan information that retours for example we are using the europaidon site for checking all the information and retours by sodan service we launch the tool with the target europaidon 256 it retours the IP address of the domain and for checking for example the host info from sodan service we can use the option 6 disconnect to the sodan service and obtain all public information it's time for questions retours all all public information that we have seen in the presentation we see that the ports open are the 18 22 and 25 and retours more information about for each service basically obtain information about the servers the banners the version the versions of the server for example the option 10 for scraping image and in a easy way we can see that extracting all the image of the site and so on ah well finally this project is available in MagiHub repository and is available for if you want to check the tool you can do free in MagiHub repository also we have as scripts as small scripts for testing each functionality separately from each order for example if we want to scan to launch an enmascam we have a specific script for this future and finally reference and leaves basically are the main the official pages of the tools that have commented the showdan documentation request Python in a map documentation and in the python security.org are available or more libraries that have comment and is a very very complete site for checking this this kind of tools and finally books that we can find the most this is the main books that we can find for this topic that for test and so on, thank you any questions?


So we just concluded our deliberations as jury members of the Indian marketing awards, exchange for medias, curtsy exchange for media, would like to really thank all the organizers of exchange for media as well as Anurag Bhattra for inviting us to be a part of this extremely insightful process. Being in jury might seem like a simple task, it is not, believe you me. Our group was focused on stages of brand building which covered categories like brand rejuvenation, use of consumer insight, brand purpose and best use of experiential marketing, content marketing, influencers, brand identity packaging and age targeted marketing. Now these are very, very specific categories. There were a lot of contenders. I must say that it was a very, very difficult choice and considering the fact that during the pandemic it was very difficult for brands and the agencies to work under these circumstances. I was pleasantly surprised to see very high quality work coming to us for evaluation and we really applied our mind for about six hours deliberated and have done the best that we could but my congratulations first go to all the people who actually participated, made it to this round and then obviously to the gold and the silver and the browns winners but remember the no process can be perfect but I can confirm this to you that exchange for media has a very, very standardized process where the chances of error or bias are almost negligible. We were really focused on the objectives, the strategy, the outcomes in each category for each campaign and for each entry. We have given all the time and the attention that was deserving of this kind of a process. I would wish all the winners and all the participants all the very best and look forward to seeing great work from the creative teams, from the marketing teams and actually creating great consumer engagement both in digital as well as in physical space. Thank you so much.