In this video, I'm going to share with you my top 10 winning dropshipping products for the month of February. Not only am I going to share with you these winning products, but I'll also show you the sellers who are selling these products, what ads they're running, their websites and where they're buying these products from, how much profit they're making, how much sales and a full analytics report for every product that I'm going over. And as an added bonus, I will also share with you my cheat sheet, including all of the links to these products to the sellers Facebook ads to their websites and so much more bonus value. So if you want that cheat sheet, just comment below hashtag February and let me know what is your top takeaway from this video? What is your favorite product? Now let's get this show on the road. The top 10 products for February coming right up quick intro and let's go. Welcome to yet another product finding video where we share with you our best products to sell along with all of the seller's informations and their analytics so that you can get a good example of what you need to sell on your store. In this video, as you know, we're going to go over the top 10 products for the month of February. So let's go ahead and get started. One second before that, if you haven't done so yet, subscribe to our YouTube channel if you want to learn about the next step that you need to take in your e-commerce business, whether it's what products you want to sell, full tutorial videos, case studies, live webinars and so much more that we have going on. Hit that subscribe button, enable those notifications and let's get this show on the road. The first product that you want to sell for the month of February is this long sleeved sweater cardigan. Let's take a look at the statistics. So right here, as you can see, this sweater is being sold for $58.98, being bought for only $26, leaving this seller with a potential profit of $33 for every time he makes a sale. So let's start with the Facebook ad. Mollilulu, shop for trendy fashion style top, find the newest casual and Q-top with affordable prices, pick yours and get more discounts. So I'm going to give you guys a quick guess, does this look like a native American speaker to you who's targeting the US audience or is this some Chinese seller who doesn't really have good English, doesn't have good copywriting skills and he's just doing his best to sell to the US audience. So option B is the correct answer here and even if you don't have real good skills running ads like this guy who doesn't even try to make a video ad but just post some images of the sweater, either way, he's getting over a thousand comments, 600 shares and over 12,000 engagements in this Facebook ad. That is because he's selling the right product to the right audience. Now imagine if you would do a similar job, not only will you target your audience well like this seller has experience in doing, but you're also going to write a good ad copy. How do we write good ad copies? Well you can use your creative mind and if it's not that creative, I'm going to show you a neat little trick as we continue moving down this video. And here's the seller's website. So here he's doing a little bit of a better job when we compare it to his Facebook ad, but it's still kind of messy and kind of spammy with these $40, $20 off. It's a good thing to give discounts, but here you're kind of confusing the buyer because what the seller is doing right is he is using upselling and cross-selling apps to help him increase his average order value. But the bad thing that he's doing here is you don't want to overdo it because if the customer landed on your product page, you want him to now check out. If he adds another accessory, that is great. You're going to make more profit, but don't try to overdo it and don't try to throw them out. So what's happening here is I understand that if I buy at a certain price, I'm going to get a discount. That's fine. A little bit too big, but that's fine. But then as I move further down, I can see frequently bought together. And it's like, oh, so there's these pants, but then there's these jackets too. And there's this shirt. And then there's this. Okay, you know what? Let me just click on this. Okay. Yeah. Nice scarf. I forgot where I was before, but you know what? I don't know. It's a little bit too confusing. Thank you. Bye bye. So that's what a lot of customers are going to experience when you throw too many things in their faces and you just kind of confuse them, give them new ideas and then they kind of forgot why they were here in the first place. Or if it's a good idea to check out with that product when now we have so many more ideas, let's go back to searching the internet. So we don't want that to happen and that's how you can do a better job. But at the end of the day, he is making sales and you can just see that by the amount of engagement. So out of the 12,000 people that engage with this ad, just imagine if only 2% of them or 1.5% of them actually clicked and went to the website and bring down another 1% or 2% sales conversion rates because those are the ones that actually converted. So you do the math and you'll see that he is making sales and it's fun making sales when you're making $33 in profit for every time you make that sale. Here's how he's making that profit because on his website, we saw that he's selling it for almost $60 for every sweater and let's see how much he's buying it for. Okay. So let's choose the jacket, the right variation. I see that the price increases as we increase in sizes. Now let's choose the variation on AliExpress. So let's go with the large version and we can see that it really does cost $23.5 with tracking available. I'm always checking if the tracking is available in the shipping options so that when the buyer asks what's going on with the package or I didn't receive my package, you can prove to them what's going on according to the tracking event. So we're always going to add shipping with tracking information and even when we do so, we notice over $30 in profit for every time we make a sale on this product. That is product number one. Let's move over to product number two, 1080p Wi-Fi magnetic cameras. So this is like one of those spy cams and it has a magnet to it. So you can stick it to more parts, more places. Let's take a look at the stats. This camera is being sold for $35, being sourced at less than $9, leaving this seller with a profit potential of about $26, $27 for every transaction. Now do keep in mind that we have selling channel fees. So it does depend on where we're selling. If we're selling on places like eBay, Shopify, Wix, SpooCommerce, we have an average of about 15% break even that we will pay there, but you really have to check your fees on your selling channels. So even if you take down about 15% from the selling price, you're still going to make a great amount of profit selling these products. And one of the things that I want to show you in these videos is that you do not have to sell for cheap prices. I'm showing you sellers who are selling at a high profit margin range because I want to show you that anyone can do it. And the dropshipping business model, especially with the low risk and high reward that can turn from it, if you have the right knowledge, tools and skills, then you can make a substantial amount of profit even when you're not putting any money down on inventory because this is dropshipping and that's what we love about it. And once again, if you want access to this cheat sheet with all of the links and all of the products and all of the hard work that we did to shave off hours and hours of time from product researching successful products to sell in your stores, just leave a comment below hashtag February and let me know what is your top takeaway from this video? What is your favorite product? And I will gladly send you this cheat sheet. No charge. Just enjoy the value and start making sales. Now, when I'm recommending these products, remember, don't just add one of them to your stores. You want to add many so you can really test out this niche and these categories. If you only test out one product, like one camera and one jacket, you're most likely not going to make sales and then you'll think that this is not a good product. But if you test the market correctly, you will find winners and then you can continue scaling your success by adding more products similar to those that are selling well for you. So here is the Facebook ad for the spy cam. I'm just going to turn down the volume. And as we can see right here, he's got a good video ad running for this product. So it shows the product in action and right here it can show you it's kind of like a CCTV camera. So it's showing you all of the things that you can do with it. You can even add it as a dash cam to your car inside your home to see what your pets or what your children are up to. And we can see that it has a micro SD slot just by looking at this video. So this video is a good one that shows you all of the use cases or some of the best use cases that we have for this product, which is great. And as we can see, it worked out pretty well for this seller because he's got over 17,000 engagements here, over 1,200 comments and two and a half million views in this ad. And once again, not the best copy. So take a look, outdoor security camera, no power needed. You can get instant live stream from your mobile device, get yours. And he even linked like the RS and get yours. He put a minimum amount of effort into this ad copy. The video was easy for him because he most likely got it from his supplier. So he just wrote a pretty bad ad copy because he's not that good at it. And he just targeted his audience pretty well. Again, you can do a much better job than this and get more sales, more views, more engagements and more profits. Just take these product ideas, add them to your stores, start marketing them, learn to do it the right way. If you want to learn how to run Facebook ads much better than this seller, let me know in the comments below and I'll send you an ebook on the eight steps to run successful Facebook ads. Just leave a comment right below and I will send you this ebook that you see right here. Okay, let's continue. So we can see the really nice engagements that we have here. And you can just see by the customer's replies that they are purchasing this product. So Brenda's saying, well, I get them and they are very helpful. One for my home, one for my office. So she's purchased a few. I will buy four. I love it. I want one now. Very hard to configure but fast delivery and here's something that we can learn. We can learn from the comments what the customer's pain points are even with this product. And then when we look for it using our dropshipping suppliers, we can look for different cameras, better ones that have an instruction booklet in English. We can talk to the suppliers and ask them if it's easy to configure this camera. And this way we can even sell a better product than what these sellers are doing by just learning from their mistakes. Okay, so let's continue. Let's take a look at their website. So here we have the product at $27.99. And after we add the shipping price, if you just add it to your card, add the shipping information and check out, you're gonna notice about a $7 shipping charge. So that's what you're gonna get right here. And then you're gonna make it to $35.54. Now let's take a look at the supplier. Okay, so I'm gonna switch over right now to AliExpress. And lo and behold, the same camera for just $8.03. Same camera. Let's see if we've got tracking information right here. And yes, we've got free shipping with tracking information available. So that is great. $8 when we're selling it for $35, leaving us with $26 for every sale and we can do a better job than this seller. In this cheat sheet, you guys also have a target audience, which I'm not going over in this video, but when you run ads on these products, you'll also have some of the target audience information that will help you target the right audiences, like their genders, age range, marital status and general interests. Let's move over to product number three, Bluetooth fairy string lights. Let's take a look at who's doing a good job selling this product. So these fairy string lights are being sold for $28.40, being sourced at $14.50, leaving us with over $13 in profit. So 2022, new do it yourself, festive ambient light by two free shipping, sound activated, decorate lights, DIY your home light show. Okay, I gotta show you guys a good trick to write better ad copies and I'm gonna do it for this product. All right, so bad ad copy, maybe a good video, let's take a look showcasing the product in action and yes, it does do a good job showing the product and that's actually pretty cool. We've got this motion effect right here, making it look like the lights are just shooting out of nowhere. This is actually a pretty cool product. Haven't seen this one around. Besides regular string lights. All right, so let's take a look at the engagements. Over a thousand engagements, 40 comments and 240,000 views almost. So yes, this product is popular. Let's take a look at his website. So here's a seller's website. He calls himself Not Wood. It's obviously a general store. He's selling all kinds of random products and not just this one product or not just in this specific niche.  The price will stay the same. We can see that he's using other apps here showing how much he's selling a month, how many people recommended this product, how many people are viewing it right now and we've got the variations right here which is kind of nice. The images on the left side are from the supplier. We can tell that he's not adding images by himself but he did add some nice gifts or gifs or whatever you wanna call them. It's very much debatable but he's adding a few of them to really show the product in action. So he does know how to do a good job in showing the product, which is really important because we wanna show the customers exactly what this product is, what problem it helps them solve and why they actually need it. So he does do a good job there. He even has an FAQ section here so questions and answers. It will help him get some organic traffic from Google but also answer questions along the way that customers may have that might be blocking them from purchasing and now that they have the answers they can continue to the checkout process with this product. Okay, so we've got some trust badges here and a whole bunch of other texts but let's write a better ad copy. So what I'm gonna do here is I'm gonna go to the suppliers website which we can see right here. You can get this same product on AliExpress for $11.5 where as we saw he's selling it for $19.99 and after we add the shipping price we're gonna make it to $28.40 cause that's how much he charges extra for shipping almost $10. So let's go back to AliExpress get the product description and try to write a better ad copy. Okay, so what I'm gonna do in this case is I'm just going to take by the way we can see all the gifts from here from AliExpress. So the seller didn't even create it himself he just took everything from here and yeah, instead of pictures it says pictures and more bad Chinese seller stuff. But hey, we're helping them make sales and it's helping us earn some extra profit. So now let's do it the right way. So we wanna write a better ad copy for this product. Here's what we're gonna do. I'm just gonna take all of the text that actually explains what this product is what it does and so forth. So that's up until about here. I'm gonna copy all of that. Now I'm gonna ask artificial intelligence to help me out with this. So hello chat GPT please write an ad copy for this product using bullet points and emojis. Okay, and I'm gonna hit shift enter paste all of that text and enter. And let's see what it's going to generate for us. Okay, so here we go. So we can already see that we have the emojis and it's being split into bullet points brighten up your space with these easy to shape wire and wide uses LED lights. I maybe I wouldn't use wide uses but we can always revise this a little bit with a flexible copper wire design and waterproof construction. These lights can be used indoors and outdoors making them perfect for any space from teen girl rooms. I don't like how that sounds, but of course we're gonna revise teen girl rooms and bedrooms to bars and windows. So we will have to revise it a little bit but it is doing a good job taking bad texts with bad English and turning it into something that can actually convert a consumer. Let's continue. These lights are also great for creating amazing decorations for holidays like Christmas, Halloween and parties plus the built-in sensitive microphone allows these LED lights to sync with music creating a dynamic and romantic atmosphere. And now we've got more product features like IP65 waterproof, USB to control it with Alexa, Echo, Google Home or Siri and more product features. Upgrade your decor with these smart lights with music sync today. So we are going to maybe change a little bit of the text around but we already have a much much better ad copy than sound activated decorative lights DIY your home light show gift from you love. Artificial intelligence guys that's what it's all about this year start taking advantage of it. Moving down to product number four we've got yoga meditation steel drum. Let's take a look. This one is being sold for $67.80 being sourced at $27 leaving the seller with a potential profit of $40. It's not potential cause that's his profit $40 for every time he makes a sale. So let's hit play. We've got a nice video ad here.  And yeah, it's creating music. We can see what it does. So that's pretty cool. There is absolutely something magical to this. It's like talking to my soul. I release all of my stress and anxiety every time I start playing it in my meditation sessions. Five stars from Dove. And then we've got some bullet point emojis helps to release your stress and anxiety helps to bring clarity to your mind rejuvenates, calms the emotions opens the heart to happiness. Each of our drums is handmade with a story behind it. Each and every tone is elaborately crafted to make you feel your inner peace and enjoy yourself better. That's not a bad ad copy considering everything else that we've seen so far. But we can still do it better with the help of chat GPT. So over 2,000 engagements, 126 comments and over five, almost five and a half million views for this product. So I would say that this seller has hundreds of thousands of engagements of views on his product page. And of course he's making lots of sales from those website visitors. Meet Balmi. So he's gonna call this Balmi. The coolest, most fun and best educational gift for your children. Let's see if it's a one product store. So I'm gonna go to his front page and yes, he created a one product store from this product. Only this seller is actually doing a pretty good job. So check it out, I'm on the front page of the website and I got this cool graphic motion thing going on showing me the product zooming in and out, celebrate the best Christmas with Balmi. So he's really going all out even though Christmas is behind us but it's probably still making sales or most people just land on his product page so they don't have to worry about the front page. But in any case, it is a one product store and he did optimize it to the best of his ability. Not only that, but I can't even highlight the text or right click on his website because he blocked left and right clicking. Does that mean that we can't find his product? We don't know where he's getting it from? Nope. Of course he doesn't want other sellers stealing his product because he's not the only one who has it. That is why he's not allowing left and right clicks. But I'm gonna show you everything anyway. So let's continue. His product page looks really good, really optimized. So in order to do a better job we're gonna have to create a website that's probably not gonna be a one product store around just this one product. And if you're just getting started, don't create a one product store. Start with a general store. Get to know the selling products, the selling categories and only then should you consider creating a niche store around that product. But Star General, learn to test the market and let me help you with the product research part. So as we can see, he's selling it for $59.90 and if you add shipping another eight bucks you're gonna make it to $67.80. Let's see the source. So I'm gonna go of course to AliExpress and here we can see the same product in all kinds of different colors for $24.36 including free shipping with tracking available. So that is just great. And that is even cheaper than what we updated inside the cheat sheet when we created it. So now it's $3 cheaper leaving you with more than $40 profit for every time you make a sale on this yoga meditation steel drum which as this seller did he completely created a store around it and he named it Balmy. Okay, so that's pretty cool. Let's move over to product number five. LED laser flashlight. What's so special about this one? Let's take a look. This small handheld flashlight is being sold for almost $62 being sourced at just $19. Again, leaving the seller with over $40 in profit for every time he's able to sell a product that he's only purchasing for $19. Let's take a look at the statistics and see how well he's doing. Quick reminder, you can find all of these products in the AutoDS marketplace. So if you're an AutoDS member just hover over the marketplace on the left side and here you'll have lots of trending products that you can import to your store in just one click. Just hover over the import draft and add these products to your stores. And this way you can continue testing the market with more and more trending items that we are adding from our database full of many, many trending products that's updating all the time. And all of this is being done so that you guys can make more sales on your dropshipping stores. If you click on any item you'll get even more product information, more images, the specifications, the description, product reviews, the seller's policies and so much more. And also from inside you can choose different variations and import it to your store in just one click and start selling. So this is a good place to find trending products. After you're done watching this video get the cheat sheet, get the products, come back to the marketplace and start adding more products so you can continue testing the market and make more sales and profit. You have a nice filtering tool so you can change what countries you wanna ship to, what warehouses you wanna ship from, price ranges, all of the best dropshipping categories and so much more. So check out the product research tool. It's already included inside your AutoDS subscription. Enjoy that, get those winning products and start making those sales. So back to that LED rechargeable flashlight. So we can see the Facebook ad right here. Let's take a look at the product in action, just turn down the volume. Look at what this guy is doing. So just look at the amount of light that you're getting so it can have a really narrow light that goes really far or you can really bring it in and zoom it up and get a huge, huge flashlight view of what we have ahead of us in the dark. So just by watching a few seconds of this video you can really get a clue of what this product does and it really stands out, right? You haven't seen it before, you're seeing it now for the first time and that is what's gonna make a lot of people impulse purchase this product. Okay, so let's take a look at the ad copy which is really, really bad. LED rechargeable tactical laser, flashlight, 90,000 high, dot, dot, dot. So I mean, he just copied and pasted a title. He didn't even look to see that he has way too many words and then it just turns into dots and nobody knows what 90,000 high means and even with bad ad copies, as I showed you in this video, you can still become really successful. 26,000 engagements, 2,600 comments, 4.8 million views on a really, really bad ad copy because this product is cool but what if you're gonna sell this cool product with a better ad copy? So imagine you're gonna get 10 million views, 5,000 comments and 50,000 engagements and even more. It all depends on the amount of work that you're willing to put in and today with the help of artificial intelligence, we can get all of this to us in just a matter of seconds. So today it's all about automation and artificial intelligence, prove me wrong. Okay guys, this is the seller's website. He's selling it for $40. But that is not the variation that I'm looking for. I wanna make sure that we are testing the same exact variation. So right now we're looking at the P70, that's this one right here. Okay, so we're looking at this upgraded version and he's selling it for $60. Right here we can see that we can purchase it for only $16 and the shipping is included, hopefully with tracking information, yes. So shipping with tracking is gonna set us back just $16 but hey, selling it for $60 and having 4.8 million views is just completely crazy because many of these views are clicking and many of those clicks are purchasing and when you're gonna make $40 for every transaction and you sell 10 items per day, which isn't that much for this much engagement, that's $400 from just one product every single day. Okay, so that is what he's doing and now I wanna show you guys something cool. So I told you guys that you need to test the market, right? And you can test with just one product. So let's say you wanna test this one right here. So what you're gonna do is search for the same product, rechargeable flashlight, zoomable P70, powerful LED flashlight. Let's just take all of that text. Okay, I'm gonna copy that. And now you can go to any supplier that you wanna search from. Now my recommendation is to head over to autods.com and hover over suppliers on the top and click on that. And then you'll get a list of the best dropshipping suppliers that you can use today. All of these suppliers, some of them are retail, some of them are wholesale but all of these suppliers have tens of thousands of products that you can resell and you can search for these products using these suppliers. So in this example, I'm just gonna go with Walmart. Okay, so let's head over to Walmart. This is a retail supplier, not a wholesale but you can of course go with wholesale and make your lives just a little bit easier but let's paste that title right here. Okay, and check it out. We've got many more flashlights that look pretty much the same. Now, if I can purchase it for $15.99 on Walmart then there is no reason to go to AliExpress. But I'm gonna show you examples from AliExpress because this is the go-to supplier for many people who are just beginning with their dropshipping journey and it's really easy to find the products there. But we can see the same flashlight right here for $15.99 and you're buying it from a US supplier meaning you can ship to the US audience in just one to two days instead of waiting about two weeks for AliExpress to bring it there. Okay, so what you're gonna do here in this case, let's say you wanna add all these products to your store. Okay, you wanna add all of these rechargeable flashlights but you don't wanna do it one by one. You don't wanna spend all day just importing products because then you won't really be able to grow your business. So what you're gonna do here is download and install the AutoDS Chrome dropshipping extension. It's free and once you have it you'll notice the circle right here. Stand on the product search page with all of the products, hover over the icon, click on it and click on extract. Now all of the 42 flashlights from this page got extracted into the extension. Now what I can do is I can continue moving to the next pages and continue extracting more and more products but I'm gonna stick with this one and click on export as CSV and as you see a CSV file was downloaded onto my computer. The next step is I'm gonna head over to AutoDS. I'm gonna click on add products. I'm gonna go with multiple products and then upload CSV and here I can drag and drop or click on it to open up my file browser window, navigate to that CSV file, double click on it, processing CSV and file is uploaded in just about a second. So the supplier here is Walmart and the region is the United States. Then we're gonna click on add as a draft and that's it. All of those products inside the extension are going to move to the draft section of my store where I can optimize these products before having them go live. One more neat trick before I show you guys the rest of the products in this video. I'm sure that this is gonna help you out so don't forget to use it. So here is another example. I wanna take a bad product description and turn it into a good one because we're importing so many products and now we have to write so many different product descriptions. So let me just look for a product that has a bad description. Okay, let's take this one right here. So we've got these LED strip lights right here and we also talked about LED lights in this video. So let's say this product right here, okay? We've got a bullet list right here talking about the product and what it can do. Wanna give you a Roma total makeover? Wanna add some ambience to life? Maybe add some color could refresh it. Right, bad English. Let's redo this. We don't need virtual assistants or our virtual assistants can just save a lot of time and do it this way. So what I'm gonna do is I'm gonna copy all of this bad text back to our good friend chat GPT and tell him rewrite this product description using bullet points, shift and enter, paste that text and hit enter. And now it's starting to write that description. Didn't even pass a second. I mean the speed of artificial intelligence and how fast it thinks and reacts to your questions or queries is simply amazing. So here we're getting a much better product description and we don't need all of this text because we have all of this information right here. So it's more readable. More people are actually gonna read it, connect with what you have to say, understand it and purchase the product. Okay, so what I'm gonna do now is go back to the product description and simply paste the much better bullet points that I got. So again, the old one, the new one. Old one, new one. Much, much better. Let's save this one. So that's another thing that you can do with artificial intelligence to help you with your website. And that's also how you can add many, many products to your drop shipping stores and start testing out the market and see what's gonna start to sell. All right, let's continue on with the product list. Moving on to product number six, wireless lavalier, is that how you say it? Lavalier microphone. Let's take a look at the stats. And remember, this cheat sheet is up for grabs. Leave a comment below this video, hashtag February. Let me know what your top takeaways from this video, what your favorite product is, and I will send you this cheat sheet so that you can start optimizing and scaling your store right after watching this video. So here we're back to Notwood USA. I put my focus on the seller because he's running a lot of Facebook ads and he's actually doing a pretty good job at it, even though he's not writing good ad copies. And this is a good example again of how we can do a better job. Okay, so here he's got a video and here we have testimonials from people who are talking about the product, you know, where they actually purchased it and they're giving it some review. And we can see, again, a bad ad copy. I'm not even gonna go over it. And the most important thing is 7,000 engagements, 300 comments, almost 300 shares. So yes, this one is popular. Yes, he's making those sales. Here is the seller's website, selling this wireless Lavalier microphone for $29.99. Add another about $6 in shipping. You're gonna make it to $35, $36. That is the total price that you're gonna pay unless you purchase for over $40. Then he's gonna give you free worldwide delivery with no space between the $40 and the worldwide. Okay, so we can see the product right here. And as we scroll down, we can see the images, we can see the products information and pretty much all the information that he's getting from the supplier, he's just pasting it right here. But his product page does look good overall. We do see the product, there are inconsistencies, there are mistakes, he's not perfect. Like this image right here, I'm gonna see the same one as I scroll down. So yeah, it's easy to find things that we can improve and do better. One of the things that I do like about what he's doing though is the sticky add to cart banner on the bottom. So no matter where I'm at on the product page, I can continue scrolling and scrolling and scrolling, but at any time I can add this product to my cart. And of course, the review importer, which he's using an app to do it. So you use a review importer app that will take the reviews from AliExpress and import it to your website so that you'll have the beginning social proof until you'll start making sales. And then your customers will start adding their own customer reviews and then you'll have your own original reviews. But until then, you can use a review importer app which you have many apps that do it for free today. So just go to the Shopify app store, look for review importer apps and you will find it. Okay, so of course we can do better than this guy and we can make more sales. So just take this as a good example of someone who's doing a good job, but not that good because we can improve and make it better. And last but not least, selling for $36, but purchasing for only, what are we looking at here, $10, $11. So $8.38 on AliExpress with another $2.85 for shipping. So that's gonna set us back about $11 and that's actually a good bargain because we're gonna make $23 in profit for selling this microphone. So good job seller, thank you for the information. Now allow us to do a better job. Product number seven, insulated hooded winter jacket. Let's take a look. This one is being sold for $57, being sourced at just $25, leaving us with a profit of $32 for every sale. Hey man, is selling this product. So that's the Facebook page. Here is his Facebook ad, really bad once again, just one sentence, not really eye catching and not even a video, just an image. But still the right product to the right audience is still going to make sales. How much sales is it gonna make? That's a question of how well you're able to market the product. But sell people what they're looking for and you will get purchases. So here we're looking at 2,300 engagements on the Facebook ad, almost 200 comments and over 200 shares. Let's take a look at his website. So right here, $39.99. We've got the different variations right here, different sizes, but we do need to choose the right variation in order to understand exactly how much he's profiting. Okay, so let's also look at AliExpress and let's choose a matching variation. Okay, so let's go with light gray top and pants. Okay, let's look for that variation. So that looks to be this one, light gray and it's the same exact variation. Yes, let's go for the extra large. So now we're looking at the same one, light gray top and pants, extra large. So this one is being sold at AliExpress for $18.79. So let's round that up to $19 and round this down to $6. So 19 plus six is $25 with shipping with tracking information. $25 is the same price that we're seeing here from the cheat sheet. So $25 being sold at $57 profit $32 on this successful product. Product number eight, anti-blemish skin cream. Let's take a look. So here we've got a video ad showing what the cream does. Okay, so, ooh, that looks bad. But yeah, this is supposedly what it does. I'm not really sure if it works like that. And if you ask me, it's better not to show these before and after. I don't know how Facebook allows these ads because every time I try to run an ad on a health product and showing before and after, like what you look like before and what you look like after, Facebook hates that and it's against their ad policies. So be careful when running ads on these types of products. Good job to this seller for running under Facebook's radar. But let's take a look at the stats. Okay, cause that's the important thing here. So a painless natural treatment that removes all spots, moles and freckles. Get yours here. Again, really, really bad, but 8,800 engagements, 1,500 comments, 1.3 million views. So this one is selling a lot. And here is the seller's website looking a little bit better than his ad. So $24.97 and add another $5 in shipping. You're gonna make it to $30.96. And as we scroll down, we can really see the product in action. So he is doing a good job showing what this product does. The bad thing here is look at all of these duplicated images over and over and over. So these sellers, they're working really, really quickly. They're just like, okay, I gotta add this product to my store. I'm not gonna review it. I think it's good. People are gonna buy it. I know it's popular. Let's move this one forward. That's fine because it works for them. They're making sales. They're making profit. But I'm sure that you wanna make more. You wanna make more sales. You wanna make more profit. You want to do a better job. And these are perfect examples of how it's working for other sellers where you can even do a much, much better job. I know that I said that a few times, but it's really important to understand this. If you look at a seller who did an excellent job and there's nothing that you can do better than him, well, first of all, most likely there is something that you can do better. You don't have to have a better looking website. Of course it's gonna help and you will probably end up making more sales, but you need to target the right audience. And even if they're targeting the right audiences, you can target more audiences that maybe they have not thought about targeting or they haven't put their focus on there yet. Because one seller's doing a good job doesn't mean that he's doing a perfect job and there's nothing that you can do to make it better. But this video will help you understand many successful sellers, what they're selling, where they're selling from, how much sales they're making and also what you can do better to make even more sales. Okay, so the product on AliExpress right here, $3.38 at another $4 shipping. So that's almost about seven, almost $7.5. But what's $7.5 when you're selling for $31 and making $23 for every time you sell this war remover ointment cream. Okay, product number nine, fast charging USB cable. So take a look at this one. The seller really branded his store around this product and I wanna show you exactly what he did. So for those of you who thought that you cannot brand a drop shipping store and we have content showing you that you can like just check out private label versus white label drop shipping. So here you have a brand called Titan Power. This is the product and look at the website. It's got pretty much everything down. It's got the color scheme, right? So we've got the whole green and black theme going on. And this product really looks like it is from Titan Power, right? It looks branded. It looks like you're buying from a real big business, from a brick and mortar business, from a retail business. And we've got all of these icons right here. Again, complimenting the green theme and more green and black texture. So tired of expensive cables constantly breaking. We have the solution for you and just talking more about what this product does, what problem it helps the user solve, why the consumer should buy it. And we just have a really good job being done here when it comes to creating the website. So everything here looks good. Everything here looks professional, even though he is drop shipping this product. He even has a frequently asked questions section here on the bottom. Okay, so overall he did a really, really good job, quite close to perfect for a seller. And just about a minute ago, I spoke to you on how you can always do a better job than other sellers, but sometimes you will find a seller who's doing an absolutely great job. Even though you see that he's drop shipping, you see what he's doing, you see his secrets, but now you have to do a better job if you want to take those sales away from him or even make more sales than he's making. So in this case, even if you can create a better looking website than this, what you can do is work on audience targeting and you're at copies because those are two different languages that you can speak that this seller may not be speaking at least not at the moment. Okay, so that's a good way of coming in and taking out more and more sales that this seller is making. Plus, you can always add products that are similar to this one and not just this same exact product that he's selling. But let's take a closer look. So we've got this great looking website. We've got the variations right here. So for the iPhone type C cable or micro USB and different lengths, and we've got the prices right here. So we're looking at $50 for these cables. And let's take a look at the seller on AliExpress. Even though do notice that he is giving you a buy, forget one free because he's making a lot of profit. And in this case for $50, he's giving you one free cable. So he's actually gonna purchase two from you. But for $50, on the other hand, when you're purchasing it for only $2 plus another $2 and 26 cents for shipping with tracking information. So it's gonna cost you less than $5. So if you buy two items, it's gonna cost you about nine something less than $10 and you're selling it for over $50. And that doesn't include the shipping price. Even though it has free shipping, but still purchasing for about $10. In our case, when we wrote the cheat sheet, it was 14 but nevermind purchasing for 10 and buying for 50, he did raise the price up a little bit. So anyway, it's over $30 in profit selling this fast charging USB cable, which isn't such a huge brand as you thought it was because here's the product right here. Nothing here is called Titan smart cable. And that is exactly how you can brand your drop shipping store even though you are still only drop shipping. Cause many people thought that it's simply not possible cause you don't really have control over the stock and so forth. But do keep in mind, even if you're drop shipping, when you have a product that's selling really well, you can consider actually using a 3PL service provider and purchasing this product, even 50 or 100 units just to start off with from stock while continuing the drop shipping business model to test more products because it's easy to test when you don't have to spend money on the products. You don't have to spend money to test them because you don't have to purchase the inventory. And if the products are not working, you can easily remove and replace them. So continue using drop shipping to test products and then purchase the ones that are actually selling well for you into your own stock. So you will have control over your stock and it'll be easier to not only brand the product but also brand the box, the packaging and so forth because you actually manufactured it. Okay, so this is called the hybrid drop shipping business model and we also have content on that if you wanna learn. But now let's move over to product number 10, cat tunnel bed. Let's take a look. This one is being sold for $70 being bought at around $34 leaving the seller with a profit potential over $35 per transaction. Let's take a look at the Facebook ad, the seller's website and of course the supplier slash source. So here we have Wendy Pet Shop. We have an ad copy that talks about the product. This one is actually pretty good, not that bad free shipping and 50% off expires soon. So kind of giving me that rush feeling. And here we have a video showcasing the product which is in my opinion, the best way to showcase a product adding a video instead of just images. So over 12,000 engagements right here over a thousand comments and 720 shares. So another really popular product that audiences do engage with here is the seller's website. So $49.99, but we're gonna look at the large variation. Okay, so $70 high ticket price around the medium to high, which is better because it's more profitable. So $70, as we can see right here $70. Now let's take a look at the source. So here we've got AliExpress $7.80, but we wanna look at the large variation. So $18.20. Now shipping is a little bit more expensive over here. What some sellers do on AliExpress they show a really low price on the product. But then when you get to the shipping you'll see that it's a whole different story. But that's fine, that's a part of the game. We've got tracking information available which is what's important here. So $14 plus $18 is $32, which is what we're looking at right here. So $34.20, so the prices do change every now and then dollar here, dollar there. But let's say it's $34 and we did see the $69.99. So $35 in profit and even more when the prices move. So that is a lot of profit for selling this product and we can see how popular it is just by looking at this ad right here. And even from the comments you can see right here this guy loves it, this one is waiting impatiently he can't wait to get it and looks pretty interesting. I have one, my cats love it. Finally something awesome for cats and so forth. So you can see that this one is popular making lots of sales and this seller is enjoying the $35 that he's making for every transaction. That sums up this video for the top 10 drop shipping products for the month of February. I hope you enjoyed it. Leave me a like and a share if you appreciate the value that you have here. And once again guys, the cheat sheet right here with all the links, all the information that I went over in this video is up for grabs. Leave a comment, hashtag February with your top takeaway from this video what your favorite product is and I will send that cheat sheet right over. Thank you for watching. And if you wanna talk to other drop shippers who help each other succeed join our Facebook community group and our Discord channel links right below this video. Good luck with product research for the month of February and I can't wait to hear your success story.


Alright hello everyone and thanks for coming to our talk today. We're pretty excited to be speaking in the ICS Village for the first time. We're going to talk about an ode that we found in an ICS system. This is actually going to be somewhat of a redo of our talk in track two yesterday, but we're going to focus a lot more on the challenges we faced specific to ICS systems.  My name is Doug McKee. I work for McAfee's advanced threat research team. I've got a little over eight years of experience in vulnerability research and pen testing and... And I'm Mark. I'm also a researcher at McAfee's ATR team and I have a little over eight months of experience in vulnerability research and pen testing. Alright so first, what are we here to talk about today? Obviously it's going to be an ICS system and for this it's Delta Controls and Telebus Manager and this is a building controller. So what building controllers do is they're installed in commercial and industrial buildings to manage things like HVAC, access control, pressure rooms and hospitals and they do this largely over a networking protocol called BACnet and that stands for Building Access Control Network and this is not proprietary to this Delta system. It's actually the same across the industry and so when we started looking at this system specifically we kind of ran into what we like to call the perfect storm and that's also why we started looking at it more heavily is one it's network connected and these are connected to the internet right so we always like to look for vulnerabilities that are network accessible because they tend to have a higher impact and they give us a really good place to start from an attack surface. Also when we started looking at the device we found that it had an active UART header on the board that did not require any authentication and it dropped us right into a root shell which allowed us easily to extract the firmware and to start debugging and when we extract the firmware we also found that the developers had left symbols in the binaries which again made things easier for us to move forward and I'll reference this throughout the presentation and last but definitely not least as most ICS systems do that run Linux anyway they have a busybox installed and in this instance of busybox Netcat was left installed on the box which became extremely useful when you get to our post exploitation phase. So I'm going to give you a really short high level understanding of the vulnerability in the exploit and then we're going to focus on some of the challenges. So first what is the vulnerability we found using broadcast packets, backnet broadcast packets that we were able to find a buffer overflow. We were able to turn that buffer overflow into a right what where condition and use it to gain control of execution. So once we gained control of execution we were able to redirect execution to shell code which we were able to put on the heap because of the buffer overflow. So a little bit more about the vulnerability this is what it looks like in code using the I2D compilers I've got it condensed here. The simplest way to look at it is we have a standard buffer size mismatch so in the top diagram we've got a buffer that I've labeled source and it's being set to 1732 bytes and it's being read directly from the wire. A little bit later down in the code we have a second buffer being allocated to 1476 bytes. As you can see those sizes are not the same it's eventually used for a mem copy and therefore we have a buffer overflow. And in order to get that to use to gain control of execution we used the right what where condition to do something called a GOT overwrite. So if you're not familiar in Linux the GOT stands for global offset table. This is a table which is populated at runtime therefore it's generally writable and it basically is a match between function name and address. So in our case one of the functions in memory was scdcode backnet UDP and this function was very close to where we found the volume and so therefore by overriding the address for that function we were able to redirect that function pointer to shell code which was stored in memory. So the shell code as I said we controlled a very small amount of memory on the heap which made shell code slightly difficult but we were able to do a little bit of hopscotch in memory to jump around and actually get our shell code to work in memory. We did a classic return to libc attack and leveraged the netcat that was already installed on the system to gain a reverse shell. And with that reverse shell is kind of the basis of where everything starts and from there we're able to exploit further infect and exploit the system. So I said one of the things we want to focus on in here is the challenges we face during our process for ICS specific systems and so like most people static analysis is great but we would really rather use dynamic analysis as much as possible in the exploitation process it's just simply easier. So in order to do that we compiled gdb server for the appropriate architecture and we dropped it on the system. Now as soon as we started using gdb we are instantly messed with a bunch of error messages and then the system rebooted. Now that the error messages have been redacted here at the request of the vendor but the gist of the error messages were that there was a washdog timer that failed, kicked the system and therefore the system reboots. For those not familiar in ICS systems washdog timers are typically used in order to ensure that if the system hangs for any period of time an action is taken to stop it to get it back going again and in this case the action is rebooting. Well that's obviously going to be not good for dynamic debugging but because we had symbols and because we were able to use those error messages it's actually pretty simple for us to find in software where this is taking place. Here you can see that there's three areas where a counter is being decremented by five and when the counter hits zero it reboots the device. Well from a software research perspective this is actually not a hard problem to solve. All we have to do is a technique called binary patching and we only need to patch one single byte in the entire binary. So what we did is we changed that five to a zero and by changing that five to a zero the code still executed but it effectively didn't do anything. In fact you can see up here on the screen on the right hand side that IDA doesn't even recognize that that line exists anymore because it's completely ineffective. So we did this and started debugging again and we were successful for an entirely total of three minutes and then the system rebooted again and the watchdog got its revenge and Mark and I were trying to figure out we just washed the watchdog why is the system rebooting? Well again something specific to ICS systems is there wasn't just a software watchdog but there was a hardware watchdog which is very common in ICS systems and we had forgotten to handle the hardware watchdog and so you can see that here on the UART screen that the hardware watchdog is being set every 180 for up to 480 seconds every time on boot and otherwise three minutes. So the important thing about hardware watchdogs is that if we have to deal with them as security researchers the developers also have to deal with them which means that the developers have to deal with them there's already code on the device to handle this problem you just have to find a way to get access to it. Well in our instance the developers had exported all of their functions to all of their SOs meaning that all we had to do was write a very small C program import their libraries and call their functions and have that boot up on start and now we were able to kick the watchdog in the exact same manner that the developers did to prevent the device from rebooting. So by employing both of these two methods to handle the software and the hardware watchdog we were able to debug the system dynamically which was extremely helpful for the rest of our research. So we faced a couple more challenges and for these to explain these I'm going to hand this over to my co-worker Mark. Thanks Doug. So yeah the next challenge we encountered is that we wanted to get this exploit to work remotely up until this point we have been conducting the attack on the same network that the device lives on using broadcast traffic which was nice because we didn't need to know the IP address of the device but ultimately we wanted to expand the impact to be worldwide. Okay so how would we go about doing this as you may or may not know the attack as it stands uses broadcast traffic which normally does not travel over the internet. Well thankfully a certain backnet technology came to save the day and that backnet technology was BBMD or broadcast IP or sorry backnet IP broadcast management device which is sort of a mouthful but suffice it to say it allows for the communication of backnet broadcast traffic the exact kind we're using to travel over the internet and so in the diagram on the right you see two such backnet networks connected over the internet and each of them have a BBMD system so the way this works is that traffic intended for a foreign network is first sent to the source networks BBMD it uses unicast traffic to send it to the destination networks BBMD whose job is then to rebroadcast that traffic that's all well and good but how does this help us well ultimately by using this technology we discovered through testing that this actually gets our exploit to work entirely over the internet. I want to reiterate at the time of this writing any eBMGR device connected to the internet with its default network settings could be pwned 100 percent remodel using this exploit. Now this is not the end of our challenges not by a long shot because after all up until this point all we have is a root shell to the device and persistence this is very nice but ultimately we're more interested from an impact perspective from controlling the hardware on the other end of this device whether this be HVAC systems and a data center access control in a government building or pressure rooms in a hospital that's where the real impact lies we need to get control of those. So our initial hypothesis was to use the programming already on the device and see if we can get that code to work for us. The first approach we used was to try and look at the database files located on the device's file system and these contain some information about the state of the iO hardware. Now the first thing they jumped out at us when we looked at these database files is that I had no idea what the hell I was looking at this was too complicated so we decided to keep looking. The next thing we try to look at is controlling the iO state directly from the touchscreen as you would in normal operation and then this actually generates loopback traffic and we decided if we could investigate how these packets are structured and maybe replay them we could get it to work for us and those paying close attention might notice that the packet structure is actually very similar to the database files and that I still have no idea what I'm looking at so we try something else again and what we ended up working for us is actually hooking to the functions that control iO natively and you see one of these functions in front of you from GDB output and once again symbols being left and proved invaluable here since names like can iO write output proved to be a huge indicator this is a good place to start since we knew that all the iO modules are handled through the CAN bus. Now through some further reversing of these iO control functions we discovered there's actually different functions for handling all the different categories of iO and there are six such categories for the eBMGR. You have inputs outputs variables and any of these can be either analog or binary. Now the idea would be to alter the parameters going to these functions in order to get the code to work for us but the question is which parameters do we alter and how do we do it? Well for that we did find one key commonality between all these various functions and that's the first parameter being sent. The first parameter as we discovered is actually data structure that describes the hardware being altered. In particular this data structure for the first four bytes are the ID or the fingerprint of the device and this is unique to each one. At a 12 byte offset we also discovered the holds the current state of the device which is the most crucial component but additionally at a varying offset you could also find a descriptive string of that device which is how we know this one is responsible for monitoring room temperature. So using this information is actually pretty straightforward to alter these data structures coming into these functions and thus get control of iO at least in theory. So using this information we were able to craft malware that took advantage of this hooking method in order to control the iO but ultimately we had another challenge in front of us and that challenge was that unlike the watchdog kicking code which we could just throw some C code onto the device and execute some functions that exist in memory, the handles to the iO hardware were trapped inside the program space of the existing programming. It could not be accessed normally from an external program. Somehow our malware had to be inserted into the memory space of the existing programming to do its job. How would we do it? Well the solution ended up being something called LD preload and for those that don't know LD preload is a Linux environment variable and shared objects it points to are loaded first by dynamic linker when a binary is executed. Now this is not a new or novel concept we've not come up with this it's actually a pretty common strategy in this field for inserting code into the existing program's memory space which is exactly what we wanted to do. So in our case we used persistence on the device in order to set this variable to point to our malware and thus we could insert it into the device's programming but then the next challenge was where do we put it? Well here are our options. So this is a rough overview of the the programming startup routine and through some further reversing we just decided that the best place to put it would be this function called can iO and net highlighted in yellow. The reason being is as Doug mentioned previously this is a vastly multi-threaded application it runs many threads in parallel and in order to avoid race conditions or inconsistent behavior we wanted a function that was called early initialization called once and called by a single thread and this function met all those criteria. So what does it look like once our baby malware is inserted here? Well once this function is executed our malware starts up and it runs a thread in the background that elicits for commands sent over the network by the attacker. It elicits on a TCP port and depending on the content of these commands the malware would then intercept the corresponding calls to the iO control functions all to the parameters as I mentioned previously thus granting us control of the hardware. Now we were getting close to the end here but we still had one and possibly the most important challenge in front of us and that was I couldn't afford my mortgage. Now hackers we got to eat just like anyone else so we wanted to see if we can make any money off this thing. Now obviously on stage I would not try and sell any kind of malware but purely hypothetically if I was going to I might mention that it does give you automatic discovery of all the hardware. You know sit back and relax it does all the recon for you. Hypothetically of course. Another thing I might mention is that you get remote control through an interactive TCP session. I'll quick aside how many of you here show of hands saw our original talk yesterday? Okay that's not many and I'm going to recycle my jokes good. Alright so my therapist always told me that two-way communication is key to any healthy relationship and so we decided to let you communicate with the malware through an interactive TCP session. Now am I saying this malware is going to fix your marriage? Yes that's exactly what I'm saying. Okay next you can actually see the state of all the hardware in real time so you can see exactly how much damage you're causing and last but not least send a single command and you revert the device to its original unhacked state wiping of any traces the malware is ever there. Best of all if you call in the next 30 minutes hypothetically you can get all this and more for 30,000 easy payments of 1995 shipping and handling not included and if you're still not convinced be sure to walk over 10 feet that way after the talk is over to check out our presentation you can see this bad boy in action but to show you a quick overview of how this looks in the real world I'm going to hand it back to Doug. Thanks Mark and just to be clear we're having some fun joking around but we are no way releasing the exploit code malware code or selling anything. I don't want to get fired and my boss is sitting right here so okay so now that we've gone through a little bit of the challenges and a really high overview of how this works what does this really look like and one of our missions at McAfee's ATR team is we really want to demonstrate these in means that it's really easy for everyone to see what the impact is and because of that we actually took the time to build a fully working demo unit which as Mark said is literally 10 feet over there afterwards if you want to see it in action but I'll show you the demos as videos right now since they won't let us will it over so what we built was a fully functional HVAC unit that's cooling a simulated data center because why a data center because this is one of the uses that this exact model is used for we actually hired an installer that installs these on a regular basis to build this and one of the main things they install it in is in data centers again we're going for is realistic as possible so if we take a look at what this unit looks like excuse me on the back hand side you see all the components of a normal working HVAC system so we've got the valves we've got fans we've got pumps and all this is doing is taking cold water from a cold water bin and using that to send cold air into our simulated data unit you might say there's not usually cold water in a real unit but again my boss was too cheap to buy an actual condenser so that's what we were working with on the other side we have a a glassed in simulated data center we've got all this again HVAC equipment that you'd seen a data center and we even have a raised floor with a server sitting on it generating heat for a moment I'll bring your attention to one of the sensors it's the independent temperature sensor this is the only component not hooked up to the delta system because it gives us ground truth through what the temperature is regardless of how much we messed the system up on the top we have some simple LEDs indicating different states of the equipment again mostly just so it's visually easy to see but the most important one is the alarm and the alarm's an interesting one because in in real life the alarm could be done one of two ways it could simply be an LED it could also be a siren but more frequently the alarm is used for like email notifications so that somebody knows this system's in distress so anytime we talk about the alarm that covers all of those the attacker would be able to manipulate that as well so i'm going to show you what this would actually look like if someone attacks it using our premium malware which will help pay for marx mortgage on the upper left hand right hand side of this left hand side of the screen you see the attacker running the exploit script that we ran on the top right hand side of the screen you see the the delta controller picture in picture as the attacker launches the exploit leverages the vulnerability it downloads our malware and a few other fun images and this reboots the system to leverage that ld preload method that marx talked about here you can see we've replaced the images just to prove that it's hacked we no one would ever actually do this in the world but we're just going for a visual representation also we're logging in to follow along but no one actually needs to touch or log into system so what are some of the things the attacker might do once they got to this point on the system well one thing they might do is control the outputs of the system the system takes input and based on those inputs have a reaction to the outputs so here what you're seeing is the attacker manually override several of the outputs on the screen one at a time in the bottom right hand corner of the screen you will also see that the the system is reacting accordingly so the pump shutting off the alarms turning off etc on the top right hand this part of the screen you'll notice that the delta screen is not reacting in fact it's still staying in its previous state this is because our premium malware has the ability to adjust or not adjust the screen based on the attacker's will and if we start talking about alerting alerting or whether the user's aware this is a very important feature so what you just saw turning stuff on and off that's binary outputs these systems typically have both analog and binary outputs and all analog means is they're set to some floating point value instead of just on or off so to manipulate those the attacker can do that as well through the same interface this time that you will see the delta screen update because we're going to have the attacker choose to update the screen so they're going to change the fan speed in the valve and you can again see the component reacting appropriately in the bottom right hand corner of the screen also what we've done in this malware is provided a reset function which sets everything back to its original programming without removing the malware's presence so it kind of gives the attacker a way to put things back and still have persistence on the device so controlling outputs are fun but it would be a lot of work for an attacker to have to manipulate all the outputs in order to get the outcome that they wanted remember I mentioned that these systems are designed to take input and based on that input perform some action so instead of controlling the outputs if we manipulated the incoming data then the the rest of the system would just function as normally so in the world of HVAC if the system thought it was reading 30 and 40 degrees from the data center what would it do do the same thing your house HVAC would do would turn itself off because it has no need to cool that's exactly what we have going on here and you can see our independent temperature sensor in the bottom right hand corner the temperature is rising alarms are not going to trigger because the system doesn't think it's in distress you know all the all the valves are dampers and stuff they're going to close on their own without the attacker doing anything because that's how the system's program to react when it thinks it's only 30 or 40 degrees inside the data center so this is actually a more impactful scenario controlling the inputs than the outputs and a data center again is just one example of how this how these would work in the real world but think about you know a major corporation insert your favorite name cloud provider and if their data center was to lose HVAC for a couple days without their technicians knowing being any of the wiser we could probably impact a pretty large portion of the internet but we want to highlight that data centers are not the only thing because this is a building controller and it's not agnostic to one industry these are installed across a very wide range of industries and I'm going to highlight a couple examples in those industries just so we understand the impact beyond melting down a data center so the first one I'm going to talk about is the healthcare industry and the healthcare is slightly unique in the fact that building controllers control pressure in individual rooms throughout a hospital and what do I mean by controlling pressure well by setting the pressure in different room I'm sorry by setting the pressure in different rooms at the different degrees they prevent the spread of disease so by having a positive pressure in one room and negative pressure in the other room diseases can't travel from one side of the hospital to the other side the other unique thing about controlling pressure is it's just a very very small change in pressure unlike temperature you might walk into a room and say hey it's kind of hot in here if you walk into a room you're not going to say hey the pressure is kind of low in here right so the human's not going to notice that change it's a very small change and if an attacker was to get on one of these systems make it so the hospital pressure was even throughout everything I'm talking the OR you know the quarantine room the rooms where the housing critical diseases or samples all the sudden all that stuff would be able to travel airborne throughout the hospital and that could actually affect human life that's a pretty interesting scenario if we talk about the government sector for a moment I only bring up this one because we did some online open source intelligence research and found that these systems are actually being used in certain government buildings state governments for access control well if you think about that impact if we were able to access one of these systems in a state government building for access control I don't think I have to go into too much detail on at a hacker conference of what that would provide right I don't know we would be able to access maybe sensitive data that we weren't supposed to get to social or way into engineer our way in and last but not least I'll take a moment just to highlight education just because I think it's a one that people are probably asking themselves why do I care if you can affect the HVAC system in my kids preschool and what I would argue is we often forget that because this system is completely accessible over the internet and remotely exploitable it's not only about the system itself unfortunately we don't we find that a lot of times these are not air-gapped and there's other systems on the network and being able to control a system that's on the network in a hundred percent capacity would allow us to potentially view video feeds that are going over the network or compromise other net other systems on the network which all of a sudden you might care a little bit more about than just the temperature but don't worry no one would actually connect these systems to the network so we really have nothing to worry about from a network perspective that I forgot about that Shoshodan does show us this a little different than the optimal version so at the time that we originally did this the exact model that we exploited with that has the vulnerable firmware there was about 500 these on the internet at the given time and the thing was where we discovered this vulnerability is pretty low on in the firm pretty low on the stack and it's because it digest the backnet protocol it's actually used across multiple delta controls devices it's not specific to this delta control unit and if we we look at those numbers connected to the internet all of a sudden we get to about 1700 connected to the internet that has the potential to be vulnerable to this vulnerability that give or take 10% or so for honeypots that we found on the on the internet as well so with that said I think we've covered just about all of the the impact and the details of the vulnerability I want to take a moment to highlight the vendor for this uh per McAfee is responsible disclosure policy we disclosed to the vendor uh that we found this bug and everything that we were able to do with it in late 2018 a cve number was generated and then we worked very closely with the vendor and in 2009 June of 2019 the vendor released a patch ATR has tested the patch and is happy to confirm the patch fully does mitigate this issue and I also want to take a moment just to say that Delta was awesome to work with and they were kind of what I'd like to see is the gold standard going forward for security researchers releasing bugs a lot of times I've been in the industry for almost 10 years and there's a lot of bad blood sometimes between uh security vendor or security researchers and vendors uh these guys were extremely receptive to the information we provided them with they worked with us very closely to make sure the system has been patched and this is really what we want to see from all the vendors in industry we're not trying to do anything negative we're actually trying to help you and find these bugs before the bad guys do and this this is really a good example on how effective that can be if you can get a good relationship with the vendor because they're actively patching these systems right now and we've actually already seen the patch being applied and less of these systems vulnerable on the internet so uh thank you for your time Mark and I will be here at the our demo station which is again literally 10 or 20 feet over there we're happy to answer any questions you might have further or and I think we're probably we're gonna get off stage here so any questions you have please come to our station thank you


Welcome everyone to config s code by Nilesh Mewada. We are glad he could join us today. Without further delay, over to you Nilesh. Thanks Bridgen, thanks a lot. Welcome folks, welcome to the session on code configs code, the worry free config deployments. In this session, I'll be talking about problems with configs text. And then I'll be talking about the solution using dial, which is configs code tool. And in the end, if time permits, I'll basically try and address the question and answer session questions and try to address those as well. So before we start, let me talk about the typical code and config combinations that we see into most of the projects. So essentially, you know, I'll be, I'll be using PHP and engine x configurations that as one combination during my presentation for Java and spring. We would have, so if Java and spring is the backend system, then we will have application properties and the Kubernetes configuration that can go along with that code. So just to give a quick understanding on the configurations, this is how the engine x configuration looks like. So it is essentially a simple text file with a nested and hierarchical structure being it.  I think most of us already know, it's a simple, you know, list of key value pairs and it can, you know, contain pretty much what the Java app would require to run it. Right. The Kubernetes configuration also probably most of us are already aware. It is a YAML file. So that itself means that it's a nested structure.  So we have looked at the different config files. Let me also talk about the environment workflow that I'm going to discuss during this presentation, throughout this presentation. So, you know, it's a very simple setup that I've, you know, considered.  And if you want to deploy those code and conflicts, basically, they have to run, you know, what you see on the right-hand side, the three environments. It is like the integration environment, which is system integration test environment wherein we, you know, combine different components and see whether the wiring is, you know, correct or not and we test that and then we move ahead to the pre-prod environment where again, we do, you know, some sort of a sanity test, etc. You know, PT, etc. And then if everything goes good, we basically go to the production environment, right? So if I have to, you know, give the holistic perspective in terms of the typical CI CD flow, in terms of the environment workflow that I just, you know, talked about, here is how it looks like. So we have the local environment. As I said, you know, the code and conflicts are basically, you know, checked in the grid or any VCS for that matter, which essentially, you know, triggers a particular built pipeline. So we are talking the CI CD pipeline. So CI flow will basically trigger on a commit, which essentially, you know, creates a code and a config artifact for us, right? In general cases, the config artifact is basically part of the code artifact only, but just for the simplicity and to explain things better throughout this presentation, I have kept them as a separate entities over here. So let's assume that we have a two code artifact and the config artifact separately created by a built pipeline. And if you want to deploy that in the environment workflow that I just talked about in the earlier slide, we basically, you know, add this combination into an environment first. We test it. If it looks fine, then we, you know, introduce the same combination into the next environment and so on. And that is how we ultimately reach to the prod environment. It is basically that we test this config and code together into a particular environment, right? So that is the main point out there.  Now let's, you know, start looking at the configs text, right? And how it basically influences our typical setup that we just talked about, right? So why it is a configs text? First of all, let's clarify that. It is configs text because all the configurations that I just talked about, right earlier, those are basically text files as simple as that. And we basically, humans are basically managing those text files. And in configs text, since we have, so for each environment, we will be having a separate copy, right? So for example, local will have its own copy of the country. And so is the case with the site pre product. So just to give the clarity, if we are talking about PHP engine X combination projects, we will have PHP code plus we'll have four, you know, copies of the engine X configurations, one for each environment. And similarly for the Java, for the Java projects, we'll have Java code plus four application properties, one for each environment. And KDIS configuration for each, for KDIS configuration for each of those environments, right? So now having this particular setup for managing configuration, what are the problems you see that might happen if you can add it to a chat box? Copy paste errors, yes. Correct. So yeah, most of you are right. I mean, all of you are right in fact. So let me run through those problems once again. So as everybody has mentioned, you know, duplication is one of the prime problems. It's like, as you can see on the screen on the left-hand side of that, you see a broad config, broad engine X config on the right-hand side, you see, you know, pre-prod. And as you can see, there is like nearly, you know, 93% duplication in this. And so is the case with, you know, the Java application properties that we see. And so is the case with the Kubernetes configuration as well. By the nature of how we manage this config as text configs, right? We generally, so we have such huge files to manage, right? So generally, as humans, we basically, you know, try to copy paste to manage all these different versions of files, right? So it naturally falls into, you know, having some sort of a copy paste errors. So something like, you know, pasting, pasting at the wrong place. So as we see in this example, we have pasted something before this operator rather than after it. Similarly, we might have a problem of, you know, pasting it multiple times. Like in this example, we have by mistake pasted it couple of times instead of just once.  So how we do a typical copy paste is control C, control V. But just in case while doing control V, if you forget to press control, we might see, you know, problems like, you know, having this letter V, you know, spread it into our config files randomly. It is quite hard to spot over here, but then yeah, this is also one of the problem. On the similar lines, again, if we, the cycle is like control C, control V and control S. So after we paste, we basically save it. And again, in that case, if we, you know, forget to press control key, then we will have this letter S also, you know, appearing into a configuration file, which is also might create a problem, right? Another one is like, you know, having a wrong values. Like in this case, instead of rolling update with double L, we have a rolling update with a single L, which is a, which are like the misspelled values. Another, you know, common problem is like, we forget to add update or remove a particular piece of config into other environment config files. Like for example, in this, on the left hand side, we see that there is one block that was supposed to be part of a pre-prod configuration. But then in production configuration file, we basically, you know, forgot to paste it, right? Just to summarize, we talked about duplication problems. We talked about typical copy paste errors like pasting at a wrong place, pasting with multiple times, missing to press, you know, control key while pasting or saving, you know, misspelling values and forgetting to add or update, remove a particular entry in the config file. So having talked about this, right? Do you think, so these are the problems, right? But do you think we have any more problems to deal with when we are using CAT? Just think about it. I'll take a quick pause. The answer is yes. We have more problems to deal with.  So it is like a release pipeline. When we hit release pipeline, you know, we are dealing with, you know, a code and a config, a misconfigured config. And we will only come to know about this misconfigured config at the time when we deploy our code, right? So, for example, let's say that when we deployed code and config into SIT system integration environment and we tested and everything worked fine. This combination basically worked fine into this environment, right, for the test that we run over here. Now, when we reach, when we promote the artifacts to pre-prod environment, the same code artifact with the pre-prod configuration, it did not work well.  So we can get all those surprises. So let me talk about all those problems now. So, yes, as I just mentioned, we might get, you know, unpleasant surprises while doing this deployment. So even though we see green, you know, all the test go green into lower environments, we might find problems into higher environments. That essentially means that we have to resolve those issues, right, the deployment issues. And that might, you know, result into extended downtime window, getting the approvals, et cetera, which is the case in majority of the organizations and it is not a pleasant experience in my opinion. Again, if this deployment, basically if we cross the threshold time to deploy or resolve the issues while we get during deployment, the next step is basically to roll back by deployment, which is also not a pleasant experience because we'll have to go through this entire process of, you know, deploying this code in config once again, all over again, right. The worst of the lot is basically the last problem that you can see is basically we were successful in deploying to a production environment, but with the wrong config values and we never caught those particular error, right.  So all those, you know, random issues might drop in, which we call it as a production once, right. So these are the typical problems with the config as text. So what do you think is the root cause of the problem if we try to minus config as C-A-T way. Again, I'll take a quick pause if you just, you know, try to reflect on that, right. So let me run through it. The main problem over here, as I just mentioned earlier as well, once is that we have a different copy of this configuration files for each of these environments. So when we basically run through this particular environment workflow, right, the idea is we test this code and config into a site in one of the environment, lower environment. But as we reach the next environment, we get the different copy of the config, which is never tested before into all the earlier lower environments, right. And same is the case with the prod environment as well. Once we reach over here, if this is misconfigured, we have never tested that particular piece ever before. And that is what the problem is, right. That is the root cause of the problem. So how do we solve that problem, right. The idea is essentially, we should have a single source of tools for all this configuration files. What I mean by that is we should have a single source from where we can create all this configuration files for all the environments rather than managing a different copies of those, right. For that, essentially we use config as code as a tool. And what we do into that is essentially as we write code like Java code or PHP code to create your business functionality, right. Essentially we write a code using the CAC tool to generate this configuration file. So it is basically a code that we managed rather than the raw configuration files that we managed. So let me show you how do we do it. Let me show a demo, quick demo of that. So here what we see is we have this production. So I'm going to show this Nginx configuration. I'm going to create this configuration, Nginx configuration files for two environments. One is the production. The other one is the pre-plot environment, right. Now, let me actually go through a bit of Dal details. We may not be able to basically cover in the interest of time, but then let me just go through it. So in Dal, basically we have this small, small functions that we write to create a particular. So this particular function is basically responsible for creating a small snippet into this particular configuration file. So similarly, we have this set of files, all these files, which will ultimately together will create the final file for us. Each one of them will create a separate snippet. And ultimately we will have the final configuration file. So let me run it for you. So let's say, you know, we created for a production environment. Okay, first of all, let me basically get rid of this existing conflict, right. So let's get rid of this existing conflict now for the pre-plot as well. I'm deleting those. Now let me create this. So as we can see, this is the Dal command to create this text configuration. And we are creating it for the prod environment. And we are directing that output to the prod-enginex file. Let me hit the enter. And yes, we have found that, as we can see, we have actually generated the enginex config for prod environment. Similarly, we would be, you know, doing same for the pre-plot environment as well. So as you can see, I'm using the same command, Dal command to create a text configuration. But this time I'm using a pre-plot environment to create it. And I'm just, you know, directing that output to pre-plot-enginex config, right. And again, as we see, the pre-plot-enginex config has also been populated, right. So this is how we basically, you know, generate the configuration. So just to, you know, recap on that. We, so we use Dal. We use those Dal commands in the build pipeline to create this code and the config artifact. So instead of managing it manually, this is basically a CSE tool, which basically creates all these configurations for us. And how we are addressing the problems of cat is like this. So duplication is gone because we are basically using the code. So as we manage the backend code, we manage this using the simple functions and structures and that promotes visibility. So there is duplication is gone. Again, copy paste errors and, you know, other errors are basically gone. It's gone out of the picture because it is the same code that is going to create a config for us for each of the environment. So if it is creating the right file for one environment, it will create for the right file for the other environments as well. Typing errors might happen, but the idea is that we would be able to cache those errors earlier in the environment. So basically, instead of the late feedback, we will get a earlier feedback. And yes, Dal basically provides more benefits. So there are a lot of benefits as, you know, listed over here. You can basically get more, much more information on this particular site. So basically go to dallang.org and you'll find more information about it. So that's it from me. I hope that you enjoyed this particular short session on this. Thank you, Nilesh, for sharing your experience with us today.


work is becoming really understanding and embodying systems thinking, you know, it's not something out there. It is what happens in our body. It's the way that, you know, the sound that comes out of my mouth right now is the result of systems, you know, a living system that everything is happening. And so having the space to really look for the interconnections and interconnections of everything about ourselves, our identities, and then we then expand that out to our communities and, you know, local communities because during the pandemic, the rise that there was a resurgence of hyper local communities. So you've got that. And then it gets bigger and bigger and bigger. Realizing the interconnectedness of all of that has also been quite grounding for me anyhow.