The following is a presentation of TFNN. Trading Hour with your host, David White. Call now toll free at 1-877-927-6648 internationally at<PHONE_NUMBER>. Now, David White. And welcome all to another exciting edition of the Power Trading Hour with me here, lovable, squeezively soft host. Once again, do we go into the breach dear friends? The following takes place between 2 p.m. and 3 p.m. Well, we're down 13 points on the S&P cash. We're to that two o'clock hour on the day when option market makers go delta neutral. We'll know a lot more. I did a run at about 130 and not much has changed. So I'm kind of sitting on my hands. I am fairly bearish. The question is, we've got light volume up even lighter. Well, we've got light volume up and just slightly less light volume down. But on the day that they go delta neutral, the whole idea is to reduce the risk on any options that are still out there that expire on a week from Friday. The idea is they want to make sure that if the market goes up or down that they don't have much exposure. Well, they didn't have much exposure. We talked about that before. In fact, they've got about the least amount of exposure that they've had in many, many, many, many years. The markets would literally have to go down to really for them to get hurt, maybe 200 points on the S&P cash. And that's a week from Friday. So these guys I think are scared. It may be that someone's told them something that's probably not true, which is maybe some kind of political issue coming to bear that would hurt the market. My view, though, is it's more economic, at least from what I can see now. Anyway, when they do this on Wednesday by about two o'clock, if the market is going to see a big move, if it was the effect of them going now delta neutral that really caused the movement on that Wednesday, whether it's up or down, you see a great deal of action in the last couple hours. So we'll watch it closely. I don't see much as of yet in what's changing. Now, 80% of the time the market is higher than today's low week from Friday. It's just got a bullish bias. There's a lot of reasons why, but mostly people buy a lot of puts. And the easiest way is to just hold the market up for a little while, tell the monthlies, which are far superior in number and dollars. And of course, the weeklies have such high decay rates that you can hardly ever make money on those. But when you actually look at the monthlies, much more of the decay is out of them. So there's a lot more risk. So they don't ever really try to keep the weeklies up. The price for the premium in there generally rules out any chance of ever making any money in the weeklies. You can, but where it's fairly low on the monthlies, it's extremely low on the weeklies. But I will tell you this, that where on a monthly basis, it's an average of 1% higher a week from Friday. When it is lower, it's a average of about 3.5% lower. So if we start moving down, if we can't start to rally, if you start looking at this market and we go through a few days and we just continue to creep down, look for a day where the thing is just going to blow out all the people on the lows. That's normally, if you're short, where you want to cover in on the short term. But, you know, we're off 13 points. I don't see a lot of buying the volumes fairly light. Again, it was light down. It was just lighter up. So it's not like we've got a dynamic distortion in the market between up and down volume. We just have light volume, both up and down. As we start off the show today, the CBOE volume on the way down early in the day. I had a lot of juice and we're already doing 4.4 billion shares. That's compared to, what was it, about 6.9 billion shares, 6.8 billion yesterday. So we are doing extremely much more volume than we saw yesterday. There isn't a lot of movement in what we're looking at in the dollar. And of course, if we want to look to the TLT, this thing's been banging on this 120 level, which has been support 122 kind of at resistance. But the question is, if we just can't continue to hang around this area, the question is, do we just gap down into the 118s? And certainly it looks like it could do that. Looking at a lot of other stocks, yeah, you're down on lighter volume, but both at long-term bottoms and long-term highs. It's not uncommon to see 3 or 4 or 5 days as the market moves lower until you start really getting some serious volume on the way back. So while I'd say intermediate and short-term highs and lows can come off with volume, and those things may run 15 or 20 days, when you're actually looking at very long-term highs and lows, it's not uncommon not to see volume for a while. I remember seeing, I think it was the S&P low at about, I'm going to say 759. That sounded about right. I think that was about right. And one of my original mentors, Tim Ward, said, you know, it's not uncommon to see volume, especially off that first leg. No one wants to buy it, and the volume just doesn't come in. And the volume then comes in later. And certainly all the way from that, I think it was about a 759 low all the way up past 1,000. One of the biggest percentage moves in a long time, there wasn't any volume. Everybody didn't believe it at all. And of course, now we're up 2,700 and change years later. But it was one of those things where that was an incredible long-term low and some of the incredible long-term highs that we saw that led to that low also had very, very light volume for a little bit of time. Not an instant.  It just was very, very light. You can give me call it<PHONE_NUMBER>. A polar bear vortex. Oh, I'd like that. That was my Hollywood name, polar vortex. That's us. But most people don't know that. Anyway, yeah, we're kind of drifting up a little bit. That's not uncommon this time of year. And the very light volume coming in to the last day of fun buying. So we'll probably have a slight bullish bearish, bullish bias into the end of the day. But if we don't get much out of here, look out for tomorrow. We'll be back in a minute. The Taz Profile Scanner is the most revolutionary piece of trading software that you will ever try. Wouldn't you like to approach the markets with confidence as you begin your trading day? It's likely that you'll be faced with lots of decisions. In order to make the best decision, the first thing you'll need is a strategy that will help you minimize your risks. Whether we're in a bull or bear market, a good strategy is to have the tools needed to help you scan and analyze the markets before you trade. The Taz Profile Scanner instantly scans and filters over 2,500 global financial markets such as stocks, ETFs, commodity futures and forex. Headed by Steve Dahl, president of Taz Market Profile, the Taz Profile Scanner understands that in today's technological world the use of top-flight software applications, automated trading algorithms and technical analysis expertise is essential to successful trading in today's market. Whether you're looking at the trade matrix, the ETF heat grid, the market breadth, the landscape charts or the many other features of the Taz Profile Scanner, this is a piece of software that will revolutionize how you look at the markets and set up your trades. The team at Taz has even put together a 12-part video series to walk you through every aspect of the Taz Profile Scanner which you can find directly on the Taz Order page at tfnn.com. Sign up now for only $97 a month with a risk-free 30-day trial so you have nothing to lose and everything to gain. See for yourself how you can harness the full power of the Taz Profile Scanner by visiting the front page of tfnn.com today and you'll find the Taz Profile Scanner under the Services section. Remember, with a 30-day money-back guarantee, you have nothing to lose. Don't let another day pass you by without trying out this amazing piece of software that will revolutionize how you look at the market and how you place trades. Sign up today. Absolutely free for 30 days and greatly enrich your knowledge of these markets and how to make your money work for you. Details on The Tiger's Den are on the front page of tfnn.com. Tfnn has launched our brand-new website. You can still visit us at the same tfnn.com URL but when you do, you'll see a new and improved homepage with a much simpler navigation whether you're watching Tiger TV live in high definition or just accessing your newsletter subscriptions. We even have new pricing in six months and yearly options. Check out the new tfnn.com now and experience all the upgrades. tfnn.com, educating investors. Call now. Toll free at 1-877-927-6648. Internationally at<PHONE_NUMBER>. We're back and we'll do a little history and then we'll get into charts. And it's all just competing. On this day in 1889, Bear began distributing aspirin in powder form to physicians to give it to the patients one gram at a time. Brand name came for A for acetyl, spur from asperia plant, and the suffix N commonly used for medications. Again, quickly the number one selling drug worldwide. Bear, of course, ended up giving up the trademark for aspirin as a result of the First World War and reparations. It continued on to be a huge drug. It was until 1996 until Bear actually bought back the company that had the rights to aspirin and bearer in the United States. They got their name back. So a long, sort of, cutest route for aspirin as one of the best pain relievers of all time. It had been around for a long time as a white willow bark, but it really, in that form, really messed with people's stomachs. But it would bring down fevers when they made the synthetic version of it. And of course, the buffered version of it happened a whole lot or actually ended up being much better. And of course, still the number one drug worldwide. On this day in 1889, what else do we have? Well, again, we're just kind of playing around here again. This is one of the asymmetrical trade weeks that you normally get, and that is it's either going to be up a little bit or down a whole lot. The chances of it being up a little bit are fairly good, probably 75%. The chances of it going down, though, are maybe three or four times what it can go up. That's at least my reading of what options are. One of the other things you can look at is whether or not they can rally it today. This is one of those rare days when we both have the end of fund buying and options expiration going delta neutral on the same day. So maybe a few cross currents out there, but man, we've been going through it for almost over two weeks. How many stocks have come up here and just died at these levels without much push? I would suspect that we'd have to have some kind of catalyst, and that catalyst has been selling the sizzle, not the steak, i.e. selling a trade deal. And trade deals never come this quick. I think they're going to come about four times quicker than traditionally. But that still means that it could be a year away. I certainly don't think we have anything in the next three months that are going to do that. 2777 on the SMP cash as we start the show and again, the volume actually was pretty good this morning. Kind of tapered off in the afternoon as it normally does. And of course, the thing to watch is probably the last 30 minutes of the day. Four and a half billion shares as we get into the first segment of the show. Let me call<PHONE_NUMBER>. What else do we have here? Looking at some other things going on here. We'll keep it on it. One of the things I wanted to get to first was Micron, and we'll look at some other charts. Saw that that one was off fairly strongly. I suspect this is a offshoot into a lot of articles I was reading about how much memory that the NVIDIA AMD crowd is going to be buying in the near future. But certainly down volume is going to be more than yesterday. And what do we have today? 36.2 million shares on Micron. 29 million shares yesterday. Let's take a quick look at the SMHs. I mean, you had some good news in this sector, but I don't think it was enough. AMBA, which is type two quick AMBA, which is the company that makes the compression video chips for cameras like GoPro's and that kind of stuff. It's still finding people buying those kind of cameras. GoPro just at the price point that they're at isn't getting that kind of business. You finally got back up to this gap down, which goes back to what is that? June 6 of 2018, we came down with almost 11 million shares. It came up, touched that gap again today on 3.3 million shares. So again, while there may be stocks outperforming others, it still is one of these things where probably only 20% of these stocks that have big volume are able to break out and hold the previous highs. What else do we have? I wanted to go back to some of these other ones. Again, you can give me call it<PHONE_NUMBER>. OK, XLU, which is the utilities sector, is up at previous highs and hanging here. This one looks fairly horrible, like a great deal of the market. And that is the energy off these lows. In the case, this one's the December 26 low is 30, 35% off all the way back up to the December 13 tie. That was $57.17, 26 million shares. Yesterday got into it with 12 million shares. Today you're into it with 8 million shares. Again, there's just, you can go back to the 22nd of February where you got into 15 million shares, but you're still 11 million shares short. And there just isn't a lot of juice. Ideally, if you really bullish on these things, you should wait until they pull back. If they pull back on light volume, then you get an opportunity to take another run up there. Maybe the next time it will break and get some volume on the way back up. Right now, I just do not see it. Almost all of these stocks we've been talking about for the last two weeks have come up and run out of juice. Talking about juice, Aqua America water company WTR down on the 23rd of October with 3.3 million shares got into it on December 13 with just 770,000 shares. You got the highest $623,000 shares at $36.40, and that's just enough. Now that rolled back down on very light volume today, but talk about incredibly light energy except for one day in this one, it's had it. Now, not a lot of money to be made on it, pretty small range in the stock, but one of the ones that shows that there's very little juice to go after these. VV, which is Vanguard large cap, just hit the highs did so on a little lighter energy, slightly lighter than the December 3. So again, what's really disturbing me is the energy off these December 26 lows that have been light, light, light. We're going to go to the break. But like said in the Philippines, any years ago, I shall return. The path of least resistance is David White's daily trading newsletter, and if you're looking for active trading ideas, then now's a perfect time for a 30-day free trial to this powerful daily trading advisory service. David uses his years of trading experience to offer his subscribers his trading ideas each morning in his path of least resistance newsletter. Using a combination of equity trades along with options, David keeps his subscribers up to date with all pertinent market information with intraday afternoon updates when warranted. Don't miss out on this great chance to get a 30-day free trial to David's daily newsletter, The Path of Lease Resistance, with no obligation to pay anything. David has been delivering solid recommendations for his subscribers recently, and if you'd like to see the type of newsletter he delivers every morning, then visit the front page of TFNN, and you'll find The Path of Lease Resistance under Trading Newsletters. For all the details and to start your 30-day free trial today, log on to TFNN.com now. Hi folks, Tom O'Brien here. If you'd like to get my daily newsletter, Market Insights, then now is a great time to sign up for a 30-day free trial. Every morning by 9.30, I send out my morning letter to subscribers with market commentary on a variety of markets, currencies, and commodities to keep investors up to date on the day's trading action. Included in Market Insights are specific buy and sell recommendations for stocks, ETFs, and even options, with stops and price targets included for every trade in my newsletter. If you'd like to try my newsletter at risk-free for 30 days, then head over to the front page of TFNN, and you'll find Market Insights under Trading Newsletters. I use my years of trading experience to bisect and dissect the market every morning, and give my subscribers the most important information they need to know for the day ahead. I even issue afternoon updates for my subscribers whenever warranted with important market action. I'm always scouring the market for the next great trading opportunity. Sign up for your 30-day free trial to my daily newsletter, Market Insights today by visiting the front page of TFNN.com. Well, go get them, folks! TFNN is excited about our new software charting program, the Art of Timing the Trade Chart. In collaboration with Tom O'Brien and using his best-selling book The Art of Timing the Trade, your ultimate trading mastery system, David White has programmed an outstanding piece of software that will complement any trader's methodology. Using this first-of-its-kind program, the Art of Timing the Trade Chart allows you to scan thousands of stocks for Fibonacci formation setups including guardleaf, ABCs, butterflies, and much more. The Art of Timing the Trade Chart is designed to help you when scouring the markets for stocks just beginning to form the trading patterns that many investors spend days, weeks, or even months searching to find. And right now, we're offering licenses available at only $79 a month. We are so confident that you're going to love this new charting software that will even give you a 30-day unconditional money-back guarantee. Don't miss out on this incredible new piece of software. Get your copy of the Art of Timing the Trade Charts today by visiting tfnn.com. This segment is brought to you by Think or Swim. For more information, just click the Think or Swim banner on the front page of tfnn.com. And we're back. What do we have here? We've got to find this. It's buried back here. First one is Pete and San Francisco Thought on Western Digital Puts. I kind of like it. I would have liked it a lot better yesterday and the day before, which is rarely where you needed to pull the trigger on them to make something. I looked at everything real quick. You've got about three days to cover, which isn't bad in Western Digital because eventually this company is going to zero. The question is timing on when spending hard drives get replaced by SSDs. And I think you can make a case that that's really moving along, especially in the data center business and in PCs. Kind of rare to get one, except in the cheapest out there. I would love for this thing to bounce maybe to 50, 51, 52 and a half would be perfect. I don't know if you ever get it. I do like the way it came up. You had a lot of shorts. It's pretty much wrong. Most of them out of the market. But, you know, about $43 is the next target. That only gives you about five bucks out here. And you've had some high shorts in the last couple of days upwards of 25%. That would suggest that you are going to get an opportunity to see this bounce. You may not. But, I mean, the odds aren't real clear. They're probably like 60%. You get a bounce. 40% you don't. So as far as I'm concerned, the risk reward, especially for options, would be probably 50s for April. And I would say that you want those... Yeah, you want those in April. But you want that when the stock is probably 51, 51 and a half, you might get one real good day where they run the shorts and then you get an opportunity to set the hook. But, yeah, it's already looking fairly weak. You did get your clothes under the nine-day moving average today. So you could have taken it at the open. That would have been tough. But again, anywhere above the nine-day moving average with light volume, you could pull the trigger again. You can give me a call at 877-9278.<PHONE_NUMBER>, he said. What else do we have going on out here? We're off 13 points kind of hovering out here, not seeing anything in the way of volume. Again, volume just slightly more on the downside today than it's been on the upside. So it's still a slight bias. And we may see that bias run into Friday now. I suspect. What else do we have going on here that we want to take a look at? We'll go back to the list of stuff. Trimble, we looked at that earlier last week. A lot of these are starting to roll over today. That's a 1.6 million share high on December 3rd. $40.35. You went to $40.80 a couple of days ago. You've got your confirmation today that it's made the high with what's going to be a solid close below it. Again, you want about three days out here for this thing to actually give it up and confirm that this thing's going to sell off hard. SXC, which is Sun Coke Energy. I don't know why that one's in there. Pump. That should be pump and dump. This one $20.91 on May 9th. And you got that with 3.1 million shares. Got into it with 1.8 million shares two days ago and instantly rolled back over. A lot of these energy stocks look very, very weak. And of course, that's what really kind of held the market up over the last few days. Without the energy segment, it is now going to be very tough for this market to rally. Another one, Portland General Electric Company, P-O-R, up at its highs. And again, when you start looking for electric companies, that tells you people are looking for places to hide their money. December 12th, $50.40, 883,000 shares. Got into it yesterday with 600,000 shares. Today, you just have 235,000 shares. Again, it's not that he's going to make money. He's going to lose less money. And fund managers thinking a market going lower that have to be 100% invested shift as much as they can to the stocks that will lose the least. Power shares preferred portfolio PGX. I had a high with almost 10 million shares on July the 6th of last year, 1445. You got into that the last couple of days. 1.8 million shares, 1.6 million shares. Today, you have about 1.8 million shares. So again, you're back up to those highs. And any close below 1445 puts another pin in the sell column. Liberty Global, LILA, L-I-L-A is a symbol on that one. Testing its previous November 8th high, $20.01, 742,000 shares tested a couple of days ago with just 279,000 shares. Yesterday, 326 today, 154,000. This is really, like I said, we're in a very good position here. Oh, is that right? Oh, no. Yep, it is right. I just wanted to check these because there have been bad quotes out today. So we're now down 15 points. And yes, there's another email here. Yes, a deliberate downside move today would probably be a fairly good indication that we will sell off all the way through options expiration. Liberty Global, L-I-L-A, again, we saw that energy wasn't that bad on the way up. Just never really amounted anything for these highs. We've been talking about some of these rail companies. And there was some news earlier in the den about the price of taking containers back and forth. Both that and the Baltic Dry Index both look rather bad. Kansas City Southern up on very, very light volume today. Again, any closes now below the nine-day moving averages tell you that these things are probably ready to give it up. HPQ, blew apart, no big deal there. HAL, Halliburton again, this whole energy sector on sale now kind of was holding the market up. Let's see what else is out here. Four Scout technologies. I think we talked about this one last Friday. Another one that was going above previous highs. In this case, it was the September 21st high at $40.96 with two million shares. Got into it with about 900,000 shares, maybe a million, but certainly testing the previous highs with a lighter volume. Now, this one did have a big boost of energy back into February 8th. And if we have a horrible market, I wouldn't be shorting this one. But I would tell you that around 30 to 50 on light volume, this may be one of the better looking stocks out there. If it gets thrown away and thrown out with the bathwater of a market pulling back. CSX, we talked about this one. It still hasn't quite made it up to its eye. And so the energy on this one, a little bit better on the way up. Certainly a little better on the high end, but not very good either way. Give me a call<PHONE_NUMBER>. The investment is anywhere from $30,000 to $75,000. The interest paid is 7% yearly paid on a monthly basis. According to bankrate.com, the best rate for a four-year CD in the country as of February 20th is 3.1%. A $50,000 investment at a normal four-year CD rate of 3.1% would give you income of $1,550 per year or $6,200 over the four-year period. The same $50,000 investment in the Tiger First Mortgage Program would give you $3,500 per year or $14,000 over the four years. What should you prefer? $6,200 or $14,000 of interest on your investment. If you'd like more information about the Tiger First Mortgage Program, you can call me at<PHONE_NUMBER>. That's<PHONE_NUMBER>. It's affordable and must have for every trader looking to gain a competitive informational edge in today's markets. TFNN Newsletters cover every aspect of the markets to offer you the very latest in market news. Plus, new subscribers get to test drive our newsletters risk-free for 30 days. From all aspects of the markets including stocks, bonds, metals, commodities and tech, there's a newsletter to fit your needs exclusively from TFNN. Stay informed each day you trade and get the competitive edge that will help you stay ahead of the game. Visit our newsletters page by going to tfnn.com and click the newsletters button near the top of the page. TFNN.com Educating Investors Will the S&P 500 continue to climb? For bold trades on U.S. large-cap stocks in either direction, trade SPXL, SPUU or SPXS. Directions daily, S&P 500, bull and bear, leveraged ETFs. Direction leveraged ETFs. An investor should carefully consider a fund's investment objective, risks, charges and expenses before investing. A fund's prospectus and summary prospectus contain this and other information about direction shares. To obtain a fund's prospectus and summary prospectus call<PHONE_NUMBER> or visit Direction Investments.com. A fund's prospectus and summary prospectus should be read carefully before investing. An investment in the funds is subject to risk including the possible loss of principal. The funds are designed to be utilized only by sophisticated investors such as traders and active investors. Distributor for side fund services LLC Don't forget you can listen to TFNN live on your mobile device 24 hours per day. Go to TFNN.com and hit watch Tiger TV. That's TFNN.com and hit watch Tiger TV for the latest market information. And we're back. Oh, had a question come in from the email about NVIDIA and AMD and my comments on memory. And yeah, this thing did have a little bit more volume and broke the previous high of January 25th on February 25th. That's NVIDIA, by the way. But it instantly couldn't hold that high. It slowed out. In fact, they said a couple of things over the last couple of days about how it's probably going to take them all the way to the end of the summer to roll through a lot of the inventory. So just look at that. I think if the market did pull back, you could see 131 on this one, maybe 125. And, you know, you've got about the right amount of time for them to turn around a market looks about six months forward. So any dip down here that could be found with light volume might be the buy for NVIDIA for a fall rally, but you may need to may take all the way into May to find that low in it. I'll take a quick look at AMD. So on AMD, you're down not much volume now, but certainly it looks like it's going to come back to $20.50 somewhere in that area. Of course, these folks don't do just video cards, but also processors. Things have been doing a little bit better for them. They are calling it coming back down on lighter volume to these nice gaps up back on January. But again, you know, I don't know if you win that low comes in if it's going to take a while or to come out. But $20.50 somewhere in that area is where you want to be looking at AMD. Serious moron, not a lot of stocks out here that are testing lows at the present time, but there are a few. The Sears one co and he did it with more volume than the January 2nd low energy off the top to. So I'm looking for these also is confirming signals over the next few days as they bust through those original lows to see what else we have out here. Some of the other stocks that are out here at lows are carbonite. I talked a little bit about this one yesterday and how beautiful it looked on light volume. Yesterday, you had 881,000 shares compared to the 4.4 million shares you saw on February 8th. So you got a little bounce, but again, probably not going to be enough, but it may tell you that if we get a couple of weeks of selling and this thing can hold out at 22 bucks, you might find some kind of viable low. Let's go back and check in the markets. Still off 15 points in the S&P cash off 125 in the Dow Azdex off 59. Let me update those just to make sure they're right. Yep, looks like it.  Yeah, off one and a half percent. Okay. Anything else going on? I want to look through there. Someone asking about Intel by NTC. So we'll go to that one. As I said, I was looking for 53 on this. I got up to 5410. Let's go a little bit farther back here, which was just the gap. It pretty much did exactly what it should. It's down back on June 18th with 40. It's called 41 million shares. You got into it with 21 million shares on the fourth. And now you're rolling back over. Support's going to be somewhere in around that $52 range. Alibaba back up against the August 23rd high. That's 18650 at $79 million shares, excuse me. You got to 1804 with 15 million shares. And you just never really had a sign of strength above that. This looks like it easily come back to 177, which is just the gap this one had. But I mean, you're basically 80 million shares to 15 million shares in around numbers. Energy wasn't as bad as you would think on the way back up. But again, extremely light test of that high, which makes you think you're going to at least go back and test the last gap up in that one. I had some questions about Netflix, Netflix and Apple. We'll talk about that. Netflix never got back up to its high. There's some arguments out there today with numbers about Apple and Apple paying Netflix the drive through to be able to use their product on iPhones. Kind of interesting to see that Apple makes about $3.50 each month because them allowing you to use the app on there and buy it through their app store. It seems somewhat confiscatorial, in my opinion. Apple just kind of gone sideways here for about 15 days. I continue to think this is a setup for Apple that is distribution. And I don't know, maybe it's Buffett on the other side of that, but every time it goes a penny higher, he's there to sell. But it's hard to tell. Other questions on Amazon. Amazon continues to act poorly and badly in public. Their stock, kind of the same thing. To January 31st, you had the big spike high, the island reversal, and a gap lower. That came on 11 million shares. Got into that with 6 million shares on the fourth. Yesterday, 3.7 and on the sixth came into it. You did actually today, you did 3 million shares. So there just isn't really much juice out here. Now, this one for a big cap has maybe the biggest change in energy on the way back higher. February 8th low up was about 30% less than the energy off that January 31st high. So pretty risky hanging on to it here. We'll look at Microsoft and what else? You never really made it up to that. Well, you kind of made it up to that high on December 3rd. We were thinking that maybe push it up to the October 3rd high. Again, I mean, massive less energy on the way back up off this December 26th low. This is extremely telling for the tech side of this market. And of course, you're about 7 and about 6 million shares light off that December 3rd high on Microsoft. The big thing for me has been this energy that just never really has come back in.  Now you're getting to the point where you're starting to look for cell signals. And Microsoft may be a little bit longer and pulled up a little longer than the rest of the market, but is extremely weak at this point. We'll be back in a minute. I'm certain you are or strive to be one of the best of the best at everything you do in life. It's the most common trait that we tigers and tigers share. If you're looking to become the best of the best when it comes to managing your money, let me teach you to do what most wealth managers tell you can't be done, which is how to time the markets. I'm Steve Rhodes, author of Mastering Probability. And for the last 12 months, Timer Digest has been tracking my newsletter signals, which have earned me the ranking as their number one market timer in the nation for the S&P 500 for the last 12, 6 and 3 months. Timer Digest also ranks me as the number one market timer for gold as well. The fact is, markets can be timed. And I'll teach you the exact set of tools that I use that has transformed me into one of the best at what I do. Sign up for Mastering Probability today by clicking on the newsletter tab on the homepage of TFNN.com and get immediate access to workshops where I take you step by step how to use an extraordinary set of tools as well as provide great market calls too. Sign up today. This newsletter is the vehicle from TFNN to capitalize on these opportunities. This is the go-to newsletter that identifies, monitors and profits on mostly little known cutting-edge companies with great long-term prospects. David's experience is as an inventor of Emmy-winning animation products for TV and Hollywood that propelled a company public. Match that with 14 years as a full-time trader and he's uniquely qualified to guide you through the light-speed world of ever-evolving high tech. If you're ready to ride the next big technology bull market for less than $40 per month, log on to TFNN.com and get your two-week free trial to the Technology Insider. Get in on the ground floor of the next big thing today. Since 1984, Basil Chapman has been using the Chapman Wave methodology to advise traders of his expert market opinion. While originally hand-drawing charts from the late 1970s into the 1980s, Basil noticed that prices under most circumstances virtually always had a certain number of legs to the upside before declining sharply. Later Basil found that computer software which included the standard market technical indicators enhanced the degree of accuracy in calling price turns as well as market trend calls. Thus was born the Chapman Wave sequence. Using the Chapman Wave methodology along with other indicators, Basil Chapman advises his subscribers of his expert market opinion each market day with his opening call newsletter. Right now you can get a two-week free trial to the opening call, Basil's daily trading newsletter, by visiting the front page of TFNN.com. Cancel at any time during that trial and pay absolutely nothing. Get your two-week free trial to Basil's newsletter, the opening call today by visiting TFNN.com. Next on TFNN. And I had a couple more emails come in. Yeah, I'm still looking for $26.50 as the pullback spot for this market. And maybe it's just on incredibly light volume and it takes a while, drives everybody nuts, but it certainly looks to me like we can get it. Options continue to look like the market makers are incredibly worried about any kind of or not worried about higher prices, but really kind of worried about lower prices. One of the markets where they could lose maybe the least amount of money in many years, they've set it up to where the stock, the SPY would have to get down to $235 before it would really change much in the risk reward for those folks. So, you know, am I looking that we could go down another 20 points? I think we could on the SPYs through expiration. May not get there, but I think that will continue. As we said earlier in the week, I had a couple of theories. One of them is that the many people, not all, but many people in the five highest taxed states are going to be selling a lot of stock to make up for those taxes that they told us all how much they wanted to pay, but when it actually gets to the point of actually paying higher taxes and why the rich should be paying tax more are kind of grumbling a bit. We were just saying it, we really didn't mean it. And of course we had the governor of New York out job boning today telling everybody how horrible it is and why everybody's fleeing the state. So, I don't think there's anything new in the information there other than the fact that how long it takes for these guys to sell enough stock to pay their state taxes that they used to write off. And a lot of people can make the case that that was a way of pushing taxes to other low tax paying states. But that's it. I just expect that we're going to have a little while of tax selling continuing. In the meantime, sell when you can, not when you have to. We'll see you here tomorrow. Same back channel, same back time.


antisocialist behavior. I've been watching the John Adams documentary drama on HBO and a quote that he made in the court of Madame Helvintus, while an assignment to Paris really stuck with me. Quote, I study war and politics so that my children may study mathematics and philosophy, and I hope that one day their children may have the right to study poetry and painting. That's an end quote. That's an amazing quote. That is fantastic. I want to read that again. I study war and politics so that my children may study mathematics and philosophy, and I hope that one day their children maybe may have the right to study poetry and painting. What an amazing world it would be if that was the motto of the world, right? And studying war and politics should, I would have said I study economics and politics because war is a byproduct of our current totalitarian, inverted totalitarian states, right? So economics, politics lead to mathematics and philosophy lead to poetry, painting, right? And gang, don't forget, Fréassange, Fréassange, Fréassange. Julien Assange is a publisher and journalist that has been crucified for trying to bring transparent and accountability of capitalist power to humanity, something that humanity, our society desperately needs. For more information, see wikileaks.org, defend.wikileaks.org, or our Julien Assange and Wikileaks playlist on CensorTube.


Pneumonia is one of the leading causes of death globally, with viruses, bacteria and fungi being responsible for its development. It is difficult to diagnose pneumonia based solely on chest x-rays, so researchers have developed a new deep learning framework for detecting pneumonia. This framework uses pre-trained models from ImageNet to extract features from images, which are then fed into a classifier for prediction. Five different models were tested, and an ensemble model was created that combined the output of each model. This model achieved an accuracy of 96.4%, with a recall of 99.62%. This model has the potential to improve the speed and accuracy of pneumonia detection, making it more accessible to both experts and novices. This article was authored by Vikas Chahan, Sanjay Kumar Singh, Aditya Kamparya, and others.


Let's do it. So can everybody hear me OK? Is this mic picking me up? Awesome. OK. Well, thanks for having me. So Stuart gave me a pretty good introduction. Usually I pull this slide up. So this is my profile page, like my staff profile page on Wikipedia for being Wikimedia Foundation staff. And so there's this statement that I have beneath my title there, and I like to point to it. Because it can help me contextualize the kind of talk that I'm giving. So it's think big, measure what you can, and build better technologies. And this talk is going to be a lot about how I see us building better technologies based on a lot of thinking and a lot of measurement and that sort of stuff from the past. So there's going to be a little bit of discussion of the measurement side of science around data. But I want to talk a lot about engineering things and how they affect the social context of Wikipedia. So before I actually get to my outline, I want to put us in a little bit of context. So Planet Earth has about 7 billion people in March of 2012. But Wikipedia has about 500 million readers on a monthly basis. So not quite the scale of the entire planet. It's about 1 to 10. The editor population of Wikipedia is a minuscule fraction in comparison to the number of people who edit Wikipedia. There's about 100,000. And so that's about 1 in 10,000 ratio. And so there's actually a dot there. It's important that you understand this. That dot is 1 pixel, and it's way too big. But I still had to circle it so that you could actually see that there is, in fact, a dot there. So I want to zoom in on this spot because we're not done going down in the scale yet. So I showed there's this 1 to 10,000 ratio between the people who edit Wikipedia and the people who read Wikipedia. Well, the Wikimedia Foundation, the company that I work for, is yet substantially smaller than the group of people who volunteer their time to work on this project. There's about 200 and 230 of us who work at the Wikimedia Foundation, depending if you have contractors and part-time workers or not. I certainly do. And so we're tiny compared to the community of people who are working on these projects that are around Wikipedia. And so I work on the research team at the Wikimedia Foundation. And we're a lot smaller. I didn't want to actually make a dot for us. But we have the team roughly split between people who specialize in quantitative methods and people who specialize in qualitative methods. And so that's research and data and design research. And I emphasize specialized because we all dabble on both sides. But I primarily bring a data and analysis focused approach to my research. OK, so now it's time for the story. And that brings us to the outline. These things need to be in threes. And so I have three things for you. So first, I want to talk to you about Wikipedia as a socio-technical system. And I'll get into defining that. I'm going to try and get you to think about Wikipedia in a certain way. So I'm going to try to bring in some systems thinking and some biological metaphors. Are there any biologists in the audience? OK, you're going to hold me accountable. Usually I don't have biologists in my audience. So if I say anything crazy, yell at me. OK, then I'm going to get into a critique of something that's happening within the system of Wikipedia. This critique of how Wikipedians use algorithmic quality control. And in order to talk about this, I'm going to draw from standpoint epistemology. And I want to talk about how we encode our ideologies and the technologies that we build and the values that we bring to them. And then finally, I want to talk to you about this project, the system that I've been working on that I envision as an infrastructure for socio-technical change. How it's aimed to be a catalyst to move us forward from a place that we're sort of held up. And I'm going to talk about some things that are drawn from the feminist literature, specifically in Elmorton and this idea of hearing a speech as opposed to speaking to be heard, empowering people rather than having power over them. And specifically, too, the dangers of algorithms operating in subjective spaces. OK, so let's get into the first part, the socio-technical of Wikipedia. So what is Wikipedia? Well, it's the world's largest encyclopedia, arguably. It's about five million articles in English. But we have a lot of articles in other languages. English is by far the biggest. So in order to show you the kind of things that are in Wikipedia, I want to show you one of my favorite things. So this is a list of lists of lists. It's exactly what you would expect that it is. Wikipedia's create these wonderful things. To show you how this works out, I want to let's just dig into that first link there. This is the list of ancient kings. So if we go to that article, you get this next list. Of course, this is a list of lists of lists. So this is a list of lists. So we have, at the very top of this, the list of pharaohs. And so if we click on that link, we get, so this actually looks like an article, but it is a list. It's got an introduction to explaining what the list is below it. And so if we actually pick on the first pharaoh in the list of pharaohs, we get to pita. And note that I actually know that the p is pronounced because Wikipedia has these cooler pronunciation keys. People get really excited about making sure that they're good. And so I wanted to give you this, because first of all, I want you to check out the list of lists of lists, because it's really cool. It's a fun thing to just go dig into on Wikipedia. But it helps give you a sense of the depth of things that are in Wikipedia and the things that volunteer contributors like to build. They like to organize this information. They do a really good job of getting the details right. So Wikipedia is also a Wiki. Anybody can edit their shared authorship. It's based on this software called Media Wiki. And it flips the publication model. So you publish first and review later.  And so if there's a problem with that, we have to review it after the fact and clean up damage or clean up vandalism and that sort of thing. And so this is a big reason why people thought that Wikipedia wasn't going to work when it was first coming about. So it's also an online community. Like I was saying before, there's about 100,000 active editors. It's actually more like 110,000 active editors. And they have forums. So this is this village pump section. So the main forums for English Wikipedia. So we have forums for setting up policies, for solving technical issues, for writing new proposals, for the idea lab is for setting up proposals that might need engineering resources or a new big initiative or something like that. And of course, we have miscellaneous because not everything falls into that category. There's also subject matter focused groups like Wiki Project Video Games and Wiki Project Medicine that focus on specific subsections of the topic space in the Wiki. But so when I think about Wikipedia, I often think about it as a system. A system that has inputs and outputs. Wikipedia is a system that converts available human attention of people who are just sort of interested in this stuff on the internet into high quality encyclopedia material. And the nice thing about this formalization, so of course it's limiting in how we look at Wikipedia. But by formalizing how we look at Wikipedia in this way, we can ask questions about productivity and efficiency and that sort of stuff. I can start applying things from my computer science background to understanding the socio-technical dynamics of Wikipedia. So I've said socio-technical a lot. I should probably actually define what that is. And to define this, I want to talk about the history of this term socio-technical, which is actually quite common in my field. So a long time ago, when computers were not really a personal device, they mostly existed in offices, we sort of saw the social as something that might happen next to the technical. We've got the technical dummy terminals that connect to a mainframe. And social would be when you get up from the mainframe and you decide to go to lunch with your coworkers or something like that. But after a while, we realized that we can solve some sort of social coordination problems with technical things. And so I have some screenshots of some really old calendar apps and email apps. And so we realized that we can engineer things that help social dynamics. And so we were bringing the social and the technical closer together. But when I say socio-technical, I mean something that's even more tightly integrated than that. And that's the sort of space that Wikipedia operates in. And so, darn it. So I'm a technologist. But for a moment, I'm going to pretend to be a technobiologist. And so I'm going to use this to try and help you understand what I mean by this sort of tight integration between technical and social things. So these are my own sketches, by the way. So here on the screen, we have a bacterium and a paramecium. And so I want to draw some contrast between the kind of organization that we have and these type of complex living systems. So a bacterium is pretty simple. It's basically just like a sack of salt water. It's got a few things that float around in it, but nothing really clear. The DNA is basically just like floating around in the salt water. There are a few things called ribosomes that help convert. I mean, essentially they help DNA convert to proteins and vacuoles that might hold a little bit of fluid that you don't want just floating around inside of the cell. Whereas a paramecium is incredibly huge and complex. And so for the biologists that I saw in the back of the room, you would also tell me this paramecium is way too big. There's no way. You would not be able to see this bacterium is way too big. You would not be able to see that bacterium next to a paramecium. A paramecium exists at a way bigger scale. And the only way that it can survive at the enormous scale that it does is by having subsystems that solve important problems for it. So we have endoplasmic reticulum, which holds the ribosomes together and allows the transcription from DNA to proteins to work much more efficiently. We have a complex set of vacuoles, like the contractile vacuole, that allows a paramecium to operate in freshwater environments. Because if you know anything about osmosis, the freshwater is going to rush into the saltwater cell and make it explode. And so it actually has a pump. These things are wonderful to look at underneath the microscope. Go search this on Wikipedia. We have great animations of contractile vacuoles pumping away. But the important thing that I want to, so I want to equate this to how social systems are organized. So who here has heard of Dunbar's number? So Dunbar was a social scientist who was looking at how humans organize around villages and social systems. And what Dunbar observed was that generally villages don't get above 150 people. And so what Dunbar thought was that there's some sort of cognitive limitation that we have to organizing above a certain scale. And so when I think about a fishing village in this limit at about 150 people, I think about that bacterium. It lacks simple substructures that can allow it to operate at scales. And so it just doesn't operate at that big of a scale. Nothing wrong with a bacterium. Nothing wrong with a fishing village. But they just don't get that big. Whereas Wikipedia is enormous. I was just saying earlier that there's 110,000 people who are working on the site. And it mostly works. It almost entirely works without very much coordination at all. And so when I talk about the integration of the social and technical, I want to talk about the baramecium, which is the peramecium without anything in it, and the organelles that help solve these critical sub-processes that allow the peramecium to operate at these enormous scales. And so the baramecium of Wikipedia is the crowd of people, this enormous group of people who are working on this project together. And by the way, this photo is taken from one of our main conferences for Wikipedia, and that sort of stuff called Wikimania. I think this one was 2014. But the critical subsystems are not just technology, but they are a lot of technology. We have the Media Wiki software, which is that flower logo. We have a lot of things that we call robots and tools that Wikipedians use to organize their processes. And we also have policies and guidelines that set up how these processes should move forward and allow the subsystems to come together and work efficiently together. And so coming back to why I say that sociotechnical means a more tight integration, you couldn't understand a peramecium by just looking at its membrane and the saltwater fluid and some of the vacuoles, or looking at the organelles separately. It's a complex system. A living thing is a complex system where each subsystem relies on other subsystems performing and operating. So you would never study a peramecium by just looking at different parts of it. You would study it by looking at it together in the same way that you can't study Wikipedia by just looking at social things or just looking at the technical things. They're so tightly integrated that you have to look at them together. They affect each other substantially. And so here we have it. The point that I'm making or that I'm working towards is that a peramecium is a system with specialized subsystems and Wikipedia is a system with specialized subsystems. So let's talk about the specialized subsystems. So there's a problem that Wikipedia has to solve in order to operate at the scale that it does, work allocation. How do we identify, prioritize and assign tasks? Who's going to write what Wikipedia article? So we mostly get this free due to Linus's law. And this isn't Linus Torvald's law. It is actually named after Linus, but it's the one that Eric Raymond framed. And that's given enough eyeballs, all bugs are shallow. So Eric Raymond was talking about open source software. And then if we make the source code of software open, then anybody can come and fix it. And so if anybody can come and fix a bug, then you're likely to have the person with the right expertise able to fix that bug more easily than if you just have a small group of people that have to fix all the bugs whether they have the right expertise or not. Oh, shoot, I never set up that joke. So essentially, yeah. So we have this idea that if we have a large enough group of people who are looking at something, somebody's going to know the right way to solve this type of problem. And so this is insightful because it's saying that visibility is critical to open collaboration. And so my corollary for Wikipedia is that given enough people who see an incomplete article, all potential contributions to that article will be easy for somebody. Obviously, we have some subject matter expert who can take the time to make that contribution. And it turns out that the research support this. It works in theory. It's part of becoming a Wikipedia editor. We can support it, support the visibility of Wikipedia and the information needs that it has with technology. And really bad things happen when we take it away, when we make things less visible in Wikipedia. Okay, second subsystem that I want to talk to you about obviously this is a problem if we have an open space is regulation of behavior. We have to come up with norms. We have to propagate them. We have to enforce them. So this is probably one of the most studied things in Wikipedia. This screenshot is actually kind of old. And of course, Google's estimates of the number of results that it has aren't perfect. But if you go to Google Scholar and you type in Wikipedia and governance, I just want you to know there is a ton of research out there that describes this stuff. And just to nail this down, so I clicked on page 10 in the Google Scholar results and we're still getting tons of relevant results that are about studying Wikipedia and governance systems. So I would just want to summarize how Wikipedia generally puts these things together. So they have a separation between a prescriptive norm which is an idea for something that we should change and a descriptive norm where we're observing how we do things and we're writing that down so that newcomers can understand it so that people who disagree with it can actually point at yes, we've been doing these things this way and we now want to change it. And so we formalize both things that we want to do and things that we have been doing. So you turn that into an essay. That's the primary formalization that we do. And then we file a request for comments and that turns into a big debate and some of these are pretty easy to go through. Some of them are very contentious. But if it passes the request for comment stage then it becomes formalized into something that we call a policy or a guideline. A policy is essentially a law and a guideline is something that directs you towards good behavior. And you can use these policies and guidelines to enforce these things on other editors. If you don't pass the request for comments then it goes back to essay stage. You might make modifications and try it again. You might just leave it as an essay. And so these things exist in parallel. We have lots of essays that don't become policies and guidelines. We have lots of policies and guidelines. And they sort of, the formalized norms that are the policies and guidelines are like the laws that you can enforce around Wikipedia. Whereas the essays are things that you can bring up to try and argue a point that's been argued before. An essay helps you make the point in a nuanced way because we've been building up the essays for a while. And so a good example of an informal norm is don't stuff beans up your nose. Which doesn't tell you actually to not stuff beans up your nose. It's don't tell people not to do things that you don't want them to do because maybe they haven't thought of that yet. If you tell a child, don't put beans in your nose. They're gonna put a bean in their nose the next chance that they have. But a good example of a formalized policy is verifiability. Wikipedia is not truth. Wikipedia is verifiable. That means that it doesn't matter what is real in the world because everybody can debate what they think is actually true. Wikipedia only contains information that can be cited. And it goes with the reference material. And so if there's a debate about what an article should say, we're always going to follow what the reference material says. If there's two different branches of reference material that say something different, then we'll report that there's two different branches of reference material that say something different. Wikipedia is verifiable. This is essentially the first law of Wikipedia. Okay, so I wanna talk about the growth dynamics of these regulations because it can kind of get into what we're talking about when we talk about a complex system here. So essentially there was a steady growth in the guidelines and policies until about 2006 in Wikipedia which is around the time that it got popular and we stopped growing the guidelines and policies or at least they grow at a slower rate. Whereas the rate at which we produce essays, new ideas about behavioral norms or how we should behave or how we should think didn't slow down at all. In fact, it's sped up quite a bit. And I'm actually gonna get back to that a little bit later. I wanna tell you a little bit more about the subsystem of citing policies at each other. So there's a study by BuzzChastnik et al. who were looking at the governance systems in Wikipedia and I would love to describe these graphs but really I'm just showing them to be pretty. I'm gonna summarize the results as Wikipedia's governance system is inclusionary. By formalizing our rules, we empower people to use them to get stuff done to protect themselves from people who are like asserting ownership over a page or to enforce that this is the thing that we need to do next. And so the fun thing is that if you have an administrator in Wikipedia cite a policy at you, that might be frustrating and they just wielded some power over you. But now you know about that policy and you can go cite it at somebody else. And essentially those graphs that I was showing you are showing that that happens all the time. People adopt the policies that are cited at them. Okay, so next subsystem I wanna talk about another important one, quality control. Wikipedia is an open Wiki, anybody can edit it. We have to control the quality, this is very hard. We need to identify and remove damage. So this benefits from the many eyes of Linus's law. So given enough people observing a Wikipedia article, there's gonna be somebody who detects the damage and removes it and this has worked pretty well through the history of Wikipedia but it doesn't scale that great especially because you can have a lot of people that read a vandalized version of an article before it gets cleaned up. And so we have some automated systems that help make sure this quality process goes fast. And I like to split these into two categories, the fully automated vandal fighting systems that use machine learning to detect vandalism. They're very fast, they revert a damaging edit on the scale of about five seconds. It requires no human effort other than of course maintaining the bot that actually does this. But it can only catch obvious vandalism because machine learning isn't perfect, you can't have a machine read a sentence and understand it. So for everything else, everything that's not super obvious, we use a human computation system. So essentially this still uses a machine learning model but it takes the things that are likely to be vandalism but we're not that confident about and shows it to a human and lets them very quickly decide whether this edit is good or bad. Still pretty fast, happens at the scale of about 30 seconds. It's designed to minimize human effort and the cool insight about this is that human eyeballs are actually really good at catching subtle vandalism. Humans can catch most vandalism at a glance. And so it also turns out that all these systems work together in a really fascinating dynamic where you have people who are reverting vandalism just because they saw it on the page, people using these automated systems and we have these backend notice boards where administrators can find out about vandals and ban them so that they don't continue vandalizing the wiki. And I like to go back to the biological metaphor here because this looks a lot like an animal immune system. It has innate characteristics which is reverting vandalism as it happens. It's fast, it's general, works across the wiki and it's local to the actual problem, the vandalized page whereas we also have an adaptive side of this which is the administrators banning people. It's slow, it's specific to the editor that's actually the problem, but it has global effects. Once you ban somebody who's vandalizing wikipedia then the vandalism stops. It's like getting immunity to a particular virus. Okay, moving on to community management. We have to socialize the newcomers, we have to train them, we have to mediate disputes, these things are going to happen in our community. So English wikipedia gets about 6,000 new editors a day and I chose this firehose image for a reason because I think this is how a lot of wikipedia's look at the new people coming in. Who here has heard of Eternal September? So this is like an old computer science thing. So back in the day when everybody wasn't on the internet you would get on the internet when you came to the university and join the news groups, the mailing lists and that sort of stuff and every September you would have a crop of new people who had no idea how to operate on the internet who would show up and it would be chaos for a month or two and then things would go back to normal as the newcomers learned how things were going on. Well once AOL or America Online and that sort of stuff started to come up and people started getting online at home then you had a constant influx of newcomers and so that's commonly referred to as Eternal September. Like suddenly now it's always September. So anyway, wikipedia has an incredible Eternal September thing going on and so we have to route these newcomers. We have to figure out who the vandals are and who the good faith newcomers who need some help or training or are already doing a good job. And so we have some technologies that route them like a bot called HostBot which tries to find good faith editors and route them to the spaces that newcomers can get help like our question and answer forum called The Tea House. Okay, finally this last subsystem is one that may be less intuitive but it's something that wikipedia really needs to actually solve. I work for the Wikimedia Foundation. We do not tell wikipedia what to do. They have to figure that out on their own. They don't have a president, they don't elect it. I mean they elect people to solve problems but they speak of it like a mop, not a hammer. The way that wikipedia's figure out what to do next is a group process. It's a collaborative complex process and so we need to have a subsystem that supports that just like the other complex collaborative processes. And so this reflection of where are we going? Where do we want to go and how do we want to get there is really important and the community has to figure this out. So coming back to this graph on talking about formalized norms which are these policies and guidelines and informal norms which are these essays I like to think about actually formalizing something into a law as adaptation. We identified something that was valuable and we're going to codify it so now we can enforce it across the wiki. But essays, it's a lot of reflection. A lot of essays ask the question, where are we going? How do we want to get there? Is this really who we want to be? Or maybe we're doing something awesome and let's talk about that more. And so it's very reflective. The whole set of essays contains a lot of reflection. A lot of trolling and a lot of humor. So I think this is something that I think is concerning about the adaptive characteristics of wikipedia and we're going to get into that a little bit so I don't want to spend too much time on this slide. But so I want to come back and summarize this now. So a paramecium is a complex system of the interactions of chemicals. And so there's no higher order than the chemicals interacting with each other and a paramecium emerges from that. Wikipedia is a complex system of the interactions of people, the technologies that they built, whether it's media wiki, the core platform piece of software or the third party software like the vandal fighting bots and the human computation interfaces or the norms and policies that they build in order to make this stuff work. There's no one in charge, this stuff comes together and wikipedia emerges. Okay, so back to looking at this thing like a system now. So the socio-technical system with inputs and outputs and that brings me to the critique of the algorithmic quality control system in wikipedia. So there's this paper that Stuart and I worked on back in 2012, The Rise and Decline. And I'm going to describe what we learned about wikipedia by showing you this graph. This is a graph of the number of active editors who are working on English wikipedia over time. And I've labeled some parts of that time period early growth and decline so we can talk about what was going on in that time period. So in the early part of wikipedia, there were just, there were less than 150 people who were working on wikipedia and so they could use the social dynamics in order to solve their systemic problems. Wikipedia was very small. Most of the complex interactions around wikipedia were just people working with each other a little bit the media wiki software itself. But between 2004 and 2007, wikipedia started growing exponentially and became a fire hose. You know, wikipedia's eternal September got into full steam and wikipedia's didn't know how to deal with all of the newcomers who needed training, all of the vandalism that was happening around this time. And so they built technologies to help scale these things up, to help them deal with damage and the other sort of problems faster. And so these technologies are largely based on a machine classifier that predicts which edits to wikipedia articles are vandalism and which edits to wikipedia articles are good. Essentially, these things take a set of statistics about what was done in an edit. Was the editor logged in or were they anonymous? How many characters did they add? How many words from a bad words list did they add? Like curse words and racial slurs, that sort of stuff. Does something and makes a prediction about whether this editor is good or bad. And the really cool thing that you can do with a model like this is you can take the 160,000 edits that wikipedia gets every day and you can split it into the smaller proportion of edits that should probably be reviewed and the huge proportion of edits that don't need to be reviewed. And so to put this in real human numbers here, so without machine prediction, reviewing this many edits per day would take about 267 hours. There's 33 people working eight hours a day in order to review vandalism and wikipedia. But with machine prediction, we can cut that down by 90%, reduce it to 27 hours and now we only need four people working full time, which is actually what wikipedia looks like right now. There's about four to five people who are working full time catching most of the vandalism. And so this is great. It makes the whole thing more efficient. But after we developed this, things went kind of bad. And we saw that the population of people who are working on wikipedia entered an abrupt decline. And so there's a lot more to the analysis that we did around this. We did a lot of qualitative work talking to newcomers in wikipedia. We did a lot of quantitative work modeling the things that predicted whether newcomers would stick around or leave and what we learned. I wanna get into a little bit of a discussion about how we got to this point before I actually tell you what happened because it'll make a lot of sense once we get there. And so in order to talk to you about the context of this, I wanna draw from Donna Haraway and talk about these terms standpoints and objectivity. So Donna Haraway spent a lot of her work studying scientists who were studying apes. And so essentially there were a bunch of research groups who were looking at apes and she split these research groups into male scientists. And I put a star next to female scientists because the female scientist groups weren't really dominated by female researchers, but rather the research groups were informed substantially by feminism and thinking about power structures and that sort of stuff. So they both looked at the same subject. They were both looking at a behavior, but they asked different questions and came to different conclusions about what was going on. So the male patriarchal scientist tradition asked things about reproductive competition and dominance, whereas the feminism informed research groups asked questions around communication and social grooming patterns. And so to define these terms, your standpoint gives you a view of what's valuable. Are we gonna look at communication patterns or we're gonna look at dominance? And an objectivity is what you construct to reify this value. What methods are we gonna use in order to explore this?  And so, but the critical insight that Donna was pushing on with this work is that standpoints and objectivities, they're not just different, they can be merged. It's good when we have many standpoints and apply many objectivities. And so if you're reading a textbook on ape behavior, it should not just tell you about either of these, it should tell you the whole story about reproductive dominant or reproductive, reproductive competition, dominance and communication and social grooming. It's better that we had more standpoints. We learned more about what we were looking at because of that. Okay, so stepping back to Wikipedia during this massive growth period, let's talk about the standpoint of Wikipedia editors.  Bad edits need to be reverted. We need to get them out of the encyclopedia and preferably we wanna minimize the effort that we waste just removing damage and spend time constructing the encyclopedia. And so we built an objectivity around that with these quality control tools that use machine prediction to make the work easier so that we could split the good from the bad fast. And this was a massive success. Wikipedia is still around. It hasn't been destroyed by vandalism. This worked really, really well. And so essentially you can think about this as there's a filter between the internet and Wikipedia that reduces the workload by about 90%. And this is sort of what the influx of edits in Wikipedia looks like right now. So it took us a little while to figure out that something was going wrong. There was a study that was published in 2009 by Bhanwansa at Palo Alto Research Center. And they noticed that Wikipedia was declining. This was published in about 2009. You can see that's just part of the way into the decline period that I have mapped out here. And they're like, whoa, whoa, whoa. The newcomers are leaving Wikipedia. The population is declining. We don't know what the heck is going on here but something bad is happening. And it took us a few years to figure out what the heck was happening here. There were a lot of major hypotheses. Maybe we can get to them in the questions. But it turns out that after looking at this for a long time we figured out that we forgot to value socializing the newcomers. We just were valuing splitting the good from the bad. And it turns out that newcomers who were making good faith mistakes were lumped in with the bad. And so they were getting ground up by the quality control gears. And so essentially if we go back to this plot of how the machine learning model was splitting things, it wasn't splitting things into good and bad. It was splitting things into good and bad but mostly newcomers who need help. And so as soon as we deployed these tools we saw a massive effect on the retention of newcomers working on Wikipedia. So essentially our quality control system got really big and powerful. It squashed part of our community management system and we were throwing out the baby with the bathwater. And so great, we have this study now and we realized like, hey, maybe we should also value newcomers having a good experience and design our system around that. So this is now reflecting and deciding what we wanna do. So let's review.  That didn't change with this analysis. Bad edits still need to be reverted. We still wanna minimize the effort wasted on quality control work but our standpoint has now been extended to realize that we need to socialize and train the newcomers. We value this because our community goes away when we don't do this. There's lots of other reasons too but that's a good one. And so between when this study was published and well, when I made this slide in 2015, the conversations around what we're doing in Wikipedia incorporates this new standpoint. So more newcomers in solving this retention issue becomes a major Wikimedia Foundation goal and there were spaces that were explicitly developed to help newcomers such as the co-op which is a mentoring space in the tea house which is the question and answer space that I referred to earlier. And so when, right, we realized that we had a problem. We adjusted our standpoint. We created new objectivities. Well, so I'm gonna pick on Huggle for a second which is one of the computational interfaces that helps people fight vandalism to show you how we didn't totally change. So first, the sugar. Huggle is actually an amazing piece of software. I have no doubt that it represents the state of the art in distributed quality control processes. This software and its users are responsible for critical work in Wikipedia. Without them, Wikipedia would not work. Its developers and its users are wonderful people. They're my collaborators. I work with them a lot. We owe them a lot. All of us do. All of us who use Wikipedia. So, thank you to them. Now the medicine. So this is what Huggle looked like in 2009. Before we knew that Wikipedia was declining. So here it shows you the diff. We have these good and bad buttons on the top. And so if you're using this tool, you click either one of those. There's actually keyboard shortcuts. People are really fast about this. And so this is how you split people into good and bad. When you click that bad one, it'll send them a warning message that says stop analyzing Wikipedia or get out of here. On the left here, we have the machine learning model sorting edits by probably bad to probably less bad. So you can focus on the probably bad stuff first. And so right after this came out, we had these studies published that there was a major problem. And we started talking about it and doing something about it. Well this is what Huggle looked like when they released the version 3.0 in 2015. Here's the good and bad buttons. Here's the list that sorts from probably bad to probably less bad. And to their credit, they did add a button to send a welcome message to newcomers that you run into. People mostly don't click on that button. It's not a big part of the workflow of this user interface. And so the fact remains that quality control on Wikipedia is still not designed with newcomer socialization in mind.  And so we're still not seeing the gains that we wanna see in the retention of good faith newcomers who are trying to work on the Wiki. Why? Like we adjusted our objectivities based on our new standpoint elsewhere. What the heck? Why do we see changes in some of the subsystems but not in others? Why not the quality control subsystem, the one that was the problem? And so this brings me to part three. Infrastructure for socio-technical change. How do we actually make changes to one of our subsystem that doesn't wanna change? So this machine classifier. It turns out that, so as I was saying, this is something that takes statistics about an edit and says whether the edit is good or bad. Each one of the dominant quality control tools in Wikipedia has its own individual machine classifier that was developed independently. And so let's say that you were to come to Wikipedia right now and you wanna build a new interface that's gonna make quality control better. It's gonna incorporate newcomer socialization. You're gonna use this extended standpoint. First thing you're gonna need to do is build a machine classifier to work with your tool because you can't really use the ones that the other tools use. So in order to do that well, you're probably gonna need to read about 20 plus research papers on damage detection in Wikipedia. You know, and a lot of the people who develop tools around Wikipedia, they don't have a computer science degree. But even if they did, machine classification is not a big part of a standard undergrad computer science degree. And also just hosting one of these things is extremely labor intensive. There are huge performance considerations.  So it should be no surprise that each one of these tools was authored by a computer scientist with extensive skills in machine learning and distributed systems. So in order to talk about what I think the real problem is here, I'm gonna borrow something from chemistry, this reaction pathway graph. And so essentially what this graph is showing is the amount of energy that you need in order to get some chemical thing to happen. And so in the case here, I'm showing that you essentially need 149 degrees Fahrenheit to turn a transparent egg white into a white egg white. And so, but there's also this line beneath that that's labeled as a catalyst. If we had an egg yolk whitening catalyst, then we could fry an egg at room temperature. And so that's essentially what this graph is showing is the idea that you need activation energy to cross certain thresholds. And with a catalyst, you can do it easier with less effort. So essentially, I think about the progress in Wikipedia as this huge activation threshold. Before we get to better quality control, we have the current tools. And in order to try something new, you have to cross this huge threshold of building a machine learning model that actually works in real time. But if we could have a catalyst that would reduce the amount of effort that you needed to put in in order to experiment with new quality control, then maybe we can get there easier. Maybe we can have people who put less effort into this. And we can actually get to better quality control faster and easier. So this is a system that I've been working on for a couple of years at the Wikimedia Foundation. It's a scoring platform. We're actually just changing the name of the team. And so if I find the other name in there, I'll tell you about it. So essentially, the idea is we're taking the machine learning model out of the tools and making a centralized resource that all of the tools can use. And so there are actually a bunch of tools that are already using it that are beyond the quality control tools right now. But the real point is that if you show up with your idea, based on the extended standpoint about how you wanna make quality control better, you can use it too. You don't have to worry about setting up a machine learning prediction model. You don't have to worry about getting it to work in real time. You can just use the one that we've made publicly available. So essentially the way that this works is you can actually go to this URL right now on your browser and this will work. It'll actually load up a score. So with this URL, we're saying that we want, we want a prediction about something in English Wikipedia. We wanna check to see whether something is damaging. And this number here corresponds to a specific edit in Wikipedia. In fact, in this edit, somebody was fixing a link in Wikipedia and changing it so that rather than linking to what we call a disambiguation page for the old woman, it links to the specific page for the play titled the old woman. So this is a good edit. This is something that Wikipedians do a lot cleaning up the link structure. So if you give this to Ores, Ores says false. I do not think that this is damaging. I'm 94% sure that this is, this is fine. Cool. Well, let's give something else to Ores. And so in this edit, we're replacing a citation with all caps. Lamas grow on trees. Okay, so let's see what Ores thinks about that. So it predicts true. We're 92% sure that this edit is damaging. At least somebody should review it and do something about it. And so essentially this is the technology that we've put in place for people to use. It's fast. It has a median delay of 0.5 seconds. It's way faster than Clubot, which was added at five seconds. It's scalable and redundant. I'm a computer scientist. I work with people who specialize in distributed systems. We can do this. You don't have to. And we're roughly comparable to the state of the art in the fitness of our predictions. And so essentially what we're hoping to do with this is provide a progress catalyst that reduces the activation temperature of getting better quality control by about 20,000 lines of code and one advanced degree in computer science. And hopefully we can get a lot of ideas that move past the ideas that we have now about how we're going to objectify our quality control and newcomer socialization processes. And we've actually already seen some of the tools are moving over. And so Huggle is a good example of a tool that's already using our scoring platform. And they're making changes to how their system looks right now based on some of the predictions that we make. Oh yeah, and of course we have a ton of other tools that are operating in the space. There's 20 volunteer developed tools that are using this prediction. There's three major Wikimedia product initiatives and some really cool data science that I'd love to show you guys later if we have some time. But the interesting thing here is that we've seen a ton of progress. Not all of it directed towards the newcomer socialization experience, but a lot of it in that direction. Okay, so I told you there were three parts but there's actually four. And I want to tell you about the feminist inspiration and how we can bring the critiques that come from the feminist scholarly tradition and apply them to the design of algorithmic technologies like this scoring platform. So first I want to wave my hand at subjective algorithms. So this is something that Zanep Tufaki and a lot of other people have been talking about in the media. I'm gonna use Zanep's definition really quick. So algorithms often aided by big data now make decisions in subjective realms where there's no right decision and no anchor with which to judge outcomes. So like what's good, what's relevant, what's important, what's desirable and what's valuable, that's based on our subjective opinions. And notably there's been a bunch of buzz in the media recently because Google released a machine prediction AI and didn't explain very well what kind of predictions it's making. This machine AI is making predictions about which comments on news articles or on Wikipedia are harassing or personal attacking and people are pushing back on this because it flags things that are very obviously not harassing like the statement few Muslims are a terrorist threat as very toxic and they're not. And so being able to deal with these machine learning models is hard, they can change conversations, they can label you as a toxic individual even though you're saying something that's actually quite reasonable. And so in Wikimedia, this is a big problem for us. We want to gather the sum of all knowledge and we wanna make sure that everybody can read it. When we're building machine learning models that decide who can edit Wikipedia, we're asking questions like who's allowed to participate? Who gets labeled bad faith like they're trying to vandalize Wikipedia and what types of contributions will be labeled damaging? Our machine learning model predicts good and bad. We'd like to have it mostly do a good job but it's gonna predict good stuff and some important bad stuff. It's gonna make mistakes. And on the other side too, it's gonna predict bad stuff but some important good stuff. But those mistakes aren't necessarily going to be evenly balanced. So this is a post that a Wikipedia editor made in response to one of our announcements, please exercise extreme caution to avoid encoding racism and other biases to the AI scheme. So a lot of Wikimedians are actually pretty sensitized to this and they wanna talk to us about this. But I just wanna summarize essentially what WNT was saying here which is that we might not just predict, for example, harassment in civil discourse, we might predict harassment and for some reason we also lump the kind of language that South Africans tend to use into that harassment category. And so people who are using our AI to patrol harassment will now start harassing South Africans for being harassing even though they're not. This is something that we have to be really careful of. So I wanna tell you two stories about biases that we accidentally encoded into our prediction models that predict good edits and damaging edits and how we were able to address those by working with our community. And so specifically I'm gonna talk about how the Italian word ah and anonymous editors get lumped in with damaging edits in a biased way. So let's talk about the Italian word ah which is literally not a laughing matter. So these are some screenshots of some pages on the Wiki for these were Italian Wikipedia editors who are telling us that there's a huge amount of predictions where the system is wrong. By the way, the system is called or is our team is called scoring platform name. So they're telling us that in a whole bunch of cases, Aura's is saying that when somebody adds the Italian word for have, this word ah, to an article, Aura's is saying that it's vandalism and that somebody should deal with it and it never is vandalism. So in order to tell you what was happening here, I got to talk to you about how we build our informals and bad word lists which are some of the statistics that go into our machine learning while they predict good and bad.  But when I say informals, I mean casual speak that would be welcome in discussion in spaces that we call talk pages but not within an article. And so for that we get you know hello, ha ha ha and as you can see, ha ha ha, we're leading somewhere. So it turns out that people vandalize Wikipedia, like Italian Wikipedia in English language all the time. In fact English language vandalism happens all over our projects. And so we use the English language informal words as part of the signal that our models use. So this shows you like the regular expression that we use to catch ha and the examples of things that we're trying to catch. These are our test cases and so that's supposed to catch hi hi hi, ha ha ha, hee hee. And so I think we're basically there. Ha is laughing in English. Doesn't belong on an article page but ah is not laughing in Italian. So we removed that specific informal term from the Italian damage prediction model, rebuilt our models and got back to the editor who reported this to us and said hey, it looks like we've reduced the false positives that you've shown to us. Can you check out the updated model and see if you're still having a problem and Rotpunk came back to us and said no, this is great, nice job, thank you. Win. So the next problem that I wanna talk to you about is anonymous editors. You don't need to register an account to edit Wikipedia. You can just show up and click edit and save it. It'll save your IP address. Totally not like somebody can use your IP address to find out where you're editing from. Register an account if you wanna be private but with anonymous you don't have a name. You're just an IP address. So we got a lot of reports from different wikis that we have these models deployed for that there seems to be a strong weight against annons. Annons seem to dominate the false positive reports on all of these wikis. But maybe anonymous editors are really bad and the prediction model is actually just doing what it's supposed to do. It's predicting things that need review. So generally anonymous edits in Wikipedia are twice as likely to be vandalism. But 90% of anonymous edits are good. So why are we harassing these people if it's 90% good? And so we actually built a system into this prediction model system where you can actually feed it a feature and have it try to make the prediction again based on a different feature. And so here I'm adding a feature. Well, how would you predict the score of this edit if it wasn't an anonymous editor making it? Well, we think it is 30% percent likely to be damaging. But what if it was an anonymous editor making it? Well, we think it's 45% likely to be damaging. And so essentially just by being an anonymous editor, we think the edit is 11.5% more likely to be damaging to the article. So in order to dig into this and try and figure out what the heck is going on, we actually had to change a larger set of features than just is the editor anonymous. And we wanted to look at a few different user classes. So we wanted to spit in the features that are an anonymous editor, are a new editor and my username on Wikipedia is epoch fail. And so I used myself as an example, like what if I were making this edit? And we experimented with two different types of modeling strategies. Our old modeling strategy was a linear support vector machine and our new modeling strategy was a gradient boosting model. And I'm just gonna wave my hand at this. Like I'm gonna show you this edit is damaging and this edit is not damaging. We get this from a labeling interface that our Wikipedia's used to tell us what they believe is damaging or not. And so that you're gonna see these into the graphs in just a second. So what this graph is showing you is the distribution of probability of damaging scores for edits that are damaging in blue and edits that aren't damaging in red. And so for these two different models. And so we can see that edits that are not damaging tend to score low and edits that are damaging tend to score high. But we see a little bit of weirdness in this graph where edits that are not damaging sometimes get scored very highly. So this is just taking a random sample of edits and scoring them and seeing how that corresponds to the labels. What if we tell ORs to make the same predictions but assume that every single edit was from an anonymous editor? What does it look like then? Well for the gradient boosting model we can see that edits that are not damaging still tend to score low although there's definitely some overlap. But for the linear SVM, edits that are not damaging score very, very highly. And so we can see that the linear SVM model can hardly differentiate damage when it was saved by an anonymous editor or not. And so we can see that these sides look very much the same. So we weren't, like I said, we were wondering what this looks like for a newly registered editor and it's basically the same story. Again the gradient boosting model can differentiate most of the damage from the not damage but the linear SVM model just can't. And so, but what if I saved the edit? And I've been around Wikipedia since 2007. And so I have a long history of editing Wikipedia. What does the model think then? Well, it turns out that I can vandalize Wikipedia and the model will never catch me. All of the scores, whether it's damaging or not are way below the threshold that we set for review. So, I get to vandalize Wikipedia. But this is an interesting point because this is not based on the actions that people take. It's based on who they are, how they approach the system, what sort of class that they operate from. And so this is a problem and we wanted to solve it. This is why we dug into this work. So on December of 2015, we deployed this gradient boosting model and substantially reduced our false positive rates for anonymous user edits. But the bias is still there. You could see in that gradient boosting model it still wasn't differentiating an edit from me as vandalism or not and it should. So this bias still exists and it's not gone. We need new signal to get around this stuff. We've been publicly documenting this problem in our process for solving it. We actually have two new sources of signal that we're working on deploying as soon as we can. So, the final thing that I wanna talk to you about when it comes to issues that you can have in these systems is empowerment versus power over. So, and here I'm channeling Nell Morton in this idea of hearing to speech versus speaking to be heard. So for hearing to speech, I wanna hear what you have to say. So I'm gonna make space for you to say it. Actually right now I'm gonna talk at you so I guess right now right now. But generally that's what hearing to speech means. Whereas speaking to be heard is I'm gonna talk first. I'm gonna set the tone of the conversation so that I make sure that we talk about what I wanna talk about. So this is sort of a Nell was making a point about the difference between empowering other people versus making sure that you have power over the situation. And so when it comes to conversations about advanced Wiki tools, I'm a pretty powerful guy. I've been doing software engineering work for about six years that beyond, on top of the research that I've been doing. So I have a PhD in computer science. I have a substantial background in psychology, social science, systems theory and HCI, human computer interaction design and practices. And on top of that I'm a staff member of the Wikimedia Foundation. So I have advanced privileges around the site. I don't want to tell people what quality control should look like. Instead what I wanna do is I wanna empower them to explore the ways that it would look like. I want more people to be involved in this technological innovation conversation that's happening here. And I think this is dramatically different from how we look at intervening in systems like this right now. And so here's me saying, I think that we should talk about quality control and socialization because I've done all this research and I think that this is important. By the way, this is a cardboard cutout of me that they have at the Wikimedia Foundation. I love putting it behind people's desks when I come for a visit. But what if somebody from the community, somebody who's less empowered than me says, you know, I think that we should talk about something else when it comes to advanced tools or something like that. So that means that we're gonna need to have a machine that predicts that something else. So we're gonna have to train the machine on what that something else is. And so this labeling system that I was showing you before, we specifically made it adaptable to new types of questions, new types of training that we might wanna train models to do. And we've made it easy for other people to get training data and start working on building these machine classification models. This actually helps us deal with a couple problems. One is feedback by having people label these edits in an interface that's outside of Wikimedia. We can remove, like, they don't know whether this edit was saved by an anonymous editor or registered editor so they can't bring that bias or it's at least difficult to bring that bias to it. But the cool thing is we bring this hearing to speech with this that we made the labeling system open and configurable so that other people can try to train machine learning models on other problems to support other types of processes. And hopefully with this system, essentially what we've been doing over the past couple years is iteratively working with Wikimedia editors who wanna have new machine learning models gathering their judgment with this labeling interface, training new machine learning models and then fitting that in the socio-technical infrastructure of Wikimedia to make it, try and see if we can make things work better, be more efficient, be more welcoming. Okay, so in summary of what I've just talked to you about, we talked about Wikimedia's socio-technical system. I brought in systems thinking in these biological metaphors. We talked about this critique of algorithmic quality control where we're throwing out the baby with the bathwater and standpoint epistemology and how ideologies get encoded in the technologies that we build. And then finally, we talked about this infrastructure for socio-technical change, the progress catalyst and how it's hard to build a machine learning tool because you had to build a machine learning model that works in real time in the past, but you don't anymore and now we're seeing the progress start to happen. And about like these insights that we got from feminist theory around designing for empowerment rather than power over and making sure that we engineer against the problems around subjective algorithms and make sure that our processes are open so that we can actually address these things. Thanks.