Ladies and gentlemen, please welcome co-founder of Databricks, Andy Konwinski. Thank you. Welcome back to the summit. This morning we're going to jump right in. Our first talk is going to be by Reynolds Shim, who is the chief architect of Spark at Databricks. Reynolds will be talking about real time and Spark. Let's welcome Reynolds. So yesterday, all of you have heard Matisse Kino, and one thing he said is we're going to double down on streaming this year.  So why real time? I think this question is pretty obvious in general that for some of the audience here, it's typically useful to actually make decisions faster and sometimes it actually leads directly to bottom line. For example, credit card fraud detection. You probably want to actually flag the fraudulent credit card swipe as soon as it happens rather than 10 days later where you don't even know where the guy that's actually making the fraudulent transactions are anymore. There's many other use cases. So in response to this, the industry kind of started this new trend or new generation of systems called streaming engines. And streaming engines typically just takes an input stream, sometimes multiple input streams, and produces an output stream or maybe multiple output streams. And what is the stream? Stream is really just an infinite list of data that grows over time. About three years ago, we kind of straw this trend in Spark as a result. Actually, to try to design Spark from ground up to actually respond to this real-time demand. One example of it is Spark SQL. Spark SQL can actually answer queries interactively very quickly. But the other more important component in Spark streaming, which was actually introduced three years ago in Spark 0.7, this is probably way before most of you have heard of Spark. And according to a survey we did last summer, actually half of the Spark users thinks Spark streaming is actually the most important component of Spark. So the popularity is certainly on the rise. Spark streaming was our first attempt at unifying what we call batch computation, basic computational static data and streaming computation. So this is a real-time aspect limit. It has many nice features that were first of its kind in streaming engines, including like building stage management, exactly one semantics, which I'll explain in a minute. And there's also a lot of features that are very nice for scaling out the computation to very large workloads, such as regular mitigation, low balancing, and fast forward recovery. Now, just to touch upon exactly one semantics, what do I mean by that? For some of you, they are less familiar with streaming computation. Typically, given some operation you want to do in streaming, there's three different ways you can do it. One is what we call at least one semantics. So you actually apply that computation at least once for a given event. The second one is at most one semantics. This means you apply the operation at most once. So imagine credit card fraud. Actually, just imagine credit card swipe. Whenever a swipe happens, in the case of at most once semantics, you basically charge the user at most once. So sometimes you might not charge the user, and sometimes you might charge the user. It doesn't sound very great. What about at least once? It means you charge that user at least once, but you might double charge, you might triple charge. It doesn't sound very nice either. You could actually get an angry email from the users. The first case, you would lose money. The second case, you might get angry emails. Now, both of these cases are actually really easy to do. I can give you a solution that requires no work. For example, for the case of at most once, just never charge your user. Just return immediately in that computation. Never run anything. In that case, you have at most once. Even though technically, you're always returning zero, but it's doing at most once. So now, typically what you really want is what we call exactly once, which means if the user swipes a credit card, you charge the user exactly once. So since Spark Streaming was introduced about three years ago, we worked with hundreds of streaming deployments with a lot of Spark users and customers. And we've learned a lot in this journey. And one thing we learned is streaming computation typically don't exist in isolation. It's actually typically combined with a lot of other kind of computations. So let's take a look into one use case again. Just credit card fraud detection. In the case of credit card fraud detection, typically you have a stream of data that's just credit card swipes. And then you want to determine for each of the swipe whether you actually have a fraud or not. So this is our anomaly that happened. Now just imagine I'm a credit card company and I've decided I'm going to implement this whole system in streaming. And I'll be using a very simple algorithm. And the algorithm is if the credit card transaction happens, outside of the 15 miles radius of my user's residence, I'll flag it as fraud. Now, can I get a show of hands if any of you have gotten a credit card transaction denied because you might be traveling somewhere? Everybody. See? Big data is still not really solved. Now, this model is obviously not very good, but sometimes you would be surprised how far you can actually use this model to get going. But what really happens is as soon as you get more user complaints, you probably want to look into how you can actually improve the model. Now, how can you improve the model? Actually, you typically involve some data scientists or analysts or modeling team that's actually go and look at historic data to actually understand the trends and spending patterns better so you can build better models. And at the same time, you might actually have a different stream that's actually updating some machine learning model that's actually in real-time to actually improve the model you're already using in production. And typically, this streaming pipeline might be actually different from the streaming pipeline that's actually happening with the credit card fraud because in the case of fraud detection, you want to actually flag the fraud as soon as it happens, maybe in a range of milliseconds or hundreds of milliseconds. Whereas, in the case of training a new model, you probably don't want to be actually deploying your model in a hundred millisecond interval because at some point, you might want humans to actually step in and take a look to see whether the model actually makes sense on that. So this use case itself involves multiple streams, involves actually looking at data, historic data in an interactive fashion and also involves running a lot of batch computation. You're looking at a lot of different use cases. This is not unique. As a matter of fact, a lot of the streaming use cases actually involve many, many non-streaming components. As a result, we actually decided to actually call this class of applications continuous application to give it a new name. So what is a continuous application? Continuous application is just an end-to-end application, the axon real-time data. It's a very simple definition. With this definition, you could almost call any application a continuous application. So since we studied this pattern, wanted to look at how Spark can actually simplify the building of continuous applications. Now, what makes building continuous applications difficult? And I think there are two reasons they are difficult to build. The first is it's very difficult to actually in general integrate streaming systems with non-streaming systems, which is actually typically necessary in continuous applications. It's difficult to actually have interactive analysis on live streaming data. It's very hard to run bash computation on historic streaming data with a consistent view of the data. It is very hard to actually output streaming data onto some relational database in a consistent fashion with exactly one semantics. It's also very hard to do machine learning on streaming data. And the second one is streaming computation model is extremely difficult. It's difficult to understand, it's difficult to learn, it's difficult to reason about. Let me give you one simple case. You can't really get simpler. We have a stream with some two traffic logs. We pipe this stream into a streaming engine, and all we want to do is aggregate some count. So I want to see how often each page gets visited at any given moment. And maybe I'm doing just some simple analysis after this. Maybe I'm actually charging my clients based on how much traffic I have generated for the website. Now, this seems really simple in this case. What can go wrong? It turns out there's a lot of things that can actually go wrong. For example, one of the processes that's actually collecting the logs might be slow and send you the records or the logs that are actually delayed. Some logs might actually literally show up two days later. In the distributed system, you might actually have some nodes that are outputting your data and the other nodes are actually not functioning and not outputting, so you're actually getting partial data. The whole pipeline might fail and when it recovers, it might actually give you inconsistent results. There's a lot of other problems that could happen. Which actually leads to streaming computation model being extremely difficult in three different dimensions. The first dimension is data. It's very hard to reason about data in streaming computation because data might arrive late and have varying distributions over time. Which means one algorithm that might work in a particular given instance of time might not work in the future. Processing is fairly difficult because over time your logic might actually change. You want to apply new machine learning algorithm, you might want to actually change your whole algorithm completely. There's also new kinds of operations such as windowing and sessions. And last but not least, the output is actually difficult to define and now you have some sort of time spending to define what output actually means over time and how do you define correctness. So for the past few years, we've been thinking about how we can actually simplify streaming computation because we don't want only the PhDs in computer science that can actually understand this and build robot streaming applications. We want to really take it to a scale to the masses. And I think finally, we have a satisfying answer and what we're calling the structure streaming. And it comes with one simple realization which is the simplest way to perform streaming computation is to not having to reason about streaming. And you heard me right. I did drink my coffee this morning. I'm not making this up. The simplest way to perform streaming analytics is to not having to reason about streaming. What do I mean by that? So in Spark 1.3 which was early about a year ago, we introduced data frames. And what data frames exposes is a very simple API that's similar to the tools that people have been using on their single node tools. You can use data frames so interactive and in a bash mode deal with historic data. So we thought, what if in Spark 2.0 we just expand data frames and generalize it to deal with infinite amount of data and introduce a new concept or new API called streaming data frames. So you can run the same data frame operations just on a stream. So a new API in Spark 2.0 with streaming data frames. But then we thought nobody really wants that. It would be so much easier if you have a single API with just existing data frames. So that's what we actually did, structure streaming. It's essentially extending the pre-existing data frame API that builds on the Spark SQL engine. And what it does, you can actually run the same data frame operations over a stream with new operations such as event time support, windowing and stashing, and streaming specific sources and things. And this is our second attempt and I think maybe the best so far to actually unify streaming interactive and bash queries so we can substantially simplify continuous applications. Examples such as aggregating data in a stream and actually output it into some MySQL database should just work out of the box. Or even better, maybe just within Spark you could actually create directly the state that's being accumulated or aggregated. And you should be able to pipe it directly into the machine learning library in Spark and use that on a live stream. So for the next five minutes keep warning. I'll dive a little bit more into the details which is probably atypical for Kino. For those who are with me, for those that are slightly higher level and I think for those developers you'll probably actually like this. So let me explain the model. First we have time, this is just your normal time there's nothing special here and we have a concept for a trigger and what the trigger is it just defines how often you should run some data frame operation. In this case I have a trigger of every second so every second something supposed to happen. Now from the beginning of time to the first second my stream has accumulated some data. In this case I've defined a data frame query let's say it's just some simple aggregation so the simplest one just count. I want to count how many records I have. Now this will output basically one record when I run this query on the data that's accumulated from time zero to time one and I just want to output all of this which is a single number out to some output. Now in time two I've accumulated more data so at time two I have all the data from time one plus the data from time one to time two and have my same query again just count. Now run this count on all the data and then just get me another number and output this number again and same thing with time three except this time I'm running all the data from time zero to time three. Sometimes in this case we actually output all the data all the time for my computation. Sometimes you might actually want to output delta so this is for example if I only run if I'm basically just doing some ETL and have new data arrive maybe I only want to output only the data not all of the data. So just to recap this very simple model there's four concepts there's input streams or input sources it's really just a table that's a pen only so it keeps growing over time it becomes infinite and then there's queries which are just typical normal data frame or SQL queries with new operators for window and sessioning and there are triggers which indicates how often this query gets run and there's output modes and I think we're going to support three different output modes includes complete which is always output all the data and then delta which is output obviously only the differences between the different triggers and then update in place and then make sure that you have a directional system in place directly. Now let me give you a couple more examples to slightly more concrete to help you understand this. Let's say in the case of if you're building a streaming ETL which is actually turns out one of the most frequent use cases for doing swimming is to ETL something continuously. So I have data that's in F3 on some bucket and then I have some data say so the query here is very simple just some simple math that turns JSON into parquet and for the purpose of this application I just want to run every five seconds and output modes I want to own basic deltas only for new records and I'll put it directly back into some F3 buckets so this is how I'll define this job. Now the second one which is similar to an early example is to run page view count and then I'll run it three times for each minute each of the page on my website has been visited. So the input might be some locks that's generated by my HTTP server in Kafka it's coming from Kafka and then the query is very simple I'm using sort of a SQL query to actually explain it it's basically just counting grouping by the page in a minute and trigger is again my SQL database I have so I don't need to query in Spark or anything I can just query my SQL database now the nice part about actually this one is sometimes data might arrive late which I was talking about in my early example and conceptually this should actually automatically correct my output if I update in place when the data is arriving late so if some data arrives like two days later my SQL database that contains actually all the accounts you get automatically updated so all you are suspicious like I see from your face first I walk up stage and told you the best way, the easiest way to do streaming is to not think about streaming and now I'm telling you we should just run all the data all the time if I want something to happen at time three I run this query on all the data from time zero to time three because this guy is smoking so what I've just explained to you is actually the logically how you should reason about this new model structure streaming essentially you're just applying data frame operations on static data and think about data from zero to some time when the queries run but when we actually execute it this is actually very similar to how data frames always been executed what this logical programming model is giving you is making it as simple to specify streaming queries as just static data and then what it really generates is for all the operations you're doing we generate a logical plan and spark actually through the catalyst optimizer it's going to turn this plan into a continuous and incremental query that will actually run in an efficient mode rather than running through all the data all the time let's go back to some aggregation example let me show you some code so this is sure how you run some aggregation in data frames looks fairly for those of you that are familiar with data frames it's probably self-evident what's actually going on we're just reading some JSON data from S3 logs and then I'm grouping by a few user IDM just aggregating which count for example or sum up the total amount of time I usually have spent on each of the page or each of the user and I want to write it out to JDBC database this is how we express this against static data so under this new model the way you actually express this for streaming continuous aggregation is to just replace those two calls with .stream and now you can actually drop new files new logs, their ship from maybe some other system into S3 and every time a new file appears we'll actually pick it up and run through this pipeline and they actually output it in place in this case you'll be bad if you run this query a year from now on all the data from now to the year later so the way it actually works is the first time Spark SQL sits actually a JSON file in the S3 bucket it's going to do some aggregate the second time a new block of data arrives Spark SQL is going to do the aggregate except in incremental fashion so we'll only be aggregating on the latest data and we'll actually combine automatically and we'll be able to run on historic aggregates without actually users having to worry about this at all and when new one arrives again the same thing will happen over and over and over again so this is actually not something we're only doing about streaming as I said continuous applications really needs more than just streaming and as a result rest of Spark will actually follow and be revamped to support this so for example we're using the same data frame abstraction there's only a single API and the second is Spark we'll introduce a new Spark data source API that can actually support exactly one semantics end to end and the reason this is worth highlighting is actually a lot of streaming system I think Spark streaming might have been one of the first streaming systems that support exactly one semantics so you don't want to charge a user either less or more times it turns out a lot of the streaming engines are work with exactly one semantics within the engine itself but as soon as you actually want to involve some output operations it's actually very difficult to guarantee you have exactly one semantics so within the engine where I'm doing all my aggregates and all of that I can guarantee records get processed once but as soon as I want to actually write it out to say my SQL or Oracle any other things exactly one semantics breaks here we actually want to have one semantics end to end which means from beginning to end for all of this and we have different output mode supports that supports actually writing out all the data just partial data in the delta fashion or updating data in place and machine learning algorithm will also be updated to actually support all of this so besides a very simple programming model that you only need to really think about batch data because you all have already know how to do it just by using the rest of Spark it's actually very difficult without the engines at the moment I think there's a few things one is worth highlighting it's actually very difficult to interact with queries directly on the stream in a consistent snapshot fashion and the second is dynamic changing queries I touched upon this earlier when I was talking about credit card fraud detections maybe sometimes you want to change your algorithm completely and you don't want to bring down your entire streaming applications or continuous applications and you don't want to bring down queries automatically, dynamically and of course you also get the benefit of Spark that you can have elastic scaling can have fault tolerance and all of that in this new engine so we keep all the nice perks Spark always had we're adding many new features with this much simpler programming model and really what we want is if you need to build ever something like this or even more complicated it should be a lot simpler a lot of things should just work out of the box without you having to worry about all the many great details about streaming so I'm just going to end the talk with a little bit of timeline as Matee announced Spark 2.0 will probably be released at the end of April or in the beginning of May in Spark 2.0 we'll be having this API foundations built up and really it's not a lot of new things from the user's point of view because there's really just data frames there's no new thing to learn it's just a new data frame API and there will be in the first version CAFTA data sources with file system data sources file system data sinks and also databases because we thought a lot of you probably have some relational database running who want actually to integrate out of the box with that with basically the correct semantics so you don't have to worry about doing that and you'll have to also support for event time integrations and Spark 2.0.1 and beyond it's a little bit hard to predict where exactly each version will have a lot of efforts into actually making this expressible in SQL so actually even some of your analyst friends can actually use this without using typing any code and then there's BI app integration that's built in plus other machining sources and things and streaming sources and things and machine learning alright so this is the end of my talk thank you Michael Ambras who will be giving a talk I believe it's one of the first technical session of the day where Spark 2.0 is going to connect with the rest of Spark structured I think the title of the talk is called structuring Spark from streaming all the way to data sets to data frames if you want to know more about details of this I encourage you to check out that talk thanks a lot for coming Spark 2.0 is very exciting our next speaker is from Capital One it's great to have folks from the enterprise come to talk about how Spark is bringing value to their use cases and the vision of the Summit yesterday and today our next speaker is the VP of technology for big data and credit cards at Capital One let's welcome to the stage to talk about their use case for Spark Chris Dagestino good morning thank you for the introduction I'm Chris Dagestino and as you mentioned I'm one of the engineers responsible for Capital One's big data implementation across credit card so today I want to talk to you a little bit about how we're using to better serve our customers, build better fraud defenses, and in particular what we're doing with Spark to help enable that. So let me give you some context about Capital One. So we've got a lot of customers across a broad range of financial products. Many of you are familiar with our credit card products, but we also have products in banking, mortgage, auto, and commercial. In total, we have over $200 billion in assets and deposits. And we've got 72 million customers that use our products each year. So this makes us one of the top 10 US banks. You can see a list of some of the competitors that we've got. We were founded in 1988. We went public in 1994. And in fact, we're the only bank founded in the last 100 years. So let's talk a little bit about what we do from a digital standpoint. We have inherently digital products. So from the transactions that are done online and with our mobile products, as well as the card present and EMV card swipes, we've got a ton of data that's coming into the company. And we have a range of both mobile and native mobile and responsive web applications to support those. You can see here that we've got customers that predominantly contact us through digital channels. In fact, 75% of our customers interface with us through these. But the other channels are important from a fraud defense standpoint, because it's these applications and these systems that traditionally are siloed and are points of attack for fraud rings. So if you take our customer transaction data, you take our servicing touch points, the instrumentation we have in our applications for clickstream data, if you add in our new credit card applications, which totals in the thousands per day. And if you take the third party data sources and feeds that we use to process customer information and build our models against, we quickly scale up to petabytes of data. So for us, it is all about the data. We're a data driven company, our strategies are based on having hired some of the best statisticians and data scientists in the industry. So everything from real time credit card authorizations with advanced fraud detection, our customer servicing touch points and our marketing campaigns to drive new customers to our products. All of this leverages massive amounts of data, massive amounts of customer history, and it's our job inside of Capital One to build systems that leverage it and produce fantastic results for our customers. This is a chart that we use a lot when we evaluate the development of a new system and we try to take into account the importance of the data that's coming in, both streaming data, landing into the system in real time or near real time. Combine that with the data that we've got in aggregate and understand that intersection. And so this is where the Spark platform has been really valuable for us, where we've been able to combine large data sets for historical information in both SQL and graph format and be able to query those systems, apply them to the models that we build and execute those models to make scoring decisions. So we operate in a really sophisticated environment with fraud rings that know a lot of different attack vectors. I won't get into all the details about what we see internally at Capital One, but we do see a lot of activity around synthetic IDs, hijacked accounts, stolen IDs, things like that where fraud rings will attack, come in through one of those different channels that I mentioned earlier, try and get some action to take place and then exploit that through another channel. So the systems that we're building are trying to combine defenses around data sets that combine from those different systems. And one of the things, we're here in New York City, we've got a lot of our competitors probably in the room, certainly nearby in the city. And so as a publicly traded company, we were very sensitive about what we could talk about. And so I wanted to give you just a quick flavor of how we build our fraud scoring engine, and then I want to move on quickly to some prototypes. So here's some background. This was what they were telling me, I was allowed to say publicly of how we compute things. So if you work for any of our competitors, I need you to leave now, and we'll talk about some of the prototypes we're building. All right, so in truth, we use a lot of different technologies in our big data ecosystem. We're a very innovative company. We would like to experiment with different approaches. And so you'll see a full range of things here. In addition to that, we toy with all the different public cloud providers. So let me talk a little bit about one prototype we've built in Amazon specifically, but we've actually done this in other cloud providers as well. But we leveraged some of the work that Databricks has done. They've written a great blog. You see the URL posted down there. We've taken data from our data centers. We've piped it into Amazon, into Redshift specifically using the parallel copy capabilities that you can run through S3 and into Redshift. So you see that there. We then take that data and using the Spark Redshift package from Databricks, we actually parallel copy the data back out, run it through a Spark cluster. You see the transformations that are taking place to produce the data frames. We then take that data set and we create both tabular and graph views of that data. We then take those views of the data and apply the models and the machine learning algorithms and run it against it for fraud detection. So what we're looking for here is those hijacked accounts, synthetic IDs. We're looking for nearest neighbors for people who are applying for credit cards. Have they applied before? Are they actually existing customers looking for a new credit card or an upgraded credit card? Are they people who have applied before and have been declined for one reason or another? Are they applying because they're part of a ring and there's some attributes about their application, whether it's click stream data or information that we collect through the application process that tips us off that they're somehow connected to someone else that might be fraudulent. So let me talk quickly about a few highlights for what we've seen with our Spark development journey. So we're prototyping new and sophisticated models. In fact, we're able to run models in parallel in the cluster and be able to score things and see different results from different scoring engines. The tabular and graph representations of data simultaneously is an important feature, and then we're doing things like auto deploying things out, which is pretty common using Chef into AWS. We're writing code internally that some of which will open source, but some of the things we're doing internally for auto scaling, our clusters are currently based on CPU utilization and we're trending more towards outstanding tasks and scaling up as the task load increases. And another thing that we've done to externalize the data pipeline workflow is we've developed our own mini language or DSL, if you will. And so I'll show you an example of it here. It's a JSON file that describes the workflow. So we externalize the steps that you need to follow. We can have data sets that are pulled in. And so in this example, it's just a sample workflow where we would pull data in from an S3 bucket, run a filter on it to filter out customers, say for a marketing campaign. To find those customers that have over $100,000 in annual income. And then we would write out the results for our marketing teams to use as part of their campaigns. So Capital One is quickly becoming a world-class software company. We are modeling ourselves after many of the great software companies that exist out there. And one of the things that we've adopted in the last few years is a full agile strategy for how we build software. So this is beyond just our software engineers. We are actually in partnership with our business. The business is going through and defining the priorities for the applications we build in an agile manner. We are getting our teams to be multi-skilled. So you see here we've got full stack development teams where for any project we will pull engineers from different tiers of the application stack and bring them in as part of an overall team. One of the things that we've got to worry ourselves over a lot, especially as we look at various public cloud providers, is the data encryption data security around PII and PCI data. While at the same time, encrypting that or tokenizing that information, trying to preserve the analytic value that's important for our analysts and obviously the fraud models that we run. So we've got world-class talent. We continue to hire. We hired thousands of engineers and data scientists this past year. We'll continue to do that. And this is kind of the prototypical model of what we're looking for. We're trying to take our existing statisticians who are very strong in SQL and have some general programming experience. Educate them on the Spark environment. Educate them in Python, Scala, Java. And try and get everybody in the company thinking about distributed computing. What it means to process both structured and unstructured data. Be able to be multi-lingual program and different programming languages. And then clearly understand the importance of algorithms and machine learning. So there's a short talk today but wanted to thank Databricks, especially we've been working with them and training and obviously some of the work that they're doing around the Spark community and then the blog post that they posted that we used to leverage our prototype. So that's it. Thank you very much. Thanks very much Chris. Our next speaker is from, our next speaker is from what was recently Razor site, but is now Synchronous. And he is the VP, the Senior Director of Big Data Platforms. Excuse me. Please welcome with me to the stage, Seren Nathan. Good morning everyone. It's great to be here at the Spark Summit and I'm excited to have this opportunity to talk to you today. Before I start, I just want to do a quick audience poll. How many of you are currently in or have been in the trenches doing data munging, wrangling, cleansing, and all of those good stuff? Okay, so hopefully this talk is for you. I can assure you when this journey began, I had a full set of hair. But then again, correlation is not causation, so. All right, who am I? I'm the Senior Director of Big Data Platforms and framework at Synchronous. We were recently acquired, I mean, I was with Razor site recently acquired by Synchronous late last year, been in this space for a long time. And then my goal is to solve real business problems with the latest technology, that's what everybody wants to do, but let me say that anyway. So how many of you have heard of Synchronous? Okay, Synchronous is a publicly traded company. We've had quoted right 45 minutes away in Jersey, and we offer personal cloud and activation platforms for large enterprises and communications providers around the globe. So what does that mean? So if you use a mobile device, any mobile device, chances are Synchronous is actually working behind the scenes for you.  Synchronous is platform and software is enabling all of that. So our solutions help operators connect to their customers. So whether you're onboarding a customer in the form of a device, whether you're allowing them to synchronize data back and forth from the device to the cloud to other devices, or if you're driving a connected car. The latest cars have 4Gs in them. Synchronous activates the connected cars. If you have a connected home, Synchronous probably activates a connected home. So that is the ecosystem we are in. And Razor site used to offer predictive analytic solutions to the communications vertical. So the marriage is all about applying our platform and products and models to the solutions that Synchronous offers. So we are part of the Synchronous analytics group. Just to give a sample of what is big data at Synchronous, what do we do? This is just a sample for one operator, large operators deployed the personal cloud solution. We're talking about 30 million active subscribers on the app, about 8 million daily active subscribers or users. They are uploading anywhere from tens to hundreds of millions of pictures every day. So the data size is staggering. We have deployed the solution across five data centers, running on multiple clusters and all the good stuff. So this is truly big data, all those events coming from those devices that can be used to improve the customer experience. Whether it is looking at better application functionality, whether it's looking at crash analysis, predicting failures, rolling out of applications and release versions. All of those good stuff can, the data can be used for. So what does my team do? We are responsible for the big data platforms and frameworks that is used to generate those consistent analytics. The platform is deployed both on a private cloud and public cloud AWS infrastructure. And when we talk about analytics here, we talk about the full range. So it starts with the traditional descriptive analytics or BI world and the advanced predictive analytics. So both end of the spectrum are there that we have internal users, we have customer users who consume this insights generated from the data. You should be very familiar with this in order to make any meaningful use of the data it has to be processed. So right from ingestion all the way to profiling, parsing, transforming, enriching, aggregation, et cetera, down to the downstream processes which may visualize it, which may apply models on top of it. This is what we're talking about data pipeline process.  It's not simple, people waffle over it. This is where we spend most of our lives. Data is not necessarily clean. Data is not necessarily structured, semi-structured. The definitions are missing, legacy systems, there's all sorts of things happening in here. So our journey started with version one back in the day and you folks should be very familiar with this. This is the day of the multiple ETL jobs running outside the context of data. Storage and processing were separated. Things were running in long running batches. And now we encounter any large volumes of data set, the latency increased. There's no support for unstructured data. Historically speaking, these sort of solutions took a year to put in place. And it's pretty expensive, inflexible, large teams working across. We could not store large amounts of data online because of obvious restrictions, so this was the life back then. Then we entered into an appliance world where, okay, we'll put the storage and processing together in one verticalized appliance. It was great, performance improved, latency reduced, but cost increased. Still we have to do it in batches, although low latency batches. Still we couldn't support unstructured data. Still the costs were so prohibitive that we couldn't store the data there. We had to do it just in time processing. Maybe store a limited amount of data and move it out somewhere else. Didn't work out. So move on to the next version. Then the whole Hadoop thing came about and said, let's go there. We looked at that, and we said that there's a big skills gap here. To have a bunch of people who are familiar with certain technologies that migrate into that. We said, let's take a pause and see where this is headed before jumping in. Then we saw MapReduce, a big hive, and a whole bunch of other acronyms. So we said, let's take a pause on that. We didn't want to do a technology migration for the heck of it. What we realized was the benefits would not be there immediately. So a couple of years ago, mercifully, out came Spark. So Spark provided a promise. It had everything required for pipelining, it had streaming, it had batch, it had SQL access, it had rich features in memory storage. So performance was better. So we said, let's take a look at that and let's centralize our pipeline process on this platform. So this is what we call our V4 data pipeline. ETLs are closer to data. We can process stream or batches, superior performance compared to MapReduce and other options. So what we did this time is we didn't want to open it up for every single developer or app developer out there. We abstracted it and built a framework. So we said, all the components needed for data pipeline processing, let's build components and then expose those components to those app developers for them to hook it up in a data pipeline process. So it simplified the design. It significantly reduced the time for us to roll out a solution. And it was highly flexible for us too. So with that, I'm going to go into data profiling. This is interesting. Good old days, we had volumes of data and when you want to do profiling, especially for the modelers, they would say that take a sample and then we'll profile the data in a sample and then utilize it to build the model. Then in the big data world came all the conventional wisdom is, no, use the full population. Don't use a sample. Run your model, train your model on the full population, okay? Then there are others that said, put everything in the lake and then somehow everything will work out. How does that work? We still have to go through what the data is to still need to understand the construct. So why do we still need data profiling? We need to understand what is in those data sets. We need to understand metrics. We need to understand the risks associated with creating rules. I mean, when you want to create an analytic data set, oftentimes you have to stitch the data to create that analytic record, which then could be used by the modelers. So when you stitch it, how do you generate the right rules? How do you make sure the quality of the data is good? Can we identify the metadata from the data set so that we can create those configurations that can be used in the pipeline process instead of manually hooking it together? How do we understand the challenges of data and inconsistency ahead of time? So anything you find later in a cycle is always more expensive and tough to fix. Also, another category of solutions came where you want to do ad hoc search, ad hoc full-text search. So in order to do those, you need to tag the data. How can you do tagging of data, categorize the data without profiling? So profiling became a key aspect of that as well. So when we looked at this challenge, and this is where most of the time was spent. If you broke down the project lifecycle, munging, wrangling, whatever term you want to use, that's where most of the time was spent. And if you really broke it down, there was a lot of touch points. So from the time the ingestion location data was moved to some other location, and there were many policies, security concerns. So moving data here and there was not possible. And as a result of all of that, there was just the interest level in any project is how soon it can be delivered. It takes months, multiple months, and the opportunities typically lost. So we wanted to address this particular challenge. So the typical scenario, I'm not saying everybody has this, but I have seen this many a time. We have analysts, business analysts, want to get a bunch of data into an Excel or somewhere else and look at it. Big data, you cannot do that. Okay, so we'll put that in a database and run our profiler, but then you cannot put it in a database unless you know what the data is, what the schema is, what the structure is. We get data from customers and they say this is what it is, but it is nowhere close to what it is. So that went cycle time there to figure out what exactly did you send us that but then you couldn't move data back and forth. That's the fundamental problem. You could not move it from a data lake location into a database or back to some other store. So all of those dependencies were causing a huge headache. So what did we need? We need speed. We need agility, we need automation. How do we automate this thing? How do we put the power back with the business analyst or data analyst? So we set out with this is the minimum data profiler requirements. We said all data is going to reside in the data lake. So you should be able to profile the data in the data lake. You should be able to review and validate the data. You should be able to review the statistics of the data. You should be able to use that same results to create metadata to run your data pipeline processing. You should also potentially be able to create downstream schemas. So if you're going to load this data into an index or into a downstream database, you should be able to create the schemas automatically. This were the goals that we set out to achieve. So Spark came to the rescue. We have large data sets, multiple data objects. We can move that into an RDD, split it up by field and run all sorts of metrics. We can use built-in transformations by Spark. It was very nice. Performance was great. So how does this work? So the simple flow is we have a very usable web application. The user does this points to a data lake location. And say, pick up a set of files based on masks, the full set of data objects, and then launch the Spark application, which then runs in the background profiles the entire data set and publishes the result to a repository which is viewable in the web application. Pretty simple.  There's a whole bunch of things that is needed by the data scientists to say how many nulls, can we create imputation rules out there? What's the health of those various attributes or the histogram? Ritosis, mean, media, and all those good stuff comes out, which can be used. This can be for an individual data set. It can be for any data in the data lake. It could be merged data, stitched data, enriched data. At the end of the day, these things are important before you can start the modeling process. So this is a sample screenshot. What it looks like is a simple Angular web app. You can go in there and pull up a result of a particular data set profile, give you a color-coded health of a data field, green, orange, red, however you want to set the threshold. I present all those statistics about the data field, whether it's amount of null, histograms, your box charts, things like that. So in language that is very usable for the data analyst or business analyst or data scientist. It also generates, as I mentioned, a full-fledged JSON metadata. So when you process the profiler, the profiler looks at all the fields. It does not only, it understands, it generates the data type. It generates the content statistics. It also generates the JSON metadata, which then is used by the data pipeline workflow. So if you want to operate on that data set to transform it and enrich it, you could use this metadata to drive that. It also generates schemas, downstream DDLs automatically from the profile output. So user doesn't have to go in and create all of that. Some data sets are very large and there are 40, 50 of them. You can see how much time can be saved by just profiling it, creating the DDL. The advantages are you can, all the source data already is in the data lake. It's been dumped into the data lake per the data location. All the profiling can be done in the data lake. There's no need to move the data back and forth. You can profile the entire data set, you don't have to work with the sample. You can integrate the results into a metadata configuration or a downstream DDL. All of this saves tremendous amount of time. It might sound trivial, but for those of us who go through this for a living, it's a lot of time. So objective is to send cleaner data down to the modelers because at the end of the day, if you want to generate roots, if you want to generate enrichment, the data pipeline process can be built accurately to cater to the needs of the data scientists downstream. So we've seen significant improvement as a result of this sort of approach. What used to take weeks, sometimes days, now it's cut down to hours. The overall data pipeline process has been reduced 80%, I would say. Which is why we say from the time we receive the data, we can put out full fledged metrics in the form of, let's say dashboards and descriptive insights. And under a month, we have identified data quality issues sometimes that trip us up ahead of time, empower the business analytics as well. Anyway, so I want to quickly go through this. This is just one component of our pipeline process. Profiling is the first part, but when we built the stack from the ground up with Spark, we said, you know what, we need a multi-layer architecture. Each layer logically performing a particular function, right from ingestion to data storage to data processing to modeling to integration to consumption. It's a pretty layered infrastructure there. And this is the architecture we have in place today. We just talked about the data management layer. So the framework components, they are all Spark components, at least in the profiling, parsing, transformation, integration layer. Each component has a set of functions. These components can be hooked up in a simple Uzi workflow, completely configurable through metadata. So the building blocks are available for the app developers. They don't have to sit in and write all those transformations. In fact, in the profiling and parsing, we have our own scripting engine integrated into that. So it's very easy to transform data, right? Some cleansing rules, lookups, substitutions, imputations, all of that are very easy to do with this sort of framework approach. If you look at the architecture, we have the data lake. We have the orchestration layer, which is Uzi. And then we have all the green boxes there are components in the pipeline, whether it's a SQL engine or a data prep engine or a database loader using scoop or a partitioner. The whole thing is built in in a component fashion. And then if you look at the stack itself, we have a certain software in there. We've used Elasticsearch for index storage for quick retrieval ad hoc analysis. We have the data lake, just a mapR distribution. We use Spark, extensively in the data processing arena. We have our own AngularJS visualization layer. What's next? We expand it. We continue to expand our component set and move more into the value aspect of it, bivariate analysis, multicollinearity, all of those things that typically is done on the data. We want to componentize that and string it together in the data pipeline as after the univariate. So the variable creation of the analytic set creation. I'm going to zip it through here. So the lessons learned here is let the business drive technology adoption. There are a lot of hidden costs. Plan incremental updates, deliver something to the business periodically. Simplify the whole thing. Framework-based development is very, very helpful to speed up delivery and also reduce our overall cost. I mean, at the end of the day, what our customers need is what is on the right. I know when a customer calls into a contact center, they want to know what is the lifetime value, what is the churned risk, what is the profitability. That's the kind of information they want. All the stuff on the left, that's the big data stuff, right? So we are all about delivering what's on the right to the customer to use the data insights to better their business. With that, I will end my talk. Thanks for the opportunity.


Adam Abusala worked on President Biden's 2020 campaign in Battleground, Michigan. Yeah, I was working on Arab-American engagement, getting people out to vote for Biden. So committed that his Palestinian-American parents put Biden 2020 on his birthday cake that year. But today, as President Biden shows unwavering support for Israel in the wake of Hamas' terror attack, Abusala says he's devastated. Right now, I have family in Palestine who's afraid for their lives. And Biden is doing nothing to stop it. I'm hurt for the betrayal that we feel from Biden. But I also feel like a little bit of guilt for what I've done. You just listened to part of a Meet the Press report by Shaquille Brewster, who confirms that Arab and Muslim Americans do feel betrayed by President Biden's unequivocal support for Israel and continued support for Israel, even as they indiscriminately murder Gazans who had nothing to do with Hamas' attack on October 7th. Now, this story comes as report. After report. After report indicates that Biden is burning any goodwill he had left with Arab Americans to the ground. And to make matters even worse, there is a mutiny brewing within the State Department over the administration stance, with high-ranking Muslim appointees reportedly considering resigning as a result of his decisions lately. And even though Biden has slightly adjusted his rhetoric to acknowledge the humanity of Palestinians a little bit, he still has not condemned Israel's war crimes or demanded a ceasefire, and he's now even downplaying the civilian death toll in Gaza. The Indians are telling the truth about how many people are killed. I'm sure innocents have been killed. And it's a price of waging a war. I think we should be incredibly careful. I think when not we, the Israelis should be incredibly careful to be sure that they're focusing on going after the folks that are propagating this war against Israel. And it's against their interests when that doesn't happen. But I have no confidence in the number that the Palestinians are using. Now put aside the fact that the Gaza Health Ministry just released the full list of deaths with names and ages that can be verified independently now by journalists, or put aside the fact that his administration is using those numbers internally, which indicates that he's lying there. But this point has to be made. When public officials and pundits deny and downplay what's happening before our very eyes, that is how genocides happen. And to see him say that is absolutely tragic. First of all, he says that Israel should be incredibly careful to go after Hamas in order to limit civilian casualties. But they're not. Limiting aid, shutting off electricity, and water is collective punishment, which they admit that they're doing and that's a war crime and you haven't condemned them for that. Second of all, Biden is rejecting the death toll being reported by Gaza's health ministry because it's run by Hamas. And it makes sense in theory to not trust what they're saying, right? But as Adam Taylor explains in an article for The Washington Post, many experts consider figures provided by the ministry reliable given its access sources and accuracy in past statements. Quote, everyone uses the figures from the Gaza Health Ministry because those are generally proven to be reliable, said Omar Shakir, Israel and Palestine director at Human Rights Watch. In the times in which we have done our own verification of numbers for particular strikes, I'm not aware of any time which there's been some major discrepancy. Now Shakir stated that they would not use the numbers if they were inaccurate because that also hurts their credibility as well. So despite the fact that Gaza's health ministry is Hamas run, like all entities in Gaza, since they are the governing body of Gaza, like it or not, the numbers have been relatively reliable, which is why people use it, which is why Biden's administration internally is citing those numbers. But I mean for argument's sake, let's just say that Biden is correct and they're embellishing those numbers. Well as Kyle Kalinsky or Dusty Smith says, well let's just pretend like the numbers are half or even a quarter of what they're saying. That's still a lot of deaths. That's a lot of dead civilians, a lot of dead children. But let's go even further. Let's pretend for a moment that we don't have access to any numbers. No numbers at all. We have no idea. Well still, even in the absence of numbers, there are enough high-profile deaths of civilians that we can see on social media. Enough examples of carnage that maybe might give people the perception perhaps that Israel isn't being as careful as you want us to think that they are. For example, former lawmaker Justin Amash announced that two of his relatives were killed by an Israeli airstrike while they were sheltering in a church. On top of that, moments after Secretary Blinken says he asked Qatari's prime minister to rein in Al Jazeera's coverage of the war in Gaza. An airstrike on the new Serat camp in central Gaza wiped out the entire immediate family of Al Jazeera's chief Gazen correspondent. His wife, his daughter, his son, his grandson, they were all killed. And they were taking refuge in this camp after they fled from northern Gaza following the IDF's evacuation order. And on top of that, when you look at satellite images taken before and after Israel's bombing campaign, you'd be naive to assume that this destruction isn't also accompanied by mass deaths. I mean, look at this. If Gazens are lucky enough to survive this, they will be permanently displaced. Many of them already have been displaced. But I mean, entire cities are being leveled. We can see that. Furthermore, with a lack of electricity that Israel says they are denying to Gazens and with fuel running out, meaning generators aren't going to be able to operate, babies and incubators and hospitals could literally die unless Israel either allows in additional aid or turns the electricity back on. So putting aside the numbers for a moment, don't you think that these tragedies, just a couple that I pointed out, there are many more, but just these tragedies that I highlighted along with confessions from Israeli officials that they're focusing on damage and not accuracy or that Palestinians are culpable for a mass's crimes too, or that they're human animals isn't necessarily instilling the most confidence in us that Israel is being as careful as you want us to think that they are? Have you thought at all about the optics and how they don't look good for Israel currently and that your unequivocal support for Israel kind of makes it seem like you're complicit in their genocide currently?  Apparently these aren't concerns for the Biden administration and if they are concerned, he sure as hell has done a poor job at showing that he's concerned, but the message to Arab Americans is coming in loud and clear, they don't think that he cares. Now, we kind of glossed over the fact that Secretary of State Anthony Blinken is pressuring a government to censor a news agency at the behest of Israel. That would be terrible even if Al Jazeera weren't credible, but they are credible and their reporting is mostly objective. But see, that's the problem. And former Israeli Prime Minister, Yair Lapid, explains why that's a problem. If the international media is objective, it serves Hamas. If it just shows both sides, it serves Hamas. In other words, support for Israel hinges on media being explicitly biased in their favor because if people actually saw the objective truth and the barbarity of Israel's war crimes, they just simply would not support it. So they have to lie to manufacture consent. They have to disseminate propaganda in order for the public to support their genocide in Gaza. And our government is assisting them with that narrative even after these hypocrites attacked Trump for calling press the enemy of the people. But now it's perfectly fine if the Secretary of State pressures a government to censor a news agency that is doing good work. It's despicable. But I do want to get back to the NBC News report because there's another clip that provides us with important insight that the Biden administration is apparently missing. They said that we had to save America from Donald Trump. And now we feel that we have to save Palestine from Joe Biden. More than 300,000 Americans from the Middle East call Michigan home. A state president Biden flipped in 2020 after Donald Trump won in 2016 by just 11,000 votes. And although not all Arab Americans are Muslim, an exit poll from the Council on American Islamic Relations showed nearly 70% of American Muslims backed Biden. We're predicting this could be another 2016. Nattah al-Huti runs the Michigan chapter of a nonprofit that studies and works to increase the political engagement of American Muslims called Engage. 145,000 Michigan Muslims went out to vote in 2020. Biden needs the Muslim vote in order to win. And right now it doesn't look good. The White House in Biden's campaign says they are aware of the concerns and are working to address them. We mourn every innocent life lost. We can't ignore the humanity of innocent Palestinians who only want to live in peace and have an opportunity. But the publisher of the Arab American News, who last year vocally backed Democrats in the midterms, is calling for his community to withdraw its support. Did you vote for Biden in 2020? Yes. Would you support his reelection? No, I have already made the decision and we are now going to endorse him in the paper. It's not only disrespect, disregard to our lives. In other words, there's an iceberg dead ahead and Biden is sailing straight into it at full speed. And those aren't the only Arab Americans who've expressed profound disappointment with the Biden administration. NBC News reports, quote, Joe Biden has single-handedly alienated almost every Arab American and Muslim American voter in Michigan, said state representative Al Abbas Farhad, a Democrat whose district includes Dearborn, which is home to one of the largest Muslim and Arab American communities in the country. Farhad said he has constituents and neighbors who have family members trapped in Gaza, including some who are American citizens, and they feel completely abandoned by the U.S. government for not doing more to help get them out, get aid in, and pressure Israel for a ceasefire. What I'm hearing now is people feeling completely betrayed by Joe Biden, said Amir Zahar, president of New Generation for Palestine, a Dearborn-based network of mostly young Palestinian Americans. People are feeling that the Democratic Party is unequivocally with the few exceptions supporting Israel and not humanizing Palestinians and looking at the humanity of Palestinians. Now, as Politico adds, may soon Zayed, a Palestinian American comedian, did everything she could to get Joe Biden elected in 2020, but after watching how Biden has navigated the Israel Hamas war over the last two weeks, Zayed said she can no longer vote for him in 2024. Quote, the man broke my heart, she said. I never in my life thought the empathizer in chief would sound the way he did. The Palestinians were given no humanity, she said. Joe Biden should spend every breath he has condemning Israel's genocide with the same zeal he condemned Hamas's massacre of civilians, that same zeal, and we get nothing. 1,000 children are dead and we get nothing. So you have a plethora of anecdotal examples of Arab Americans saying they're just not going to vote for Biden again. Full stop.  Now, I think internally they know and they're scared shitless, but they're still not reverse in course. Therefore, they are willingly bleeding support and not actively trying to change what's happening currently. Now, I know that most liberals and even a lot of leftists don't agree with that sentiment that they're not voting for Biden. They think that they should vote for Biden, because obviously Trump would be worse on the issue of Israel-Palestine. He'd be worse for Arab Americans. He'd be worse for every marginalized community, for poor people, for everyone across the board. And as someone with trans family members, I'm horrified at the prospect of Trump winning. I think he's an existential threat to the existence of my trans family members. And I don't think American democracy can survive another four years of Trump. Having said that though, you don't have to agree with their decision to not vote for Biden, but it's very important that you understand where they're coming from. You can't just lash out in anger at them, which is what liberals usually do when it comes to voters disillusion with the Democratic Party. You need to listen to what they're saying. They feel hurt. They feel like they're disposable. They feel used. They feel betrayed. They feel like they're not seen by this administration. And if Biden loses Michigan in 2024 because Arab Americans choose to stay home, he can lose the entire election. And guess what? That's on Biden. That's not on them. And Ellie Mestal explains why in an op-ed for the nation that Arab Americans aren't the only voters that Biden is alienating with his behavior here. He writes Biden risks labeling himself as a president who is in favor of colonization and one who will turn a blind eye to ethnic cleansing and war crimes. And those are tough labels to shake once they take hold in communities of color. Voters of color are strategic and willing to swallow a lot of nonsense and vote for the lesser evil. But there are some who will simply not pull the lever for any president in any party who stands aside while an oppressed people is besieged, starved, and bombed into oblivion. Biden can entirely appropriately call terrorist attacks against Israelis pure evil, but he can't seem to fix his mouth to say turn on the water to civilians living in the desert. He can say that Palestinian people deserve dignity and respect, but can't unequivocally say cutting off electricity to civilians is a human rights violation. I'm sorry, but it's hard to see Biden's personal decency when he can't speak up for the thirsty. Biden is asking black and brown voters to trust that he values all lives equally, but that becomes almost impossible to believe when he endorses the treatment of Palestinians as collateral damage to a counterterrorism campaign. Moreover, I feel like white commentators too easily forget that we live in a country where Trump is a looming threat only because a significant majority of their own cousins and uncles and spouses are set once again to vote for a raving orange clown no matter how many times he gets indicted for crimes. White folks are the ones threatening the future of American democracy. To save it, Democrats need super majorities of Latino, Asian American, Pacific Islander, and Arab American voters, and they need black people to continue supporting Democrats by well over 80% margins, and they need all those groups to turn out. So just maybe the white anti-Trump crowd could listen to voters of color with some respect when they try to tell you what's wrong with their political approach. Now, Ellie goes on to make a really important point. It also must be said that Biden's lack of support for Palestinians is being viewed in the context of his enthusiastic support for Ukraine. The situations are far from analogous, but Biden literally linked his support for Ukraine and support for Israel together in his Oval Office address. It's a politically tone-deaf comparison for Biden to make because linking the two doesn't work in his favor. It's simply not lost on voters of color that when a white country was invaded, Biden sent guns and missiles and money. While all he seems to have for Palestinians is some bottles of water chucked at them from Egypt and the hard-won pronouncement that not every single one of them is a terrorist who should be deleted from existence. So there's a lot of really important points made here by Ellie Mistal and another point that he made that I didn't read to you was if Biden does indeed approve more aid to Israel and another aid package to Ukraine, then voters at some point are going to begin to wonder, what about us? Where's our aid? Because we can barely afford to feed our families. We struggle to pay rent. So when do we get a turn? When do we get aid, right? And I think that that is something that he needs to address, but he doesn't seem concerned at all. But one of the most important points that Ellie Mistal made was that Democrats, they rely on a strong, multi-ethnic, multi-racial coalition to win elections. And if they fail to inspire confidence in voters of color that they actually care about the plight of marginalized people, then they may stay home. They risk losing those people. And it's not just Biden. I mean, take a look at members of Congress who signed on to the resolution calling for a ceasefire. You'll probably notice a couple of things. So one is that the list is embarrassingly small. And two, these are exclusively black and brown members of Congress. So it looks really, really bad to see virtually every white Democrat in Congress seemingly turn a blind eye to the suffering of brown people. And it looks even worse to know that they are all dogpiling on the only Palestinian American in Congress right now, lambasting her for daring to speak out in favor of Palestinian human rights. It communicates to voters of color that these Democrats don't actually care about their concerns. They're just votes whose utility is gone after each election. And none of them. Not a single Democrat seemed self-aware enough to course correct.  We supported you monetarily. We volunteered for you. Please support a ceasefire. He has not done that. He released a milk toast statement. So I don't think that they understand how bad they are hurting themselves right now by being so tone deaf, by refusing to listen. But the issue goes even deeper than that because Biden stands here is not just alienating every Americans and voters of color, but he's also alienating young voters. And he's already doing terrible in that regard as well. But this could hurt him even more. As Alexander Salmon of Slate explains, a recent Quinnipiac poll underscores Biden's disastrous standing with the youth vote. The president's favorability rating has cratered out at an almost unbelievable 25% among registered voters under 35 years old. Now Biden's Israel policy threatens to deepen that divide even further. Again, it's wise to not overreact to polls, but the data is not terribly ambiguous on this. On this issue, young voters are far from the president, who publicly remains hawkish and unstinting in a way that has not kept pace with their political attitudes or even those to a lesser degree of the Democratic Party. Biden has continued to pledge unquestioning and total support for Israel, even as human rights groups sound the alarm about the threat of the Israeli military committing ethnic cleansing against Palestinians, human rights groups inability to deliver anything resembling sufficient aid to Gaza and comments from Israeli military leaders that indicate a willingness to target civilians. Biden's Thursday night public address on the matter, which broadcasts right around the same time that Israel was bombing one of the world's oldest churches, killing 16 people, was cheered by Fox News, which is not incidentally a youth outlet. But none of the ways that Biden has responded to the conflict have been received favorably by young Americans. That same Quinnipiac poll found that 51% of voters under 35 say they disapprove of the United States sending weapons and military support to Israel a much higher figure than the 28% of Americans who oppose such a policy. Only 21% of voters under 35 say they approve of Biden's Israel policy. 42% of voters across all age brackets approve. A CBS news poll conducted last week came to an even starker conclusion. One asked if the U.S. should send weapons and supplies to Israel. 59% of respondents under 30 said it should not. An even more resounding 64% of those between the age of 30 and 44, a bracket more likely to vote that carries the whole millennial generation and part of Gen X said the U.S. should not. So little by little Biden is driving a wedge between him and the people that he needs the most to win in 2024. But he's not adjusting right now. And Simon also pointed out that young people were already disillusioned with Biden for restarting student loan payments without the $10,000 to $20,000 and promised cancellation. I mean after the Supreme Court struck down his initial plan he said he would pursue cancellation through his authority under the Higher Education Act. We've heard nothing to this point. And this handling of the issue certainly has not helped. And you've got to remember that millennials like myself we grew up during the Bush years. So we're very sensitive when it comes to U.S. foreign policy especially in the Middle East. So in response to 3000 Americans dying on 9-11 we watched our government butcher more than a million Iraqis who had nothing to do with 9-11 in response. And now it feels like we're already watching history repeat itself. In response to the tragic attack in Israel on October 7th thousands of Gazans who had nothing to do with that attack are now being killed. Over a thousand children now the number is around 2000. So many young people are just not going to accept what Biden is doing. A lot of people aren't going to including older people as well because people don't have an appetite for war. And even if this gets us embroiled in broader conflict in the Middle East it's going to be even worse for Biden. Now a Gallup poll released on the 26th also found that his approval among Democrats dropped by 11 points which is a stunning new low for his presidency and it's a huge red flag and correlation doesn't necessarily equal causation. So we can definitively say that that drop that we're seeing is specifically due to his mishandling of the situation but it's not unreasonable to think that it could contribute to that. It's not unreasonable to assume that he should right now start to win back the voters that he lost. So I need to be very blunt when I say this. Biden's inhumane one sided response to Israel's war on Gaza could literally cost him the election and as a result killed democracy in the United States permanently. This is serious.  But the good news is that it's not too late to turn it around.  What disturbs me more than the policy that you call a policy is the rationale for the policy. The rationale for the policy you set out four principles that you adhere to and then you and I will go over them in a moment then you say in page 14 we must not become part of South Africa's problem we must remain part of their solution we must not aim to impose ourselves our solutions our favorites in South Africa damn it we have favorites in South Africa the favorites in South Africa are the people who are being repressed by that ugly white regime we have favorites our loyalty is not to South Africa it's to South Africans and the South Africans are majority black and they are being excoriated it is not to some stupid puppet government over there it is not to the Afrikaans regime we have no loyalty to them we have no loyalty to South Africa to South Africans and the fact of the matter is we I mean I listen to this rationale first of all it is the leaders of South Africa and their people black and white who have the majority responsibility they must rise to it well they are rising to it they're rising to it the only thing left available to them is that repulsive repugnant regime of Afrikaans there and it's the only way they have they've tried everything for the last 20 years they begged they borrowed they crawled and now they're taking up arms the second thing progress toward peace requires a timetable timetable elimination of a part what's our timetable what are we saying to that repugnant regime are we saying you've got 20 days 20 months 20 years we ask them to put up a timetable what's our timetable these people are being crushed and we're sitting here with the same kind of rhetoric the same thing we heard we heard go slow we heard we have to take care of the problem afterwards we heard we can't impose you are totally misconstruing the testimony that I gave read first furthermore senator let me say that I hate to hear a senator of the united states calling for violence I'm not calling for violence that's what you're doing I hate to hear exactly what you're doing I hate to hear an administration and a secretary of state refusing to act on a morally abhorrent point I hate to hear this country I'm ashamed that this country puts out a policy like this that says nothing nothing


The theme of migration is quite important for the area we are visiting this time, which is Europe. Most of the countries are receiving thousands of people every single day, and they don't have the capabilities to support them as a human being. So this is a problem that all the countries in the region, including those that are not part of the European Union, have to solve. And I think that it was a great opportunity for alumni to go back to their critical thinking skills and the tools they learned to use in the world college to throw ideas. And I think from that point of view, everybody, not only the panelists, but all the people in the audience were participating actively in sending and creating ideas.


Hey, what's up you guys welcome back to my channel if you're new here. Hi. Hello. I'm Lydia It's a heatwave currently and I am sweating a lot It's not fun. I Don't like the heat I'm a winter baby Sun help Save me today. We're gonna be talking about my experience with this young So I've been on lithium for four months now and these have been the most stable four months of my life So those who don't know lithium is a new stabilizer I started it because I was in a very severe depressed episode while I was in hospital and I wasn't improving so they always started lithium and almost right away and my mood Changed. I was more positive to more awake Lithium basically saved my life. I highly recommend lithium as a drug in itself Especially if you're going for a depressed of episode whether it's because you have BPD or because you're bipolar or just generally speaking I would highly recommend lithium Because it works very quickly and it's a medication that you can be on long term I do believe if I hadn't started it, I'd still be in hospital Lithium is one of the more common medications that you have prescribed when they're bipolar. I know so many people on it I think we all agreed that it saved our lives I know for me that if I hadn't started taking lithium, I'd still be in that depressed episode And still be anxious all the time. It just wouldn't be great It's something that I think I need and I wish I'd started it sooner Because it's been so beneficial to my mental health and I wouldn't be sat here today Doing as well as I'm doing if I was not wasn't on lithium So I have a big part of my recovery to lithium It's an amazing medication that works so incredibly well when it's used right The only downside to it I guess is you have to have lithium blood taken every so often So they can check your lithium levels and make sure that you're on the right path I had my lithium levels checked a few weeks back Lithium works by changing the way your brain See use chemicals it changed the release of chemicals in the brain Eight of you have ever done lithium? What's your experience on lithium? I'm interested to know let me know in the comments. For me, it's been a lifesaver lithium for me Changed everything to the consultant who started me on it Thank you. The only issue I had when I first started it is I couldn't sleep It was too alert. I was awake for two nights or but I don't really have anything negative to say about lithium It's worked very well for me And if you're on it, I hope it works as well for you Thanks for watching this video and I'll see you guys in the next one If you have any video requests, please leave it in the comments down below