Γεια σας, πρόκειται για να αρκετήσω τη μία εξαιρετική μου, δε θα ξεκινήσω λίγο πριν. Θα προσπαθώ να τα κομμάσω, είναι το τέτοιο εξαιρετικό που ξέρω, πιστεύω ότι όλοι είναι γεγκά. Λοιπόν, σήμερα θα είμαστε εδώ για να πω για να βρήξει από τέτοιες μεταλλακές μονοδικές ποιπλίσεις, με δύο διαφορετικές προς τεκνολογικές από απατσίο, απατσίο Φλήνκ, τένσο Φλο και Hobsorx. Hopsurgs είναι η ανοίκης πλατφόρου της ανοίκης που εργαστούμε σε σχέδια λογικές. Λοιπόν, πρώτα θα πούμε τι είναι οι τελικά πλατφόρου της ανοίκης που εργαστούμε, ένα πολύ σκέψενος παράδειγμα. Τι είναι οι Hopsurgs, το τελικά πλατφόρου της ανοίκης που εργαστούμε για αυτό. Τι είναι το απατσυπίμπωτας που να πει στον κόμμα και πώς εργαστούμε με Φλήνκ. How we can build machine learning pipelines on top of all this with TensorFlow Extended and we will explain later what that is and we will finish with a short demo. So what machine learning pipelines are. Here we can see that we have, we apply essentially transformations on data, so we get data, raw data from different sources, it can be on the internet, it can be IoT devices, transactions, it can be many different things, so we need to ingest them into the platform, the platform of your choice. There you need to ingest and then do some data preparation, some validation and transformation. For all this typically you need CPU compute power. Now then when you go to the machine learning part, typically you need GPU compute power. Now then after you train your model you need to be able to serve it into production for different clients to be able to use your models actually. Now one way to do all this would be to have a single pipeline where you have all the stages within the same let's say processing framework. There is a project in the Apache Spark community called Hydrogen which tries to do that, it hasn't been released yet because it really is a complex problem, how to keep everything in the same framework. The other way to do it is to have some distributed storage where you actually separate the data preparation transformation part and the machine learning, the training part effectively. What we do with this is we have the concept of the feature store where it is a place to manage your feature data. So you do your feature engineering on one end and then you need a very scalable distributed file system which is part of the platform. You can get your feature data in different formats from the feature store or whichever way you want to actually do your, install your feature data and then you continue and do your training and serving. So that's a very like gentle introduction. Now what is Hopstorx? Hopstorx is an open source project. It has been going on for a few years now and this is a short timeline of what we have been doing so far. One of the most recent ones is the feature store and what we're also showing here today, how to run TensorFlow Extended on Hopstorx with Apache Beam and Flink. Now Hopstorx as we said is a platform for doing data intensive AI and on one hand you have all the different data sources that you bring in the platform. You apply your transformations, you do your machine learning, your deep learning and then the output is different applications and APIs to access all this data and functionality. So if we zoom in a little bit, we can see the different technologies that we use to implement this. So at the bottom, at the down layer we have Hopstorx, it's our distributed file system and the unique thing here about all these technologies that we integrate is not just a bundle of technologies but we keep the metadata of all these different technologies in the same meta store, in our file system. So then we support streaming with Beam, Spark and Flink. You can do also bots with the same technologies, the feature store which is built in-house and different distributed machine learning and deep learning technologies. Really any Python library that you might want to use. But the main ones are TensorFlow and PyTorch as we have been seeing. And then you have your model and then you can serve it and you can do that with Kubernetes in an elastic way. You can ask for more or less containers and then the platform will manage to automatically do this for you. Now all this comes with a nice web UI, you access it from your browser. But the principle is that everything needs to be accessible via a well-documented REST API. So these are some of the resources of the platform so you can create projects. It's kind of like a workspace, you can imagine it. Data sets where you actually store your data, jobs, your compute jobs. It can be Spark, Flink, Beam, feature store, etc. So it's very important to have this well-documented and follow best practices to allow actual users to be able to use your APIs from different clients. Now that was what Hopsox is, the generically say platform that we are building. And the main part now starts of the talk about Beam. So Apache Beam is a framework to do distributed processing. And it's also like a programming model built on top of Apache Spark, Flink and different runners. So what it gives you really is a way to write your programs, your distributed processing programs in one language of your choice. And then this Apache Beam will manage to execute your code in different runners, whichever runner you have in your platform. Now in our case it's Apache Flink also because it is of the on-prem, let's say, solutions is the more mature one for Apache Beam. If you're on GCP you will use probably Dataflow to do that. So the idea here with the portable runner is that you implement your program, let's say in Python. This program will be converted into the pipeline that will be executed later on by the runner. So you can choose Apache Flink, Dataflow, Spark, there are a few others like SAMSA I think. Then the actual execution will be pushed down on each machine. So the idea here is that you can distribute your execution, you can have a scalable engine to distribute your execution. You don't need to just be restricted on your laptop. You can just take your code and distribute it on a cluster. Now this needs a lot of manual labor to get it done, to run it. And there are a lot of things that need to be done to make it, let's say, not production ready, but to have something that fits your production ready needs, let's say. So what we have been working on is how to have Beam as a service, let's say, in hopscorks. And what does that mean? That means we try to do things easier for people to actually start, implement a Beam program and run it. And that means that you can develop different Beam pipelines in Python from your Jupyter notebooks. So the portable runner that I just described is kind of a recent development in Beam and a very important one. Because the Beam concept from whenever Beam was incepted kind of, but only just recently there was being improvement and progress on that. So you're not restricted just on Java. You can write your Python, it is very important for machine learning, because you can really use all these Python libraries for machine learning on top of a distributed framework that can give you different runners. And that one is Beam. You can develop your Beam pipelines in Python from within Jupyter notebooks so you have this familiar environment of notebooks. Now, you need to have to run something as a service. You need some tooling to simplify deployment and execution. That means that the platform where you run Beam as a service will manage to deploy artifacts to collect logs and all that. We will see later on more details about it. Then you need to manage the lifecycle of the Beam portability job service. We will describe again later on, but these are different components of the Beam portability framework that you need to manage either on your own or you have a service to do it for you. Logging, as we said, and then the important part. If you have a Python program, then it will be pushed down different executors on Beam. But there you need to have your Python environment already either on the machine or a container. We will see what's the trade-offs there, which are the trade-offs. And in Hopsocks at least we provide Flink and also Spark is a work in progress because these are the main runners if you don't run Beam on Dataflow. These are, and especially Flink is the one with the more complete functionality, is the way to run Beam, let's say, on-prem or if you're not on Dataflow. Now, we have developed some APIs. We have listed them here that help with all this distribution of artifacts, logging, et cetera. These Hopsootl-py and Hopsootl-python and Java, et cetera, respectively, simplify deployment by doing the following. They set all the security configuration parameters because we have this security model we will explain again a bit later by using certificates and then these APIs have some default configuration and they provide getters and setters for all your security configs. We provide service discovery, so when you're in the platform, you have other services that are running, not just Beam and Flink. You might have also, we have Kafka for ingestion, you have elastic search available for free-text search on your logs and all these things. You need to know where these services are located, if you want to access them from within your Beam program. So we have service discovery from these APIs, you get back the details of where these services are running. Helper methods for the Hopsootl-Rest API, so from within your Beam program, you might want to do something more Hopsoot-specific like run another job or upload a file, create a dataset, something like that. And a very interesting ML Experiments API, we'll show later how you can run machine learning experiments on Hopsootl-Rest and with TFX with the Helper APIs that we have built. Now, the question we posed before was, which one is better? If you want to run a Python program in a distributed way with the Beam portability runner, then you have two options really, either the process or the Docker one. Now, this really boils down to how you want to manage your infrastructure and what the rate of your willing to take depending on the use case. Now, if you go Docker, then you have to, what you need to do is build your image with all your dependencies or you have to download them later on when the container is actually deployed. Now, if you make changes to your Python environment, then you need to update or modify your image and you need to have additional infrastructure components to deploy the images and all that. Now, the process one runs natively on the file system and you skip all that, but still you need to be able to have consistency of your Python environments across all different machines. Python is notoriously hard to get it right in Python when you uninstall and you install different libraries, you get dependency conflicts and all that. So, we went with the process one, it fit our model better, but it really is a matter of how you want to manage your infrastructure. Now, how we solve the Python problem, when you want to have the same environment on all the machines in the cluster, we are managing the environment with Anaconda, so you have better workspace, each workspace has its own Python environment and its fully self-service. You go in the UI, we'll solve it later on, you search from the UI, I want to install this library like BIM, BIM also has a Python library. So, it will go, search for the library and it will install it. We have some services running in the background that will manage to replicate and install all the environments and have it available on the cluster. Now, this is the infrastructure part, if you want to start actually a Jupyter dashboard, Jupyter notebook, you go in the Jupyter dashboard in Hopstorx and you can select different settings if you want to run pure Python or Spark, for example. I will go a bit quickly through the screenshots, as we have also a demo later on. Now, as I said, Jupyter and Jupyter lab in Hopstorx, you can execute a BIM Python pipeline. That can happen in Hopstorx in many different ways. So, you can run your pipeline, as we see here in the example, yeah, it's a bit small letters, but you can run it in pure Python kernel, or what we do, we use Spark to actually orchestrate execution of the Python programs in the back and this is transparent to the user, the only thing they need to do is select the PySpark kernel in Jupyter. But if they go with Python, the Python kernel will run in a Docker container managed by Kubernetes, which is in the Hopstorx installation, so you don't need to manage that. An important aspect of this is that you use notebooks to actually develop code to develop your BIM program. That's good for your development. What do you do when you want to deploy this in production and run the notebook really as a core component of your production pipeline? So, the first way is that you have your data scientist and they use Jupyter to have interactive programming in Python. So, you start Jupyter and then we go to LibyServer, a service we have in Hopstorx to actually start your notebook. Now, if you want the other way to orchestrate all this, we have Apache Airflow as an orchestrator, which does the following.  Now, the notebook, automatically, by Hopstorx will be converted to a Python program and then it will follow the same, let's say, route of materializing all the certificates, all the security secrets and everything and it will be deployed on the cluster. Now, both notebooks and the Python programs that are converted from the notebooks are stored on the distributed file system that we described before. So, consistency metadata, we have also the notebooks there, so everything under the same roof. Now, if we dig a bit deeper in the portability framework of Beam, this is an image taken from, there is a link over there. This is a very good figure if you want to understand how the portability framework works. So, what happens is you have your Python program developed in Beam, you have your Beam pipeline and then you would need to run this job service. This is a standalone service that you need to start and you attach it to a Flink cluster. You need to have started the Flink cluster from before. This is like the way that you would do it if you wanted to run Beam on Flink, let's say, on your laptop. Then you need to have a staging location. Here we see DFS, S3, whatever. It really depends on you. Where Beam stores the artifacts and the state and the logs of the job service and also of all the task managers in Flink. Now, to simplify this, to have something that is as a service as we said, we have hopscrux at the top, which is the entry point to this architecture, and we have hopsfs as the backend storage, the distributed file system, where we store the artifacts. Now, the runner, we automate the process of starting the runner and the job service, sorry, and we have compiled it with the dependencies that come from hopsfs, so they must be compatible in order for the job service to be able to talk to hopsfs. To help do all this, as we said, we have these helper APIs that we can compile to the Flink cluster. Now, the important part is that you need to start all this. You need to start the job service, you compile it, you need to start it, and you need also to start the Flink cluster. We have, again, helper methods where it's written in Python and you just do from hops import Beam as hops Beam, and then you do hops Beam.start. What this will do, it will start the Flink cluster automatically. There are some default values there in the call, in the start call, and also it will start the job service that you see in the middle of the screen. Then you will be able to, the API attaches the job service to this particular Flink cluster, that's the important part. So then the job service will send the task to be computed on the Flink cluster. So there is a method here get portable runner config which returns all the portability, some default portability configuration and you can add your own, you can override the default ones. But these are the ones that come out of the box in the hops works. And the last part is that the SDK workers these are the actual workers that will run your code on the Flink cluster have access to the Python environment. That has, as we said before, has been created and synchronized across the machines by hops works itself. So this is a view like a screenshot of integrating Apache Flink in hops works.  Now the API in a bit more detail you define your runner name this is really the name Flink cluster that you are deploying you can override that and then the default configs that BIM can actually set for Flink. Now you have a different runner like Spark or SAMSHA then this would be different then we have a different like start method for the same start method but with different parameters for every runner but for now you can say I want this heap size for the Flink job manager this number of task managers you can increase it if you want to scale out more memory, the slots this is Flink specific, the number of slots in Flink and kill runner. This kill runner means that when you actually stop your jupter notebook when your BIM pipeline is finished there is an option here to shut down also the textual runner because we try to make it transparent to the users you do not really need to care about the runner, the Flink or the Spark or whatever the framework should be able to handle that and start the runner and shut it down gracefully and collect all the logs and everything. Now optionally you can start the BIM job server it's done automatically by the start method but if you want some more control over it this is the API Now logging is very important because with all these different frameworks and services you have logs scattered kind of everywhere so you get logs from the Flink job manager and the task managers which in our case run on yarn you get logs from the BIM job service itself sends the pipeline down to the actual graph down to Flink the tasks so there are two ways to run this in local mode and cluster mode in hopscorks that is and then you might get your logs in your jupiter directory or in a cluster somewhere in a machine where this PySpark container will be running you also get important logs from the actual SDK worker your Python program will run in the worker so if something goes wrong you really need to go deep and see ok my Python program failed because the dependency was missing or segmentation falls you might get also with all this transferring back and forth of pickling and pickling etc and to collect all these we use the elastic logstash kibana stack the elk stack so we collect all these we aggregate them into the workspace and then you can from kibana then you can view we have an index pair service kind of and it looks like that from within your workspace you have kibana and then you can see the logs for example these are the logs of the beam job server it says ok manifest to the manifest location remove directory and all that now about security in hopscorks all the services like the store ads, the execution we use TLS certificates to talk to each other and also for clients to talk to these services this is this system is inspired by a Netflix system called bliss it was open sourced a couple of years ago and internally we use TLS certificates everywhere short lived TLS certificates now for users to actually access the platform you can log in with your username and password or LDAP and all of with support for Kerberos that's to enter the platform and then everything is automated now about the machine learning part so we did beam we did we did beam we did we showed how to run your programs and how we set up everything now about the machine learning part so there's a very famous paper by Google on the hidden technical debt in machine learning so of all this that you see here I hope you see because here I lost the monitor actually so of all this the actual machine learning, the training this white box in the middle all the rest is infrastructure development related you need to do all this to do the central part properly and there is a lot of progress in the community lately with different frameworks that help data scientists to automate engineers to automate these processes and all this pipeline so one of the one of the most popular ones is TensorFlow Extended or TFX which as we see here this is taken from TensorFlow's webpage you can provide a set of components and with these components then you can do data validation that means ok I got my own data and then at scale I run this on beam I want to run a program for these terabytes of data I want to see I want to get statistics on my data I want to see if I have a lot of kernels or some values might be empty, things like these then you can run the data validation component then you can run the TensorFlow transform so you can transform the values and compact them let's say in different if it's numerical you can compact it in a different range and then you actually go to the training part TFX of course uses TensorFlow to do the training part after that there is TensorFlow model analysis so you train your model but is it performing well is it like valid is it correct, do I need to retrain maybe I made a mistake so TensorFlow model analysis helps you with that with these two components evaluator and model validator so after you are sure that your model is correct then you can go and push it to TensorFlow serving so your clients can actually start making requests against the model now this is the pipeline of TFX how does it actually run now TensorFlow data validation and transform they use, you write a program in Python of course and then we utilize BIM on Flink to run it same goes for TensorFlow model analysis so you have all the data in the distributed file system you have your Python code that is using TensorFlow model analysis component and then again BIM on Flink for the training part in Hopsorx whatever library but here basically we saw that in this case with TFX you will use TensorFlow and then you can run it on GPUs so in Hopsorx you can do for example distributed training with GPUs and the GPUs are managed allocation of GPUs it's managed automatically by the platform and TensorFlow serving as I described before is using Kubernetes with Docker to make it elastic now when you do your training you need to be able to track your experiments that means that you have your notebook you have your data you can get it from the feature store or you can get it from this TFX pipeline after the transform part when you do your training and you want to manage the metadata of your experiment if you want to go back in time and actually run it again so this is tracked by all the helper libraries that I described before and shown in the UI in this manner there is integration with TensorFlow so you have your experiment let's say you run an experiment a week ago and you want to view the TensorFlow we kind of bundle all the experiment artifacts everything that you need to know about this particular experiment code and you can have version data from the feature store and you can open TensorFlow and view your past experimentals now about orchestration there are a few ways in Hopsorx we use Apache Airflow and we provided kind of as a service which means that when you are in your workspace everything is set up for you to actually use Airflow in a multi-tenant way that means you cannot access all the services in the workspaces in a multi-tenant you cannot access services outside of your workspace you can invite other people other members in your workspace but you cannot manipulate other people's like workspaces so same goes with Airflow as part of your workspace your project and then what you would typically do you would write or develop via the UI you would have your Airflow DAG which is a python program in the end that's what it is and here is an example of it this is the DAG that I will show later in the demo to actually run the tfx pipeline so we will have one notebook per tfx component and Airflow you schedule it and you say ok I want to run it periodically per week, per day, at this time and all these things it is quite a powerful tool and this is how the pipeline would look in the trivia of Airflow but then you have different tools to actually monitor and monitor the progress and everything and this is the view where you have all the jobs so these jobs we call them jobs we convert it into python programs and run on beam and flink so it's notebook here corresponds to it's part of our pipeline that's one way to do it there are many many ways to do it but this is like a basic example so you have compute statistics in first schema, compare statistics etc now to put it all together there is this nice architecture in the beginning we had kind of a structure but now we put in all the components that we described so we ingest data you can use on the platform Apache Kafka for example and then you can do your data preparation with beam and flink but in this example we go with beam and flink runner feature store or tfx transform to transform your data and then you can store it in the distributed file system then you continue with training actually the interesting part and then when you have the model you want to analyze it, again beam model analysis on flink and then serving now the interesting thing also in serving what you can do for online serving is that we track all the inference requests that go to model serving and then you can have streaming job let's say that gets all the inference requests in real time and monitors performance of the model so then you can decide ok I want to retrain I got new data or my model is not performing very well and all that is backed by this metadata store now demo let's see if that ok can you see ok so we have this instance is running on AWS right now we'll go here pick things and let's start with so the example is based on the Chicago taxi ride it is on TensorFlow TFX github so we got it from there and then we developed the notebooks here so if we open here yes so let's do first a simple one word count ok so this is an example of how you could do before TFX like something even simpler some word count let's say on beam so we have this from hops import beam as hops beam then hops beam start here for example I wanted to override the default memory and set 8 gigs for it so let's go we'll go back we will see that these are the jobs this is the runner and now it is initializing so for me it created the link this cluster description it submitted it on the resource manager and it's starting to to run it so it is running now so we can click here and then hopefully yeah so this is the link dashboard if we go back to our nice notebook then here we defined word counting workflow ok we just import this stuff and then here is the actual do fun that we have in beam and here we provide the arguments for so this now got the default beam pipeline options that come from hopsworks and you can override them for example SDK worker script this is the actual script that invokes the worker python process and all the like ports and everything so what we should see here yep so now the request went to flint cluster with the beam portability framework it was submitted on the flint cluster which is computing now a set of tasks so in total so in total here it has to compute 15 tasks as we can see it already has done 7 it takes a bit of time and because I want to also do the tfx example I will cancel this so we can continue with the tfx which is more interesting but the output in the end just to show you if we let it run because I ran it before it should be in resources here is a sexpear output so we got sexpear and we did some word count and then this is the word count, this is the output if I let the flint job finish now for the more interesting part we can go to let's say there are two ways to run the notebooks here there is one big notebook that runs the entire pipeline in an interactive way and this is mostly done for development so let's say ok this one also tfx we did this a while ago but tfx recently released their own interactive notebook so here what you would do is again you start the runner, the runner is already started so it should be fast it returns back the job server B job server parameters like the port the host, where the log is then we do the imports then we download the input dataset we clean up the previous run, these are our parameters and then we start the actual pipeline here is to compute the train stats in the data validation part so let's do this yeah train stats is notified ok name option ok let's run it again I think I need to restart it yeah ok so that's the kind of interactive way this takes quite a while to actually run it's a lot of tasks, if we scroll down let's keep it running then I can show you then we have to infer the schema the schema of the data that you have ingested and you want to continue with the training later on you check the eval data not just the train for different errors and then if we scroll down you freeze the schema you say ok this is my data now I'm ready to transform it and then you do this part with the pre-process inputs and then if we continue we compute statistics over the transform data we need this later on in the pipeline when we also go to model analysis here we actually define the experiment, the training code and here we submit it with the run local experiment and this is the part where you run TFMA to compute the matrix so you run TFMA as we can see is a beam pipeline then you enter the data from the file system or in memory you might have it in memory here it's an interactive notebook and then you run the visualization so that takes a while let's see if it actually so I have the output already from a previous run I can show name options and define ok never mind so let's go here the last thing is the jobs so what we want to do we have defined a set of jobs, this job is a notebook and the notebooks are here in JupyterLab so if we go to pipeline we can see the notebooks this is the same, these notebooks are from the interactive one but we split them in different steps so we can schedule them with airflow actually so we can see here it's exactly the same but we can go to airflow this is our DAG and here we can see you set up some properties like the project name and all that and then this is the actual DAG I want to run compute statistics after that I want to infer the schema compare the statistics freeze the schema, preprocess the inputs transform the actual data do the training and then model analysis this is actually the lower level API of TFX there are you can also run a notebook I can show you here this one which uses higher level components of TFX and you can actually build a pipeline not with just airflow but you can run this pipeline here with the beam DAG runner which is brought by it is a nice way in TFX to skip if you want like more complicated services like airflow and you can go just with the beam runner it will do the DAG for you now let's go back to airflow let's open it so we have the pipeline and then we can try to do this yes I am sure so if we go to graph view it will start as you can see it started now this is the DAG compute statistics everything it started with the first one it will run all the jobs which is one job one notebook everything in the distributed file system now this takes a while let's see if something happens so here I run the DAG and you see already it started running the first step of the pipeline but we have the results already we don't need to wait so here are the visualizations so we have this notebook to visualize the stats we read the stats from the file system so we can do this no such file ok it's because I run this example which wipes kind of the visualizations so we have we would have to wait let me see if I have other ones ok so that will continue and then you can monitor also the progress from here that's it if we go back and data sets let's see the last thing taxi data that's just the input data as the last thing and then the resource pipelines and resources last modified so as slots are being computed they are being resisted here this is the actual file that was computed ok so let's go back to the presentation and conclusions so Hopsox 1.0 is the first on-prem horizontally scalable platform to support BIM and FLINQ portability framework which is very important for machine learning because it brings support you can run Python really on all these runners and then we now our future work is to finish integration with Spark runner as we have support for Spark as well and integrate the feature store with the effects that would be a really nice thing to have and last thing also export metrics of the FLINQ runner to InflexDB and visualize with Grafana different metrics that the actual execution engine exposes contributors and how to get started you can get a free account on Hopsox to try it out with AWS or you can deploy it on GC as well thank you


Till oh, what's poppin? We are on twitch. We are not live unfortunately But you can still leave a like comment subscribe turn on your post notification bells. Let's continue to grow the family from Chicago to the UK As you can see we are doing a music reaction You got to be at least a seven to get on this list to be considered for this list, man So let's check this out man. What we got we got six seven monkey. It's called.  You don't want to be positive You want to be negative at all times? I don't know why they envy we I don't know was it cuz they're friendly see When I look at the mirror, I see a G when I took it ever got back on my feet I think a little from a to be because five times in a day is one fun cool You can be with junks in a separate hammers. We never knew life could turn this way We top the bar from boys to men.  He's actually killing this beat. Okay But nobody checking for monkey no more wise three days under 37,000 views What's going on? Are we not appreciating drill no more Sinking raising my cup when I'm drinking You ever I Bet you they ain't by not one of them watches and if they did my bad salute But I just watched a video where it was a whole crime ring around stolen watches man. I don't know what's what I Got the right level of negativity. They got the right level of flow. They got the right level of you know, I'm saying Demo demand. What's the word I'm looking for? I don't even know it got it got the right levels though Oh word play. I don't know where I started the D from you get pause Wow, they got the right level of word play like I like this like I said man on a scale of one to ten I Still give about a seven though. You just it's on the cusp. It's still I was still right I'm still putting it on the list though I like it till I leave a like comment subscribe turning a post. I'm gone man


So in the previous video I gave you some intuition about this idea of group action which I said can be slightly difficult. So that was the intuition and I glossed over a few proofs but I think there's one we really should just spend a little bit more time before I get to the promised non-faithful group actions and some examples there. Remember in the previous lecture and if you haven't watched it this is not going to make sense so I'll put a link up there. So I had my set with three elements and I had the G's. I'm going to write G set that's the set that is part of the group G and that was this e tau 1 2 3 or tau 1 2 1 3 2 3 sigma and sigma squared and the idea is how do we the idea here behind group actions is that these are represented by permutations of that set. So group actions is to have the elements of a set that makes up a group onto another set and those are permutations and what we mentioned quickly was this sigma G function that maps A to A. Now think about what happens when I map a set to itself. So I have a set of objects and I map it to itself that is the definition of a permutation. I'm just going to scramble the order of those of those elements that is a permutation and we define sigma G as mapping A to the action of an element in G onto A. So remember this symbol that doesn't mean this A and this A is the same thing it just means I'm running through all the A's for every G. So if we if we think about what sigma was what the mapping of the sigma was it mapped sigma there it mapped two there's the two and that is the one there and then the three and the two so see these two A's it doesn't mean they the same thing when I use the symbol it's not equal to each other it's something different. So two was equal to then sigma dot one and three was equal to sigma dot two and one was equal to sigma dot three. So it's an action of the group set one of the elements in the group on one of the elements in the set the action of that same element G so this G is the same G G G and one of the G's I picked here was sigma so it's this action of sigma on one the action of sigma on two the action of sigma on three and that's what it did to this side but I'm showing this sigma the other way around now and what we could see here is that really this was a bijection this was a bijection because I have all the elements there and I have all the elements there they are not repeated this is a one-to-one and onto mapping and really so it for our example here at work but it would be much more powerful if I can show in general that this is a bijection because if it is a bijection I'm really dealing here with a permutation because every time I take a different one of these one of these and it acts on there you know I'm going to get this or this is equal to this action on this side so really if this is if this is a bijection this is a biject function then I am really you know representing each of these with a permutation and that is the power of group actions here and I mentioned to you very quickly we use proofs from from set theory we're going to just show that this is actually a two-sided inverse those functions are two-sided inverse and I just want to give you some intuition behind those proofs as well so to show that it's a if it is a two-sided inverse it is a bijection so I'm going to use that form of a proof to show that this is a bijection and what you do is you say that if I have sigma of the inverse of one of the elements so whatever the inverse of that was with that on a that I get back to a if I have this composition of two of these and such that one time I use the the inverse in g and an element in g and it's inverse that I get back a so what I'm saying is I'm mapping the whole set a finally through the whole composite of two actions back to the whole a so I must get all of the elements a here take it through a composition of these actions and I must land up with all those elements once again and the other way around as well so there must be an inverse there so if I have this then the inverse I land up up back with with a again equal sign so I land up with a as as well now I can show you that this one is an injective mapping so imagine here I start with all the a's I start with all the a's and then so my first map is this one that's the sigma of g and then finally I have this one this one is the sigma of g inverse so there's my first map I do this one first and then the outside bit there and then this is the composite now the composite must be an injective must be injective because that's what it's saying there if it's can I prove that this is injective well let's assume to the contrary that it is not now remember what injective is injective means if I have lots of elements on this side each one of these on this side maps to something there's nothing left out here and they they will just map to one to a single element maybe I shouldn't put that in it let's put it in here so that would mean by to the contrary that I would have two elements mapping to the exactly to the same element and even if this part is then one to one so each one of these would map to a unique one there in so that's assuming to the contrary that this composition is not injective that means I cannot have by the composite all of the elements in here being represented in here because certainly it's only one of the two of these that is going to be represented there not both of them so there's going to be a is in here remember each of these represents a there's going to be some a is in here so you know to the contrary then I'm going to have these only that one is mapped and that one is mapped this one's not mapped at all so these two can definitely not equal each other this says that that it should be equal the composite means that all the elements a should eventually appear here but if I let this first step not be to the contrary not be injective so two would go to the same one all of these will not eventually make it to there so by assumption to the contrary proves that that this is injective that doesn't make it so objective specifically if you think about if my set here is infinite in size now let's do it this way so again I have the whole a the whole a the whole a and my first mapping here is now the g sigma g inverse my second mapping here is sigma of g and I'm suggesting that this the whole thing must be that the all of a is all of there but let's assume to the contrary that it is so this way around you look at to show that it is surjective that this is surjective now if I if I say to the contrary let's not make it surjective now it not being surjective means that there's elements here on the second side which don't get mapped to so there's a whole bunch here so there's a whole bunch here that I can just take away they are not being mapped to so to the contrary I assume that this bit is not surjective so there's some that are not mapped to well then you can think of the fact that if there's this one mapped to this one mapped to there through the composite here going through first through this one then through that one there are some here that are not going to land up over there in other words not all of these not all of these make it into there so it's not the whole a that I'm talking about so if I assume that it's not surjective then the composite doesn't work because there are going to be some in here which you know don't end up there so it has to be surjective so this thing a composition of this thing must be injective it must be surjective therefore it is a bijection so if we have a two-sided inverse and that's a two-sided inverse this must be and we saw a beautiful example here it of course it is a bijection because it is a bijection and that links that is the real link between each one of these being represented by a permutation of the set on which it is acting it's a permutation of the set you can see the beautiful example there you can get the intuition to the proof this two-sided inverse effect why it makes this thing a bijection so there's a one-to-one and onto mapping between these two so all of them are there all of them are there in that set and that just means it's just a scrambling of them all the time this just acts to scramble these up each one of these that one doesn't scramble it up that one scrambles those two and that one scrambles those two and that one scrambles these two et cetera they are just a permutation they represent permutations because this is a bijection so you can understand what we mean by the action of one of these on it is a it just scrambles them up that is the action of the elements in a group that make up a set on another set group actions permutations it's as simple as that


You are out of here. What's up, money geeks. Mr. V here. Welcome to another video. Today's video is a warning. Tax season is coming up. And so the IRS and the government are changing the rules. And if you are somebody that is an immigrant and you are on a visa or you have a green card, this is definitely for you. Or if you know somebody that is on a visa or has a green card, please definitely share this video with them because it's very important. And again, it's a financial video, but it impacts your status or impacts your state in the United States. So the IRS has changed the rules working with the government as to what are the violations that can get you deported. So in today's video, we're going to examine two key changes that if you miss them in 2020, if you don't report those changes properly, you are going to be kicked out of the country. Very, very simple. Two key changes. So the first change that you want to be aware of is when you file your taxes, if you're not a U.S. citizen, if you are a green card holder, if you are on any kind of visa, you definitely have to state that clearly on your tax return. Your status here in the United States, failing to report that status to the IRS and the government will get you deported out of the country. It's just that simple. So you have to be very careful. Make sure that you check the right box when you fill out that tax. And again, if you're not impacted by this, but you know people that can be impacted by this, please make sure that you show them. And one of the reasons why I'm really pushing this is because a lot of these people don't really even understand the tax system. They don't even understand how to file their taxes. So they'll either have maybe another relative that's an immigrant to help them file their taxes, or they'll go someplace and try to pay somebody to help them. But that person doesn't really understand that their status is different from any other person that's a citizen. So you need to be very, very careful. If you doubt, please ask questions. Reach out to people that can help you share videos like this so people can use it to actually learn and not get kicked out of the country. So again, like I said, this is a very, very important video from a financial standpoint which impacts your immigration status here in the United States. So point number two is that if you don't report all your income, again, if you make a mistake not reporting all your income, you are out of here. It's just that simple. The IRS will report you to immigration which they're going to track you down and give you a notice to leave the country because you didn't report all your income.  That is very risky. Report every dime that you make. Is it worth you getting deported out of the country to keep an extra $1,000 in your pocket here in the United States? I don't know about that. You can make that $1,000 even triple that amount if you just report it. So if it's even a gift, if somebody gives you some money, report that as income. If you work several jobs in Uber, if any work jobs that get paid you cash, report that income. Make sure that you capture everything so that it doesn't impact your status. So those are the two key changes that the government is actually implementing 2020 moving forward that if you fail to, it's going to get you kicked out of the country. Again, I'm going to repeat.  If you don't report all your income and they kind of audit you and find out that you didn't report all your income, that will get you kicked out of the country. So again, guys, share this video with people that would find it useful, people that can benefit from an information like this because you never know. Like I mentioned, the downside of this is that most of those people are going to be impacted. Don't even understand how to fill out the tax forms. They don't. So they are very, very vulnerable to making mistakes. So please, please share. I usually don't ask people to share my videos a lot, but this one is very, very important because we're talking about lives at stake here. There's people whose lives being here in the United States has been transformed and for them to get deported would not only impact them, but it might impact a whole generation, a whole family tree. So please definitely share. So question of the day, do you think that these new laws are crazy or you think that the government is out to get immigrants? Let me know in the comment section. Like, share, subscribe. And as always, stay motivated.


The sun has set. It is dark outside. Thank you to the panel, obviously this day, and to colleagues at the back who stayed. Thank you for seeing that. We're here to talk to you about our proposal, a Type 3 proposal, which is not very sampling entitled, I would have to say, enhancing the digital teaching capabilities of experienced online instructors and the digital learning capabilities of their students. I've listened to some of the other project teams that have their gorgeous acronyms, A-I-T-E-A, and I wish we had come up with something snappier. But this is coming from, I suppose, our experience of delivering a wholly online course. So, Trace and I have been involved in an online programme at Trinity, the postgraduate diploma in social policy and practice, which is delivered wholly online. It's a one-year postgraduate programme delivered, I suppose, really quite intensively over a year with six modules. We take two classes a year. This is one of those classes graduating. And I think that experience, which is part of Trinity's strategy to go online, part of the 2014 strategy to go online, has developed, I suppose, among our staff, a very intensive acquisition of knowledge and skill. Very much focused on, how do I get my materials online? How do I facilitate my students' engagement? And I think what we now have would be quite a technically skilled group of instructors who are also, I think, actually quite good at the design and development elements of the curriculum. We've been through that experience. We also have a very good set of established supports, I would say, in no small part to Trace, supporting students. This is part of a centralised support unit, where all of our students have access to that technical support. And that, I suppose, has been really where our focus has been, in getting online, going online, and requiring those technical skills. But I think it's fair to say that that's really only one part of the picture. And if I'm to look at the all aboard map of the digital skills, mapping those in our higher education, I'd have to say I think we've probably been quite focused on some of the tracks in this lovely map. Probably very focused on things like tools and technologies, certainly focused on creating and innovating. But one of the things I suppose now our experience of running that programme has brought us to is to realise that for us, and for our students, it's certainly a lot more than simply the technology. It's not just about design and development. And good design and development of digital resources does not make necessarily for a good, rich, vibrant, digital learning experience. And that's where we are now. We're realising, as opposed to this point in our experience, we have four cohorts of students graduating. We're now in our fifth year. But actually, really for us, there's a growing need, I think, for students and instructors to focus more on the, I suppose, the interaction within the classroom. What is going on within the classroom that is, I suppose, important. So that live part of the online course is something we're beginning to reflect on now. And we're reflecting on that, I suppose, guided by, despite how obvious it might seem, I think for a while we probably thought that online learning was simply replicating what you might do in a face-to-face environment. And the truth is, of course, that it's not, that it requires new ways of teaching and new ways of learning. And it's taken us a while, I suppose, to get to the point where we're beginning to ask, okay, well, what is it we need to be doing in the classroom that will stimulate that debate, that will get students to engage with each other? Now, I think that's probably of particular importance to us because of the program. The program is a program of social policy and practice. We have a lot of people coming to us from the world of social services, people who run services and manage them. We have people coming to us from the community and the voluntary sector who are advocating for social change, and who want an understanding of the bigger social policy picture to make that change happen. We also, actually, some years of people coming from the corporate sector who want to learn a little bit more about social things to fulfill their corporate social responsibility. What that means is that we have a melting pot of people from all kinds of different backgrounds. Some of our students are operating with English, certainly not as their first language. And yet, in order for them to achieve learning outcomes at level nine on a social policy program, they need to be able to discuss and debate with each other in a very robust way. So, how do we make that happen? How do we make sure that when the course goes live that the discussion forum, certainly, is live, interactive, vibrant? And how do we make sure that the tutorials we use? We have synchronous tutorials once a week. On a Thursday evening, the students would come together, enhanced with webcam and microphone, and we talk as a class with each other. We debate and we engage. So, in that environment, I suppose one of our experiences, and this is a very local project, arises from very local need, one of the things that we can see can happen is that when the classroom isn't vibrant, it is, I suppose, a natural instinct for instructors perhaps to move into didactic mode and start teaching and filling the space where maybe they're wanting discussion to happen. The same can happen, I suppose, in the reverse talking earlier with colleagues from DCU that sometimes in the synchronous tutorial, if a student does speak up and they start to dominate the discussion, you're almost so delighted that they're talking, you leave them at it, perhaps for longer than you should. So, trying to figure out, I suppose, what's the right balance? How to promote that engagement? How to make it safe for students to share with each other and often in social policy to argue with each other about what social policy might be the correct response to a social problem? We think we need some new ways of teaching and learning on this programme. We think we need to be spending a little more time thinking about the roles that teachers and learners play in the classroom. A little more time thinking about the behaviours, the teaching and learning behaviours. So, not just thinking about the material, not just thinking about the stuff, but thinking about how once you've made that available, how do you signal to people, how do you build the right kinds of behaviours, the right actions and the right attitudes in the class? So, what you're building is, I suppose, the competence but also the comfort and the confidence for that classroom to be vibrant. So, this is what we reckon our need is and we have an opportunity now because we've been running this course since 2014. We have four graduated cohorts of students and one currently in line actually having classes, synchronous tutorials this evening. And one of the things we'd like to do is we'd like to go and find out what those students thought about the experience they had online. We'd like to find out from their perspective what were the digital learning behaviours that they thought were good for them, that were good practice from them, what were the digital teaching practices that they liked. So, we've done variants of this in very small scale surveys before. So, we know for example that students really like it at the digital classroom when the instructor calls on somebody to contribute, but they don't like it when it's them that's called. So, we know therefore that there's something that works when you pull somebody out of the group, but there's some barrier, some attitude, resistance perhaps from the individual's perspective. We also know from having conducted, we've tried a few approaches for example in the discussion forum. So, one year for example we decided to comment on every single post on the discussion forum. Every single thing that was posted to see what that would do. And not only was that a fairly significant endeavour from a time perspective, we could also see that actually it encouraged a very unilateral conversation between the instructor and the students. And that actually when a course goes live, we need to step out of it a little to allow the students to connect with each other. But some of this learning to be honest is a little ad hoc. Some of it comes from, you know, what we have done of some of our annual evaluations of materials. Some of it comes from discussions between the instructors themselves. What we'd like to do is to really get a good grasp now on the student opinions of digital learning, digital teaching, their experience, what was good and what they think is needed. We'd like to do a survey with the students. We already have a survey planned for this year, a destination survey. We'd like to add in some digital teaching and digital learning questions into it. We'd like the students to respond to that survey to indicate whether they're willing to participate in a focus group. And that's something that we will conduct online using the same methodology, the same technology that we use generally to support the course itself. What we hope we will get out of that, as well as with some interviews with our instructors, we want to develop some additional artefacts, some too new teaching and learning sessions. We already have quite a good suite of resources for our students and for our lecturers, but we think it needs to be enhanced. And we think with some online resources that we can add into the induction sessions that we can support a shared understanding between lecturers and between the students of the kinds of roles that people might be playing in a digital environment and the kinds of behaviours, actions and attitudes that I suppose foster a good rich environment. We think that this will allow for deeper learning experience for our students. And we think it will capture something that's happening on quite an ad hoc basis at the moment. A lot of our instructors, we talk regularly about what's the best way to get a classroom going, perhaps when it has slowed down. We think we could do with formalising some of that and certainly locating it within an evidence base. Now, if we can get the student voice activated one of the potentials of course is for us to use it even more. And this is based on the comments from the panel. We think that there's the opportunity certainly for us in engaging with students in this way to get them to reflect a little on the meta skills of learning digitally. For them to think about how they're learning and not just as many. Most of them are busy people. They all have jobs many of them have families and caring responsibilities to zoom out a little and to think about those meta skills. We also think that this would be a way for us to get some qualitative input into an evaluation of the curriculum going forward. So we evaluate the curriculum at the moment in a fairly standard way using evaluations, survey based. It would be really good if we could get a solid group of students who are willing to give us feedback on our curriculum on an ongoing basis. So if this was to work for this project, it would be something that we could hope on and use for further improvements to what we do. So how do we plan to go about doing this? This is the idea we have in mind. So we'd like to obviously have a look at the evidence base. We know there's some evidence there. We haven't been looking at it in a very concrete way or a very focused way to look at it. Particularly to look at what's out there with regard to the roles that people can play in an online environment. So we can see for example that some of the students very clearly play a supporting role for other students. So very often they'll help students find where resources might be online. That's a role that they're playing. Sometimes they're playing a display role where they're letting you know what they know and their post is very much a performance rather than in any way engaging with anybody else in their class. So we'd like to have a look at the evidence base. Then we'd like to try and capture the student voice through a survey distributed to our five cohorts of students. That's about 200 students since we've started. And then from that to work further with those students in focus groups. We also want to use our instructor group. We're fortunate we have a mainly full time instructor base in Trinity. This program is staffed with full time lecturers. We meet each other regularly as a course committee to use those inputs in group and also in individual formats. And then to develop as I say these two artifacts. An online session in digital learning and one on digital teaching. This is the outline of the project plan. Over the year we really would like to get these materials ready for the next start of the program which would be here in September. So that means we would be starting the secondary research phase there in January and starting primary research towards the middle of February. Now we've already received medical approval for a destination survey so we've a little bit of work done there already. So we would be able I think to go back and look for some revised ethical approval in order to add these additional elements to the project. Then after we have engaged in the survey and the focus groups with students the interviews with lecturers to analyze what's coming out. You know what are people who have been doing this kind of learning in a wholly digital environment. What are they saying are some of the things that work for them. The behaviors and the actions that they associate with a good learning experience. Then we look to develop these materials and I'll talk about the team in a moment to develop those materials for online implementation and then what we'd like to do is to test those materials. We always have a testing phase anyway. We'd like to use our student focus group to give us some feedback on some of the elements of our program. The team basically is a team that works together already quite a bit. The staff and Theresa and our colleagues in Trinity Online Services work together on a weekly basis to deliver online sessions for this program and for others. We would also see a role for research assistant here to assist us with primary collection and also we have a predefined group of students and instructors that we will be able and hope that we would be able to access. I think one of the things that this would do would be it would add to the existing digital skill resources that we certainly have available to us. It would supplement them and complement them. It's something that I think when it's in online format as I think many of the groups that I hear today have said it facilitates that national sharing. Definitely for those of us involved in teaching this supports your professional development I think and it's a way of researching what you're teaching and then teaching what you've researched is that lovely virtuous cycle. Putting the students at the centre of that evaluation of what works. We think for students and also for instructors it will facilitate that meta-analysis and really ultimately though we'll know if it's working. I mean that sustainability in terms of impact will we see in our classrooms more competence in terms of participation in class. Will we see greater confidence and comfort and I think then we'll see whether it has facilitated a vibrant digital classroom. That's our proposal. Thank you very much. I'm going to stop now and take questions from the panel.