From such a long time for around two years now, you can say that the love and the entire, the way you have connected with us, it gives us the zeal to keep on moving. And as they say that everything has to change for the betterment and we all did sessions originally in English and thereafter with the feedback when you did few sessions in Hindi itself. And now we have also started a series wherein we take sessions which is a combination of English and Hindi. We made a request to Justice Kuldeep Singh, who is a former judge from Punjab and High Court, that the sessions were a combination of Hindi and English so that people can understand, especially I would say who have been deprived of the education of the best of the cities but after the pandemic, we have also seen that the best of the speakers and best of the knowledge can be shared on the YouTube and different social medias. While taking this endeavour forward, today's session is about the principles of natural justice under the administrative law. In principles of natural justice, there are two basic principles which have been called out. One is that nobody can be judged in its own case and second is nobody can be condemned unheard. The different facets and these two aspects have been given different contours under the law, but the bottom line continues to remain the same. Though Pramab Shah appears that these two principles have been day in and day out being given in the various judgments, yet we find that the administrative authorities, quasi-judicial authorities, sometimes in their orders, past orders which are non-speaking, cryptic, and these two cardinal principles have been violated. But understanding this for common people, for lawyers, for students, and for quasi-judicial authorities which have been called out. To understand these things, today we have Justice Kuldeep Singh, who has done several sessions before. And the last session which he did, which is the Hindi-English combination, it is doing tremendously well. And since it is late hours, I'll request Justice Kuldeep Singh, a former judge of Punjab and Aykut, to share his knowledge. Thank you Vikas. These days, we work with authorities in some places. We go there. So the job of the authorities is that they exist for the public. The administrative authorities that have been created have been created to give justice to the people. That is why it is important that the faith of the people in the authorities is increased. That is why it is important that on the administrative side, one person's problem should be heard completely. And after hearing it, make a decision that there is no scope for error. Sometimes there are two parties that should be heard. Today, in our democracy, the right of hearing is very important. Take our constitution. Article 311, which is the removal of public servants, has been given a procedure on how to inquire, how to give it a hearing, how to do it. Then Article 14-21 has been emphasized by the principle of natural justice. It has been imposed on it, such as the right of equality, your independence, your personal liberty, your trade liberty, your religion and so on. So in today's age, when people are very conscious of their rights, the importance of principle of natural justice increases so that in the democratic process, people can make more faith and study further. The principle of natural justice basically, it is a fair exercise of power by the administrative authorities. The administrative authorities had to adopt a fair procedure and do it. Today, the number of administrative authorities has increased a lot. Their duties have become very different. Sometimes, it happens in law that a procedure is given in its rules that you have to follow this procedure. That is the complying with the principle of natural justice. But it is not possible that every work should be made into rules and given a procedure. So where the procedure is given, you have to follow it. If it is not possible or not given, then the principle of natural justice comes. You have to do it again so that people can get justice. That is why you have to follow the principle of natural justice. As I told you, people's faith increases by following it. Sometimes, big mistakes happen because of not listening. So that can be avoided. When you listen to someone, the first party tells you about it. You have seen it in some files, it is not in the file. If you listen to the second party, you get an idea of where it is. So you can reach a rational decision. Otherwise, it becomes a blunder. Let me give you an example. There was one case where there was a crack. There was a conviction in the corruption case. It was less than three years ago. The court bailed it the same day. So he quietly came and joined the duty. The next day he came to the duty. As the procedure goes on, when the judge's copy is done, or the prosecutor is sent, it takes time. He went to a branch and kept it there. So that person continued to work. He filed an appeal and his sentence was suspended. He continued to work. During the appeal pending in the High Court, he died because of his natural causes. When he died, his family gave him a pension. He told the department to give him a pension. When he started giving him a pension, he came to the judgment that this person had been convicted. The government's instruction was that if someone is convicted, he should be dismissed from service. Article 311 says so. If the appeal is in his eyes, he should be reinstated. What did the concerned authority do? It was a diplomatic mission in that case. He didn't listen to the party. He said, we had to do this. He was left with us. He dismissed the debtor who was there to hide the mistake. Now the order has been challenged. So you know what action you can take against a dead person. You can't dismiss him from service. Obviously, he is not alive. Now the question is, is the order of the Deputy Commissioner valid? So I asked him, did you dismiss him? Can you dismiss a dead person? He said, I can't. I said, how did you do it? I said, I had to do it. I said, I could have dismissed him. When he was convicted, you didn't do it. He kept on working. How can you do it? The problem was that before he passed the order, he didn't listen to his legal absence. When he was listening, he was dead, how can we do it? So maybe in his mind, we can't take a department election against a dead person. So you don't listen to such a blunder when you just look at the file, it happens many times. So he has to do it. Now the principle of natural justice, it is applicable to the judicial authorities. Everybody knows that the judicial authorities have to listen to both the first party and the second party. So it is applicable to the city. There are its rules and laws. Second, there are quasi-judicial authorities. Those quasi-judicial authorities are applicable to them as well. Now there was a gray area in this, what is the principle of natural justice applied to the administrative authorities? So first I had a little doubt about this in the courts that administrative action is not applicable to it. But now that line is very clear now. That administrative orders and administrative authorities are also the principle of natural justice. Suppose the administrative authorities have to pass an order, like in the UNISTIC exam, someone was caught copying unfair means. Now the question is, if there is a disability, if they have to take action, they have to cancel the exam, they have to do the debar. So in that, the principle of natural justice has been applied, you will have to listen to it. Administrative authorities also have to listen to it. For example, in the election, if you have to cancel the election, if you find a material, you have to listen to it. This is an administrative order, of the election commission, but you have to listen to it. For example, the municipal committee wants to supersede the authorities, there are allegations against it. So they have to listen to it, that this is something against you, and you have to supersede it. And many times it happens that there is a dangerous structure, if you want to destroy it, you have to hear the other party and you have to listen to it. In some cases, the administrative action on which the principle of natural justice is applied, is applied to everyone. So the courts had a very famous case, Menka Gandhi versus Union of India, Mahindra Singh Gill versus Chief Election Commission, they examined everything. In that, the Supreme Court had ruled that the civil consequences of the action, in which you had to follow the principle of natural justice. What are the civil consequences? Pecuniary, you may have a penalty, you may have a loss, you may have a property effect, you may have a personal liberty, you may have a personal loss, you may have a civil consequences, you may have to follow the civil consequences, the administrative authorities will have to follow the principles of natural justice. The principle of natural justice is not a fixed concept. It is not a fixed concept. Time to time changes. Whether it was in 1900 or 1950, or 1970, it is not necessary that it is the same today. Our society is developing our society is developing and where the principles of natural justice were not followed, we have to follow them. So it is a flexible concept, it keeps changing with time. The principle of natural justice is that nobody can be judge of his own cause. Nobody can decide their own problem in which they are involved. The other thing is that justice is not done. But it appears to have been done. Justice is not done but people also feel that justice is being done or the party felt that justice was done. For example, if you want to take action, the civil consequences of which you first notice the party, call them, listen to them. After listening to them, if there is no other party, if they order, if they give reasoning, then it seems to have been heard. If the authority is involved, if it has personal interest, then they cannot be judge of their own cause. Suppose there is a selection for a higher post, a selection committee is formed and one of the candidates himself is a member of that committee and he is deciding that he has to be promoted. In court, you cannot be judge of your own cause. You are involved in that and you don't defend yourself. The maximum is called the party, the party here. Now, the rules of hearing are that you have to hear the opposite party. The first thing is that the authority has to issue notice to the affected party. So, what should be the notice? This is also that you issue a notice and if you have a problem, then it is very uncertain. So, the notice should be an adequate notice. You give it enough time for 2-4 days. So, give it enough time for 2-4 days and then what kind of nature of hearing is there. Then the authority will tell you which authority is there and what is its jurisdiction. And the material you have should also be given to the other party. So, after giving the notice, the opportunity of hearing should be a reasonable opportunity of hearing. It is not that you should not let me see or hear you. It is over. You have given a notice of 24 hours. So, it is not like that.  It should not be in haste. Now, the hearing can be your oral if you call the party and after that it is necessary for hearing that the material you have should be given to the party which is relevant material. Then whatever the party's effects should be disclosed. I should emphasize that this session is a mix of Hindi and English in hybrid form for the benefit of all the viewers. So, whatever effects should be disclosed whatever evidence should be disclosed and the material should be given to the party. Then the party will answer that. Then after that it should be given a chance to rebut it that your first party is doing such a thing that it has such documents.  So, the other party listens to it. Both parties have the right that they should know the evidence which is being used against them. So, after that if the party is being elitrated then you have to see if the party is being properly explained or given or if it is being given legal representative's permission or if you are taking your assistance from someone. Sometimes there are technical matters where you have to give assistance. If you are being trained then you can give help to the liar. So, sometimes the rules are provided that how to give legal help. So, then you have to follow the rules. Then after giving the party an opportunity your decision should not be in haste. There is no decision in haste. There is a mistake. Sometimes the party makes a big emotional appeal to the administrative authority that I am poor but they are making mistakes. So, they have an art and a retreat. So, they make you emotional. You make emotional decisions and sometimes you make mistakes. So, my suggestion is that when such a thing happens that the party makes an emotional appeal or you think that you are emotionally involved then you should give it to them for a few days. One or two days if you do more work then the administrative authority will put in more cases then the reasoning mind will start working then your decision will be made. Now, the decision to give the administrative authority that should be reasoned speaking order. What is the speaking order? Speaking order for administrative authority is that you have to give sufficient reasons. Judicial authority does not have to give detailed reasoning. Maybe it can be brief but not sketchy order. In that you have to give brief reasoning that it is like this and like this. And the matter of the appeal that you are doing then you have to give the grounds that you are doing. Now, the appellate authority does not have to give brief reasoning or detailed reasoning. Now, many times you have to take a decision due to public policy. In the case of Main Ka Gandhi you can also do post-decision hearing or even after taking a decision. For example, in the case of Main Ka Gandhi his unit of India has revoked his passport in the interest of what we are doing. So, the Supreme Court has never said that you cannot do this. If you have passed the order then now listen to this and pass the fresh order. Personally, I think if we examine the academic side then there is no post-decision hearing you are revoking his passport. So, the petition is not so dangerous that he will do something like the Raj Paltak then you have to give the hearing first. It is just an emergency case. Public policy case. If there is a smuggler, he is running. You can revoke his passport and then listen. There is an element that is going to crime on the foreign soil you can do this. But not the ordinary citizens not in that. So, on the academic side it is that in which you feel that there is no danger to the public peace or safety you should give the hearing first. Now, the next question is how can biases be? How can there be bias? So, how many types of bias do you have? So, the first is pecuniary bias that you have financial interest suppose you share the company so, your property is involved so, your property is involved so, in which your pecuniary interest is involved the administrative authority has a personal bias the principle of natural just requires that the authority should not hear that matter or decide that matter. Next kind of bias is personal bias suppose your relatives your friends your business associates they have a matter or someone with enmity or grievance or professional rivalry you have a matter of that matter so, your personal bias is involved in the favour of your friend or your rival or an MP so, business associates so, the authority should not hear it is against the principle of natural justice in which your interest matters should not hear and decide now, next kind of bias is subject matter bias or your view on subject matter your view on subject matter in that normally speaking in that the principle of natural justice does not apply it is a very rare case that replies suppose the subject we take about reservation your personal view your personal view your personal view doubt looks your decision effect but you have to listen to both parties you have to see the law so, your personal view is not necessary to overshadow that matter so, subject matter about bias normally speaking the principle of natural justice does not apply next kind of bias is departmental bias departmental bias is a bias like a secretary he has to nationalize him he has to make a police he has to take his own objection listen to the parties and decide but he is making a police he is doing his own objection so, his bias is a bias that I have to uphold the decision so, in that matter he cannot listen to that matter that is illegal by the people of natural justice so, in such a matter when the order makes a concern that is a question now, sometimes a next kind of bias can be a policy bias suppose the government made a policy take the nationalization policy that you have to nationalize the transport the government has already declared that we are going to nationalize made a committee put a policy bias and put other members so, today you have to listen to them and decide they listened they made a policy bias and they reported the government by doing an arrest so, in this the police has made and ultimately the government is the one deciding so, in the fact it does not make a policy bias that he is involved in the policy, he has made a policy government he has made political bosses so, in those cases the principle of natural justice does not apply now, the question is that yes sir yes sir, you can start have you done? yes so, now the question is what administrative authorities in every matter the principle of natural justice is important to follow do you have any exceptions yes, there are some exceptions if you start doing this work in every work if you call him and do this then sometimes there are some such work in which his purpose will be defeated or there are some such matters in which there is no need so, first of all in the statute there is a provision that there is no need to listen to it like on attaining certain age you have to do an assessment of an employee whether he is to be retained in service or not and by looking at the record you have to decide about the compulsory retirement that you need not hear that concerned employee you can also decide that you have to give an extension or not because it is in the statute the second conclusion is legislative function suppose the legislative unit the legislature has to pass the bill the tax has to be paid the price has to be fixed the direction of the general character has to be given so, there is no need to call every person and start doing this work if this happens then they will not be able to work so, the legislative function there is no need to listen to it like the municipal corporation also has some legislative function which is the administrative character so, if they are doing some kind of tax the municipal act if there is no provision then you do not need to listen to it after that comes emergency actions there are many things that are of the emergent nature so, in that you do not need to listen to the other party if you understand it correctly then you can listen later there are examples that there is a dangerous building that is going to fall there are human lives at stake in that you can drop the building before listening to it and to save the lives of the people you can listen to it later you can listen to it later you have to wound up the banking companies to protect their depository interest there are many times that a dangerous person has to maintain public peace he has to detain it so, sometimes it is so dangerous that if you listen to it then you will not get it so, that is important there is a dangerous example that there is a dangerous commodity that you have to destroy so that the public's health is affected you can do it without hearing if it is important then you can listen later then comes the public interest if it is in the public interest that you have to do the work it is in the public interest that its hearing is not important then you can dismiss it there is an example there is a public safety public health is involved public morality is involved sometimes it is like a fire to save a fire from spreading it is important to drop the building if you do the work of giving notice to it then it will cause some damage because the public interest requires it sometimes there is an animal life that is dangerous animal that is killing the public then you have to take it then there is no need to listen to the public sometimes there is a poisonous food then there is no need to destroy it next comes the exclusion is the ability of impractic because sometimes impractic means that you cannot hear like the number of people if you want to hear their number it is not possible to call and hear like an example you are copying a mask in a centre there are so many students you cannot call and hear you have to cancel the exam so if you want to hear it then without hearing those it is not possible to cancel the exam sometimes if you want to do academic evaluation then you will not need hearing you have seen the academic record then you have to do the evaluation indeed not here next comes the authorities to take internal disciplinary action if you want to take pre-disciplinary action then you have to suspend it transfer it then there is no need to hear it next comes the contract if you have to pass an order if it is not provided in rules then there is no need to hear it sometimes the work of hearing that is useless formality the other party knows the case is very clear an example is the teacher took a leave to fill an M and took admission in Ph.D now the authorities know they have to take action about it now the teacher's stand is very clear that I have taken admission in Ph.D if you ask then he will tell that I have taken admission in Ph.D but he took permission to fill an M to take an administrative action in fact there is no useless formality to hear it so these are the exclusions to hear it now comes if the principle of natural justice is not followed then what is the nature of that order whether it is a wide order it is a viable order or a wide order now a wide order which you can provide if someone will do a challenge then it is illegal a wide order is shown that it is illegal from the beginning so about this the courts in different cases examined it one case state of Kerala versus M Kuni, Kanan Nambir in that case if you have not done natural justice follow then that is your order it is a wide order it is illegal from the beginning as if you have taken action without hearing without following procedure it is illegal from the beginning in another case a journalist's passport was impounded without hearing him he says he does not follow anyone in public police if you have done it then it is a wide order and that was a question as I told you earlier as you go to the doctor if the doctor listens to you patiently you say I have a sore throat I have a flu so the doctor tells you if you have a flu, take this medicine if you ask the patient what is your symptoms you have a headache you have a sore throat if the patient gives you hearing you feel that you have come to the right person he will treat you you have confidence antibodies start forming I will be fine the other is that this is not a simple flu these are the symptoms of corona I do not have to give a simple flu medicine I have to treat corona and take the precautions the double benefit is that because of the patient's hearing he has become a confidence in the doctor and the doctor he has been able to treat and heal his patient that is why it is important that the principle of natural justice is followed now of late this tendency has become that I have the power to order whatever you want if you challenge in the court then whatever will happen will take time so that arbitraryness has come in the authorities it is necessary to check the public's confidence in the administration and in the democratic system if the administrative authority goes with someone he will challenge the person but his faith in the administrative authorities will end and then he starts using extra legal methods to do this so it is very important for a healthy democratic system the principle of natural justice has all the authorities to follow that was all on the principle of natural justice over to Vikas let us see what Mr. Vikas want to add or if there are any questions that can be answered sir by chance his name is Vikas he says how expediency and interest of justice is decided as normally courts are reluctant to pass an order under a CRPC section 340 he gives power to the court if there is a crime in the court courts are reluctant of course I agree with that I have worked in the lower courts I was not reluctant I used to do that why they are reluctant they have so many cases that they are unable to handle those cases there is a lot of rush so it is a new case they are afraid of adding secondly after the inquiry after the inquiry after the court if the court complains then the court will become a complainant judge says why should I complain then the parties will ask the question will cross me that is why they add I think that fear should be dispelled your duty you are a crime if you do not prevent it then people will get hurt I once said that there was a will in the court so I did the inquiry very senior council appeared he said leave it you will complain in 340 you will become a complainant then I will ask you a simple question then it will be very uncomfortable I said I am not afraid of the questions but I will tell you that you cannot ask me the reason is that I am citing you as a witness this is not your concern you will also be a witness because the drafting you were the counsel you will prove that you will not be able to do it then he was taken aback once they have committed how much courage in the court they have done this they will have to face the consequences so I filed the complaint but judges by and large tell you the reason they are afraid of being a complainant they are afraid of their work that is why they are a little reluctant and this the high court of the administrator it is their duty to put the judges in their minds that you do not need to be afraid it is part of your duties if the case increases if you do not increase then they will have to file another file if you stop them then the case will not increase then they will be afraid of the party and the other person who is going to do the crime at least the person who is going to do the crime will stop and they will not do the wrong case what happened to me that there was a case I said no we do not want to run it was wrong so they will do the same and they will not be able to do such a case so it has a salutary effect but the judges need to be sensitized about this and this was our effort to speak in Hindi in the time of elections as we can say people like Rujan and I am seeing for both of us the communication skills being in the so much we are customized towards English it looks as if it is answered I interrupted you I had second language in Hindi I had only studied for 2 years so what I have learned in Hindi is that I have learned from my friend the circle my friend was from the university he used to speak Hindi so I had to speak with him in Hindi so I have learned from him that in Hindi like I can speak well that is the reason I have my friends because of them I have learned from them one more reason so it is better so thank you sir for sharing your knowledge and tomorrow's session would be on overview of the schedule cast and schedule tribes prevention of atrocities act 1989 by Mr. Isha Somasekar a retired principal judge from Bangalore and we all know that his sessions are doing extremely well do stick connected with us tomorrow at 6.30 thank you everybody and thank you for encouraging the 9 o'clock session as we say in the TRPs that is prime time we never expected that people would leave that prime time and join these sessions thank you everyone and we hope that the cases of COVID are getting less and people return to normal and those who didn't get the injection they return to normal and keep the mask on as long as the government is doing good but as a citizen we will also be over responsible and thank you sir thank you everybody thank you


It's Zadonna. So I wanted to talk to you guys today about Scholar PA and it is as illustrious as it sounds, I guess you could say. If you've never heard of Scholar PA, this video is for you. So let's get into it right now. So Scholar PA is a program that is based and created from RossReview. So I know you guys have heard me talk about RossReview on a consistent basis and it's just because I really love the company. And so when they have new products and things that I think are going to be beneficial, I want to tell you guys about it. I want to tell everybody about it. And so that is why I'm talking about Scholar PA today. So Scholar PA as I stated is a program that RossReview created that allows you to have 25 new Cubank questions every month. There's a subscription-based program. You do have to subscribe to it, but it is beneficial for not only like PAs that are practicing and trying to kind of just re-up on their knowledge, but you know PA students as well. And especially I would say like this is something that is beneficial for the new grad PA. This program will help you kind of to stay up-to-date on all of the information and the different types of disease processes that you'll see in healthcare.  And so having 25 new Cubank questions every month sent to you is actually a benefit. I wanted to read some of the other things that comes along with it because you're in luck. Like right now it's at a discounted rate and so that is a benefit to you to get it while it's hot and discounted right now, okay? So right now it is currently $169. It's typically $209 for the year. Like I said, you get 25 new questions and you're able to stay up-to-date with things like EKG ECG interpretation, which is actually something that you should really be very aware of because even if you're like, for me, I'm in, I do trauma and OB and there are times when like my patients like tachycardic and all these things I'm like, oh my gosh, like I need to get EKG, right? I need to get a 12 lead and I need to be able to interpret it myself. I can't just rely on like the little readout at the top. So this is actually a benefit. One of the really cool things that they do is give you 50 AMA Category 1 CME credits. Now that is amazing because you actually, that's like half of what you need to stay credentialed as a PA. So if you didn't know that a little bit of information right there for you, that one was free, okay you guys? But this will give you half of what you need because you need 100 Category 1 and Category 2 in total, so 50 and 50. So just as a heads up, okay? So that's really good. It has this thing where you can try it like essentially for free because you know if you don't like it, you can cancel it at any time and that's a thing. And then it will give you these uncommon delights. So let me read what uncommon delights are. All packages include questions and explanations, which is what I love about Rosh review. It includes the visual learning.  And then it gives you your personal analytics with respect to like how you're answering the questions, which is actually really nice. You do have like a great support system from the actual team members and there are educators on call. So if you have like questions about content and stuff, you can do that. And then there is a built in search option. So this is a very nice program, very good program for you to be a continuing learner like we all are when we joined the healthcare field. So definitely take advantage of the low prices right now because it's only going to be a benefit. All right. If you guys have any other questions for me, leave them in the comment section below. Don't forget to like and subscribe and be sure to follow Rosh review on Instagram at Rosh review and me on Instagram at edadamapie and at Instagram on Get That See University. Thank you guys so much for watching. I will talk to you guys next time.


All right, well, thanks for joining us for Escape the Data Dungeon, Unlock Scalable R Analytics and Machine Learning. And also thanks to the R Consortium for hosting this R database webinar. Let's first introduce ourselves. I'm Mark Hornick with Oracle Machine Learning Product Management. I've been involved with the R Consortium since its founding and with R more generally for around 15 years. I focused on in database machine learning since Oracle's acquisition of the company Thinking Machines in 1999. And more recently, also on Oracle Autonomous Database AI Features. Sherry? I'm Sherry LaMonica, a consulting member of technical staff in the Oracle Machine Learning product team. I work with internal and external customers and leveraging Oracle Machine Learning technologies. I've been working with R for 20 years and with Oracle Machine Learning for R since its beginning. All right. So Oracle is a supporter of the R community for over a decade now. And the R Consortium since its founding. To help R users work more seamlessly with their database data, we introduced Oracle Machine Learning for R in 2011. Then it was called Oracle R Enterprise. And with OML for R 2.0, Oracle continues to enhance the ability of R users to take advantage of powerful database capabilities, both with Oracle Database and Autonomous Database. We also maintain the R Oracle Database connectivity package and provide a redistribution of R for use with Oracle Database. Now today, we're going to start with the characterization of the data dungeon. Rather amusing analogy, but one that sometimes feels quite appropriate. We'll touch on the benefits of databases as a focal point for data access, organization, and analysis, as well as Autonomous Database, which eliminates so much of the overhead and complexity with using traditional database management systems. We'll then dive into a few use cases and demonstrations involving demand forecasting, customer churn, and product bundling before wrapping up. So on to the data dungeon. Now you have lots of amazing data, right? But it may be all over the place. Some of it's in spreadsheets, CSV files, departmental databases, cloud storage repositories, and even individual laptops. It's a challenge when there are so many data silos. You may feel as though your data is trapped in some sort of dungeon, requiring a lot of effort to figure out what you have and where you have it. If the data is large, moving that data from one place to another can become expensive, both in terms of time and money, feeling like your data is somehow locked up. Duplicating database data in local environments can also pose increased security risks. There are production deployment challenges for reliable data access. The need to integrate R engines with production applications and handling of more complex error situations, as well as overall production level scalability. Now, this is where databases and their data catalogs come in. They enable you to organize your data where it's easy to find, use, and make sense of, whether it's physically stored in the database or through external tables that reference data stored elsewhere. Combined with Oracle Machine Learning for R, you can more easily work with your database data with minimal data movement and easier solution deployment as we're going to see. And Autonomous Database offers those same benefits and more. So let's take a look at what that's all about. Now, while you're likely familiar with the Oracle database, you may be wondering, what's an autonomous database? Well, we're using the cloud to eliminate the complexity of data management while also providing a suite of integrated analytics tools, including support for R, all in the same platform. Traditionally, each database deployment was more or less unique where you needed to build, secure, repair, patch, and tune each database. And this was labor-intensive and generally not scalable. Now, Autonomous Database re-imagines Oracle Database for the cloud with automation of infrastructure as well as database and data center operations. And it takes care of database administration so you don't have to spend time and resources tuning your database, applying patches or updating software, among other database administrator tasks. So Autonomous Database helps you escape the data dungeon by allowing you to reach a wide range of data sources, data in multiple formats, generative AI, and other supporting tools. Now, of course, SQL is the predominant language used for accessing databases and manipulating data, but not everyone is a SQL expert. And using a native language interface like R makes working with database data that much more convenient. When you need data, you may end up importing data snapshots from Excel or CSV files. Some R users may have direct programmatic access to pull data from the database to the R client and push results back to the database. But this round trip can result in scalability issues, such as access latency and local memory limitations. Also, by definition, data snapshots are obsolete and often require refreshing. Data may have errors that require going back to the source for correction to re-retrieve that data. And these round trips, of course, take additional time. In addition, data privacy laws can require keeping data in secure systems or locations. So to help address these issues, our focus is on enabling in-database processing from R. And we're gonna highlight the use of OML for R from a few interfaces and how they can connect to Oracle Database and Autonomous Database. For R, we have the popular RStudio IDE and OML notebooks, which is built into Autonomous Database and supports R paragraphs. We can also use SQL Developer to invoke R from SQL. One of the deployment scenarios that we'll cover involves using REST endpoints for deployed models and invoking user-defined R functions in database spawned R engines. And we'll demonstrate that using Postman. Now, I suspect you're familiar with the typical machine learning workflow. We're gonna highlight aspects of each step, but on the deployment front, we'll highlight working with in-database models as well as native R models, illustrating these from R, SQL and REST.  Now, our first scenario involves aggregated call center interaction counts, with the goal of forecasting the volume of call center interactions or customer incidents. These incidents are categorized by the type of request like billing, coverage or policy question and the channel where they were received like chatbot, email or phone. Now, rather than build a single forecast model for all of this data, we wanna do this at a finer granularity. So we're going to use partitioned models to automate the building of a model on each partition of data, specified by one or more columns, like category and channel, but these are used and managed as a single top level model. Now, you can use partitioned models for classification, regression, clustering and other machine learning techniques as well as time series. In the demo notebook that we'll look at we'll also highlight the use of conda environments and that allows us to work with additional packages such as GG Plot 2 to customize the packages we want to work with. Now, the second scenario involves customer churn where we'll build in database models to predict likely churners. We'll see prediction details to understand why an individual prediction was made and deploy an in database model to OML services for access from REST endpoints. The second part of the churn example includes building a native R part model where we'll invoke it using what we call embedded R execution from R, SQL and REST. And the last example involves product bundling and recommendation. We have sales data on customer purchases. This will allow us to identify frequent item sets that can be used for candidate product bundles, product placement for online shopping or store layout as well as inventory management for co-occurring purchases. Then we'll use the association rules to rank products to recommend based on their support, confidence and lift. So let's move on to the demonstrations. Okay, so we're looking at the Oracle Machine Learning user interface on Autonomous Database and I'm just going to go into our notebooks listing. We're going to start, I'm going to log in, it logged me out automatically. And so we're going to go into the notebooks listing for demand forecasting. So this is our first notebook we're going to work with. Now I'm just going to run all of the paragraphs from the top and that's gonna start up the environment and continue to support all of the different paragraphs that we have here. To start off with though, for the initialization, we're going to look at the conda environment that we created that has a wide range of R packages in particular, the packages that we've added specifically that we wanted to use in this case, GG Plot 2. We see that here. We're then going to download this environment and activate it so we can use it in the notebook environment. And this allows those additional packages to be made available. We're then going to initialize ORE and the ORE D-Plyer packages by loading those. And then we're going to start off with accessing our data table. Now the table we're working with is called call data two. And using the ORE sync function, this gives us a database table proxy object that corresponds to an R data frame, but the data remains in the database. We see that its class is ORE frame, which is a subclass of the R data frame class and other proxy objects that we have available to us in the environment. Now, to give you a sense of how does a proxy object differ from a data frame, here we're looking at the structure of that. And one of the key things that we want to highlight is the data query. This corresponds to the query that one would use to access that table in the database. From the description, we see the columns and their types, as well as the table call data two that this corresponds to. Now, next we can view some of the records from call data two. Again, using the proxy object and the overloaded head function, we're retrieving some of those values and displaying them here in the table view of the notebook itself. Mark, what does Z.show do? Yeah, so Z.show is a function that's provided to allow us to take the results that we get from the R environment and map those into the notebook environment for displaying in this interface. Now, we also have other overloaded functions like distinct in this case for saying, well, what are the distinct categories and channels that we have? And next we're going to focus on ggplot2. So we'll load that library and we're going to pull the call data two data from the database into our memory and do a transformation on the date received column. Why do we need to pull the data to the client? Yeah, that's a great question. When we're using third party R packages, what we need to do is make the data available in R memory to use them. We don't change the underlying implementation of packages to work with database data. So if you're using a native package, that data has to be brought into R memory and that's where ORI pull allows us to very conveniently take that data from the database and bring it into an R data frame. Of course, you do have to be careful of volume of data that you're pulling because memory limitations certainly do apply. Okay, now using ggplot2, we're just going to do a few simple visualizations here just to illustrate that one can do that and we'll do a plot of the data using, in this case highlighting the number of calls received over time by the category and channel. Moving further to filter data, we're going to use the dplyr functionality again. We have incident categories, what we're going to get back and using this proxy object for call data two, we're going to filter group by summarize that data, essentially want to aggregate the counts for each category and associate those with the date. So when we build our time series model, the exponential smoothing model will have the data that we need. We're also going to do incidents all that does something similar but it also includes the category and channel and recall that we talked about the partitioned models. So when we build a partition model here, we'll have three partitions, one for each category. And in the case of the incidents all will have nine because there are three categories and three channels. We can also get a bar plot of incidents by day of week. So again, we're using dplyr on the incidents all and using a mutate, we're going to transform the date received by day and get the resulting plot here. Where is the computation occurring for the already dplyr functionality? That's part of the transparency layer that we have with OML for R. And so what that means is that these functions that are overloaded, they're actually translating those requests into corresponding SQL that is then executed in the database. So because we have incidents all here is a data frame proxy object, that is going to take the corresponding SQL for the mutate, the group by and the like and run that in the database and provide back to us another proxy object because we don't want to pull data to the client unless we absolutely have to to do certain analyses. Okay, so that brings us to modeling. And in this case, we want to build an in database exponential smoothing model using Holt Winters. And we're going to have a partition model here where we're going to build one sub model per category. How do we do that? Well, first we're going to delete any model that we have created previously and we have a number of settings that we can specify. Most notably, what's the prediction step? How many forecast periods do we want to have? And we're going to have four for that. The model is going to be Holt Winters. We have seasonality of 26. And this is where we're identifying the column that we want to partition the data on to form one sub model per category. And we're going to identify the model name here explicitly as a ESM incident forecast one. Now to invoke this model, we're using ORE ODM ESM to specify the formula, the data, which is a proxy object again, because this model build is going to occur in the database. So we just need a reference to the data that's there. And of course, the corresponding settings. Mark, what is the ODM in the model build function name and why are we getting a warning about the model build? Yeah, so the ODM is corresponding to the original name of the in database machine learning which was called Oracle Data Mining. And so we incorporated that into the names to highlight the fact that this is an in database algorithm that's being used. And in terms of the warning message, we see that it's saying, ORE does not manage this mining model's lifecycle. Well, that's because when we designed OML for R, we wanted it to mimic the behavior of a native R experience. So when you build models or you create data frames in R, those objects, when you terminate the R process, they disappear. There's nothing there unless you've explicitly saved them. And so we modeled that same behavior by saying if we don't specify a name for the model that uses a default name, that when you exit out of the R process, it's simply going to delete those in database models for you automatically. If you provide a name, then you assume the responsibility for explicitly deleting that model later. All right, so having built that model, we can look at the partitions from this and we see that we have the three sub models for billing issue, coverage question, and policy cancellation. And we could even build the second partition model both on category and channel. And the only thing that we need to change is the partition column name, in this case, category and channel. Again, we run that and we'll see that we have new partitions, nine and all, that have the corresponding category and channel. For those of you who have teams that are not just for R users, but also include SQL users, perhaps part of the IT organization or others, you also have access to this information through SQL tables. So in this case, we have the all mining models partitions and you can access this information not only from R, but also directly from SQL. Now, there are other aspects to the models that you might want to explore. What were the settings that were used to produce this model? There's the user mining model settings itself that we can create a proxy object for and explore the contents there. There are also other, what we call model detail views that allow us to get more insight as to what the in-database algorithm did when it produced this model. And in this case, we have global diagnostics and model quality metrics that are available per partition. So each of those partition models will have its own specific metrics and their corresponding values that you have access to. Now we can also list the model detail views that are available for the in-database model because there are several that are there. And this can be done through a select query asking from the user mining model views for the specific model that we have. In this case, the incident forecast one model and we see those that we could access at this point. The next thing is, since we built these forecast models, key is we want to see what are those specific forecasts. And for this, we're going to be using the specific model detail view to see what those forecasts are. We're going to work on the forecast proxy object and perform a mutate from D-plier, getting the date ID, the count, the forecast count and lower upper bound and arrange that sorting by partition and date ID. And so here we're seeing the result from that, but let's look at what the actual forecasts were. So here the count is NA because those are the four periods that we're predicting. And the forecast count, the upper and lower bounds. And if we wanted to do a quick visualization of the count and forecast count, we see that here. So at this point, I'm going to turn it over to Sherry and she's going to highlight how we can do very similar things through OML for R using a third party IDE, in this case, RStudio and leveraging the OML for R universal client. Sherry? Okay, so here is RStudio server and I'm going to connect to both my Oracle database and my Oracle autonomous database from here. It's called the universal client and it can connect to either one and you can work from this client. First, I'll load my packages. First, we'll create the Oracle database connection on-premises using my RQ user schema. And then Mark showed all of these commands run in the notebook and I'm just showing you that they can run from the client as well. So first I'll create a proxy object from the call data to table in my schema and then using the overloaded or a dplyr functionality I'm filtering my incidents by category. So I can see I get my category count and the date, the date received for each. And I can build an in database model from here. I'm first dropping the model if it already exists passing all of the model settings to a list and then using those settings to build the model using or a ODM ESM, the object that comes back is mod. And I can look at the partitions. Each partition for this model contains, each partition for this object contains its own model. So for the billing issue we can see that I've got the model call and the model settings. So we can disconnect from our Oracle database and create an autonomous database connection.  And then I'm gonna connect to the database using the database service level high. Sherry, could you explain what a wallet is? Yeah, the wallet is a, it's actually a file on this host, an encrypted file that is holding my OML user credentials. And so that's the reason that I don't actually need to pass my credentials in clear text here. I can just create the connection string that corresponds to the autonomous database service level that I want to use, in this case, high. Okay, so then I'm gonna create a proxy object for the ESM model view result, call that forecasts. And then I'm going to use ORI deep layer to filter the, to view the forecast for the billing issue category. And those are my results. And this is just to show you that, you know, you don't have to use OML notebooks, you can use a client. In this case, I'm using RStudio and you can connect to Oracle autonomous database or Oracle database on premises. Sherry, we have another question about if the ORI library is not available on CRAN, how do you get access to it? Yeah, and actually you can get it in the Oracle downloads, Oracle machine learning for our downloads. That will contain the script that you need to use to install OML for R upon Oracle database and the third party supporting packages. For earlier versions of the database, the script to actually install it is included with the database as well. So you'd wanna look at the installation instructions and make sure that you're following the correct instructions for your Oracle database version. Great, thanks Sherry. All right, so let's continue with the CRAN prediction example. So in this case, we're going to again, of course, import our libraries and do some data preparation. And in this, we're going to get a proxy object to the table that we're calling customer CRAN 45K, it has 45,000 rows in it. We're going to enable row indexing by assigning row names and then convert the churner column to a factor. Now, using standard R, we're going to generate the sampling indexes that we want to use and then create the train set and the test set using standard R syntax passing in the index that we created earlier. And you can see that the result of this, both of these objects are also proxy objects or reframe instances. And the training data has about 27,000 rows and test 18,000 rows. Now to highlight that, how does the in database table representation map to the R representation? I did a describe here using SQL because in the notebook environment, you can have paragraphs of all different types in the same notebook. You can have SQL, PL SQL, Python and R, of course, and markdown as well. And so here we're describing what this table looks like from the SQL perspective. And we can also describe this from the proxy object using the attribute here. And we see that customer ID is numeric and churn is a factor because we modified that up above and the remainder of the columns with their respective types. So moving on to modeling, we're going to build a random forest model. And how do we do that? Well, again, as we saw in the previous notebook, we can drop the model that we've created, RF churn model one. And we're going to use ORE ODM RF. Passing in the R formula is churn or is our target. We want to take out the customer ID. We're using our proxy object, customer churn 45K dot train, and that the model name is going to be RF churn model one. From there, we get the prediction. We're going to invoke the overload predict function, passing in our model proxy object because that's important to know. When we get finished building the model, it exists in our database schema as a first class database object. And we're actually being returned a proxy object to that in database model. And that's what we're supplying here in predict along with the proxy object for our database table result. And the supplemental columns is going to be is churner. Now, this is an important concept as well because when you have an R object, a data frame, the first row is always the first row. The second row is always the second row. However, when you're dealing with data in a database, the relational model is essentially unordered, right? There is no ordering unless you explicitly ask for that data to be ordered according to some key. And so rather than incur the overhead for that, we rely on this notion of having supplemental columns that we can then associate with the predictions that we get back in that same table. And we'll see a little bit about what that means shortly. But if we look at the proxy object itself, we have the call, of course, and the settings that were used to produce that model. And in some cases, these settings were automatically chosen by the in database algorithm because we certainly didn't specify all of those. Now, we can also build support factor machine models among others. And we're just highlighting two of the available models. So we'll again delete that model that we've perhaps built in the past. Using the ODMSVM function, we'll do basically the same thing we did before. One thing you'll notice is that the type is classification and that's because our SVM algorithm supports both classification and regression. We'll do the predict as before, but one additional item is to do the predict and ask for the top N attributes. And this allows us to return the most influential predictors that help us explain the prediction. And so we're gonna get those at the top two of those. In the case of the basic prediction result, we have is churner, the original, the actual target. And then we have the prediction that's being made for that. We have the probability of zero and the probability of one. So what is the positive case for is churner? Oh yeah, thanks, Jerry. So the positive case, of course, is going to be one in this case and the negative case being zero. But if we go further for the explanation of why given predictions were being made, we can also see that in this case, the first prediction, which is a negative outcome, that household size played a significant role. Or in the third one, that whether or not they purchased a certain application or didn't purchase it was influential and their education had a role in that as well. We'll look at this in another context shortly. If we wanted to evaluate the results of the model, we have the predictions that were made by the algorithm. And we can use the overloaded table function to compute an in database crosstab using proxy objects. So again, this computation is occurring in the database without having to move that data to the client. And here we see the resulting confusion matrix. There are other metrics, of course, that we want compute as well are a C curve, lift chart, probability densities, and these were all produced using R. Deployment, deployment is one of the key areas for how do we leverage R in enterprise applications? And if you have your data science team that is working from an R perspective, these in database models can then be used from SQL. So if you're handing these off to perhaps others in your IT environment or your application development team, they can leverage prediction operators that are available in the SQL language for the Oracle database and use the model objects that were created above because these are first class database objects, just like tables. And so we're going to get the prediction and the prediction details, which is going to, again, tell us what are the factors that most contributed to this prediction? So here we see that the first customer here is declared as is Turner, yes, and year's residence with the value of five and this corresponding weight is what most contributed to that prediction. Same thing with this product purchase, whether or not they did and if that the corresponding weight that that has as well. Now we can use this model, not only from SQL and R, but also from REST. And Sherry is going to show us how that works in Postman. Okay, so here is Postman. This is a REST client. The first thing that I need to do is to get an authentication token. And I do that by sending a post request to this token endpoint. I have saved my Oracle Machine Learning user credentials in this environment here in Postman. And that includes the OML URL, which I saved as a variable. And when I send that request, I will exchange my username and password for this access token, which will be good for an hour and it can be refreshed before it expires or you can get a new one after it expires. Now, when I deployed this model through the models interface in OML notebooks, I saved it with a model URI called RF churn model one. And if I wanna just take a look at the model deployment details, I send a get request to this model URI. And I can get information about this deployment I can see that I saved it as a version 1.0 that it's an in database model OML created by user OML user. There's a model ID associated with it and all of the model metadata. Now, to score a single record, I pass a single record, here's my record, to the scoring endpoint with the model URI. And I'm also just like Mark did requesting the top two most influential predictors. So I can see my scoring results, I get back and then here, household size had the most impact on this prediction followed by YBOX games. Can also score in mini batches of records. In this case, I have a two records that I'm passing to the scoring endpoint. And when I get my results back, I can see the results for both of those records. Sherry, how many records can we include in a mini batch? You can include up to 256 records in a mini batch. And after that, you would need to use our asynchronous APIs for batch scoring. Okay. And how did we get the model deployed to be able to use it as rest endpoints? Yeah, great question. So you can actually do that here using rest commands. But what I did was we have a one-click way to do this and I'm on the book. So let me log in. And then after I'm logged in, I'm going to go to the models interface. Now this models interface shows us all of the models that we have access to in our database. So I'm going to look at RF churn model one. Here it is. And here is how I deployed it. The deployments are in here. You can see that it's been deployed. So the way that I did that was I selected the model, hit deploy, I gave it a URI, which is the same as the model name, a version and a namespace if we'd like. And then whether I want to share this model with other users in my database. And now I get a message that the model's been deployed and I can go here and I can look at the model metadata. And I can also see the open API specification for this model. Again, you can also do this through rest endpoints if you want, but this is a simple one-click way to deploy your database model to the almost services model repository. Okay, back to you, Mark. All right, thanks Sherry. So we're going to continue on the churn prediction but we're going to look at using a native R model from our part. And so for data preparation for this, we're going to use a data set that has 4,500 rows in it. We're going to pull that data to the client. And then we're also going to pull a 45K data set to the clients as well. And you'll see how we use those going forward. Now building the R part model, this should be very familiar. If you're familiar with R, we load the R part package, we build the model and we can look at the results from that as well. But for deployment and to be able to use embedded R execution that I mentioned earlier, we're going to go through a few steps. And one is that we're going to save this model in the database R data store. And you can store any of your R objects in the database as opposed to having to store them in flat files and manage them separately. So in this case, we have the churn model data store and we see that that's reflected here. And also, if we look at the contents of the churn model data store, we see the R part mod R, it's the variable name that we had that it's an R part object and the corresponding size of that. Okay, so that's part one. Next to use embedded R execution, we're going to create a user defined R function that we'll use for scoring. We're going to invoke that user defined function locally to verify that it behaves as expected. We're going to save that UDF in the database R script repository. So we can actually store our function in the database. And then we're going to invoke that using two different functions. One is the RE table apply, which will pass in all of the data that we identify in our proxy object to that function. And also row apply, that's going to use data in chunks to that and potentially with multiple R engines that are supporting that. So in this case, we have the score data function that we're creating that takes in a data frame, our first argument that and DS name, which is the name of the data store where our model is stored. We're going to load the data store into memory. So that's going to take our R part model and bring that into our function so that we can use it. Our result is going to consist of a customer ID column and the is churner column because we want to do a cross tab after we get our prediction results say to do a confusion matrix. And then we get our predictions as well using the R part mod R model. The data that was passed in, which is an actual data frame at this point. And of course type is class. And if we look at the results that come back from this, okay, this is pretty much what we expect. From there, we're going to load our user defined function into the R script repository. So we're going to give it the name score data, just the same name as we have for our function, but you could use anything that you'd like there. And then we're going to list that from the script repository. And here you see the name, a score data and the corresponding script. Now to invoke this, we'll use first or a table apply. And you'll notice that this is the proxy object, customer churn 4500. So the data exists in the database. The function name is the name of that function that's in the script repository. And the DS name is where our model resides, the R part model. And we call that data store churn model. Now what happens when we invoke this is we're starting up an R engine that is going to process this request. It's going to load the data from the database into our memory, process that result and return that for us. And here we see that, you know, this is the prediction that we are expecting from that. Now we can do the same thing from ORI row apply. The main difference here, well one, we're going to use the 45K proxy object, our score data, the churn model data store, but we're specifying the number of rows that we want to be in each chunk. And so what this is going to end up doing is causing our function to be invoked five times. We've got 4500 rows and that we were requesting parallel of two, meaning that you should have two R engines that are supporting this. And so that's how we can get some parallelism out of that. And again, the results coming back from that. Can we use the scripts saved in the script repository outside of embedded execution? Yes, absolutely. So it's not only that you can store these scripts, these user defined functions in the script repository for invocation through the embedded R execution, but you may want to store other functions that are just available that you can conveniently load as opposed to having separate script files that exist elsewhere that you would have to manage. Okay, so the next thing is to use embedded R execution from SQL. And when you do that from autonomous database, you need to set an authentication token. Sherry showed how we did that from the REST interface a moment ago. But then once we get that token, we'll be able to use that in subsequent invocations. And this sequence here is going to take advantage of a function get token two. And we've already provided the URL username and password in a table, because we didn't want to display that in the open text here. And so we'll get back the token and then we're going to set that token. So we're ready to use that in other functions. And so the next thing is to invoke the RQ row of L2. Now this is the equivalent from the SQL perspective of the ORI row apply function that we showed a moment ago from R. And so you'll notice a few similarities. This is the input data. This is the table name now, customer churn 45K. We have the parameter list, the data store name churn model that we want this to be done in parallel. So we're going to set the parallel flag to true. The asynchronous flag is set to true because this is going to be an asynchronous invocation. Some of these operations can take a long time to process. So what you'd like to do is to set up the job, run it, and then check for the result being completed. And we're also going to use a service level of high to encourage parallel processing in multiple R engines. The output format will be JSON. The number of rows is again, 10,000. That's what we specified from the row apply function earlier. And the name of the function that we're invoking is score data. So after invoking this, we have a job ID. And this job ID is now going to allow us to find, when is this result complete? And what we will get back when it is complete is this URL that we can then process to retrieve the result. And here we see that we're opening the cursor, we're getting the job result itself, the customer ID is churner and prediction result and getting the first 10 rows of that. And that's what's coming back from our SQL invocation. But we can also do this from rest. And now Sherry is going to take us through that. Okay, thank you, Mark. So here I can score my data in asynchronous mode. I'm here are the parameters that I'm passing to the function. So you can see I'm passing that I want to run this in async mode with a timeout value of 300 seconds. You can actually, this is an optional parameter. You can say, if this runs for over 300 seconds, please time out the number of rows that I want to run at a time. So the row apply, the input data, customer churn 45K, the parameters that I'm passing to the script itself, the data store name, whether I want this to run in parallel and the service level, the autonomous database service level that I want to use. So Sherry, could you contrast the low and high service level for us? And what are the options that we have available there? Right, so autonomous database has service levels for parallelism and concurrency. And with low, you get the lowest amount of parallelism and concurrency. Medium is a moderate amount of parallelism and concurrency. And then high is the highest level of parallelism and concurrency. So this time I'm using the service level high, the default is low. We can see that we get HTTP status 201 back. That means that my job has been created, a job ID has been generated. And in the background here, I saved that job ID to a variable. So I can pull using that job ID, which is part of a URL, I can pull the job status, see if it's still running or if I'm actually given a HTTP status telling me to go fetch the result. And HTTP 302 indicates that my job result has been found so that I can send a GET request to the job URL result and get the return value for my script here. And it comes back in a JSON format and you can see that for this first customer here, customer ID, they are not a churner. And down below, you can see all the other customers and what most contributed to those predictions. Sherry, how can we get the curl command that's associated with this rest invocation? Right, so if you wanted to run these in curl, you will go to the code button and here's the curl. So you can see we're passing this token, okay, and our headers. And this is my OML URL, this is my job ID and I'm pulling the result location. So you can just copy this and paste it into your, you Mac terminal or the next terminal and run that curl. All right, so with association rules, let's go a bit further. Again, we're going to import our libraries as before. One thing that I'm going to highlight a little bit differently here than we did before is instead of doing a sync from a table or a view, we're going to specify a query. So this is going to get us a proxy object that corresponds to the result of this query. So we're going to start with a table, sales two and products two are the two tables. From the sales table, we want the customer ID, product ID, quantity sold and amount sold. With the products, we've got the product name and the product category as well. And so here we've got about 186,000 rows. We've got 72 rows from the products table and we just see a few rows coming back from each of those to highlight what content is there. But what we'd like to do is combine these two. So we want to join them, merge them using the overloaded merge function of OML for R, the transparency layer. And so we have product DF and sales DF and we're going to take the unique rows from that and we have a result of about 56,000 rows that we'll be working with. Looking at the sales trans cust, which is again a proxy object as a result of the join that we just did, we see we have the customer ID, the product name and the product category. And then the distribution, if we wanted to view that as well, we can use the overloaded table for a cross tab and look at the counts of each product category and use standard R for doing the result of that. Now with association rules, one of the settings that we can specify is the rule length. And so we're going to build two models, one with rule length two, one with rule length three. What that means is that in the case of two, we have one element in the antecedent, the if part of the rule and one in the consequent and with rule of length three, we've got two in the antecedent and one in the consequent. And so you can increase the rule length as you'd like depending on what type of analysis that you want to do. But in this case, we're providing to the association rules algorithm in the database, the fact that we have a data frame proxy object, the sales trans cust, we identify those special columns, the case ID, which is really the transaction ID, if you will. And also the item ID column, which in this case is product name. We can specify minimum support and confidence and of course the rule length. And in this case, the setting is the name of the model that we want to produce. And in each of these cases, we have the corresponding objects that are the model proxy objects because these are in database models that are produced. And we're able to manipulate them now and use them from R. So let's create a table with candidate bundles from item sets. One of the computed results from the association rules algorithm is the set of item sets. And so accessing this, we're going to order these based on the support and the number of items in decreasing order. And to highlight that, you can not only pull data from the database into our memory, but we can also take data frames from the client and create tables in them. And we can create tables explicitly materialized from other data frame proxy objects. So if we did want to have an explicit table representation of that, we can do that as well. And that's what we're highlighting here with the candidate bundles and creating the table candidate bundles all in caps. And we're showing a few of the results from that. One of the things that you'll notice is this is kind of a transactional representation where we have multiple rows corresponding to the same item set ID. And so these first three rows relate to the single item set that we're seeing here for these items. Now we might want to have a more convenient way of representing that. Perhaps we'll say that let's have one row per item set with the results. One way of doing that is that we'd like to concatenate these together. Now just to highlight that not all of the deep layer functionality that you'll encounter in R has been replicated in OML for R. So there are cases where you're simply not going to be able to do the equivalent functionality. This is one such example. Here we have the candidate bundles, right? This is a proxy object. The group buy and the summarize if you're familiar with deep plier you might have used something like this but the paste function in this case is not available for that. So to take advantage of this from R you could pull the candidate bundles into R memory. And then you'll see the corresponding results here where these are all concatenated as you would expect from R. But alternatively, you might want to use this from SQL. And so if you have members in your team that are leveraging these results they might be familiar with the list tag function and they'll produce a similar result here with those results forming the individual item list. If we want to display the top rules that are sorted by confidence and support here we're just going to pull out the rules themselves and we'll say let's get the top rules from this. And again, because this is a transactional representation multiple rows will correspond to the individual rules and we're seeing that. So in this case, we're saying that baseball is life cap plus the bucket of 24 synthetic baseballs implies purchase of linseed oils. So how can we use these types of rules to make predictions? Well, one of the ways is that we can pull out those rules that have a certain item in the left hand side. And so here we'll let's say if the person has indoor cricket ball in their basket then what else should we recommend to them? And we're just going to pull out those specific rules for that and we see that well if the left hand side has indoor cricket ball we might be interested in recommending to them any of these items that are here on the right. And that concludes the set of three demonstrations. So let's go back and just wrap up our presentation. So this brings us to a summarizing the broader context in which OML for R exists. You'll find the SQL and Python interfaces as well. The SQL API is really the foundation for the in database algorithms. OML for Pi has similar functionality to OML for R but it also includes auto ML, automated machine learning capabilities. And on autonomous database you've seen OML notebooks. There's also a no code auto ML user interface where the resulting classification regression models can be immediately used from SQL queries or deployed to OML services for real time scoring using rest endpoints as Sherry demonstrated for us. The no code data and model monitoring UIs support the broader ML ops requirement for tracking changes in data that support applications and models as well as changes in machine learning model quality. And lastly the original OML user interface Oracle Data Miner which is a SQL developer extension. So in summary, we've seen how you can use R for accessing and manipulating database data. With OML for R we leverage the database as a high performance computing environment for data exploration, preparation and machine learning modeling. And with in database machine learning algorithms from an R API, we're gaining scalability and performance in part by eliminating data movement and leveraging algorithms that are designed for parallelism and memory optimizations. We also can easily deploy machine learning models and invoke R user defined functions with system provided data parallelism and task parallelism. Now by operating in the database, users benefit from database backup recovery and security. So we don't have to handle these separately in our applications. Further, the OML for R interface is included with your autonomous database instances and Oracle database licenses. So note that on Oracle database you do need to explicitly install OML for R and as we mentioned from the client perspective this is supported through the Linux operating system. So if you have your data in or accessible through an Oracle database you can really take advantage of this R-based functionality.  You can also try this functionality in your database or on the autonomous database free tier and explore workshops through the Oracle live labs and this will give you guided tours through various aspects of the OML components. So thanks for joining this session and if there are any more questions we're happy to take them now. Yes, feel free to put your questions in the Q&A. Mark, there was an interesting question at the beginning of the presentation about the differences between the OML notebook interface and Jupyter notebooks. So with the OML notebooks, this is an environment that actually was developed by Oracle labs and has a number of separate features that allow for built-in visualizations. It allows you to incorporate and to load both Zeppelin and Jupyter notebooks and to export not only native representations but also Zeppelin and Jupyter notebooks as well. So you can interoperate with those environments and when you're opening a notebook you can open it in both sort of Zeppelin mode which allows you to restructure some of the paragraphs and how they're displayed, their widths and whatnot or you can open it in Jupyter format as well. And so those are just some of the characteristics that we've incorporated into the notebook environment. We also provide ready-to-use interpreters for our Python SQL, PL SQL and Conda for installing third-party packages. Exactly, yes. You don't have to configure anything to take advantage of those additional languages. That's all provided with the autonomous database notebook environment. And then in terms of running from a Windows client which was asked a few times, what I demonstrated here today, I took an OCI Linux compute node, installed almost for our client there and then I just exported a few ports so that I could run it directly on my laptop browser. You can reach out to me if you need the instructions to do that. Any other questions? Well, if not, thank you very much for joining us and until next time.


Hey everybody, I'm Casey and I asked to moderate this panel because I've been working on EVM and E wasm lately and I know there were there are no other experts besides these panelists So You saw Martin's talk yesterday And well it was on primea, but it mentioned the precursor to primea, which is E wasm and that started in December 2015 By The summer of 2016 the first commit was in April, but by the summer there was a pretty working prototype for the EVM to E wasm transpiler. So that was the EVM 2.0 So after the 2.0 proposal came the 1.5 proposal from Greg and and Pavel and you just saw Pavel's talk earlier about EVMC, which is sort of the the API to plug in and be able to swap between E wasm and and EVM 1.5 And Then came Earlier this year was Julia Yesterday also we saw Alex Alex's talk about Julia Which he and Christian designed To upgrade solidity and be able to make solidity target The EVM 2.0 the next versions of the EVM 2.0 and 1.5 and then since then Also Pavel has made a lot of progress on the the JIT VM So it's sort of been backwards where first EVM 2.0 was proposed then more progress on EVM 1.5 I hope this makes some sense. So I think I'll Ask our panelists what is What is E wasm what is EVM 1.5? I guess E wasm had made some progress substantial And a few things struck me one is people were sort of excited because gee whiz you could you could run C++ contracts and E wasm and as a C++ expert I said Why on earth would you want to write contracts and C++? Haven't haven't people lost enough money on the blockchain already And the other thing that struck me was why on earth would you want to hand over to an outside committee the definition of Anything to do with the core consensus protocol and I looked at You know the current EVM and said there's an awful lot of white space for more op codes and This thing's not broken. It just it needs a little work So I set out to say what what do we need to do to this to to bring it up to modern standards and make good use of modern hardware and And So I got to work on that and I got a lot of help from other people on the C++ team The documentation would not have happened without Christian And So the team put a fair amount of work into that. We've got a couple of the IPs Martin has put a lot of work into the E wasm proposal So they're sitting there and we'll need to make some choices and We might throw both of them away and say, okay, we've learned a lot What should we actually do? We might choose one of them We might stick with what we have and ask and we advance our compiler technology To compile what we have into code. Yeah, that runs better and Pablo can speak better to How possible that actually is Yeah, I would be very happy to have any of these but To add it more constrained to the control flow would help a lot there And I believe we can do much better in JIT like EVMs if you have that But you know just a step back So last year when you came around and even 1.5 came around It was still in a point of time where WebAssembly was not finalized at all, right? And there was no knowledge at least we didn't have any knowledge when it's going to be finalized There's no idea and since then this year the first version came out So that problem went away And you know like back a year ago We had no idea when it's going to be finalized and how it's going to look like when it's finalized And so that was a big concern And I think because of that even 1.5 made a lot of sense to maybe bridge the gap or have like a you know backup plan Assuming that EVM 2.0 would be finished in time But we cannot release it because the WebAssembly isn't finished And then if you could finish even 1.5 quickly enough it could be a good bridge between the two But that problem went away today. Well, you know the earlier this year. So how do you feel about the? brokenness of EVM since Did did anything change? I? Actually think you want I think he was I'm going to die What do you think that? Because there's been so many attempts to get a binary format running in browsers and they've all failed You're saying WebAssembly is going to die. Yes Wasm's going to die whatever I said, I don't know That's a bold prediction, but maybe we can step back and Say what are the problems with the current? EVM 1.0. I mean one problem is that the gas limit in each block is Not enough to do everything people would like to deploy contracts to do because One example is the Blake to be pre-compile there's a proposal for these native pre-compiles and pre-compile solve the problem of Contracts that people would like to deploy but they might take you know a hundred million gas when the current block limit is The current block gas limit is six million gas So how does? EVM 1.5 or our WebAssembly solve this problem Well, WebAssembly allows you would allow you to can just compile code Directly to WebAssembly so you most likely don't need native Native contracts because you would just write them in WebAssembly. Yes, but it's partly just that it's too slow You know, I mean Pavel was working. I remember on one of the pre-compiles That you were like pushing all the compiler flags as far as you could and trolling through Multi-precision libraries to find one that was mostly in hand-coded assembly You know, we just can't get enough speed out of the VM for these pre-compiles But then how is 1.5 and in 2.0? How are those? becoming faster than Then 1.0. I can my little grade school example You do a multiply and you're doing one instruction on a 64-bit pair of registers as opposed to you know Long division or long multiplication on a you know collection of registers I think that the current issue we have is Quite big difference in terms of speed comparing to native code So we we cannot actually effectively encode the algorithm we want there for example some hash functions and And that's makes them if you like to implement them in pure smart contracts that make them quite expensive and You have to pay for that You have to pay a lot for that because it's just The current EVM is not capable of express enough To have comparable speed comparing to native code and I believe that's what for example web assembly gives like it's it's at least comparable like if you implement The same hash function in in C and web assembly at least you have comparable for them and not having like times 10 10 to 100 times slower performance But actually if you if you take a step back to like the first version of EVM and Look at why do we have or why did we had an identity contract? just for achieve memory copying that maybe shows that We didn't had everything thought through properly and we started to introduce These pre-compines especially the one for copying memory that means Loading and storing memory was too expensive yet. We still wanted to do it. So we introduced a pre-compine and And that are bigger issue which Greg mentioned several times already is the bit with everything is 256 bit And there was a proposal even before the 1.5 to have 64-bit arithmetics in EVM You have folded that into the same the proposal as I see it there But if if we if we just look at these two problems that it's quite wide for arithmetics and we have we started to introduce all these pre-compines just to get around that that Probably shows that we didn't figure out the prices properly and with web assembly as Pavel said it's Resembles the instructions much more closer to traditional computers. So there's a much Probably much easier way to figure out what the real cost for those instructions are and by figuring out the real cost And we can probably avoid having pre-compines It's worse than 10 to 100 my graph was scaled by square root and the slowest Exponential operation compared to the fastest native code was 10,000 to one But that was on EVM to wasm, right? Actually, it was actually go versus C plus plus compiled straight to assembly so another advantage of another motivation for the you wasm proposal was be able to write contracts in other languages that Target web assembly as a as a compilation target versus only targeting EVM so We have the the EVM to E wasm transpiler as a prototype We don't yet have the E wasm to EVM transpiler so the reverse Which would Make the EVM 1.5 proposal equivalent to almost the E wasm proposal Because then you could still Write languages write the contracts in languages that target web assembly transpile those to EVM 1.5 I've asked before How we what would it take to write the EVM the E wasm to EVM 1.5 transpiler and Greg said it would be easy Martin said good luck Yeah, good luck. You told me it would be easy. It's not impossible. I wouldn't would never say it's impossible You said it would be easy Easy It's gonna be some work. It was on a fire escape in Berlin Was I sober I don't know And also one of the motivations for Just skipping EVM 1.5 and just and because the original proposal was just going straight to 2.0 The EVM 1.5 proposal came later and one of those reasons was because well, it must be hard to write a JIT a JIT compiled VM and then and then you know Pavel wrote a prototype of a EVM JIT VM is Was that easy Pavel so actually the prototype of that it's still a prototype it was it was It was done even before the launch of Ethereum. It was one of the Performance benchmark project that we want to To have to actually assess what can we do in the future in terms of smart contract performance? But yeah, it's it's it's still struggles with some some cases and As I said we can do much better in this case but And the required step is it's at least this control flow restrictions and subroutine support Directly in the in the EVM bytecode That would allow even more optimizations and but on the other hand like JIT compiles are hard I mean the the the network consensus depend on that and Like the risk is like We might might get hard or it might be never finished in terms of Removing bugs and finding education Because that's that's much more complex construct comparing to interpreter I mean another problem with with just in time compilers is that I'm not sure if there is a just in time compiler that provides a a fixed upper bound In terms of resource consumptions and this is a very important guarantee that we need in order to do to do the gas calculations properly, right? I mean usually just in time compilers generate code that is faster and takes less resources But we don't have a guarantee or do we well as I said we can't do a JIT. It's it's exploitable We have to it has to be a compiler that runs at deployment time not at runtime Which means it can be a full compiler Depending on how long we want to take to load a block then my understanding is Storage is the main constraint there anyway but I mean I remember in my testing I came up with one performance bottleneck and you said it's not a priority I don't think I can get to it and the next day you had it fixed Yeah, this is a bit bit a bit different issue. So in in in Ethereum we actually care about the worst worst cases because this is what what's the cost must be for and and So this affects more complex Optimizations, but also the big big integer libraries that actually try to squeeze the easy cases first But this is not what what we point to like we don't care if we can Divide quickly for small numbers because what we care is to have the the worst case covered So, yeah, that would be much more much more difficult to control that within the JIT because it's like you have the big at least in the in the In the in the in the EVM JIT that depends on LLVM you have a big Bucket library that that's that for you and it's really hard to tell what it's actually doing But yeah, I guess there are different approaches to that and for example He was him has some JIT prototypes that are not depending on on the on LLVM You know, of course one of my bigger concerns is not technical either of these Either of these programs technically does the job. I'm much more concerned About who controls the specification and I I really believe That the Ethereum community should completely control that specification that the web browser space Is not the Ethereum space and I would not want to get wedged with the With the wasm group moving in a direction they need to move And us shaking our heads and going no we don't want to go in that direction Yeah, I disagree with that because I think I am much rather It's much more pluralistic to go with a Larger community a larger body of people standardizing and coming to consensus on a Virtual machine instead of just using a virtual machine that can only be or was created to only be Used in one Pacific use case and if you look at the browser use case is very similar to the blockchain use case We need secure portable And size efficient By code right is exact same concern we have in the blockchain space Actually, I agree and and furthermore like It's an open. It's a It's open to participation. You can go to the WebAssembly community meetings you can voice your opinions you can submit proposals on GitHub It's very open and you know, it's easy it's easy to get involved That and within the blockchain space, I know of at least three other blockchain projects outside of the Ethereum that are already prototyping and actively have Wasm running In in their systems, so we're seeing a lot of momentum. I think pick up around it And I think that's sort of just Going to be the way it is because like once there is We start to have consensus around a VM everyone's just going to be the obvious choice Everyone's going to use it and it's going to be a recursive feedback loop, right? It's going to be a feedback loop where Since there's more people using it. We're going to get better tooling for it faster. We're going to get better implementations Etc. But it's not like Hypothetically if you introduce EVM 2.0 It's not like that a Vasem would introduce a new update and that magically would work on Ethereum As you mentioned in your talk both 1.5 and 2.0 have a verification code Which has to run prior deploying the contract and that verification code verifies the You know according to the E-Vasem specification, which is the current list of op codes in WebAssembly And so if they introduce new op codes, they wouldn't work without us making this Decision that we want to support them and so I don't really see that as a risk that we would be exposed to random new instructions being supported by Ethereum without our review first Now I'm more concerned So we come along and there's an issue with Vasem I mean bug bug show up in specs or if not a bug an ambiguity and we resolve it one way and It works for us and we can't wait You know we if something's a little weird in a browser It's not a big deal and the committee will get around to it But if there's anything that breaks consensus, we have to fix it immediately And then eventually it gets around that a committee and they go no we don't like the way you fixed it We're gonna do a different fix And I just feel like over time We will wind up forking away from that standard And we will lose these benefits of shared design and tooling anyway In fact, we're a fork to begin with we're a subset So I I mean I spent a couple weeks trying to get C++ into wasm. I succeeded, but it wasn't easy And then I think that's misleading. It's a subset, but The only subset being with the we don't let floating point operators. Yeah, that's the only subset But you have to convince the compiler not to do that Yeah, that should be pretty easy. I mean if you don't use floating point inside The compiler can decide that it's going to use the floating point unit just because it feels like it Unfortunately, we're out of time. So we'll have to keep that's fine. Good loading points What about Julia Sorry, what about Julia? Julia, I think Christian you're gonna cover Julia a little bit in the flexibility Solidity talk Not too much. I mean Alex talked about it in quite big details Yeah, well, it's a very active debate Even 1.5 verse E wasm. Hopefully we can do the magic of Transpillation Come to a united front and move forward with the evolution of the with evolving the EVM Thank you