Thanks, Mary. And hi everybody. This is a really tall podium for somebody who's only 5'2". Okay, Julia. Hi everyone. Thanks for coming to the final session. It's always nerve-wracking being the last people, but we hope that you enjoy this presentation. I'm Julia Avenall. I'm the Learning Designer at the Australian Film TV Radio School. Yeah, and I'm Jay Newton. I'm the Head of Curriculum at AFTERS. Is that better? Oh, that's better. Yes, we're from the Australian Film TV and Radio School. And we're talking today about advanced creating efficiencies with Moodle rubrics. I just wanted to start the presentation with a quote that I heard on the Moodle podcast the other day from Abby Fry that she said, Moodle is a journey of utilising the features better and better. And that really resonated with me because I think that it's not just about making iterative changes to the learning content. It's also about making iterative changes to your workflow and finding efficiencies in that workflow. So this presentation is about some of the efficiencies that we have made in our workplace using the Moodle online rubric tool. And I think basically we all want to spend less time marking and more time teaching. So this may be something that could help you as well. So who are we? So AFTERS was established in 1973 under an active parliament. So we're turning 50 next year. And our campus is based in Sydney, Australia. Our purpose is to support the next generation of Australian storytellers through teaching all facets of film production and broadcast. And so that includes writing, directing, producing, cinematography, editing, design and sound. So effectively, I mean, we have a range of award courses, both undergrad and postgraduate courses. And we also have a suite of short and industry certificate courses as well. Effectively, we're practice-based education. I mean, you know, you kind of need to be there making things in order to kind of teach the content that we do. But we use Moodle to support our learning. So what is a rubric? A rubric is typically an evaluation tool. It's to measure attainment against a consistent set of criteria. Rubrics are really useful in helping to position students against a set of learning standards. And it helps to determine how well they've met certain learning outcomes. It's a really transparent tool. So why do we use Moodle rubrics? AFTERS uses rubrics as our primary assessment tool. We also use some observation checklists and things as well. But we do tend to rely quite heavily on rubrics as our assessment instruments. A big part of the reason for doing this is it helps us to mark really efficiently and consistently across large cohorts and with multiple markers. So in the case of our Bachelor of Arts Screen Production course, we have 90 students each year. So that's a lot of students to mark. And we have multiple markers working with their cohorts across the year group. So it helps to keep things consistent for us. So then using Moodle rubrics with grades being calculated automatically in the back end, it means fewer errors and miscalculations by our markers. Plus the grades are immediately visible to students and we don't need to upload separate marking sheets so there's an efficiency there as well. And rubrics are a useful tool for students' self-evaluation. So we encourage our students to review their rubrics prior to submitting their work so that they can verify whether they've met all of the required criteria before they put the work in. So I'd just like to talk about the nuts and bolts of how you start building a Moodle rubric for those of you who haven't ventured into the advanced grading features in the assignment. So you start with the assignment activity and within the assignment activity you set the grading method to be rubric. From there you go to the settings and you can then go to, step number two is either go to advanced grading or define rubric depending on whether you're building one from scratch, whether you're adapting one that was used on that assignment before because they will roll over when you copy your courses or whether you're creating one based on a template that somebody else has pre-built for you. Step number three, once you get inside the rubric page you then build out essentially a grid that you can add rows and columns to populate with your marking criteria and the point value that is going to be assigned at each grade band level. There are also a bunch of rubric settings at the bottom which sort of effectively change how the students are going to see the rubric and I'd say get in there and play with all of those and see which ones are suitable for your students and your particular course. And the finished product looks a little bit, something a little bit like this. You have your marking criteria on the left and going left from right in our organisation. We go HD to a fail and in each cell that's associated with that there's a numerical value behind that, a positive numerical value and the rubric is going to calculate the grade based on the median of that grade band.  We have had instances where some people are doing them out of 50, some people are doing out of 100, gets really messy. I would say if you go for 100 across the board it really simplifies things at the final stages of marking. What you do find is that rubrics will calculate based on the median so students in a particular grade band will always come out at the same mark. That means if you want to bump up a student up or down a few marks you can do that manually when the grades are released. What makes a good rubric? Not all rubrics are created equally and a poorly worded or overly complicated rubric can make marking really challenging. Ideally the marker shouldn't need to exercise their own subjective discretion when awarding a grade, the indicators should support the marker in selecting the appropriate grade for each student. I suppose if you're doing a little bit of a review of your own rubrics really do look at those indicators and see whether they would be easy to award or if they're a little bit opaque and difficult to interpret. The best rubrics are those where the indicators are concise and unambiguous and each criterion should ask for a very specific intangible input. Sometimes where we've gone wrong in the past is within one indicator or one criterion we've tried to ask for too many things and then it makes it really hard, it makes the marker sort of has to go oh they've achieved that but they haven't achieved this so where do I put them? I guess trying to really simplify your criteria and have those indicators really clear just really helps with that marking. Students themselves should also be able to use the rubric as a form of feedback so they can see when they get their grade and they've seen their rubric marked up they can sort of see why there's a bit of inherent feedback built into that process for them. If we're trying to engage students with rubrics student input can really assist in the design of a good rubric ensuring students understand the assessment requirements and even having them co-determine the rubric can be a good way to engage them. It's also quite useful to provide examples of kind of rubrics in practice so if you have some exemplar student work that you're showing you could always have the marked rubric sitting alongside it so they can sort of see how the marking would have happened for that piece of work and providing fun video explainers has been one way the after-staff have helped our students to engage with the assessment tools and we're going to have just a look at a quick example of that now. Hello. Oh, hi Amber. Oh, hi. Where am I? Welcome to the rubric explainer. I've brought you into the rubric. We're inside it to explain it to our students. So what we're going to do today is we're going to talk through the last two assessments, the mood reel and the director's statement and we're going to go through the criteria with the students and then we'll talk through what an HD looks like so that they know exactly what we're looking for and I'm going to kind of talk through it and feel free to jump in and ask me questions and chime in yourself because you've got notes from how the students did last year as well. So I thought we would start with the director's statement. Now only because in my brain when I'm doing this work I start with the director's statement. It doesn't mean you have to do this this way. Like you could start with the mood reel first depending on how your brain works. Okay, so director's statement. It's worth, you can see at the top there, worth 40% of your mark and we have four different criteria. So you've got industry forms and practice delivers a well-structured, clear and detailed vision. That's, to me, that's like an overall thing. It's like I'm impressionistically reading the whole thing and going, this is a good director's statement. This is an HD, well-structured, clear, detailed, sophisticated is an industry standard. That's what I expect from someone handing this in to Screen Australia or Screen South Wales or another state agency. And I just want to say that little video was actually completely lo-fi. It was filmed during lockdown just with a digital background on a Zoom using the immersive mode. So, you know, we have a lot of technology at film school that we can use to make videos, but we didn't, we just did Zoom. Because we didn't have access to it. That was literally made at home. So, yeah. So I just wanted to talk now a little bit about some of the iterative changes that we've made. I'm sorry the text has gone a bit small there but I'll just talk you through it. So many of you might relate to this problem in that Moodle can't easily deduct late penalties when you're doing marking in the grade book. So we apply our late penalties manually at an assessment panel. But when we were using Moodle rubrics to grade, what we were finding is that our teaching staff weren't able to apply those late marks because they were marking on the rubric. So in our workflow process, what would happen is we would make those adjustments in our assessment panel. And basically it was a matter of, you know, manually adjusting the grade and then typing into the second cell that a late penalty had been applied and noting what the original mark was so they knew what they would have got if they hadn't done, you know, handed in late. What actually the result there was a lot of fiddly work with, you know, 70 to 100 students typing this in and quite often we made errors and there were inconsistencies and obviously it was time consuming as well. So what we came up with in the next iteration was that we would use the online rubric to actually deduct the late penalties at the point of marking. So instead of having a positive value assigned to each cell, we applied a negative value, a numerical value. So the criteria was the late penalty column and then we had a, the first row, the first column was zero points if they were on time. We just said no late penalties applied, zero points were deducted and then moving left to right, if they were one day late they lost five marks if they were two days late they lost 10 marks and so on. Obviously everybody has different penalties that they need to apply but that's how it works out for us, it's five marks a day. There's a bit of a game changer actually because teachers were now, or tutors even were able to apply these late marks instead of the lead teachers actually trying to do all this fiddly work at the assessment panel when you're meant to be concentrating on, you know, bigger issues at hand. Some of the other efficiencies that we have sort of developed over time is we started to publish our rubrics as templates. To do that, all you need to do is to create your template in the first instance and then you publish it as a template. It's then available for everybody in your organisation to copy or copy and adapt. You, I would say if you're going to start using templates, agree on a naming convention. Once you start getting a lot of templates in there it can be quite tricky to find them especially when you have a lot of assessments that have quite similar names or when people don't name them at all. In our organisation, our administrative staff build the rubrics on behalf of the teaching staff. To help them apply the correct numerical value within the rubric, we started out with a spreadsheet that was a pre-calculated matrix of the grade bands and the weightings that were going to be applied and the numerical value at each grade band. Then we realised we could actually build a rubric template with those pre-calculated grade band numerical values into the rubric. We built a full rubric with every numerical value for every percentage of each weight band and it's really just a matter of the administrative staff coming in using that base template deleting the rows that they're not going to use, duplicating the ones that they are going to use and again that has been a game changer because it's taken literally days off the work that they have to do at the beginning of the semester now that they have this template published. So just some positive feedback. So the feedback from students about the video explainer we just watched was extremely positive. By using humour and a light but informative approach students were made familiar with the rubric and the assessment requirements. It was also interesting a few students were noting that it really helped them reading and interpreting the rubric and how our teaching staff approached markings. So together with the video explainer it really did help them to understand how they might position their work for a high grade which is kind of what we want. And then just some feedback from the staff. So they found that markings obviously easier and quicker and they don't have to do all of this manual calculation. They can just click the buttons in the automated rubric to get the mark that they need and it's also easier to moderate across multiple markers. And as I was saying before with the example of our bachelor course it's such a big cohort that we've got multiple people looking at student work. So they found that that was great and then they were able to really feel like their marking was really rigorous as it should be and that they could trust the tool, they could trust the rubric to give the mark that they feel the student deserved. And I just wanted to add as well there's a cost saving in that as well. It's really enabled us to get people marking within the allocated time rather than spending too much time and coming away feeling exhausted and overwhelmed because they can't get the marking done in time or we have to find more money to pay more people to mark. That's right. And we actually do in addition to using our automated rubrics we also ask the tutors to provide some written feedback as well. Having a more efficient kind of rubric marking system meant that then they could spend more time on that written feedback. So that's it. Thank you. This is us. Does anybody have any questions? Yeah, thank you. That was really interesting. We do have plenty of time for questions. I was interested in your it's not a question just a comment. Your late penalty negative marking row I thought. Yeah, that's a clever idea. Okay, so if you have a question put your hands up and wait for the microphone to come to you. Hi, thank you for this very interesting and clear presentation. I have two questions actually. One is can you have multiple graders for the same assignment same student because we do do that sometimes at my school and I just forgot the other question but it'll come back. Yes, we can if the grade hasn't been released yet. Yes, we can do that and sometimes we do do that especially if there might be some sort of contentious issue or we have a new grader who isn't sure especially the point that Jane mentioned before where we've made the mistake of cramming too much into one assessment criteria and they don't know whereabouts on the rubric to mark we might get two markers in or if the student wants to question it, generally we would have one person grading but we can have two. No, it'd still be the one overall result but it's editable before you release the grade. They would have to agree which would happen through a moderation process that we have so again for courses where we've got multiple markers looking at a lot of student work there's always a moderation session that would happen so that they'll do that little bit of benchmarking and they might adjust the grade bands just depending on sort of looking at a cross section of student results. Did you remember your other question? Hi I always have an issue working with faculty coming up with what's in the criteria and sometimes they put too much stuff in the criteria and then they can't get, they're like oh they can't get enough or I had one person that had a 38 criteria rubric that they used to grade. How do you talk to faculty about what to include and what not to include? Yeah, it sounds like for 38 criteria that sounds like using a rubric is the wrong assessment tool. It sounds like that's more of an observation checklist or some other tool but yeah that sounds like you're just trying to cram too much into quite limited tool. I mean obviously a rubric really should only have probably no more than four or five kind of criteria and then with the weighting kind of the 100% weighting distributed across that criteria so that sort of calculates to 100% at the end but yeah so my advice would be to really break it down like obviously look at the subject learning outcomes first and then think about how you might get the student to demonstrate those learning outcomes and then do consider maybe having some complimentary assessment tools alongside your rubric but yeah always really it is always a challenge to try and make sure that each of the criterion is really tangible and specific rather than being loaded up with lots of things. It's really tricky. Hi so I've got a sort of follow up question to that is it realistic from what you said it seems quite difficult do people get their rubrics right first time or does it take a few kind of goes of using the rubric and then tweaking it to actually get to a rubric if it really works well? Yeah exactly so there's always that sort of continuous improvement so you give it a good go you kind of say well this is the rubric that we've agreed on for this task but then through course review at the end of each year of delivery round of delivery we always take the time to have a look at it and go is that working and particularly I mean I guess the biggest indicators are is it easy to mark against like or was that really challenging and like I said earlier does it need a lot of clarity to go oh it's a little bit of this and a little bit of that and let's try and split the difference and put it somewhere sort of you know around a credit sort of grade because we don't know where else to put it like so I guess yeah we always do a review process each year and yeah we have found that we've tweaked and refined those criteria over time and then after a while it's like yeah now that rubric is really working I'll tell you what it does help with though actually in terms of like a learning curve for people we we have a lot of industry experts who come in and teach our students and they are not teachers quite often they're quite nervous about marking students because they have never done that before and they may or may not even be coming back you know the next semester you know they're in and out and when they get into the online rubric and they realize that they don't have to do any maths they don't have to sit there and you know mull over like where a student has landed they really just have to select the cells and trust in the rubric it really does free them up to think more about the specific feedback that they're going to give and they can just write a couple of paragraphs and the learning curve is really minimal and that is great because they you know they're here for a short time and a good time obviously you remember second question do you usually mark up or mark down meaning do you usually with the criteria that you put in place give points or deduct points for not meeting the criteria for the automated rubrics the inbuilt rubrics that we're speaking about we for each grade band we take the median so it sits right in the middle with you know within each grade band which means that then potentially multiple students can end up being marked the same if they're marked sort of like an HD for that criteria a D for that one a credit for that one so it could end up calculating to the same overall grade and then we will often then do a sort of a secondary moderation sort of process before we release the grades and have a look and just decide whether we want to tweak some of those grades up or down and so we've got and Julia talked to that before there's some discretion for us to override that automatically calculated total if we want to bump them up like if they're sort of sitting just just shy of being bumped up to the next grade band we might decide actually that piece of work was really good it's deserving of a distinction rather than a credit so we'll just we'll notch them up just manually by a couple of grades just to get them over the line so that is a discretionary thing we tend to find and again speaking to the sort of the success of the rubric if you know it does a good rubric does mean that we're we're sort of usually happy with where the student ends up sitting and and then then we're able to sort of go yeah that that kind of worked but yeah there is always that element of manual override if we need to the feedback that the students get is it you didn't do this or you did do that yeah that's right so that will come through in the written feedback so obviously like I say in each of the each of the sort of indicators in the rubric there's a little bit of inherent feedback in there but also then we ask them to write a little bit of written feedback as well and it's usually just a couple of sentences that's something good something bad something for later so you know we sort of tend to yeah give them give them that sort of guidance written and we also I mean after because we're we're a small school and you know we've we've got you know fewer students than I'm sure a lot of you we do a lot of mentoring so we always sort of invite students to seek feedback and have a one-to-one with their tutor and better understand their grades so there's additional feedback there too great thank you you one last question yeah yep going going go on okay thank you just in time very last one thank you it's like an auction you talked a bit about peer-to-peer assessment the rubric you developed can it be used to in peer-to-peer assessment or does it have to be something completely different you can but we don't we haven't done that yet we did talk about it actually because we did have one assessment where we do have a peer-to-peer assessment actually what we ended up doing because it was basically like a screening where the students had to screen their work and then they've got an audience response on their work we actually chose to use a mentimeter for that and then we manually put those grades into the rubric but the students were given a copy of the rubric to make their judgment about you know it was basically like the movies one start kind of situation but yeah we haven't we haven't done any other peer-to-peer using the online rubric thank you


Well hey everybody, John Walls here with theCUBE continuing coverage at AWS Rangvent 22. It has been three really fantastic days here at the Venetian in Las Vegas, and we still have more to come. With us to talk about persistent systems is the Senior Vice President of Cloud at Persistent, Dominique Bastos and Dominique, good to see you. Thank you for joining us here on theCUBE. Thank you for having me.  All right, tell us about persistent systems. First off, core focus, what you're up to and then we'll jump in from there. Sure, sure. So persistent systems is a digital engineering solutions and services provider. They've been around for 32 years doing software engineering, innovating in several areas within different verticals. There's over 22,500 people at Persistent now as of my last count. We're in 18 countries. And in October, we hit the $1 billion annualized recurring revenue mark. Oh that's a good number right there. It's a good number, it's a great company. It's been such an interesting journey. I was with AWS for almost seven years before recently joining Persistent, and it almost felt like such a logical transition in terms of bringing what I've seen in my entire career of interacting with customers and businesses to what Persistent can provide as people are looking to make their journey to the cloud, whatever stage they might be at. And we should point out it's a SVP of cloud but your focus is AWS. My focus is AWS. Other options, other opportunities for your AWS all the way. Right, it's a multi-cloud company because we really don't believe in dictating to a customer what they need. I think one of the differentiators for Persistent is the amount of legacy history that they have across these industries and customers, I mean 32 years is a lot, and in terms of software engineering. So it's like really doing the hard work, the heavy lifting. And then seeing what can actually be commoditized, repeatable building solutions within these verticals to help customers accelerate their transformations. So. You know, when we talk about cloud, I mean this has been something that's been on the forefront for a long, I feel like a long time, right? But yet there are still many, and maybe you can tell me how with that percentage, whatever of companies who are either having begun yet, are just beginning, they're really in a nascent stage of this transformation. And yeah, I find it, I found it curious this week as we've talked with different people about where are you in your journey and so on and so forth. A lot of people are way back just starting past go and aren't as mature as I would have thought. I mean, do you find that to be the case? Absolutely, and there's many reasons for that. I mean, I think what I've started, I mean, I've been seeing it over the years, but we all know IT and business back then was very much kept separate. Two separate animals. Two separate animals, IT made the decisions, not in a vacuum, but almost in a vacuum, right? Now, obviously companies who know it's necessary and have embraced it, bring together the function of looking at the technological solutions that they're adopting to solve a business problem, but that business problem really is dictated by the customer need. So I think I have seen in terms of the life cycle of a business adopting technology post-cloud, there's a lot of enterprises that are still, they've made such big investments in their legacy infrastructure and in actually the developers and the people that are maintaining those systems and the different connections to put it in layman's terms between their systems and their customer systems, right? So that entire scenario makes it very difficult for them to move, it's like moving a mountain. So there's, I would say there's like three ways of looking at it, you have those that kind of want to revitalize their technology, right? They're backend systems, they want to optimize costs, and my background in technology is specifically in data. I kind of, I came up as a DBA and built data models and I've always loved data before it was a thing to love data. So- You were so far ahead of the crowd. I was ahead of the crowd, I was a trend setter. What a trend setter. I'm a trend setter. So I think from that perspective, they're looking at these enormous amounts of data that they've been capturing in these legacy systems that they're so heavily invested in, but they're not able to derive the insights to better serve their customers or to even innovate new revenue streams from that data. But they're taking the first step to say, look, we can actually operate more smoothly at a lower cost by moving to the cloud. So there's that. Then there are those that are looking to actually innovate and create new revenue streams, monetize their data, look at opportunities to integrate feedback that they've been getting from their customers to provide new services. So they're using the cloud journey, they've probably already moved into the cloud. They're starting to look at analytics and potentially using AIML to facilitate creating these solutions and services. And then there's those that want to pioneer and break into new inventions and ways of solving the big world problems, right? I mean, I think that's one thing I noticed in this reinvent that I thought was so special is there's like a really big focus on humanity, on humans, as we were talking earlier, everything, and I myself, like holding books and I don't like people being on their phone or having a conversation. But I think we are where we are. The reality is the world has evolved in such a way that community is no longer, it takes a small village, everybody knows each other, you have face-to-face interactions. You're not doing that with your customers either. There's digitally native businesses that have, for a long time, cropped up in the fintech space in, you know, you name the space, there's a startup that was born in the cloud that can reach customers immediately and can provide a service that an enterprise that's kind of like weighed down with their legacy systems, they can't pivot fast enough. So I think, you know, the pioneers think beyond that, how do we use quantum computing? How do we use 3D simulation to anticipate solving big world problems? Whether it's, you know, people no longer, I don't know what the statistics are, but it's very sad that elderly people, you know, the amount of human contact that they have is very little, you know, and if you could provide, I don't know, an experience, an immersive experience where their memories are triggered, you know, to help them with dementia, Alzheimer's, and those types of things, those are the things that I think that's what excites me about the launches that I see at re-invent. And I think the innovation, you know, you have to take that journey. Unless you're born in the cloud, you do have to kind of take that journey. You've got to get there. You have to get there, but it's so worth it. So how about, let's just say, if I'm a health sciences company or I'm a pharmaceutical or whatever, and so I've got this desire to create this new opportunity, you know, with a human, if I say, but yeah, but if you're also, if you're persistent systems and you're working with, you know, somebody in Fintac or somebody in Ag or whatever, you can't really understand my challenges or my problems. I mean, how do you wear those different hats so you can identify not only what the focus of that client is, but also their technology and how you're going to get them to marry up so they can achieve their goals? Well, the beauty of being, you know, in a company with teams of people that you work with, my, I cut across industries, right? So we have vertical leaders that have very deep subject matter expertise in any one of, any number of those areas. You know, we're working with Genomics, for example. So for example, you know, we engage with a customer that we've been helping over the past 32 years use technology to bring services to their customers. And now we're seeing an opportunity to help them innovate, to keep up for their business for obvious reasons, but also to supply their customers with the new, the new innovative solutions within that industry, right? Cause that's, you need that vehicle to kind of deploy and deliver what customers need. The way we do it is from end to end, right? So we can, we have in the partnership with AWS, we're a partner of AWS, and as such, we're able to collaborate with AWS and their customers or bring our customers to the cloud, all the way from assessment to planning to execution. And even within persistent, we have ways to operationalize the maintenance of these solutions. So it's really a very easy managed services type framework that we work under. In terms of like migration planning, we have competencies within AWS for looking at migrations. We have AIML, we have DevOps. So we have the various competencies aligned with AWS to be able to execute at whatever stage the customer is, but also in terms of like the accelerators that we provide or the frameworks to look at total cost that cuts across, right? And then we don't kind of like, here's what you needed and buy, never speak to us again. I mean, I think the beauty of this company and what I really loved when I was first speaking to them is the depth of the relationships with their customers and the longevity of them. So they've really seen their customers grow. And you can only do that if you're there for the long run. You've got to be present. You have to be present. So how do you handle that? If people are making this transformation and they're moving into the cloud, but the people they have on staff might not be familiar with it, right? They have great expertise in what they've been doing on these legacy systems, but now you're moving, you're migrating to a new world, new culture, new environment. And you got to get them up to speed and that's not easy, right? So what do you do or what does persistence suggest or what are you doing with regard to closing that gap and to making that bridge so that they can maintain a little bit on their own? They can execute and implement on their own a little bit. They don't need somebody there to stand over their shoulder the whole time. I won't geek out on having joined AWS in professional services way back when to migrate a major company to the cloud and having lived through painstakingly all those problems and blockers and adoption roadblocks that you speak of. I think the way persistent handles it is what I would have done myself, right? If I were to start a company and say, how do we help customers simplify their cloud journey and remove the complexity? I think that's what persistent systems does. We, there's training programs that we're aligned to with AWS. So there's upskilling of development teams, application developers. We collaborate from the top down with executives to look at the resources that they have available. Obviously mission critical systems that cannot sacrifice having engineers pulled away for a new project. You take that into account. I think when I spoke earlier about assessments, you're not just assessing what needs to be lifted and shifted or refactored or rearchitected. You're looking at all these applications that are going to move to the cloud. Who owns them? Do you have a CI CD pipeline or data pipeline built? Well, we're going to need that, right? So the continuous integration, continuous development of applications, that type of DevOps, obviously security also, DevSecOps, we look at it from end to end as well. We have a very strong security practice. So all those advisory pieces we have, but we also have the capability to execute on it. Where we're not just coming in and saying, well, this is what you should do. We're kind of in there saying this is what you should do. Here's how we can get you started. And then, you know, it's a collaborative effort with our customers to see how much they still want us to stay versus how much they want to take over. Right, it's nice to have a friend. Yeah, who doesn't need a friend? And persistent systems is your friend. Dominic, thanks for the time. Oh, my pleasure. I appreciate it. Thanks again for being here on theCUBE. You bet. Absolutely. You are watching theCUBE. As you well know, the leader in high tech coverage.


It has been a key member in the National Committee for Enterprise Architecture in India and a co-author of the Indian Enterprise Architecture Framework, better known to many of you as INDA. So a really important standard that is transforming the way that things are done in the Indian government as from an enterprise architecture point of view and delighted that our own standard Tegaf plays such an important role in that. So today Palab is going to look at the Digital India Initiative and what it's doing to transform India. So a virtual round of applause for my colleague Dr. Palab Sahar over to you Palab. Thank you Steve. Good, good, good day all of you, good morning, good afternoon, good evening. I'm happy to be doing this presentation. In fact, this is very nicely timed because the content that was covered by me and some of the, you know, some of the ideas and some of the concepts that were already covered provides me a very good quote unquote platform to kind of move into my presentation, which focuses more on Digital India. I think from, you know, for those of you who are joining from India, you have definitely heard of this big program, which started in 2014, so to speak, which I would, you know, term as the first generation of Digital India. And you know, I have made a couple of presentations already in our conferences over the past three years on this topic. So I will not cover, you know, that material already because those that have been recorded and they are available. So what do you see to be a continuation of what is in progress program that is currently unfolding in India as we speak. So last year in 2019 towards 2019, there was a, you know, let's say, you know, saying that we need to move to the level of transformation in India and therefore it was, you know, unofficially as the India version 2.0. I'm going to show you some of the, you know, the material and some of the work that is going on. And please understand, as I've mentioned earlier, is work in progress. So certain things I will mention are going on. So even as we see things are progressing. Let me move on to the slides. You heard, you know, you've already heard Remy and some of the questions that Steve came through with his presentation. So some of these terms are being captured here. So this is the transformation framework that kind of summarizes everything that is important. You know, Steve, you asked one very important question that is the role of enterprise architecture in digital transformation. So in this, you know, picture, you can see the key layers and the key elements and the key, you know, segments that, you know, really form the digital transformation framework that is being currently followed in, within the digital India initiative. Right on the top, it starts with the various delivery partners and please understand that because this is pertaining to digital links, there's a lot of involvement from the open side, which is quite obvious. However, there is a lot of key take-holds which are coming in from the citizen and the consumer side, that is people who consume the digital services, the providers of those digital services. And there is an understanding that all the providers don't have to be a government ministry or a department, we are on a PPP basis in partnership with these. And I think we all understand that today, given the crisis we are facing, of course, we have the government, we have the aggregators, the platform providers and the investors are also involved in that. Having said that, the second layer is the most important part of the digital transformation because one thing that we are always thinking about is what digital services mean to the people, what does it mean to the different stakeholders that have been identified on the layer on top. So some of these terms have already been discussed before. So this is the speed of a digital service that we have defined services that are personalized, that meet the requirement of the citizens and the stakeholders, certain services that are paperless. I think we all understand that and we all need that today. Presence-less you don't have to see a human being to get the service end to end proactive. Why do you have to wait for the government to provide a service? Why does to request for a service? Why doesn't the government actually proactively say, this is the kind of service that we believe citizens need and therefore we provide that service. Perceptive and pervasive is important to use of other digital technologies like AI, like mobile technologies to provide services in a more inclusive manner. One thing that is very important from a government perspective is inclusiveness. If you provide all citizens don't have the same level of literacy, if you will. Now it is important therefore it's important that even a person in the area is able to consume a digital service who is not be on a smartphone for instance. So this is something that a corporate entity will unlikely face because there the consumers or customers are a lot more homogeneous. Then you have the architecture. I think I will cover that a little later, there are a couple of slides there. But what I'd like to highlight here in the architecture layer are the six domains or the eight domains that we cover in terms of the architecture starting all the way from performance to integration to governance. So that's the architecture planning layer and then the bottom three pillars talk about the different implementation layers that are required to execute that architecture. Number one is it's very important to have good institutions and good governance and all of the success factors of institution and governance. Number two is definitely when you provide services we need to know citizens are asking for and therefore it's important to provide citizen insights. And third is we need back and delivery capabilities. And this is where the couple of questions that came up earlier in terms of how do we combine architecture with practices is something that we have we have dealt with and I'm going to cover that in a little slide later. In a nutshell, various building blocks, if you will, of the transformation that digital India is currently rolling out, even as we speak. And having said that, you can obviously understand and appreciate that the enterprise architecture or architecture plays a very important role in this journey. So my intent here is not to go into the details of the frame. As I said, I have made a detailed presentation in London about the India Enterprise Architecture Framework and that session has been recorded. So you could always go to our channel, open group channel on YouTube and have a look at that. But in a nutshell, this framework consists of three parts. So the number one is the framework itself, which focuses on the eight reference models and the eight domains are the same ones that I've shown in the previous slide. Second is the guide. And third is, as you can see, this is where the marriage of the practices with architecture rigor comes in and with the latest update, which is called the Agile India Framework. So I'm not going to go into details, but I, you know, this is a basic understanding of what is currently in from an architecture perspective. But what is more important is that in the digital India, digital transformation journey that is currently underway, we have a set on the United Nations Development Goals because this is for us and for any country because this is defined, you know, basically defines the goals and objectives that digitalization, so to speak, should be should be targeting. So because these goals are already available and they are best, because we thought let us not reinvent the wheel and let us try to, you know, build our architecture in alignment to the UN SDG. OK, so these are the UN SDG as such has got 17 goals and more than 160 targets and indicators. So it is a very, very detailed framework that is available in the United Nations if you follow from the government domain perspective, you should be familiar with this. Having said, what you see here are the 17 goals. They look very independent of each other. Actually, they are not independent. So this takes me to the next slide, which shows that these goals. So this is a snapshot of the same goals, but it shows the relationship in the different goals. So, for instance, if people are, you know, if people are in poverty, for instance, I cannot provide education and so you can see there are many goals which are actually connected to one another. So this is a snapshot of the of the SDGs in a connected manner, unlike in the previous slide, which is basically the boxes, which kind of, you know, were placed there in a different manner. And you see the numbers there, 17 goals, 169 targets and 241 indicators. And the reason I'm showing this slide is because it's the basis for integration. When you talk of an end-to-end business process, when you talk of data integration, when you talk application integration, or any other kind of technical integration, it is the SDG which provides the basis and the rationale for that integration. It provides the answer to the question, why do we need to integrate? Whether it is API based integration or whether it's the integration based on some kind of a service, but in a particular detail. But when you talk to the lawmakers and the strategy level bureaucrats, you have to tell them that, you know, if you cannot address a goal without addressing other interconnected goals, and that's the very essence of digital transformation that is currently going on. So this is what I was talking about. You see how we have actually how the, you know, how integration is being, you know, let's say, realized using data level information. So right on top, you can see the integrated views which are based on that is being created through a set of integrated data that is created from multiple independent systems. And please understand that many of the corporates are operating in silo. You can imagine the governments operate in silos to a factor of 1,000 or more, right? So the level of silo that exists in government, because we have a, you know, federal level of government as a central government, there are state government, the local government and, you know, and it plays out, this is not specific to any country which has multiple level of government. Very, very difficult to get them on board on a single goal or a single, you know, mission or single objective, given the way they operate. So anyway, that cannot be addressed, you know, in this section. That's a different topic, but I think you get the point in terms of how we are using data to provide integrated view based on the sustainable development goals, which I talked about in the previous slide. Now, I am not going to go into the details of the architecture, but being part of the journey, these are obstacles that I all, and I'm going to address those obstacles that is coming out. The first option, the first question that I always get asked is architecture tends to be too rigid, and we, and the government operates in a very dynamic environment. So how do you plan for changes? Which is very important. It's all good to do a blueprint, which two years to do, but then the government has changed or even the process has changed or, you know, something has happened and therefore, you know, things need to be brought into the architecture blueprint so that it's been able to reflect in the reality of how the government is operating on that day. Second is it takes too long. We need something bigger. Absolutely. We cannot, you know, it is impossible for us to go to a state government or federal government or a ministry and say, I'm going to take five years to come blueprint and by the time we have a new election. And third is the IT function itself needs to reinvent itself in the digital. In many places, this is on our own personal experience that even though the court and court, the business of the government is talking about sometimes it's IT, which is slowing down the transformation. So therefore, there are certain things that need to be covered. And I'm going to cover these three major obstacles that are important from a from a trans perspective. First is, you know, what we have done as part of the architecture is that to take care of major changes. We know what changes can take place in this slide, actually, on your right, shows you the different kinds of changes that takes in government. The government coming up with a new scheme or a program, which leads to need some of the continuous management program and the life of a specific technology or moving from existing infrastructure to government cloud incorporate a new standard, merger of two or more departments, creation of a new department or a district, the functioning of a mission, critical application. So based on the historical knowledge that we have gathered, we know that fundamentally there are this limited number of changes that typically take place. So what we have done in the architecture framework is that we have tackled these changes through SOPs. If these changes take place, which is more or less what happens, then the state government or a ministry exactly knows what needs to be done. Therefore, it provides faster reaction time. It is not as if every time these changes happen, any of these changes happen. And of course, you can imagine that this is not an exhaustive list, but I think we get the idea. So if there is a change that happens, I know exactly the SOP. I, you know, pick up the SOP and know what changes to be done to the architecture and to the implementation of the audience. The first wave, we have tackled the dynamic aspect of the business operations of the government or the business of government, quote unquote. The second part is we have come up, if you remember my earlier slide, the poetry of the NDA framework talks about bringing data practices into enterprise architecture. So these are the different interventions that we have recommended. This is still a progress because please understand that governments in more or less around the world are not really structured to take in agile practices for various reasons. Of course, you know, I'm not doing the details of the interventions here, but I think, you know, you understand most of the points because given that we have had two questions on some discussions on agile practices also in the previous, you know, session, but the changes we have classified into changes required at the organizational level, at the process level, at the technology level, at the governance level. Now, having said that, it is still not easy because agile, as you all understand, is a paradigm shift. It is not a technology modernization, and governments at certain points, it's very difficult for the agile for various reasons. For instance, even at simple fact, like a language gap, you know, in government, people don't use the word sprint. You know, in the philosophy, we all understand that sprint survey, you know, is a fundamental vocabulary that we adopt in governments. People don't use sprint. So even as fundamental and as basic as that, those changes are not in place, but there are five key changes we have recommended in terms of, you know, procuring agile projects. Number one is we have recommended that we need to move from being contract centered. Government typically becomes very contract centered. And in most countries, what they do is there are three contracts and you look for the L1, the lowest price one. And therefore that's where it takes up the project. Here it moves from being contract centered to, you know, project centered. So there is a statement of objective. Second is we have also recommended that the vendor and the customer in this government ministry or a state needs to work, you know, as partners. You know, you don't have that customer-vendor relationship where you have an arms-length relationship with the vendor.  You have to work in a partnership mode. The other important thing which I'd like to highlight is that typically, you know, most governments around the world at this point focus on lump sum, you know, fixed price. So this is the project. This will be requirements for the project and let's do a fixed bid so that the vendors can actually respond to the RFP versus incremental pricing, which is, again, a major change of policy level needs to be brought into the government. In terms of pricing options, there are multiple pricing options that we have recommended, but the most important and the most successful pricing option that we have seen at this point is entry, which is hybrid pricing. There is a mix of fixed and time and material pricing for appropriate segment or transaction-based pricing, which means the success of the success of the program or success of the, you know, or the project is also, you know, related to the vendors pricing and vendors charges. So this is, you know, you can understand, there are certain, you know, paradigm shift that needs to be brought in when agile is adopted in the government and we are working towards it, not reach to a point where agile is being adopted, but we are working towards it. Okay, let's see. The third one was, you know, whether the IT is ready. Now, I know that there are going to be sessions on IT for IT. So this is basically a variation of IT for IT that we have created for the government and you can see some of the words who are familiar with IT are very similar. So we have strategy to portfolio requirements to deploy, request, fulfill, detect, to correct. And these are the same pieces of IT for IT, but we have identified to the next level of details what needs to be done. So this provides a good framework for IT departments within the government to modify, to transform, to modernize themselves in terms of their practices, in terms of their policies, in terms of regulatory changes and, of course, technical changes to ensure that the digital transformation of the digital India that is being talked about is being realized. Now, these are some of the, we have talked about digital platforms. Remy was talking about digital platforms. So these are the, you know, some examples. Again, this is not an exhaustive digital platforms. Please, we call them as public digital platforms. And many people who come from India definitely are using these platforms and we have benefited from these platforms even as citizens. And you can imagine, given the size of the number, we have all of these within the top three in the world in terms of the volume of information and the volume of transaction currently going on. So you get an idea in terms of where the, where the government is working to. And there are a few more added to this list, but I think you, the basic idea is there on this slide. Now, one example, so if you see these digital platforms, they tend to be more infrastructure oriented platforms, which means these are the building blocks that are required to provide the basic technology capability to provide digital transformation. But at a business level, one example, how is how, you know, enterprise architecture is being applied is to improve the ease of business. So World Bank has index called EODB, it's called ease of doing business. People who understand the government domain will definitely be familiar with, you know, in terms of the different phases of ease of doing business. But fundamentally, it talks about how easy it is for, for, for, for a business to operate in a country, starting right from initiating the business all the way from closing the business, the business goes, if the organization goes out to see the bubbles, their number from one to 10 and the different activities to improve, you know, each of the phases. So some of the states is that they are using architectural practices to improve these activities, which is quite obvious because the index already is very easy for them to show the some benefit that comes out of architecture. The next slide shows you the linkage. So if you see the previous slide, these are the 10 bubbles, which are the 10 different phases, so to speak of EODB. And the next slide shows you the mapping between your EODB parameters, which is on your y-axis, which is on your left. And on top are the different architecture outcomes that is being achieved using the NDA framework. So you can see the mapping here and then the next slide, you see how it is working out. So on the bottom is the enterprise architecture value chain. The transformation outcomes are in the center, where what transformation outcomes the states are looking for, goals and goals. So states are using enterprise architecture, you know, driven digital transformation to improve their, you know, if people who come from India, you definitely know that states are now being ranked on UDB and countries and World Bank. Actually ranked the countries also on UDB. So this is an example of the infrastructure building blocks, which I showed you, the public digital platform are being used along with the architect to improve on a certain parameter on a certain goal, which is business oriented. Okay, so this is another example of a digital platform. So currently we are in the process of developing a public digital platform for higher education. So this is kind of a summary of what is going into it. So you can see the, you know, on your left are the existing systems that already exist. So you have the Raj, Chhattar, Shikshabian, loans, vocational skills, research, professors, SIP, so on and so forth. Then you have the education policy and you have the India architecture and the API. So all of the integration is based on API, which is being consumed and put together into a single which provides these services which are provided, which are listed on the right of this slide. So all the way from education loans to promotion of research to ranking, regulation, management of government institutions, so on and so forth. So this is an example of how the fundamental of the infrastructure building blocks are being used to create a total building block, so to speak. So one was the either doing business example. The second one is higher education. We have created one for health and that's currently being implemented. Implemented the other one that we're looking at at this point, agriculture. You can obviously imagine that health, education and agriculture are the fundamental sectors or the fundamental focus areas for almost any government in the world. And that's why these have been identified as the higher areas as part of digital India. And the reason it's quite obvious because these services or these sectors touch the citizens directly. So it's direct citizen contact. So if there's an improvement in the agriculture service, the farmer is directly benefited. If there's an improvement in the health service, you can immediately see, you know, benefited from a perspective. So that's what these are the priority platforms that are currently being developed even as we speak. And I'm sure we will have an update in a year or so when these platforms become take shape and they are adopted. So. Okay, so the last slide which shows the panoramic view of our digital landscape. It starts all the way from infrastructure on top, right? So there are places which are not where we have to provide broadband connectivity, you know, moving on to providing the capacity and capital, providing the policy, legal and the regulatory changes that are required to show the digital, the transform services. We spoke about platforms already, you know, that is contributing to the digital economy. And at this point, we always understand that many, many parts of the economy is being digital. In fact, India has a target of being a $1 billion digital economy by 2025, which is not very far. And we are definitely going to get it. And of course, all the way into smart living and smart city. So you can see that given the heterogeneous landscape that we operate in, the entire digital landscape which Digital India plans to cover, you know, spans all the way from basic infrastructure all the way to smart city and smart living. So I hope it gives you an idea of very, you know, summarized idea of what is currently going on. And thank you very much. Palav, thank you very much. It's fascinating stuff going on there, world leading stuff, I think, going on there in India and lots still to do, of course. I don't know if you have the ability to be on video as well, Palav, at the same time, but by all means, if you start your video, then folks can see you too. Been a number of requests for your contact information. And so there it is on the screen. That's that answered. And also, few people are asking about the slides. There you are, Palav, great. A few people are asking about the slides. They will be available. So obviously there's a lot of information on those slides and there's a lot for people to digest. So they will be available. So let me move to some questions. What are some of the key considerations to establish a real business IT coalition inside government organizations? So the fact that we look for is creating a common working group, so to speak, which has the representation both from the sectoral perspective of culture, education or commerce and industries in case of doing business as IT. But in most cases, what we do is we have tried to make sure that there is a lot of buying and direct involvement from the business side. Business as in quote unquote, the government operations side, so that they take ownership of it. So it's an entirely IT driven issue and we know that it's not easy in the government to push something through for reasons that some of us would probably understand if we have acted with the government. So one of the considerations that we definitely look and the most primary consideration is that we would like to be respect the minister or the ministry or department to take ownership of these transformation projects. Because as you would understand that these probably help in their own quality of governance, that is very important. And agriculture ministry exists to provide better services for farmers and better foods. IT is definitely enabling some of that journey. So this is the primary consideration that we need to take into refactoring when we design a platform or the architecture around that. Right, OK. With such a big percentage of the population not having basic needs, how do you achieve digital culture and behavior, the challenge? Yeah, it's very interesting because a lot of people did tell us that yeah, if people have some portion of the people are physically literate, do you expect them to be digitally literate? And how do you ensure that the services are inclusive? But what we have found in the past five, six years ever since transformation took off is that people become digitally literate before they become physically literate. Today it is not. And as we can see that in our own kids, right? I mean, they know how to use a smartphone even before they know how to read alphabet. So it has never been an issue in terms of whether the services are being consumed in the urban level, urban centers are being consumed in the rural centers where people might be semi-literate or some of them, a few of them could be illiterate. But what happens is we do have this concept of a citizen service center where somebody who's unable to consume a service on a smartphone can still go and get a service in an alternative means which could be through a kiosk or a physical office or a government outlet, so to speak. Yes. Right. OK. Well, we just time for one last question. And then we'll try and get the other ones answered. I know you've worked in private practice as well as government Palab in your career. Do you think this approach is applicable to the business sector as well? Or is it just something that is particular to the government, do you think, the approach that's being taken here? In my view, because I, as you said, Steve, because I worked in both sides of the equation or the table. So yeah, my own personal experience is definitely applicable to the business sector as well with certain modifications in respect to the alignment that needs to be applied. It is a lot more difficult in the government sector because we understand it's not, it is a humongous task to get a state government and local level government aligned to the national government or the industry. Yeah. And some of us do understand that if you worked in the government, it's likely to be much easier in the corporate sector because there's a lot more alignment. Even though the corporate sector did talk about, you know, Remy was talking about operating in silos, we all understand that the silos that exist in the government are humongous, it's very, very difficult. So yes, the question definitely it is applicable to the business sector with certain modifications.