Hi everyone, thanks for joining this session. So today we'll talk together about high performance multi-region messaging with NATS. So before to do that, let's talk about who we are. So I'm Cyril Baker, I'm a editor of infrastructure at XBTO, where I lead and manage the infrastructure teams since 2022. I started as a Linux, HP, CEC, Sennmin. I have more than 15 years experience in open source infrastructure and services. I just discovered Kubernetes and AWS in 2017, and I designed previously many highly scalable platforms in the public cloud for the attack and the mobile game industry. And I'm Vincent Bernaud, I'm a DevSecOps at XBTO for more than three years now. I'm mainly focusing on dev and trading teams, and I have five years experience in sysadmin and cloud native applications. So XBTO, who we are and what are we doing? So we are crypto trading firm. We were built in 2015 as a proprietary crypto trading investment firm. But today we are a full service company offering different services around crypto and investments. In 2023, we acquire a custodian trading platform that is called Stable House. And then now Stable House secures some, oh, sorry for that, secures some big deal with the fifth largest firm that is called Apex. I think there's a world thing with the TV, sorry for that, should be good, sorry. So maybe some of you already know us because we signed Lionel Messi at the Miami football club Astures. And we are also the main corporate and jersey sponsor of Miami football club, co-owned by David Beckham and George Mass. We are licensed by the Bermuda Monetary Authority and have presented five locations. So Bermuda, Miami, New York, London, Paris, and Abu Dhabi next. So what about the agenda? First, we'll present our needs and why we choose Nats, then we'll present quickly Nats for those of you who don't know the technology. Then we'll deep dive into the platform and the architecture and the change we faced. And at the end, we'll present a quick tool that we developed ourselves and we'll answer all of your questions at the end. So let's talk about our needs and our use case. So what we want to do at XBTO is to share collected data between processes located in different regions around the world. So what we want is to have a full mesh technology that can connect several local message booths together to our messages. We want also something that have clustering technology to have redundancy and availability, something stateless that do not require writing to disk and that store all the data in memory, something secure with modern authentication, authorization and multi-tenancy and also something really scalable that allow to ingest a big quantity of messages that will increase over time without changing our current architecture. So let's talk about Nats. So how many of you are already familiar with Nats? Ok, nice. And now how many of you are using Nats in production in your current company? Great. So a quick intro on Nats for those that are not familiar with this tech. So what's Nats? Nats is a simple, secure and high performance open source messaging system built for the cloud and microservice architecture. It is available since 2011. It's used by top major company around the world. It's written in Go, so it's cross-platform. There are multiple client libraries available for most languages. And its core design principle is performance and scalability. And it's developed mainly by Cynidea that provides multiple different types of services around Nats and also some support. So why it's cool and why it's great? There are multiple advanced features included in Nats, but we will focus on four great advanced features. The first two features are linked together. The first great feature is the supercluster. So what's a supercluster? A supercluster is a set of clusters, sorry, that have gateway connection established between them. This means that, for example, if a client connected to cluster A wants to subscribe to messages that is hosted in a cluster B, this will be possible seamlessly. Messages will transparently flow across all clusters without the client needing to have any knowledge of the physical location. So let's say, for example, you have three different local message bus split in three different regions. You will be able to connect a supercluster with all this cluster connected together. For doing that, the supercluster is using the gateway. So what's a gateway? A gateway enables connecting one or more clusters together into a full mesh network. So this allows the formation of superclusters. Gateway exists to reduce the number of connections required between servers and optimize the interest graph propagation. Another great feature that is included in Nats is the lift node. So the lift node extends any existing system of any size and optionally bridging both operator and security domain. The lift node will transparently route messages from local clients to one or more remote Nats system in this version. And the last one is the monitoring. So what's great with Nats is that you have a built-in HTTP server that can provide some JSON endpoints. So this is really cool if you want to scrap some metrics with Prometheus or something else. Now, let's talk about our architecture and platform at XBTO. A few metrics from about Nats at XBTO. So currently, we have more than 100 nodes, 34 clusters, five superclusters, four lift nodes, more than 5,000 clients. We reach around 80 billion messages per month. So this means that we have approximately around 60 terabytes of traffic each month. And about the stack, so we use mostly Nats 2.9, 2.10. Kubernetes, I'm sure to deploy on Kubernetes. But we use also some standard Linux binaries. We'll talk about that later. And most of our clients are Python, are written in Python.net and JavaScript. We use also Protobuf. It's really cool. We have built some custom Prometheus exporter. So we create new features and extend the current Nats Prometheus exporter available on GitHub. And our monitoring stack is running on Mimir, Grafana, and the on-call services hosted at Grafana Labs. And we will also show you, at the end of this talk, homemade tools that we created, that is called SnapMap, to allow us to have a custom monitoring interface. So here, you can see our current architecture at XBTO. So we have three main regions, one in the US, one in Japan, and one in the UK. We are using a hybrid infrastructure. So we have some private cloud, and we use also AWS. So between the US and the Japan, we use a direct connect link. So it's a dedicated fiber link between AWS and our own data centers. This is really great for having stability. And between the US and the UK and Japan and the UK, we just use a gold old VPN on the internet that is doing the tricks. On each region, so all our processes are running into multiple Kubernetes clusters. So we use EKS on AWS, but also on prem RKE, on short distribution. I think soon we will migrate to Talos. This is in the roadmap. But for the moment, we still use RKE. And on each region, we have this NAPS bus that is local in the US, in Japan, and in the UK. This is a great setup. And the great thing is that with the direct connect with AWS, this allows us to route any traffic from the United States to Japan with what they call the direct connect gateway. So we can reach all the region that we want with the same link in AWS. So now let's deep dive into the supercluster and how we built it. So first, the goal that we had is to interconnect the different regions. We started with only one region, and then we had to expand to new regions. And we keep expanding and adding more regions during the coming month. So the big thing is, it allows us to have a full mesh network. So we have a full mesh network in your regional cluster. So as you can see on the schema here, on region A we have a full mesh cluster with NAPS, sorry. And we have a second cluster in region B which is also full meshed, but every node in cluster A is connected to one node in cluster B. It's not completely full mesh, I would say. And it works with inbound and outbound gateways. So it can have inbound, so it receives connections and outbound. This is a really important feature for another thing which is scalability and decentralized architecture because it allows to quickly add new clusters as you expand your regions. And also minimize round-trip delay because we are trading companies, so latency is key for us, and we need to have the best routes and optimize the routes as we go. So two important features that we use is the interest-based routing. So for example, in your schema, you have the market data process A which is publishing on a subject and it's consumed by the trading process A and if it's local, you keep it local. So it will never be propagated to region B, it reduce costs, and also the number of messages. So that's great, but if you need to exchange messages from process C to the trading process C, which is on another region, it will be completely transparent. You won't have to even know on which cluster your process is deployed, it will be fully transparent. And at the end, the go-sipping. So the go-sipping is a really interesting feature in NATS where you can dynamically update your cluster without any interaction. Meaning, if we want to add a region C in this setup, we would just have to put in the config of the new cluster at least one node of cluster A of cluster B and you will have a full setup which will connect to every cluster and nothing to change in cluster B or cluster A. So that's really great for us. Now, let's talk about one of our superclusters. So we have five of them, like this, and we have also other smaller cluster, but this is the main one. So we use EC2s to run on AWS. We'll dive into this later. We have a bunch of processes on Kubernetes, so EKS on AWS, and on-prem we have also processes on Kubernetes. We have some time series database and are also connected to NATS using all the clients. We have standalone processes. So we have really a lot of processes with different stack, different clients that are connected to the same infrastructure and communicate easily between the regions to build really a decentralized architecture. So if we lose, I don't know, region B, it will still be functional. So let's talk about the monitoring now. So how do we monitor everything? So one thing that is really important for us is if we have an issue on one of these core regions to be able to see what happens and continue to monitor the other one. So we decided to set up our monitoring platform outside the current production infrastructure. So for doing that, we use a Mimir, self-hosted. We deploy that into two different providers, so in two different countries. So in America, in Canada, in North America, and in Europe, in Germany. We just choose some single instances, some really high CPUs and memory with fast local SSD storage. We use also KIFU-S for doing that because we wanted to stay in the Kubernetes ecosystem because it's easier for us to deploy Mimir to maintain the lifecycle, this type of things. Et on each node, we have a local Mimir object storage. So we have great IO to write on the object storage locally, the data from Mimir. On this platform, so it's not just for NAT, we reach 5 million active metrics. So this means that we ingest around 90,000 metrics on the ingest, on the ingest controller that reach the Mimir API. And also we decided to use Mimir because maybe some of you know that this is maintained by Graphanalab. It was called Cortext a few years ago. They rebuilt that. So if Graphanalab is using that for their customer, I think it can scale really well for our use case. Now, how do we collect the data from each node's bus? So as I said previously, if you remember, we have this JSON endpoint with the NAT metrics that is available inside NATs. So for doing that, we use our custom Prometheus exporter with advanced feature that scrap the data on the JSON page to get the metrics. And then we just use Graphanalagent to scrap the data from the Prometheus exporter and push it back to the Mimir API with the Prometheus remote write features. Why we use Graphanalagent and not just a big Prometheus server that scrap the data from the Prometheus exporter is because Graphanalagent is a small Prometheus. So it's really light. It's, for our use case, it's better. So we have this agent on each node. It doesn't require to have the big Prometheus server where you need redundancy to have at least two replicas if you have a failure, this type of things. So this is really great. And for visualization and alerting, so we don't want to care about maintaining our own Graphana, okay? The guys at Graphanalagent are doing a great job. They create advanced feature that are only available on the Graphanalag platform. So we use Graphana, we see the SAS version at Graphanalagent to visualize and create some alerts. And also we use a great server that is on call. So for MNGertianert and critical alerts that can send some SMS or you can have also a bot that call you if there is an emergency on the system. And the last great feature that the guys from Graphanalagent are providing is the synthetic monitoring. So basically it's like the Prometheus black box, okay? That is officially available on GitHub. The only difference is this, with this version from Graphanalagent, you can create all the configuration on the portal, okay, on the web UI. So let's say that I want to check something. So synthetic monitoring is doing, is mostly using, used sorry for doing some prob checking. So I just have to go to the Graphana portal, say, okay, I want to pick this prob so you can set up your own prob or you can use also some prob provided by Graphana. So this is really great all around the world. You just have to say, okay, I pick this prob, I want to check the latency with this service. If it's an ICMP protocol or TCP protocol or also an advanced HTTP protocol where you need, for example, if it's an IPI token, you can set up that on the platform and you just have to click on set up and it's done, okay? And you have automatically the graph that show you what happened, the latencies or the HTTP status code if you have something in HTTP. So now a slide that some of you may not like, why we migrated out of Kubernetes. So first we still have Nats running on Kubernetes for some use case, but we decided to migrate out for the big super clusters and the trading intensive cluster. Why? First, dedicated resource because for networking Nats and latency, you need dedicated resources. We want to really have a high throughput and also don't have any CPU issues when you have shared instances and some processes may go wrong. So dedicated instances are better for that, but it's more expensive, obviously, and also maintenance is harder because you have to maintain everything by yourself. And it's not the same stack, it's not standardized. Another thing is the dependency on Kubernetes updates. So we are using EKS which has a lifecycle and a resource cycle. And for us, we have some issues with event deconnexions. So we need to have Nats bus running for a month, years. We have without any disconnections also. So when EKS forced us to update, it creates down times and we obviously don't want to do that for some kinds of clusters. So that's why also we migrated out, but it's also a harder lifecycle management because you have to do everything by yourself. The end and for tuning, it's also simple and working because you don't have the Kubernetes overhead, which is small admit, but it's still there. It's easier to identify Nats traffic because you have dedicated instances so you can directly know what traffic is going out and in. And one thing that was really difficult for us out here at some point, what the load balancer. We use ingresses and load balancer in Kubernetes to expose Nats, but we have, there is really intelligent features in Nats to optimize routing also. When you have disconnection, try to find another node, which is still up. And the load balancer was adding more complexity to that. So we decided to get rid of the load balancer and it works really well by itself. And last point, which is also more difficult, is the configuration management because we had to create custom tooling and see ball terraform and all that stuff to ensure that you have the same configuration all time in all of your nodes and secret rotations are also a bit harder. Next, the challenge we faced. So it's more about tuning and optimizations. We've been running Nats for three or four years now. We started small, we built on Nats, there was updates, there are many features that are coming and updating. First one I just spoke about, but it's the reliable networking. We kind of a downtime, even the connection is considered downtime for us. So we wanted to have something that can run for years without having to touch it. So that's why we migrated to the new infrastructure, which is totally standalone, on VMs on premise and on EC2s on AWS. And we only rely on Nats for routing optimizations and everything related to network. Basically, it's running on Docker and we have host networking. So we really try to have the simpler networking possible. Second point was scalability. We started with only one cluster, then we expanded to a new region, we are expanding to a third region and now we will expand to a fourth region soon. So we really want to have something scalable where you can dynamically grow without downtime. For that, some of you may know, you cannot out reload the routes and the gateway configurations. So you have to reload the Nats binary when you want to update those configs. So it's kind of disruptive for us. What we did, we extensively used the go-sipping system where you can dynamically grow your cluster, as I said before, where you just put an endpoint of the running Nats and it will connect all together and we use the mandatory downtime that we create to update Nats and to basically do hardware maintenance to also put the new cluster inside the configurations. So we do two thing at once. Third thing was unified authentication and authorization. We started with nothing, then we added basic authorization, sorry. And at some point when we used the super cluster, we also had to a unified authentication between all the Nats. And that was something really difficult to handle because when you handle authentication at the cluster level, regional cluster level, it's fine. You can enable it quite easily and it will work even if remote cluster doesn't have authentication. Only if you use the default account. But that was the case. We use the default accounts, so the dology. But when you run in super clusters and you want to enable new accounts, for example, in our case, we wanted to use the system accounts and kind of separate all those things logically because we have multiple clusters physically separated on EC2s but we also want to use the powerful logical separation in Nats and to enable that, we had to create a downtime because you cannot enable authorization at the super cluster level without bringing everything down and bringing everything back up.  So another feature also for authentication is the no-chooser. I don't know if some of you have used it but it's really great because we have a mix of authenticated clients due to legacy software and legacy stuff that we have. And the no-chooser allows you to give a default set of permissions that will be used in case of a client that will just not provide authentication. So it's really enabled a smooth transition from non authenticated to authenticated clients. And at the end, so the global overview, we have more than 100 nodes, we are keep expending this. So I think the metrics is great but we also wanted to have a better way of visualizing the whole infrastructure because it's really critical for us is the core of our business. If NATS is even down for a few seconds, we can lose a lot of money. So we need to know at all time what happens and we need to know at all time that NATS, if it's failing, we need to know before the business, basically. And quickly identify bottlenecks on the infrastructure because as we rely only on NATS to decide by itself to load balance traffic and everything, we need to know if at some point the node is overcrowded or has too many traffic on it. And for that, we build custom tools. So we forked the Prometheus exporter, so the official one, to add more metrics and compute everything. And we created a new web dashboard, simple dashboard to have a better observability on NATS. And I will present it quickly. So it's not a dynamic because I could not replicate the whole infrastructure without leaking some information. So it's only screenshots, I'm sorry. But basically, it's more web UI with a Python backend, some TypeScript and React for the front. And obviously, it's running on Kubernetes because we can have that time. This is not an issue. We use extensively NATS monitoring endpoints to the rest endpoints that are built in. But we want to switch to the system account and use basically NATS to monitor NATS. It's used for the infra team and the developers. At infra, basically, we use it to monitor the infra. And the developers are using it to know at all point in their processes. Where are they connected to? What are the subscriptions? And the number of messages also that are going through. You have all of this in metrics, but you can have a, I would say, visual way on the list, basically. It's easier for them. So yeah, that's it for the NATS map. So we built a graph. So it's one of our clusters. We can see all three dots, our regional cluster, that are connected to the others. Whoops. Okay, great. So yeah, this is not dynamic on the screen, but if you click on the node, basically, you'll have a list and an overview of the other nodes that are connected. It's really, really great for us when we need to do some maintenance because we can know beforehand which nodes are connected to this one. And when we reboot, we can know dynamically on which node it really reconnect because it's not something that we can predict. So that's really great on this part to really identify this. Oh, just one thing. What you don't see here is that you can zoom out and zoom in also to see, for example, share what happened between these three local bus. So it's really, really cool. Yeah, this is really a big overview, but you can have more details on this. And basically, this one is just a list, but you have really many, many details on the details space that I cannot present today. But it's really useful for us to have an overview of all of the nodes, all of our buses and everything running there. So it's really, really simple, but it does a job right now. So now let's see real conclusions. Yes, to conclude, we are really happy with this current architecture. It's really scalable. We think that nets fit well with our use case. So we will continue to use that. The guys from China, they are currently including many great features in the next release. So feel free to check the last release notes. I think the new version is coming in a few weeks. And yes, we will try to expand that and keep continuing working on this infrastructure and add more also a density feature into our nets map. It's not open sourced for the moment. Maybe one day we will push it on GitHub. I don't know. I'm not sure I want everyone to look at the code. But yeah, we will see. That's it. Any question, maybe? Hi, I'm just curious why you chose to go stateless and in memory and not on disk. Wouldn't that be safer? Yeah. First performance. And because, you know, the persistences in our nets is available now with JetStream. Before that, it was not really available. So when we decided to build this architecture, what we wanted to have is just full performance. And because we have three replicas on each region, we don't really need persistency because if a failure is happening, we still have one bus, for example, we still have the two other ones that are running. So it's why we decided to go stateless for performance and because JetStream was not available before. And basically for the training processes, it needs to go fast. Like we have reconciliation built in the trading platform, but it needs to go fast. Even if the message is not delivered, it will be delivered again, but it needs to go fast and we don't want to have writing to disk for that. No, I think we will stay stateless. We will continue to stay stateless. It's much better for us because it's a really specific trading use case, you know. On other services, on other at XBTO, we use persistences. We start using JetStream. We still have one old Kafka cluster for another project, but maybe we will replace Kafka with JetStream in the future for this project. Yeah, it's mainly for the trading part where we don't have JetStream, but we use Nats and JetStream for other specific use cases. Do you have some latency matrix within your super cluster? Between each region that created the super cluster. So yeah, so currently between Europe and Japan, we have around 200 milliseconds, if I remember correctly. Yeah. Yeah, currently we're working on a new project to increase that, to lower that. We will use a new global provider network, a private one, so we will achieve around 100 milliseconds between the Europe and Japan. And between the US and UK, I don't remember, it's around 80. Yeah, something like that. Yeah, something like that, 80. But you know, between the US and Japan, what's great is that we have the direct connect link. So we use the internal backbone of AWS to rotate everything. Why we did that, it's because when you're in the AWS backbone, it's really stable. So you can reach all AWS region. We've not the best latency available on the market, but it's really stable and we don't have any issue. Yeah, and that's from the infrastructure part, but where the processes are, the most latency sensitive, usually they use the same region. We put everything in the same region and so regional clusters are really latency sensitive, but super clusters. Basically, it's okay to have a bit less, a bit more latency, sorry. When we need really, really latency sensitive processes, it's directly on the same host and really optimizing in another way. Hello, thanks for the talk, it was really great. I got two questions. Do you do a performance test on your NATS infrastructure? And if you do, do you recommend any tools that helped you so? And I wanted to know as well, how did you handle back pressure problem on your NATS infrastructure? Thank you. Thank you for the question. Basically, we have two setups. So we presented the prod one, but we have a fully similar setup for testing environment. So it's exactly the same VMs EC2s. So it's expensive, it's less used, but we do pressure tests on this one. Usually what we do is, we have all the same stuff running, obviously for testing, but at some point we just generate more and more messages to test beforehand if the NATS infrastructure will, well, we'll endow the new load, I would say. And when we do migrations, we test everything on UAT, we try all type of scenarios to really test if our clients are really able to reconcile everything. And that's also one feature that will be really great, normally in the next release, is the lame duck. Lame duck mode is a signal that is sent from NATS to your clients to say, hey, I'm stopping in two minutes. So do what you have to do, but I'm stopping and you should know. That's something that we're really working on to add this into our processes so we can directly reconnect to another node if we have an issue. And, you know, I would say later, we also would like to measure latency between the client and NATS and if we see that it's too high, try to connect to another node directly to reduce the load. Yeah, you have NATS box, that's a great tool that allow you also to do some benchmarks provided in NATS. So you can use that also to test your performance and see how many messages you can reach, everything, so that's really, really cool. Hi, couple questions about your Kubernetes setup. You're using the RKI-1, probably, and do you manage that by Rancher server and how big is the cluster? This is a bare metal node or VM and last question, why you decide to migrate to Talos? So, as you know, Rancher was acquired by SUSE since more than one year, two years now. And, first, we are just using the Rancher manager to be able to give access to our developer with the project features from Rancher. This is really great, it's basically a Kubernetes proxy that allow your user to connect to any cluster managed by the Rancher manager easily. Also, we were not using GitOps two years ago, okay, we were deploying our cluster by hand, set up in the end chart, for example, Promet to use the ingress controller by hand, so it was a pain. And what we wanted at that time is to move to the GitOps methodology. I love Flux, for example, but Rancher was providing a tool that was called Flit. And it's why also, when we explore the market, we told us, okay, we have this Rancher multi cluster manager that is great. They integrate now a component like Flux inside the Rancher manager, so we will use Flit. It was a big nightmare, to be honest. Because Flit is not like Flux, the product was not really stable, they were updating major release, so it was breaking many things on the Kubernetes cluster managed by Rancher. So we decided to stop that. Now we want to reduce our footprint with Rancher. And to be honest, we discover TALOS now last years. I think it's a mature technology now. What's great is that, as you know, you don't, it's a light distribution, okay? You don't need anymore an SSH on it, so this means that for security purpose, it's great. You don't have to take care about, for example, baiting Ubuntu each month because you have some security fix. And also, you can deploy your cluster easily just with the TALOS command line in one second. Same for maintaining the life cycle, upgrading your cluster, adding more nodes. So to be honest, I think, for the on-prem part, moving to TALOS will be a great benefit for us. And it will be easier for us to manage our Kubernetes in the future with just the TALOS CLI. And just to add on that, maybe also the life cycle because on EKS, you're always at the, almost the latest version of Kubernetes. And on Rancher, it's a bit more difficult to have the latest. So sometimes we have differences in a version of Kubernetes for the same projects. So that's also one of the points for moving away from Rancher. But we are really happy also with the project. It was working well and still working well. All right, no more question. So thanks a lot everybody. You have a QR code on the last slide. So if you want to give us some feedback, it's really cool. Feel free to do it. Thanks.


Greetings, and welcome to Books for Busy Students. I'm Chloe, one of your librarians here at the Ray Howard Library. It has been over 10 years now since the disastrous events of Hurricane Katrina unfolded in New Orleans and the surrounding areas, but its echoes are still being felt. Drowned City, Hurricane Katrina and New Orleans by Don Brown provides a really great introduction to the timeline of important events in a way that feels urgent and personal. Brown has a wonderful way of illustrating the movement and the chaos of the storm. His style is slightly abstract, but it's personal at the same time. He doesn't go too in-depth about any one aspect of the disaster, but the book has a really robust bibliography, be still my librarian heart. I would especially recommend this book to students who are studying Katrina as part of a course and could use a little help understanding the timeline. The book makes it really clear and visual. Drowned City is available now at Ray Howard Library. Thanks for watching.


How about some good news for BSU men's hockey today the Ontario reign of the AHL part of the Kings organization signed Beaver senior forward Kyle Baumann for the rest of the season he posted a career high 30 points this year 15 assists and nine goals you could debut as early as Friday night if you enjoyed this segment of Lakeland news please consider making a tax deductible contribution to Lakeland PBS


So, oh, I saw there was another person. No, okay, fine. So want to know a little bit about you. Who would describe themselves as an academic? Academics in here. Oh, great. Who works in higher education? That's not because I don't like academics, by the way. Thank you. Well, can we just have in a little chat before we start? You're not late, we're just having a chat. So who would describe, who works in higher education? H.E. Okay. Who works in something that's really different from H.E. Fabulous. What kind of work do you do? Oh, really? Okay, wonderful. Thank you. Okay. What else was I going to ask? Right, so who has English as a second language? It's not their first, their native language. Oh, that's really interesting. Where are you from, sir? You're German. And do you know, funnily enough, you won't believe this, right? So am I. But I barely speak a word. And there's a story behind that, but I won't go on about it now. Yeah, I understand that. Sorry, a long time. Okay. Who's left-handed?  Is there any one right? Okay. That's very unusual. It should be, is it 17% or something? Not nil. Okay.  Why am I asking these questions? We're kind of to get to know you, but also because, forgive me, but in a way you're my kind of learners, aren't you? Because if you leave this session as the same as you came in, then you've wasted your time, other than the fact that it's cool in here. Okay. So in a way, what I've got to do is understand you as people and individuals and learners, haven't I? I've got to do just a bit of that. Now, technology is really good at that. Yeah. And lectures, workshops, are not quite as good, are they?   If I make a cultural reference to something in the UK, I've got to look it into, oh, right, okay, that's interesting. Do you see what I mean? There's an individual element to each of you, that this kind of session is not great at. Technology is brilliant at it. Okay. So what is this session about? It's my second of the day, so I might get confused, bear with me. I don't believe the hype making informed decision about learning, learning technologies in a context of constant change. This is the sequence of the session. I start by talking about hype, and I will explain why it's important. I talk about constant change, but the number this session is about informed decisions. Okay, that's what it's about. Actually, just, I've learned a tiny bit about you, I'll just do a little bit about UFI, then a bit about me. We are a charity, charitable trust. We believe that equitable access to skills enabled by technology is vital to a thriving economy. We tend to focus, not entirely, we focus on learners that are poorly served by traditional mainstream provision, and we believe technology works in terms of providing skills, okay? We have this term vocational plus, vocation skills plus technology, and it's called VokTech. That's why we're the UFI VokTech Trust. The term, the acronym UFI has a long and strange history, but I'm not gonna bore you with it. We do three things. We provide grant funding, we invest in organizations, and we advocate through partners, and I've skipped a really important part, which is that we are a key strategic partner of the Association for Learning Technologies, which is partly why I'm here, why we're here. Maareen, my colleague is also here. I just thought I'd tell you a tiny little bit about me. I've been doing this job strange enough for 40 years, as of the end of this year. I did my first learning technology product in 1983, and I flip between very pragmatic, vocational training, corporate work and so on to academia. I like doing master's degrees. I've got an MBA from this very institution from about 35 years ago. I've got an MSc in network learning, and very oddly, an MMS in creative music technology. The only reason I'm saying this, it's not about sort of bragging, I was saying how clever I am, it's about saying that this presentation is gonna be right down the middle of those two, hopefully. I am very much on the pragmatic side, but I support my pragmatic position through what I hope is robust research and kind of effective academic practice. So, I'm talking about decision-making, but I'll intro these other two elements, hype and change. As I say, I will be consulting my own notes, simply to remind myself what I've learned over the last day or so. I just wanna do some terminology. I don't know if you read the abstract, the description, but what I'm doing is I'm using virtual reality as an exemplar of some of the decision points and elements of hype and so on that I'm gonna be talking about. This isn't primarily a session about virtual reality, okay? But I just thought we needed to sort out some terminology. Could anyone just summarize, sorry, one more thing. I can't possibly talk for an hour. You are gonna have to contribute, I hope that's okay. I want to make this as kind of workshoppy as possible. So, my first interaction, could someone just explain the difference between those three? Very briefly, yeah. There's a spectrum of reality for the immersive experience, which is the real-world present AR and the other side is minimal overlay, which might be the space seven things and then MR. Mixes the two. So, I'll just try and summarize. Virtual reality should be just, you must correct me if I'm wrong, okay? Virtual reality should be all about complete immersion, creating a simulated world in which one is immersed, yeah? Augmented reality overlays information onto the real-world, okay? And mixed reality is kind of, it shouldn't be called merged reality, should it? It should be called mixed reality. Yeah, it's mixed reality. I don't, this graphic is not one of mine. So, mixed reality kind of merges the two, okay? And the reason that it's really worth doing this is because I've had projects, I'm not gonna say at UFI. I have had projects which have been funded and started and planned and in which the main parties didn't know the difference, right? So, it's just worth being clear about that. In other words, they just, they said they were doing an MR and it was a VR. Anyway, oh, forget that one. Right, don't believe the hype. 1990s theme tonight. 1990s theme, eh? Isn't that great? I didn't have a costume until costume. I didn't have the light get up until this morning. Okay, and by the way, I don't endorse public enemies positions on all sorts of things. So, why am I talking about hype? Well, we're trying to sort out decision-making, rational decision-making. The problem with hype is that it's blurry. It interferes with the objectivity. And what we're talking about is extravagant or intensive publicity or promotion. And I promise you, it is relevant. And the reason it's relevant is because of this. Suits come across as the Gartner hype curve, just indicate in some way, and I've seen a couple of people write, okay, it's really, really important. And just, you know, the basic principle is that many things, not just technology, they have this very distinct curve where they appear and then sort of the media gets hold of them and all of a sudden we have extraordinary expectations that, you know, these things, whatever they are going to solve the world's problems. And then things crash because we realize they don't and little by little you get a slope of enlightenment and people start adopting them. And the reason I think this is such a great tool is because Gartner and others have actually broken it down. So these things are clues. Isn't that great? So you, for example, the point at which, let's find the zapper again, the point at which you start seeing negative press, you can say, oh, you think, oh, okay, we're at the top of the curve here. Yeah. When you start seeing loads of suppliers appear on the market, you know, you get shows and all of a sudden, everyone is going, I'll tell you what, virtual reality authoring tools, they're the biz, right? And you didn't see them the year before. You go, hmm, I think we're there. You start seeing failures, you start seeing, let's have a look. I like this figure, 5% of the potential audiences adopted the fully. In other words, despite all the promises, no one's using it, right? And then you get the sort of curve which goes up. So it's a really good tool, isn't it? And the reason it's a good tool is because it will affect your decision-making. So what Gartner and others, that they've applied it to all sorts of different technologies, the digital marketing, oh, look, AI, how relevant is that? Generative, do we agree? Is it? Let's think about it a minute. Do you think we're right at the top? Could it get any worse? Tell me, are we going? Sorry. Okay, so just for the folks streaming and recording, I'm so sorry, I forgot your name. Sue from disk has just said, we don't think we're maybe not at the top of the high curve yet. I think in my opinion, is that what's so fascinating about generative AI, what's so fascinating, like the next 5,000 things, are that it's got up that curve a long way, whether it's at the top or not, but really quickly, really quickly, the fastest ever, like if we'd had this conversation a year ago, we'd go, oh yeah, isn't AI cool, right? And now it's splattered all over the mainstream media, isn't it? Okay, and so I'm sorry, I'm kind of pointing at that one. And there is, there's a little bit of audience over here as well. Okay, let's just go on one more. Okay, there's all sorts of technologies on there that I don't really understand, but it's the same point. It's a really, really excellent way of helping out, helping you understand the messaging you're being hit with. And what we've learned over decades is that you will be hit by all sorts of complex messaging, right? By experts, by idiots, by media, by colleagues, by colleagues who are idiots, and so on, right? So I just wanna, oh, look at this, this is interesting, augmented reality down here. Do we agree? Where do we think augmented reality is? Now, why do we agree? I think augmented reality is kind of the signal of our third reality. Okay. Obviously, augmented reality is still there, but the height of the video, whether it's real or not accessible. Oh, that's interesting. Just create augmented reality, and that sort of thing. Yeah, yeah, that's right, yeah. That whole view of augmented reality. That's interesting. I'm not entirely sure I agree. I reckon augmented reality is kind of maybe down here, or on the way up a bit. It's becoming a bit commoditized, actually. Someone came, yeah, sorry, Mato. Yes, it will, soon. Yeah, and I confess what we were just hearing there from Mato is that, and I didn't know it was there. I'm so sorry, I feel terribly ignorant. You can get an AR overlay, what, on the screen, on the car screen, right?  I can feel a tangent coming on, but I have to resist. The tangent basically being, isn't that an amazing form of performance support? Why on earth do I need to learn how to read maps? Why do I need to know all sorts of things that I used to have to learn if I can see them all in real time? Anyway, I could go on for hours about that. So where are we, for example, I'm just gonna do a few examples. Where are we with online retailing, do you think? See? Are we on the road? Yeah, well, of course we are. It's completely commoditised, isn't it? AI-generated graphics, specifically graphics. Dali, mid-journey, that lot. Where are we on that? Somewhere near the peak? I think so. Anywhere before the peak, maybe? I think we're not on the peak, aren't we? Self-driving taxis. We're towards that, and I've got to tell you a funny story about this one. I know this is a tangent. Has anyone seen that fantastic YouTube video where there's a self-driving prototype car in San Francisco, which is stopped by the police and tries to get away? Isn't that great? And then another cop car comes in from the corner and stops it, isn't that amazing? Anyway, and the best bit is that the police get up and they say, hey, guys, there's no one in it. Anyway, learning management systems fairly obviously, way, way, way up the curve, okay? Where is the VR? I'm gonna be talking a little bit about VR. Where is VR on this? Somewhere up here, maybe? You said that it was taken over by AR, do you think? Sorry, it was taken over from AR? It was taken over from AR. Yeah, I think so. Yeah, I think so. I'll be talking a bit later as to why I think it's on the way up. So I reckon it's somewhere there. Well, maybe roughly. Anyone radically disagree? Good. Please do disagree on anything. Okay, so just to reiterate, why we're talking about the hype cycle because you need to be able to absorb messages, you need to be able to understand messages clearly. We live in such a complex world, which is what I'm talking about, excuse me. Okay, change. I love this quote, things in this world are in so constant a flux that nothing remains long in the same state. And obviously I'm being cheeky because that was actually written in the 17th century. Now, the kind of change we're going through at the moment is kind of spectacular. And I was saying to the little session this morning to Lynn others that we're not in a period of transition. We are not. We're in a period of total constant change for the rest of humanity. It's it now, change all the time. We're not transitioning from one state to the next other than stable one to an unstable one. I quite like this curve. It's much misused. It was, you may or may not have seen it before. It's the Kubler-Ross curve and it was originally developed for, it's something therapeutic use is handling of grief. It is occasionally misused in terms of change management, particularly in the corporate sector. But the reason I think it's useful is because of that. So we're not in that situation anymore where one is dealing with whatever the truth is, whether you manage change through denial and shock and anger and depression, whatever, whether or not it's true, actually that's what's happening. So just as we accommodate a particular type of transformation or change, 15 others arrive. And then we realize that we didn't understand the first one and then so on and so on and so on. So the kind of change we're going through, or in other words, the kind of change we're going to have to cope with when we make technology decisions, that change is now constant. And yeah, okay. So how do we deal with it? Well, there's this rather neat model, which I was amazed was actually developed in the 1960s by a chap called Francis Aguilar. It's amazing what you learn at conferences, isn't it? And I have occasionally used it to help very complex jobs, really problematic clients break down the nature of their change. It looks so obvious, doesn't it? It looks just such common sense. But if you are in a situation, I used to do change management for large corporations and almost inevitably, the technology folks would steam in and say, this is what you got to do. Because there's all sorts of emerging technologies, maturing technologies, copyright plans, etc. And you get this focus on that there. And there's all sorts of other things going on as well. The important thing about this is that all these factors affect what people need to learn and how they're going to learn it. They all affect it. Clearly, you're not going to use this model every time you get a new mobile phone, are you?  But very large transformations, big installations, strategic changes around the technologies you're going to use to support your whole learning infrastructure. Do you know it really helps? And actually, on a couple of jobs, I've done one of those where you actually identify what's important and what's unimportant. So I've been a bit mischievous there in that I've deliberately put the technology bit very low and the environmental bit really very high. You've got to understand, all right? If you're doing large-scale technological transformation, it really, really helps. I learned a lot at this conference. This page is almost unintelligible because of all of my notes. So at UFI, we think a lot about change. We're mainly in the vocational space. What we've, the kind of things we look at, we tend to produce a lot of reports, documents, this kind of thing. We just came up with this thing, a white paper on skills for an economy and transition. It's very interesting and it looks at the key changes that are going on in the vocational workspace. This is a fantastic publication. There's a really good couple of chapters at the back, which are about AI. The trouble is that AI changes so quickly that by the time you've printed something, you're not up to date anymore. But anyway, it's a good document. Now, I know this has got a slightly grim title, but one of our other partners, the resolution foundation, is they have an ongoing inquiry called the economy 2013 inquiry. And this particular document is excellent at providing a model that structures all the various changes that are going on in the workplace. So the reason I've introduced those is just to kind of introduce this idea that we think about, we look at the nature of the changes that are going on. And we're in a very fortuitous situation, a very privileged situation at UFI, because if you think about what we do, right, we give money to people, we give grants to people. Therefore, surprise, surprise, a lot of people like us, don't they? They come to us, they say, can we have some money, right? So we get a lot of people coming along and saying, this is what our problems are, right? These are the technology solutions we're thinking of using. These are the things which bother us, right? So we're constantly kind of getting free market research, yeah, because we see what people are interested in, what bugs them, what worries them. And yeah, like I say, it's a privileged position to be in. So I just wanted to ask you, and this is the most open of open questions. What kind of things are you experiencing in your lives and works, that your work, that you would regard as really significant changes, things that are really affecting you? Any thoughts or comments? You're gone. Okay, well, they have a high... A high, yeah, of course. So I come from a education and employability background, relate to a lot of the things that were talked about in the workplace. And you have to be hit of the hiring... Yeah, yeah, the curve, yeah. ...in the tech industry. And roughly at the same time, the AI, I came up. And I've literally seen people in running businesses speaking out loud, how can we boost this in order to cut people? Yeah, of course, of course, yeah. Going through this experience as we speak, seeing that basically, that goes up even before the technology is actually there. So we're somewhere in this kind of falling curve. Because decisions are being made in businesses based on the speculation of what the hype will basically bring in terms of business benefit, which in a year's time, not two years' time, so not to be... You're fake. So I think, correct me if I'm wrong, I think just to summarize what you said, so the change that you're experiencing is around poor decision-making and employment issues. What else is going on? Anyone else? Yeah, go on. Cost of living crisis, for goodness' sake. What else? Anyone? Lynn? When you speak, do you just use the intent of the person? Really interesting. Oh, pardon me. Choice, and I think those that are not using tech are quite right if they're a lot more overwhelmed and where do I start? Okay, so just to summarize again, a widening gap between those in FE that use tech and those that don't. Great, okay. Anything else?  I'll chuck up a few ideas. These are just mine, actually. I did ask around UFI, various colleagues. So the kind of things that just popped up spontaneously, this was the widest, most open of questions, but this is an interesting one, particularly if you've got teenage kids. Concentration spans, destroying learning or expanding flexibility. There's quite an argument to say that concentration spans declining actually means that you can jump from one thing to another very quickly. There's also some utterly rubbish data going around about it. If you come across this stat which says we used to concentrate for 12 seconds now, we only do eight seconds, it was made up. It's all over the place. But let's have a look. World Economic Forum has come up with some interesting stuff around how much people read. If you compare 1984, it would be 1984, wouldn't it? With 2021, young people read half as much as they did or spend half as much time as they did, which is interesting. So concentration spans a ruble of distance. This is a great one. So I do quite a little bit of work in prisons. And I came across a job just yesterday before that was talking about the removal of distance.  It's pretty tough to train you, okay? And then it's really tough to employ you because you're not safe, right? So I came across this job where basically we're using digital twins to train dangerous prisoners to do skills outside, all right? Complex skills, engineering skills. And then here's the really cool bit, right? They can actually do those jobs. They can actually do them using digital twins and using replicated technology outside the institution, right? So we're training them and giving them real jobs, removal of distance, micro accreditation. Sorry to be controversial. Yeah, yeah. Or, or, because if you're a, if you've committed a very serious crime, you may never be at full liberty. You may leave the secure institution, but you may live in a set, you might have a tag or whatever. Therefore, you're not gonna be on the building site, right? That's the wrong context. You're not gonna be in an engineering company, but you can still do the job. Isn't that great? Removal of different micro accreditation. Do we need big qualifications in curricula? I was just being mischievous with Lynn at lunchtime. You know, if we, if we do micro accreditation and you build up competence in little bits, do we need think, do we need universities? I don't know. And we've already mentioned, I mean, AI is everywhere. It's like bindweed or something.    Okay. The UFI perspective, so we see a lot of, because like I say, we're in such a privileged position. People come to us and they tell us what's going on out there. Greening economy, remote learning, post pandemic staff shortages. That's an interesting one. AI, of course, key industry skills shortages and system fragmentation. The white paper I mentioned is, it very much addresses this issue that the learning and training system is very fragmented. Okay. I'm just gonna have another quick drink of water. We can skip that. I'm very keen to keep time. Excuse me. Oh, actually, no, I will ask one of those questions. A lot of the discourse about technology is around coping. And I don't like that. Yeah. This is not about coping. It's about baking. It's about, I'm so tired. It's about making the best of things, isn't it? How do you make the best of what's happening? All right. So, forgive me, we are gonna have to talk about, oh, let's not talk about it. Yeah, let's talk about VR. All right. So, VR is changing very, sorry? Brilliant. Brilliant. Okay. So what I would say is that's making the best of it, isn't it? Making the best of the situation. Yeah. Brilliant. Do you know what? I'm gonna ask you to repeat that so I can summarise it correctly. Just do it again. Oh, don't worry. Then... That's accurate. No. It's about signing what you want, how best you can do that and then using the tool to pay it off. Of course. Thank you. Thank you for repeating that because what you've actually done, I was a bit mischievous there, forgive me. You've actually given an intro to my next section. So, thank you. I'm gonna go on to decisions in a minute but the kind of key message behind this first part or the second part is that if you don't understand the nature of the change and its impact on you, you're less likely to make good decisions and more likely to be distracted by the hype, okay? So, section two relates to section one. Understand the change you're going through. Right, decisions. And just a reminder here, the decisions you are making are there to cultivate learning, to generate. That's the decision we're making. We're not making a decision, thank you to get cool technology or whatever, we're there to cultivate learning. Okay, just a trivial exercise. How do we make decisions about where we go on holiday? Who's recently been on holiday? Come on, it's the end of the summer. Marine, how did you make a decision? You had to go to a wedding. I asked you, I shouldn't have asked you, should I? How do you make a decision? Is it somewhere? And then it depends on what you've been talking earlier about doing so, it's family that you're there rather than a person who tries to buy some on holiday. Yeah. Which is the other thing that's popular. Yeah, and it's really interesting, your sort of body language and the way that you're explaining it is, it's not a big deal, we just wanna have some fun, you know, that's fine. It's probably not the way you make decisions if you're gonna spend 20 million quid on an IT system. It's probably not the kind of decision, it's not the attitude you take if you're gonna say, let's invest a lot of money in virtual reality or we've got a serious problem that we need to address using this. That's not the way you do it, is it? What did I have for lunch? How did I make the decision? Anyone, what did you have for lunch? I just walked down the aisle and I thought that looks nice or I was really late as well. So I just, well, I picked what was left, to be honest. Right, buying a house, how do you make decisions about buying a house? You do a survey? You do a survey, what else? Do you have a sort of criteria list? Do you have a, sorry? How does it feel when you walk in? So there's a very subjective bit and how did you decide on it? I decided, so this and the real estate agent actually walked around with the camera so we had a thing of a little bit of space. Okay, so there's subjective elements. Well, you're very brave because of course in this bait and switch, I like this because in this wonderful era of genitive AI, it could have been the faker state agent and the non-aggressive house, couldn't it? So what I'm getting at, it's so obvious, forgive me. There are many, many models of decision-making. Of course there are, but it just helps to reflect on what are you doing? What's the element of objectivity and subjectivity here? Forgive me, I have a very intriguing friend who makes decisions about lunch on a spreadsheet, analyzing whether there's enough protein in it and carbohydrate and all that kind of thing, right? That person is down that end. Do you know what their job is? They're a data scientist. Okay, so it just helps to be aware of where you are and what you're doing and how you move up and down. Oh my goodness, have I offended any data scientists? Sorry. Sorry. Okay, cheers. So I'm so sorry. Okay, now, let's just get serious just for a second. Forgive me, I'm just gonna check the time. Okay, we're doing all right. Okay, let's just get serious for a minute. You're investing in some technology, but let's remove the word investing. You're deciding on a piece of technology in order to cultivate learning of some sort and I'm gonna talk more about what that means in a minute. It really helps to have some kind of criteria list, doesn't it? I think this is Microsoft, but it doesn't matter, you'll find lots of these around. Technical fit, maturity, time scale, all this kind of thing. Yeah, I would have thought so, yes, I think so, yes. Actually, anyone else want to criticize this list? I think that I think, no, no, no, no, that's really important. What else is missing? In advance and some might emerge from looking around like in the house, right?  Of course, of course, that's so interesting. That's such a good point. So just to summarize again, you start making decisions, you start with a list of criteria if you're a particular objective, but as you find options, other criteria emerge. That's really interesting. And that relates very strongly, very powerfully, doesn't it? To this idea that technology changing all the time, because if you came up with a criteria list three months ago, it might be different now, yeah? And also this relates to the idea that you cannot be an expert in everything that you're looking at. Therefore, the thing that you're looking at will tell you more. Now, I just want to talk about requirements, because of course requirements in our context, the requirement is that we help people learn. That's the key thing. So let's just forget about all this techy stuff and talk about learning. Forget that bit. So I have a couple of favorite phrases, and forgive me, I've forgotten who came up with them. I think they're both colleagues, ex-colleagues. I've been doing this long time. I want to go right back to first principles, right back to first principles. So I like this phrase, you can change without learning, but you can't learn without changing. I like that. And if your learners haven't changed, they're not learners. Now, of course, we're in a context where the word student is used a lot. So if your students haven't changed, they're not students. I don't know if it works, does it? Because of course, you can have students that don't learn anything. Anyway, it doesn't happen around here, of course. So the reason that I am going right back to sort of almost infantile basics is that learning is about change, obviously. So when you think about, I'm going to go back one. When you think about learning, you have to think about change, because it's the change you want that determines the technology that you use. I've got time for this little exercise. Yes, we have actually, good, right. Good, I'm sorry, it's just, I, yeah. Now I'm aware, thank you. I'm just, I was going to do a cultural cliche thing but I'm not going to now. What do these four characters have in common? Okay, so I will just explain who they are. We know who Mahatma Gandhi is. Erling Harlan plays Manchester City.  He scores lots of goals. We know who von Berger's. We know who Emily and Pam Kirk's is. There's something, there's something. Okay, there's something to do with activism, maybe. What am I talking about? I'm talking about change. I'm sorry if I'm treating you like idiots. I'm so sorry. They've all disrupted a paradigm. They had more than that. They had a theory of change. They had a theory of change. One of the words, so you go on. The what? I'll tell you, right. So each of them has or is involved in a very clear theory of change. One of the words, they are involved in this idea that there was an outcome was required. They knew what the change was and therefore there was a process that caused the change to happen. That's what learning is, okay? Just going back to explain. I'll start with the footballer because this is the one that always kind of makes people what are you doing Patrick? So Manchester City, they needed someone to score more goals. They wanted to win the Champions League, which they did and they partly did it because he scored more goals. It was a theory of change. We need a number nine to score more goals. Gandhi, his history basically, he realized that the Indian population could overwhelm the British Empire. So he picked on the salt tax, which everyone was upset by and literally got a million people to just show the British that they didn't stand a chance based on their opposition to the salt tax. Tax. Thunberg clearly, a very interesting character, a theory of change was kind of naive persona, aggressive, clear. I will create a media presence that influences this particular issue. I would actually question whether it's an effective theory of change. Emmeline Pankhurst, what was hers? This is a really interesting one. What was her theory of change? Women's suffrage, that's the end goal. But how did she disobedience? And actually, here's an interesting one, disobedience and violence. Yeah, so she felt that actually she could raise the profile of women's suffrage. Yes, no, yes, you're right, but I think he does. I'm going to defend myself. And I did say contradict me, didn't I? I did say contradict me. So, you know, I'm not going to take offense. But what I'm saying is he's involved in a clear theory of change, all right? I'm really not a fan of this whole thing. I know I've come over a bit sort of blokey about all this, and maybe I shouldn't have used him. But he is part of a theory of change, yeah? Is it the coincidence, the coincidence that three of your four change magas have been disobedient? Yeah, yeah, it's a good point. Yeah, it's a good point. Something that I'm very intrigued by is this notion that, you know, for goodness sake, this is really obvious.  They tend to be awkward buggers, yeah? Don't know. And it's really interesting that you spotted that. Incidentally, the last time I did this, Marine, I think you saw this, I made the catastrophic error that I put three men up. Isn't that awful? And I must apologize, even though you weren't there. Because I said, oh, what have all three got in common? And my boss said, they're all men. Anyway, I'm so sorry I went banging on about this, but it's really important because I have seen so many projects where people have said, for example, these are quotes from real projects, our learners will learn because we're using games. So where's the theory of change, right? And there's another one, we'll give them the information to do their jobs better. Yeah, it's not a theory of change. Those are methods, aren't they? So your technology selection's got to support your theory of change. And incidentally, UFI's got one, really complicated. But basically what it is, I'm obviously not gonna go through it, but the idea is that we underpin a vocational skills system by the very best digital tools, pedagogy and learning technology, and the outcome is around the UK economy thriving. So each of these components is part of our theory of change. And it's a really interesting document, and you can read it here. Okay, again, I'm staying with really, really basic first principles. So learning, what do your learners like now? Start point, how will they be different in the future? End point. So that's what designing learning is all about. Okay, so what is the change? You want these people to be involved in? What experience will create this change, okay? What's your learning strategy? Now, I'm very aware that I'm using the term learning strategy in perhaps a different way than it's used sometimes in higher education. What I'm talking about is the big idea. What's the big idea? What's the strategic thing that ties your learning together? What's the learning strategy? And it's only at that point that you start thinking about technology and then technology about learning strategy. You get this kind of iterative thing going on. Now, I was talking, this is one of the things I wanted to change in my presentation. I was talking to a lovely chap from Durham just before lunch called Paul Finley. And he was brilliant at talking about the way that your understanding of technology options... I can get this right. What he was saying is you need to constantly scan forward and make sure that when you're looking at technology options, you think not just now, but three months or six months, you think a lot about sustainability and development and where things are gonna go if you possibly can. And it really helps if you get experts to help you with that. So this kind of iterative thing can be very tiresome these days because you're not only like to go around it many times, you're likely to go around it with different options each time you do. And the reason I'm showing this really oversimplified diagram is to the left of the dotted line, you should not be thinking about technology. And so many projects I've been involved in, start, oh, what's the change that's used for you? And it's really painfully obvious, but I'm gonna give you another farcical example. This is not a UFI project because we would have nothing to do with this. Not long ago, I was working with a legal company and they said, look, we've got these guys and they don't, because we got these guys and they're not reading policy documents, right?  They were gonna build in a whole lot of PDFs, right? So you wandered around a virtual office and you saw a desk over there with a PDF on it and you read it, yeah? 250,000 pound project. That wasn't about the change, it wasn't about the experience, it was about the cool technology. Okay, I'm just gonna run through a few UFI and other VR examples. And the point of these is not to say how cool VR is, it's about being really clear what the learning strategy was and how they got that right before they chose the VR.  So what they decided on, what first step decided on was that really simple tasks, changing a wheel and a tire of a car. The way that the individual should would learn it was through repetitive practice, repetition of processes. Painfully obvious, painfully obvious. But the only way they could do that was VR, right? So they said, right, what we're gonna do is we're gonna have this procedural thing replicated in virtual reality and they will practice it. And as an aside, the reason it was such a great, or it has been such a great success is because these poor individuals who have got appalling lives, you put a headset on them and the world they're in is much better than the one that they know in the real world and they feel looked after. Yeah. Body swaps, here's a really interesting one. In the real world, you can't, I can't jump into Myrene's head and see what I'm looking like and that's probably a good thing actually. But in virtual reality, I can do that. So Body Swaps does that brilliantly and what they decided was in the soft skills learning, yeah? What I can actually do is when I'm learning about how to interview and how I'm learning how to be interviewed, I can swap my perspective. I can actually get into someone else's body in their head. Isn't that amazing? So they decided on perspective shifting. The only way you can do it is VR. Anagram, this is one of my projects at VR. At VR, I'm so tired. At UFI. Oh, dear. This is great.  Wouldn't be great to get inside their mind. It might be a bit disturbing, but you can actually get into a person's mind and see what they're seeing and feel what they're feeling. Is that extraordinary? So they decided really, this is a award winning before it was involved with us. It was, what's that really cool media festival in Texas? SWSX, South by Southwest, of course. They want a prize there for this, right? The only way you can do that is VR. So they decided on the strategy first and they chose VR. This is not a UFI project. It's by a company called Nanome. I don't know how you pronounce that. Perspective shifting, okay? So these are medical professionals looking at molecules living next to a molecule. That's great, isn't it? So what they decided was that the particular learning strategy they were employing at the time wasn't, it wasn't, okay. So the learners, the professionals who were being trained weren't able to conceptualize the nature of these particular molecules accurately enough. So they thought, well, let's show them. Let's show them closely. Let's shift their perspective. Another one, this is personally very important to me. This is Parkinson's sufferers. This is an immersive application where Parkinson's sufferers learn to rebalance their themselves and regain certain motion. And again, they decided physical immersion was the only answer, it's got a BPR. Simple one, Northern Irish Association for Care and the Abilitation of Offenders. This is a UFI project. And the strategy was, and in their project specification, it says our strategy is exploration and consequence, right? This is a health and safety program. And basically you wander around the building site and you take decisions around what to do in a building site. And I think this is my last one. I think so, let's have a look. Yeah, this is another great one. This is one of our projects. This is for classroom assistants. Classroom assistants have problems. They tend to be somewhat under-trained to start. They have problems really understanding the environment they work in. So again, the project spec says very clearly the learning strategy is about emotional immersion, right? So they deliberately make this quite a dynamic environment in order that the learners really feel where they are. Okay, so those are just some examples. And again, I'm not saying that these are really great examples of VR. I think they're okay. I think they're pretty good. But the reason I'm showing them is because they have very, very clear and explicit. The explicit is important learning strategies, excuse me. Okay, a few more curves and processes. Just a reminder, we're talking about decision-making. The reason I keep going about learning strategy is the learning strategy determines to a very significant extent the technology you decide to use. Okay, I really like this technology adoption curve. One of the points we were talking about earlier on was around organizational culture. So when you're deciding on your, when you're making your learning technology decisions, it helps to understand where you are. Now, I know that these terms over here are somewhat pejorative, late majority laggards and stuff, but you need to be honest where your organization is. You need to be really honest where your organization is. If you're here culturally, technologically, you need to be honest. You need to be clear where you are. Now, I've just had a five-minute warning. So we're not doing too badly. I'll just accelerate to teeny bit. Okay, another one of these processes and things. All technology, generalizing here, all technology moves in a particular way, all the ones I've experienced anyway. You start off, and this is to some extent, but not entirely replicates the hype curve. Pretty much all technology starts off with a, in a phase of using proprietary tools where everything's incompatible, where it's difficult, there are technical challenges. You need specialists, you have immature processes, development is slow, there are lots of risks. I was first involved in web development where we had a team of, I think, 15. We developed the first Marx and Spencer's website. We needed 15 people. It was rubbish, it was also rubbish, waste of time. I now do fantastically complex website building in half an evening for friends. The reason being is I have a generic tool. I'm not particularly technical. Everything's compatible. It's technically simple. The processes are established. I can develop quickly in its low risk. So VR is pretty much commoditized, right? Why? Because I can now use authoring tools. In other words, I can use a preset package to build stuff with. But assets costs have dropped amazingly. I can get libraries of three-dimensional models for like $2 now. LiDAR, do you know what LiDAR is? It's scanning technology. So let's say I wanted to scan this desk with a computer on. I'd walk around with LiDAR and it would give me a three-dimensional model. So, you know, I learned really complex 3D modeling tools like Cinema 4D and Blender. I didn't have bothered, actually. You can get really easy 3D building tools. Compatibility has increased. The cost of headsets, hardware costs have dropped. Now, I just want to talk a little bit more about learning strategy and design production. But I keep saying the same thing. If you're going to make decisions, you need to know where you are in this kind of, in this phase. Yeah? You need to know where you are there and where you are there. So very quickly, I'm going to skip over that bit. Okay, just, all right. Yeah, no, I'll do it. So design production process streamlines, all right. Learning strategy, expertise develops. Right, learning strategies, that's really important. So when you start with a new type of learning technology, you don't know how to design for it. But when you've got a really, you know, maturing one, you've got all your learning strategies ready, you've got your interaction types ready, and the general principle of the designs are established. Yeah, so not only is it easy to buy the thing and use the thing, but in terms of designing the learning experience, that's easy too, because everyone's done it before. And then finally, the application of that technology clarifies. So for example, with VR, you apply it when you've got perspective shifting, when you've got high risk, high cost. For my sins, a while ago, I worked for the military on a VR program, aiming missiles, missiles are expensive.  When you need motion in the environment, when you need a motion in the environment, that's when you should use it. And we know that now, because early on in the technology, we don't know when to use it. We make awful decisions. And then don't use VR for this. Don't use it just because it's fun. Don't use it when you've got high information, when you're wandering around an office trying to read PDFs, don't do it. Scale is still a bit of an issue. Headset costs dropped a lot, but it's not until we have glasses, like you were saying at the back, that it'll be really cheap. If you've got special accessibility requirements, it can be an issue.  And long duration. This is a very important issue that a lot of people miss. Don't use it if the learning you're cultivating, you're aiming at developing, involves four or five hours at a chart. There's a lot of research that shows that people get very disorientated, can fall over, feel sick after a certain period. And actually the data is quite contradictory on that. I've seen some data that says, don't do more than 10 minutes. And just to kind of finish, I think I'm finishing, this is really interesting. Check your assumptions. Check your assumptions. Look, everyone says VR's fun. Yeah, but teens don't want it, young people don't want it, right? Check your assumptions really carefully. Okay, here's the summary. I talked about hype, I talked about change and how those feed in to decisions. I hope that's been of use. You haven't talked anything like as much and that's my fault because this was supposed to be kind of reflective workshop. But like I say, I hope this has been of use. Just a little promo thing. UFI is running this future skills award. Have we got any flyers here, Marion? Okay, don't worry, we've got a few here. Okay, this is an award that's asking the question, there's real cash available, hey folks, real money. Yeah, not for your, is it correct me if I'm wrong, Marion? It's not for your institution, it's for you, isn't it? Yes, for you. You want 500 quid, don't, 5,000 quid, 5,000 quid, don't ignore us, okay? Have a look, there's some flyers gonna be handed out here and thank you very much to receive a copy of these slides, do scan that, do get in contact with me. And I'm so sorry, it's actually quite... No, you're not, it's a matter of fact, if I could first ask you around for thoughts, if that's what you want to say. Thank you. Questions or thoughts, any more? Should we come back? My back then, which is, I've got you two, you and me and Anne, we are, which is, you know, colleagues in New York who have made decisions about buying and selling learning resources. Yeah. And they're also investing in the technology to be able to use those, but often there's a quite a lot of hints like them. I know, yeah. So they're trying to find multiple things, but also which accelerates the cost. But also there's a timeline associated with that, particularly when the content is around where there are safety standards that are paid. And that's a really big inhibitor for the colleges in the universe. I know. So they're going to be investing. What's your view on how that could be overcome as a problem? Do you know what? I'm going to do that facilitation thing of opening this up and saying, what do we think? I can give you my view. I can give you my view. But seriously, we've got this whole institutional kind of cultural thing, haven't we? Where because of the nature of these institutions, decisions are kind of locked in. Are there any obvious answers or am I going to have to have a go? I think you've posed me a question that I can't directly answer. It's an institutional thing, isn't it? It's an organizational thing. And so in a way, all my stuff about decision making is somewhat, I wouldn't say useless, but it's tricky in your situation, isn't it? In other words, you can make these decisions, but because of the way the organizations function, your decision may not necessarily be implementable. Is that right? For the longer term? To realise that pool of investment then. Do you have a chat afterwards? I'm just trying to get my head around it. I'm not an institution. I just made it fast enough. You've just heard of it. Ah, okay, I'm sorry. Okay, all right. Any other thoughts on that? Some of it is to do with pricing models for the organizations that create this. So... Oh, you see, that's interesting. Yeah, because the organizations don't price something a particular way because if you don't start off with a low point, you don't get access into these institutions to begin with, but actually that price doesn't quite reflect the work that has gone into creating it. And so what they then do is have to price it differently later on once they have you in. Sorry, this is my sales background talking for them. So it's actually more how the pricing models work. Yeah, I thought you might have... Yeah, because they're not the core of the university, so we may apply some of their big decisions that are going with their... And hilarious how limited the decision-making is now farthest away from what you were basically describing. Part of it is also, I always thought that, first of all, there's not the real understanding of the technology and the context that's a bit. You'll have intern stakeholders who will be quite convinced of a specific technology often because they're on the same train as the person selling it to them. I have seen that. And once you're basically bought into a specific system, it's very hard to get the organization to leave it again. Yeah, absolutely. Even though, for example, it has outlived its usefulness as well, but people become really wedded culturally to a specific system. Because it's hard to change, actually. Incredibly hard to change. Really, we can say we love you. Yeah, and I think sometimes... Not only that, people don't want to resist the change, but I think within organizations, people also have kind of made their career with something. And at this point, we kind of rise when the dean lives. And he was in business talks about how he's going to have... Yes, so you've got a chance. I have one sort of half-baked thought in that I worked for PWC for quite a long time and we kept making incredibly expensive, very dumb decisions. And the way my boss... I don't think he resolved it, but he helped it, was by engaging in the most scrupulous evaluation you ever seen, right? So we were going to like, granular pounds and pens and stuff, saying, we know that this worked. We know that it worked. Can we have more of this? And he was very honest. He was saying, this didn't work.   Really, really, really granular forensic evaluation can help, yeah. I don't know if that's relevant. Any other thoughts, questions or are we done? Great. But I'm sure people can interrupt your tea break while I'm following you at the end of Patrick. Thanks to you once again. Thank you. Thank you very much.