Hello, my name is Raina Tartler and this is Yao Lin and we welcome you to our presentation Bloomberg's journey to a multi cluster workflow orchestration platform. Let me start by giving you a little background about what we are working on. We are part of the cloud native compute services group at Bloomberg where we offer a platform called workflow orchestration. This is not an externally visible product which means that all of our customers are internal engineers who generally use Bloomberg infrastructure and data. The team and system is rather new but has seen quite rapid growth and popularity. We believe that our requirements, challenges and conclusions may resonate with at least some of you in the audience and we are thrilled to share with you what makes our journey unique and get into a conversation with you afterwards. Before we dive into the technical details let me walk you through what we mean with workflow orchestration platform to give you a better idea about what we're actually offering. In a nutshell, our workflow orchestration platform provides general utility compute for run-to-completion batch jobs. So what does that mean specifically? That means that our customers come to us with two things. Coating containers on our company's internal container registry and an execution plan that defines in what pots to execute in what order. That plan may also decide on about fan-out of branches of execution, our defined synchronization points where the workflow requires several nodes to terminate before proceeding. Luckily we are able to use Argo workflows a CNCF project that has reached the graduated maturity level in December 2022 for the core functionality that our clients primarily use. Argo workflows comes with a controller, custom resource definitions, CRDs for DAX, workflow steps and most prominently a user interface that allows users to visually observe the progression of their workflows. It is also critical for debugging when containers don't execute as expected. At Bloomberg we are proud to offer workflow orchestration as a service for a number of different use cases such as machine learning orchestration which covers training and tuning of our AI, company's AI models. As an aside, AI is not something new to us at Bloomberg. We've been using AI for more than 15 years since 2009 to manage the large volume of financial data, news and analytics our customers access every single day. Now back to our system. So we have users implement custom CI CD resource solutions. In many cases we see users trigger maintenance tasks on both physical as well as virtual infrastructures. We have seen users orchestrate financial analysis tasks to build all kinds of processing pipelines. As you can see this is quite a variety of use cases and that calls, that's what calls for a platform that provides general utility compute for batch processing applications because we run all of them on the same physical hardware. While our workflows is a great platform, we do have a number of additional requirements at Bloomberg both functional and non-functional that steered our journey towards a multi-cluster solution. Let me give you a quick walk through here. Bloomberg is first and foremost a data company. The data is proprietary, substantial in size and has very specific access requirements. Our users generally benefit from our UIs and APIs to keep track of workflows, pods, inputs and the workflow logs that our users run our platform to develop new workflows and troubleshoot when something misbehaves. Additionally we provide our users means to observe the budget resource allocations so that they can make informed decisions on how to size their pod resource limits and when they need additional resource allocations. Thinking more about functional requirements, Bloomberg expects its engineers to be mindful where their applications run. The most obvious instance is to choose the correct network zone, think prod or dev network for data and compute. Additionally, Bloomberg is very conscious about production stability. In general, running workflows in prod requires approval from someone in your management chain. Our platform is tightly integrated into Bloomberg standard approval processes which makes it easy for our clients to comply with the relevant company policies. Most users rely on being able to run workflows on a schedule either by specifying a cron string or by coming with their own custom scheduler that is connected to external events, to internal triggers. One of the most challenging requirements our team had to overcome is cross data center resiliency. This basically defines how application teams must develop their application for resiliency across multiple data centers. From our perspective, that means that we essentially need to provide several installations of our group workflows at least one per data center. This however now comes with additional cognitive load and failure modes that we want to discuss in this very session. With this introduction and overview, we are now ready to embark on a journey or more specifically how we, the workflow orchestration team, have started out with what we had at the beginning and where we ended up today. This presentation will walk you through our analysis, the tools in our toolkit that we had at our hands, how we adjusted our architecture and what new tools we adopted. Finally, we're taking a look at what we found and have a critical analysis before concluding our talk with hopefully enough time to have a conversation with you. Let's start our journey with an application team that runs some financial analysis report on a weekly basis. For this, they may generate a workflow that starts generating data from various data sources, does some calculation and then persists the results to wherever they need to go and at the end send a Slack message to notify that it's done. In this case, we know that our users deal with two very kind of resources that have very different characteristics. On the one hand, we have the workflow that starts and ends at some point and after that is no longer needed on the cluster and can be deleted to free up at CD resources. On the other hand, that workflow requires a number of other resources on the cluster to work properly, such as config maps with connection strings to how to connect to the database or secrets with authentication credentials. Those kind of resources generally stay on the cluster. When we started our journey, we considered the following setup. Given two data centers, let's call them Oscar and Romeo, we need to provide means to our users to submit their workflows and deploy static resources such as config maps and secrets. Such static resources get automatically copied to both data centers. For submission, in this scenario, we would need to ask the user to choose which data center the workflow should run. This has a number of obvious challenges but most obviously it's way too easy to pick a data center that is overloaded or possibly not even available. Clearly, this approach doesn't work and to be clear, this is not a theoretical but very practical issue to us. In order to get confidence in failover scenarios, we conduct regular mandated data center resiliency tests where the service is brought down in one data center for an extended period of time on a regular basis. On one hand, this is a major inconvenience for everyone involved. But on the other hand, this is the only way to build up trust in confidence in the failover mechanism that we implemented. To overcome these issues, we have chosen to introduce a management service in the workflow cluster. Here, a service is hosted that offers an API for workload submission and deployment of static resources. This management service provides reliable placement decisions to address cases where a data center is currently unavailable or possibly overloaded. It also provides a user interface that allows onboarding of new users, submission of workflows, setting up and editing work schedules, management of workflow templates and much more. This is also the best place to record submission of workflows for bookkeeping so that our users can easily retrieve their previous workflow submissions from months ago. By the way, we use the service also to implement sharding of our tenant's workflows over multiple clusters. We have observed that our clusters are generally comparatively underutilized, but the Argo workflow controller tends to get quite busy due to the large number of workflows that it has to process. Therefore, we tend to prefer smaller clusters, which helps with scaling but also helps to minimize noisy neighbor issues and reduce the blast radius of disruptions. Now that we have covered how we handle runs in workflow orchestration, let me point out the most obvious difference to static resources. In general, workflows run exactly once on one cluster, where static resources, so think config maps and secrets, generally need to get consistently deployed and kept up to date on all participating clusters. This imposes quite a different set of challenges. At this point, I'd like to hand it over to Yao to walk you through the details and particularities on how our platform handles static resources. Thanks, Reinhardt. So we'll move on to static resources. As we can see, the management API here needs to be globally available first, even if one data center is currently unavailable. So in order to keep the management service reachable by both sides, they are deployed on each one, the Oscar and Romeo. So from the user perspective, the requests are route to the nearest location with DNS-based service discovery. But we'll just pause that topic here because itself can easily extend to another topic. Let's come back to the work, to the static resources. So similar to handling workflow runs, the API itself makes placement decisions, well, accept user requests first, makes placement decisions and push the change, push the resource to where it needs to be. However, there's immediate concerns because these are static resources. They must be consistent and persistent on the cluster. There are a few what-if scenarios. What if we temporarily need to make some fundamental upgrade to a cluster? We need to rebuild the cluster and bring back everything. So can we have, can we lose those resources and recover them? And additionally, these, think of the functionality of this API, it needs to do three basic things. Accept a request, make a decision and perform the deployment actions. Is that too much for it? And for every user to write a request, it must wait until everything finished. So that's raised immediate concern that we need some place to store these resources overnight or at least some place to survive over a disaster. Then the most straightforward approach would be store them in a database, obviously. Then this API simply accepts a request, make a decision, write it into a database. And then on the other side of the cluster, the workload cluster has a sync agent to pull things that should have been landed here. Now, it looks promising. It looks good. It satisfies our user's request on a very basic basis. But we have observed that it's become increasingly difficult to actually operate it. In particular, we start seeing the management service start to eat up our error rates, error budget, and show some increased latencies and exhibit issues when creating resource, most likely the consistency issue. Also, it sometimes just give us time out. So it's very risky for the schedule job, particularly, because that just breaks the promise of automation. So we need to do something to improve it. Let's come back to the design again. So think about the edge cases that the system might not function properly. What if the write request succeeded on one side of the data center, but not on the other? And what if now I need to add a third cluster? I need to remake all those decisions again so that the third cluster gets what it should have. So that summarized our issues into these two, the cross data center consistency, and to handle the change in cluster joining and leaving. So let's take a quick short pause here. We have explained how the workflow runs and static resource have different characteristics and they have unique challenges. This presentation going forward will be focusing mostly on static resource, although we do have considered the workflow runs case as well. So we start with reexamining what we have at our hand. So we're attending KubeCon, right? So people here, I would assume most of us understand how Kubernetes on a high level it works. It has a storage backend. SCD provides eventual consistency more like the source of truth. And it comes with the controller mechanisms, leader elections, and many more features. So eventual consistency is the key term here. Now, why can't we just rely on Kubernetes to reconcile the placement decisions and to automate the workload cluster change? It seems pretty easy from that point. So we're a big fan of Kubernetes and benefit a lot from the access control, the resource allocation, and that are all built on top of an API server that is easy to operate. However, this is true for a single cluster scenario, but we're thinking about cross data center. And stretching SCD across data center is not that cost efficient. It's actually quite expensive. It requires significant expertise and probably money. So in this session, we want to have another tool to help us on the other aspect. So in summary, Kubernetes help us with many of the problem here, but not all of them. So think about what's left. Cross data consistency. Why not just use a database? Modern database has been generally believed to be reliable and easy to operate. They're the perfect tool to store static resources and allow subsequent deployment to be executed. So with that in mind, you also have the chance to build a bookkeeping system for user input as well. It can serve auditing and tracking purpose. It also helps server user read request without hammering your Kubernetes cluster. But there's another aspect that we want specifically to highlight here. So modern database, modern relational database offers a number of different solutions for high ability across data centers. In our context, we have found the streaming replication offered by Postgres is the most cost efficient way. So basically, a database leader will be selected to accept all the reader request and the transaction log gets streamed to other replicas and eventually all the replicas will be consistent and all of them will be able to serve reader request. So in case of failure in one of the leader replicas, a leader election process kicks in and select a new leader, that is a critical process that usually happens on a weekly basis. But in case of failure, it's usually just finished in a few seconds. So that is acceptable. So let's come back to the cost efficiency part. We found this is much cost efficient and viable solutions for us comparing to stretching a real typical SED across data centers. But we are thinking of taking the advantage of API server as well. Then how can we benefit from both of them, not just one? So another pause here. So we have examined we have two kinds of tools and it's now to combine them together. And we'll introduce how we combine them. You may have heard of K3S is a lightweight Kubernetes distribution that is capable of using a debt external database as the storage backend. In most cases, K3S is discussed in context like constrained hardware, edge node or experimental environments such as your home lab itself as a whole definitely don't fit into the large scale cluster like what our system offers. But think of the uniqueness here. It comes with the interface to connect the database and the Kubernetes world together. How can we isolate that out, that particular part? That interface is called Kine. It stands for Kine is not SED. It's simply a stream layer that offers the connectors to expose things like SQLite or Postgres as a SED endpoint. So with that anything that works with an configurable API server may just function without any underlying typical SED. So for installation, you will need a real database endpoint. It's external. And then you'll install a Kine instance expose itself as SED and then configure an API server to talk to that SED endpoint. So here's the example what gets stored in a Postgres database to convince you it actually stores what you want. Now it's time to assemble things together to think about how it can work in practice. One thing to highlight here is that the API server that uses Kine itself is a real API server instance. But does not come with a Kubelet. So it does not have control plane, it's unable to schedule pod. So these actual clusters themselves have an actual stretch SED but within that data center. So the Kine API server here is more like the application you installed in this cluster and actually they're installed in multiple clusters. However, they talk to the same database back end. So they're consistent just like replicas on the same cluster. So let's look at how it operates. Still what it exposes is just an API server. It itself doesn't know how to talk to a workload cluster then you still need some kind of application, a deployer role to do all those work. You can probably select something from the CNCF community. We have heard of Kamada and OCM. You may also think about rather something in-house. But definitely you will need some specific interpreter to translate the resource anyways. So, yep. By using the same database here, one thing to highlight is that you don't need to worry about the conflicts between these deployer actions because they're talking to the same SED just use the standard leader election mechanism, problem solved. So that summarized what we designed and it's a very fundamental change for how the SED and the API server is set up. Now let's address some frequently asked questions. Most obvious issue is that this kind API server itself doesn't come with a cubelet. It does not have control plane and no worker pod. Does that limit the potential functionality so you can use against it? But think about the use case here. We're using it just for managing resources only. So it's not intended to operate a real cluster anyways. So if you clarify the use case here, then there are only a limited set of things that you need to install to actually interact with that kind. Many of the system components are more reasonable to just talk to the typical API server that's used the typical SED at the back end. Then we move on to the quantitative world. The storage back end has quite some fundamental change. Then what about performance? Well performance comparison only makes sense if you are looking at the exact same read-write pattern. So think about the use case here. We're using it just to manage the resource. So in most of the cases, resources are static. There will be writes, of course, but not from a workload side. So most likely the deployer application simply just read from this kind API server and land things on the actual cluster, on the actual workload cluster. So the usage pattern is quite different and there are a lot of means you can actually optimize that kind of call pattern for database characteristics. And then with those concerns addressed for new platform setup, there may be concerns for existing platforms like us. We have existing users. This is such a huge change. We don't want to interrupt their daily job. Then how can we do about it? Let's take the system, for example. Sorry. Originally those decisions are stored in the database as static decisions and later it's a pool model to actually land things. The placement decision has been made up front, but afterwards we're actually storing just the plain input and expect recognition to happen later and most likely it will be a push model. How can those change be hidden away? Well, just build another API to assist in front of it. So you can either build something new to talk to your old API and the new or you can just have them in the same binary. That's your call. So this API will help us to do the handover work without breaking anything. Also, it's not a throwaway work even if you finish because it's designed to be user-friendly to serve long-term goals. So right now I'll hand over to Reinhardt to do a quick recap and wrap up today's session. So thanks, Yao, for sharing your insights on the technical details, the solution that we came up with. Let me summarize and conclude. Multi-cluster federation is a very relevant and broad topic for the entire community, especially for companies and teams that offer large platforms, reliability and resiliency are key concerns. We observe that modern relational databases handle cross-data center persistence using streaming applications very well and are generally well understood. However, they alone don't offer quite the same conveniences and properties that we are used to in the cloud-native world. For the requirements that we have identified at Bloomberg, we have shown how we use Kine to achieve multi-data center resiliency with cloud-native technologies, more specifically how we persist Kates objects using the Kubernetes API server in a relational database with transaction log-streaming. Thank you so much for your attention. I hope that this talk resonates with you and we would like to hear your thoughts because we are curious whether you have maybe similar requirements or how do you achieve multi-cluster resilience. Let's compare in contrast and use the microphones for questions. So, I have a question. Awesome. So, before you're of course receiving API calls, but with this new Kine system, you can receive actual Kubernetes jammals or like this, how are the users adapting to this?  Yeah, that may not have been clear as much as it could have in the presentation. Do you remember we have this management service that sits in front of it? So technically we don't expose the Argo API server directly, but in this management service with a API that we are designing and defining in front of it. That API server will then pack up this Argo native manifest. It could be Argo workflow template, it could be an Argo workflow, it could be a number of other things and persist them in this Kine server in a way that allows us to identify, okay, this is supposed to go to that namespace and that cluster and all of that is hidden away from the users. So in order to identify the Argo version, you have to identify the Argo version details at all. And if possible then I have one continuation question. How do you then handle inconsistency between your different clusters with for example Argo version? Because before you were getting the abstract concepts and you were changing that into an Argo version, what is the management layer? I'm not sure if I'm understanding the question, is the concern that we may be operating different versions of Argo workflows or did I understand that? If you have a federated cluster with ten clusters and five of them are running different version than the other five. So the answer is very carefully. So in general, Argo, the CRD themselves don't change very much. So in general, I think we are at 3.5. We haven't actually started on that but so far we didn't experience major problems with that particular concern yet. Thank you for your talk. I have a question because you are already using Argo. And did you look at the application set to manage the static resources because with the process that we have looked at it briefly, the thing is this GitOps model that Argo CD comes from really isn't a good fit for us based on the specific data access requirements and the way how we set up the platform to give you a little bit more insight. So one of the things that we really need to handle is if you are AI team that has access to one namespace then there are other controls in place that you can only see your specific things and only in your namespace and nothing else and integrating that with external systems like Argo CD controller are challenges that we didn't have the capacity to think that completely through and we came up with this solution which we felt is significantly simpler and thought we share that with you at this point. Did I understand the concept correctly in that you are using basically the kind API service to stretch I would call it virtual control plane over multiple data centers and you are using the post-press URLs basically as a database for that. That is correct, yes. So you built in that case a virtual control plane that spans over many more classes . Yeah, that is one way to look at it. I tend to avoid the term virtual control plane. You have the deployer there . It is only provisioning data in a Kubernetes way and then deploying it onto the workload classes. Exactly. When you are saying management of this virtual control plane people generally assume that you are going to quickly end up in a solution like vcluster which shares a couple of similarities but again we are using that to store and deploy static resources like config maps, workflow templates, things like that. That does not require a Q-plat because it is storing about and we thought this is a new trick. Thank you very much. Next question. How do you consider using Carmada to manage the federated clusters? Excellent question. Carmada is something that we are actually looking at for workflows for the dynamic resources. This presentation was mostly on static resources. I understand that Carmada can also be used for static resources but we will be using Carmada maybe not. From the cost-benefit trade-off we found this is the cheaper way to go for now. Makes sense. Thank you. Thank you so much for your questions. Happy to talk to you afterwards. Bye-bye.


Welcome back to our MedSmarter question of the day. Let's get right to that question. Which of the following is the most likely diagnosis in this patient? An obese 49 year old G3P3 woman presents to the physician with right upper quadrant pain and fever that was preceded by nausea and vomiting. Ultrasound shows hyperechoic structures in the right upper quadrant. Fever testing reveals white blood cell counts of 15,000 per cubic millimeter, 15,000 per cubic millimeter, an erythrocyte sedimentation rate of 45 millimeters per hour, and a serum amylase level of 75. Which of the following is the most likely diagnosis in this patient? Take a minute, look at these answer choices, come up with your answer, and then write your answer in the comment box below. Alright, let's look through this question and find out what are the important things. We start out as an obese 49 year old G3P3 female. She's having right upper quadrant pain, had nausea and vomiting, on the ultrasound she had hyperechoic structures in the upper quadrant. White blood cell count is 1500, ESR is 45, and amylase is 75. Okay, so let's look at those lab values. A white blood cell count of 1500. A normal white blood cell count is going to be between 4500 and 11000.  Our ESR in females is typically between 0 and 20, and our amylase levels, normal values are between 25 and 125. So ESR is 45, which is greater than our normal values, so we have an elevated ESR. An amylase of 75 falls right within our normal range for amylase. So what that tells me is that we have ruled out any possible pancreatitis causes associated with this. So I can go through my answer choices and mark out anything that has to do with the pancreas. So carcinoma of the pancreas and acute pancreatitis are out. So that leaves us with two cholecystitis and a cholecystitis. So let's break those down. An acute calculus cholecystitis. Gallstones seem to be a possible answer choice here. So this is something I think I will leave in as a possible answer. De-cholesterolysis, while cholecystitis does have effects on the gallbladder, it typically doesn't show inflammatory changes. So we wouldn't see an elevated ESR in cholesterolysis and we also wouldn't probably see an elevated white blood cell count either. So I'm going to say D is not the correct answer. So we're down between acute calculus or an acute a-cholesterolysis cholecystitis. So what does a-cholesterolysis cholecystitis mean? That means there is an absence of gallstones. Based upon this ultrasound, there is hyper-echogenic structures that typically mean that we have gallstones. So I think we are dealing with a calculus cholecystitis, not an a-cholesterolysis cholecystitis. Therefore B will be my final answer. And B is the correct answer. So this right upper quadrant in an obese middle-aged, multiparous female with the ultrasound findings consistent of gallstones, including having nausea and vomiting probably after she ate a meal, a high-fatty meal, this is all indicative of acute calculus cholecystitis. So what's happening here are gallstones are getting stuck in the gallbladder neck or in the cystic duct and those gallstones are causing pain. Sometimes we can also get a referred pain up to the shoulder. Don't forget our 4 F's associated with our risk factors of acute calculus cholecystitis. What are the 4 F's? Female, fat, fertile, and 40 are the 4 F's associated with risk factors with gallstones. So this patient, 49-year-old female that is fertile, she has had multiple children and is obese or fat, fits classically into our acute calculus cholecystitis diagnosis.


Patrick Mahomes' entire career butt. He switches teams every season. In season one with the Chiefs, Mahomes struggled and lost in the AFC championship.  In year three, he headed to Baltimore, where they barely missed the playoffs. The Lions won the Super Bowl though. In year four, Mahomes is headed to Washington, where he took the number one seed, only to be upset by the Cowboys. Year five, he's a Detroit Lion, made the Super Bowl, but lost to the Steelers. In year six, Mahomes heads to Houston, but gets womped in the wildcard. In year seven, he becomes a Denver Bronco, loses again in the wildcard. Two is for her team, the Ravens. In year eight, he goes to the New York Giants to lose again in the wildcard. In year nine, he pairs up with Justin Jefferson to make a Super Bowl run, and falls short to the Panthers dynasty. He did finally win MVP though. After a long career, he played his final season with the Saints, but fell to Tray Lance and the Niners in the playoffs.


Well, there was a couple of really tough acts to follow. Roberto and Marshall had some just awesome research. That's one of the reasons it's so great to be here at MIT and be able to participate with them. David was just talking about how there's a revolution underway. And in our book, The Second Machine Age, we explicitly harken back to an earlier revolution, the Industrial Revolution, which really literally bent the curve of history. As an economist, if you look at what happened for millennia, for centuries, in terms of the basic living standards of people, the answer is essentially nothing. There was people living standards didn't change much until a certain time and a certain place. That time was the late 1700s, and the place was right here in England where the Industrial Revolution set off a fundamental transformation in our prosperity. And that was because of a set of technologies, in particular, the steam engine. And so it's good to be here where that first revolution really got underway and to start talking about the new revolution that's underway and the one that's involved in The Second Machine Age. When you look at what it was that made the first machine age so successful, it's because it transformed one of the two key ingredients that's needed to really change the world. So if you think about what it takes to change the world, one of the things is a power system. The world is literally made of atoms, mostly. And if you want to change the world, you need to be able to move them around, transform them, shift them. They're held in place by powerful forces. And in the first machine age, we learned how to use technology to do a lot of that. Instead of having human muscle power, animal muscle power, we used machines, incredibly powerful machines. First the steam engine, later electricity, the internal combustion engine. These are what economists call general purpose technologies. And each of them provided more and more power to move and transform things. And it's important to have that. And that's what led to a roughly 50-fold increase in living standards since the late 1700s. But it's not sufficient. You also need a control system. You need to figure out how to arrange and organize those things that you are moving and transforming. So in the Industrial Revolution, we set forth a set of technologies that dealt with physical power. And they were mostly a compliment to humans because the humans were the control system. We were the ones that managed how that physical power was deployed and how it was used. And I mean compliment in a very specific technical economic sense, in the sense that the technology made human labor more valuable. And I don't just mean that intuitively. You can measure how much more valuable human labor became because wages rose. So the value to an entrepreneur of hiring one additional worker went up. And that was reflected in the fact that they were willing to pay more for that labor power. Now we are, as I mentioned, in the early stages of a second machine age. And Andy and I have been trying to study and understand this. And one of the special characteristics of this second machine age is that we are beginning to use technology not just to deal with the physical power, but also the control system. Not just to automate muscles, but also to augment our brain's mental power. And the general purpose technologies of this revolution include computers, software, machine intelligence, and other technologies. It has some similarities to the first machine age. It's leading to an increase in output per person and increase in productivity, more wealth creation. It has some differences as well. One difference is that many of the technologies are evolving dramatically faster. There's an exponential improvement in the power of computers. We call it Moore's Law, which means that a child's PlayStation today is more powerful than a military supercomputer from the late 1990s. Computers get better faster than anything else ever. Now, in the first machine age, there are also improvements in the technology. And we went back and charted, for instance, the improvement in the efficiency of the steam engine. That also had a doubling time. But the doubling time was about once every 70 years. So it took 140 years for it to become four times more efficient. So a very dramatic difference in the exponents. Another difference is that as these technologies evolve, it's not as clear whether or not there are going to be complements or substitutes for human labor. You can see some ways in which they make human labor more valuable, but you can see other ways in which they really replace the need for humans in the loop in many situations. And if you look at the data, it's a bit troubling, because median wages, rather than going up, have stagnated and even begun to fall, suggesting that entrepreneurs, managers, the market is not willing to pay as much for that additional worker on the margin the way they did for two centuries previously. So let's dive in a little bit more closely and try to understand what these technologies are and how they're changing the economy. Let's go back, not way back to the 1700s, but let's go back to just a little about 10 years ago, when we were first beginning to look at some of these issues I was teaching a course at MIT. We used this terrific book by Frank Levy and Dick Murnain. And I had a great discussion with my class about what were the things that were uniquely human. Humans could do well that machines could not do. And we brainstormed and we came up with a list here. It's a fun exercise to do. We redo this exercise periodically. But three categories that we thought seemed to be pretty important. One was autonomous mobility, just being able to move around, whether it's in a room or out in the world. Fine motor control, you can pick up a pen or a fork from your table very easily. That's actually quite hard for a robot to do. Another category is language and complex communication. We speak, we interact, we write. And the third category is unstructured problem solving and pattern matching. Machines we acknowledged were quite good at solving structured problems. But the unstructured ones that weren't well defined tended to be very difficult for machines. And we thought of some different examples of things that just had a lot of these unstructured characteristics that used lots of sensory data, fine motor control, that wouldn't be very suitable to automation anytime soon. And one of our favorite examples was actually driving a car or truck through heavy traffic. We didn't see any likelihood that that would be automated anytime soon. I'm looking around.  Because of course, you know now that there are self-driving cars. So we really underestimated. In fact, here's a picture of Andy and me. We are grinning kind of like idiots, like we sometimes do when we learn something new. In this case, we just drove, or I should say rode, in this self-driving car that's behind us a couple of years ago from Cupertino or from Mountain View, California, up to San Francisco and back down again on Route 101, the highway with an autonomous vehicle. So we were happy to be proven wrong and realized that the technology is advancing a lot faster. And some of the categories that we thought wouldn't change anytime soon were, in fact, changing a lot. And this is part of what sparked us to think a little bit about self-driving vehicles. And I want to open this to questions, and I'm going to do it right now. Try that Sturro Drive. Yes, Sturro Drive? Yeah, that's a lot harder. In fact, this is a good point that Route 101 in California is an easier environment than Sturro Drive in Boston. Just any kind of driving in Boston is a lot harder. Because not only are there no structured rules, none of the human drivers follow the rules either. And it may be a long time between when cars can drive on highways to driving in downtown London or downtown Boston. So these things are evolving, but I certainly don't want to give the impression that this is a completely solved problem. But it is worth kind of assessing where we are doing an update on what we thought was true 10 years ago and what's true again. So let's come back to that list of interacting with the physical world, language, and problem solving. And I want to make the case that in all three of these categories, there have been just dramatic changes, dramatic improvements, and machines can do a much better job in each of those categories than we anticipated a while back. And this is part of what's sparking this acceleration of the second machine age. So let's take them each in turn. The first one, interacting with the physical world. Our colleague, Rod Brooks, has started several robot companies, iRobot, now Rethink Robotics. And this is one of the robots that he has. It's called Baxter. And Baxter is an amazing robot. It can work side by side with humans. Traditional industrial robots are really dangerous. If you put your head in the way of one of its arms, a traditional robot might just not even notice and knock your head right off, literally. Baxter is aware of the surroundings and is gentle and will not hurt people when they work alongside it. It can do a lot of simple tasks. Like here, it's working an assembly line, putting some stuff in boxes. It seems like a very simple routine task, but it's something that millions of people all around the world do, basically, this exact task, including people in Massachusetts, a relatively high wage state and place in the world. There are people doing this task. And Baxter works for about the equivalent of $4 an hour, according to Rod Brooks. And they just came out with a new version that's about twice as fast, 10 times more sensitive in terms of its motor control called Sawyer. So there's been some very rapid improvements there. And you can see on the horizon a next generation of robots that have even more mobility and control. For instance, this is Atlas, about six foot tall, six foot feet tall, 300 pounds. And Atlas is the robot that's being used currently in the DARPA Robotics Challenge. The previous DARPA challenge was to make a self-driving car. The new one is to make a robot that can walk up to a vehicle, get inside of it, grab the steering wheel, put its feet on the pedal and drive a short distance. It's more like a golf cart than a car, but it's sort of beginning to get the feel of it. It walks across a field of uneven cinder blocks and steps across them without falling. It reaches down and picks up a hose, attaches a nozzle to it, goes up a flight of stairs, turns a lever, goes through a door. A series of simple tasks like that that are sort of an obstacle course for robots. And Atlas cannot yet do all those things quite well. It's been able to do some of them. But if you talk to Gil Pratt, who runs DARPA's program, the DARPA Robotics Challenge, he says that the teams are way ahead of where he anticipated they would be at this point. And he seems quite confident that it will achieve those goals. And if you're a manager or an entrepreneur, it doesn't take a ton of imagination to think about, huh, if a robot can do all those steps, what are the kinds of jobs that are being done in a warehouse or a factory or elsewhere that we could put together some combination of those tasks and it would be useful, economically useful work. Another big category I talked about was language.  And in many cases, they literally are talking to their phone, not on their phone, but they're talking to the machine and they're expecting the machine to understand what they're saying. And they expect the machine to answer them back in English or whatever language they're speaking in. That's just kind of weird and amazing. I mean, think of it, 10 years ago, that would have been considered science fiction. Again, I don't think the machines are quite perfect at understanding what we're saying, but we're in the middle of, I call it, a 10-year period. Tom Mitchell at Carnegie Mellon describes this period we're in where we went from machines just not being able to understand this at all to machines being routinely expected to understand our language as we speak to it and carry out actions based on that. There are machines that are translating language. Microsoft Skype program, you may have heard, will now translate from German to French or Spanish to Chinese or perhaps someday from British to American. I'm hoping we'll be able to understand that. And those technologies are also evolving quite rapidly. Again, not perfect, but serviceably well. One of the most amazing things, I think, is the way that they're even beginning to generate quasi-original text. Companies like Automated Insights or Narrative Science are doing that. Here's a new story about the world champion New England Patriots football team. And to translate that, world champion means that they've defeated all the other football teams in the world that are based in the United States. And football means, well, it's not what you think football is. So it's a different kind of sport, but what's interesting is not this story that's sort of just a run-of-the-mill kind of new story, sports story that people read thousands of times a day. It's who the author is. Someone by the name of Automated Insights. That's right, a robot wrote this story. It took the box score, took the statistics of what happened in the game, and it inferred from that what the storyline was of the sports game. And they are generating thousands of these stories for professional sports, for college sports. There are versions of these algorithms that are being used for earnings reports and accompanied by magazines like Forbes. So you're starting to see machines actually writing. And again, I want to emphasize, these are not sophisticated novels or research reports. They're very simple, basic stories, but it is a threshold that's being crossed here. And what's the third category I talked about? It was unstructured problem solving, answering unstructured questions. Could give you lots of examples, but maybe my favorite one is just the game show, Jeopardy. You may be familiar with this game. They can ask questions like this one that involve literature, sports like the previous slide, news, geography, science.  Often the questions are a little convoluted. They involve puns. So it's hard sometimes even to understand exactly what they're trying to get at. Something that humans are quite good at, like this guy, Ken Jennings, was the world champion, won 75 games in a row. Andy said there was a rumor that he was genetically engineered from birth to play Jeopardy. I'm not sure that's true, but he is quite good at that. But they had IBM Watson play against it. And the father of IBM Watson, Dave Ferrucci, showed me this most remarkable class. Like he showed it to my class at MIT. And I want to share it with you. If you chart how accurate Watson or a human was at answering questions and how many questions they tried to answer, you get this kind of a chart. There's little blue dots up there, little mosquitoes buzzing up there. That's the data from all the human Jeopardy champions, all the people who ever played Jeopardy on television and won the game, the ones who were really good. You can see that they tended to get a lot of the questions accurate. That's why they were champions. They also are pretty aggressive, often answering more than 50% of the questions. So that's what humans did. How did Watson do? Well, actually not so good at first. When Watson first started, it was dismal, getting 10, 20, never more than 50% of the questions right. Watson's aligned instead of a dot because they have a little dial. They can make it more aggressive or less aggressive, only answer things that's really confident in. But even then, when it's confident, it still isn't all that good. But Watson had one feature that no human had. And that was the ability to learn at a ferociously fast pace. So they fed lots and lots of information into Watson. And every few months, they came out with additional information and a new model. You can see Watson just progressed at a scary rate. One point they gave Watson the entire Wikipedia to read and understand all the correlations in there. Can't really read it the way we read it, but it read it well enough to see correlations of basic facts. Another point they asked Watson to read something called the Urban Dictionary. If you guys may know what that is. It turned out that was not such a good idea. Watson started answering some of their questions with profanities, so they had to erase that part of the memory. Don't let a six-year-old just surf the internet unsupervised. When Dave showed this chart to me, you could see Watson was already better than most of the human jeopardy champions. These are not run-of-the-mill people. These are champions. Well, there's still some people up there at the top right, like Ken Jennings, who on average did much, much better. But a few months later, they went on national television. As many of you may know, they went ahead and beat Ken Jennings and Brad Rudder, who was the next best person in the world, quite handling. More than the two of them put together three times as much as either one of them, so they won $77,000. But the point, if you really think about it, was not to win $77,000, it was to take this technology and apply it to lots of other areas. So today, that technology or related technology is being used in call centers. For instance, there's a call center in South Africa, which, when you think about it, what is a call center? It's a question answering system. You ask questions and you expect whoever's answering on the other end to tell you the correct answers. Watson is powering that call center. They have a version of it that's helping with legal questions. There's asked versions of Watson that help give investment advice and financial advice. There's several versions that work for medical diagnosis. You can go up to it. This is kind of fun. And describe your symptoms. And Watson will give you a differential diagnosis of what is likely the disease that you may have because it's causing those symptoms. If Watson is not today the best diagnostician in the world, I wouldn't be surprised if it is within five years. People like Vinod Kosla, the venture capitalist, legendary venture capitalist in Silicon Valley is making exactly that bet. And you can imagine combining marrying Watson with Siri. And now you have a medical diagnosis that's available to billions of people, perhaps for free, on a mobile device worldwide. So those are some remarkable changes. And I could go through some more technologies, come to our annual conference or go to our website and you can read other descriptions of some of the things we see happening. But let me now turn to what all these changes may mean for the economy. What kinds of changes do we expect to have happen? And the first set of changes, I think, are just very, very good news. And that is that we can have vastly better medical diagnosis and just vastly more goods and services than we did previously at a much lower cost. In fact, a cost approaching zero in many cases. This newspaper ad somebody sent me was from the early 1990s. And the interesting thing about it, all these devices for sale, there's a computer, a phone, a video camera, a regular camera, a music player, there's an answering machine, if any of you guys remember what an answering machine is. All those things that cost hundreds or thousands of dollars, collectively, are now available for free as bundled in as part of your smartphone. Most of them, I shouldn't say all of them, but most of them are available. And then there's a whole set of other products that you couldn't even have gotten then. A GPS, we talked about earlier, or a way of diagnosing skin disease, things like that, that are new applications. And again, they're available at no additional cost once you've paid for the basic hardware or the internet connection. And we've been trying to do some research to understand what is the value of all these free goods that are now available in the mobile economy and the digital economy and the internet. And first, we just looked at one slice, which is free goods in the internet, like Wikipedia. What would the value of those be if you had to pay for them? Well, it turns out it's about $300 billion per year. And that's interesting because that's a number that does not show up in the GDP statistics. GDP measures the value of all the goods and services bought and sold. So if something has zero price, it has zero weight in the GDP statistics. But that doesn't mean it has zero value. Wikipedia is probably more valuable to me than Encyclopedia Britannica was, with all due respect to Encyclopedia Britannica. Wikipedia has about 10 times as many articles, it's more up to date, and the accuracy apparently is comparable. So that's good news, we're getting lots of bounty, free goods and services that aren't measured. We have to update our statistics. People like Roberto Rigobon, you heard, are showing the way to how we can get more fine-grained data to understand better what's happening in the world. But that's not the whole story. If you look at some of the main characteristics, statistics that economists keep track of, it shows an interesting pattern. So first let me show you these four, productivity, employment, median household income and real GDP. And for much of the past economic history, since the Industrial Revolution, those have been rising in tandem, tightly coupled. And there's sort of an implicit social contract that as the economy got bigger, we would share in that, in particular for instance median income. The income at the 50th percentile would grow alongside. But more recently they've become decoupled. Andy and I call this the great decoupling. Productivity and GDP growth have continued to grow, but median income has stagnated and in fact is lower now than it was in the 1990s. Employment is also falling, the share of workers falling, the employment population ratio is falling. So this is a break from history. This is different from what we saw in the first machine age. What's going on? Why is this happening? Well, to be frank, we were really puzzled when we saw this, we were like, wait a minute, this doesn't fit with our intuition. We see this wondrous innovation at MIT and elsewhere. It's creating all these new possibilities. How can it be that median income goes down in the face of all this wonderful innovation? But facts are stubborn things and we looked at it and we realized, well, this is really accurate. People who have been pointing these, this falling median income are not wrong. You can tweak the statistics a little bit and you can adjust for household income, but the basic story I just showed you doesn't change. So how can that be? And the aha moment came when we were talking about this when we realized that while digital progress does make the pie bigger, there's no economic law. There's nothing in any textbook that says that everybody or even that most people will automatically benefit. It's possible that the pie can get bigger and some people don't participate. Now, it may be a very small group that doesn't participate, but in theory, it could be more than half the population that isn't joining in that bounty. And the data suggests that recently we haven't created the kind of social and economic and technological system that has created the shared prosperity that we had in earlier eras. And the economic term for this is biased technical change. And let me quickly describe to you three kinds of biased technical change. I wanna get into some time for questions and answers as well. And this biased technical change means the technology is helping some groups more than others. The first one that's been studied quite at length is skill biased technical change. And that looks at the differential effects on people typically with more education or less education. Sometimes it's more training or experience as other ways of dividing it, but it compares those different groups. And recent technologies have affected them quite differently. Here's a terrific graph from our MIT colleagues, David Otter and Daron Asimoglu, who studied this in some depth. And really, if there's been lots of people, we wrote some papers on it and others. And this collective literature basically describes how in the early part of the century there was a rising tide that lifted all boats. Then there was the oil shock and recession and everyone took a bit of a hit. But since then, you can see things are spreading out. People who've gone to graduate school or college graduates have resumed the increase in incomes. But people who are high school dropouts or don't have as much education are worse off in terms of wages in absolute terms. This is true for males. There's a similar pattern for women. All the whole thing is skewed upward a little bit. But if you control for demography and other factors, the key characteristic that shows that comes out is this change in education. So if any of your kids are thinking of dropping out of college or not going there, you might want to show them this chart before they make that decision. There's clearly been a change there. And it's not just one particular set of industries that, driving this, we have a whole set of different industries, starting with some of the more digital industries, into some of the other industries like retailing and finance that have been very much transformed. Manufacturing, logistics are clearly in the midst of a big transformation. Professional services and others are sort of on the cusp. We don't see it quite happening in education yet, but you can see that the beginnings of it are there. And maybe someday, even domestic help and other categories are all being transformed. The second type of bias technical change that we're observing is what we'd call capital bias technical change. Another way of dividing the world is between capital owners and those who provide labor. And for much of history, there was sort of a dance there where they kind of bounced around, but sometimes one or the other would have a little rise or fall. But you can see in the past decade or so, the labor share has been falling quite precipitously. Meanwhile, profits are at record highs, both in absolute terms and as a share of GDP. So things are becoming much more skewed. And again, you can imagine that technology is part of the story here. When someone like Terry Gao replaces thousands of workers, he says he's planning to replace as many as a million workers with industrial robots that can increase output, maybe do things in terms of efficiency and quality. But it also shifts the share of revenues that are being paid to labor to workers versus being paid to capital. And that's a transition that's happening on a worldwide basis. And the last kind of bias technical change, the way the technology can affect different groups differently, is what we call superstar bias technical change. It's perhaps most evident in some of the media and entertainment industries, whether it's Lady Gaga or Beyonce or others, they're able to reach global audiences now, digital technologies. You see it also in software. Here's Scott Cook. Maybe not quite as recognizable, but he's a billionaire, in part because he's the co-founder of Intuit that makes programs like TurboTax. And they have codified some processes that used to be done only by humans, but now tax preparation is often done by software. And if you can codify something, you can digitize something. And if you can digitize something, you can make copies of it. And 10 copies, 100 copies, a million or 100 million copies. And each of those copies has some unusual characteristics. They can be made for almost zero cost. They can be transmitted anywhere on the planet almost instantly at the speed of light. And each copy is a perfect replica of the original. You can't tell the difference between the copy and the original, they're the same thing. Free, perfect, and instant are three adjectives that we didn't use for most goods and services for the preceding couple hundred years or couple hundred millennia, for that matter. But they're standard for digital goods. They're just automatic. And they lead to some unusual economics. One of the things they lead to is power laws. Andy and I and Michael Spence, the economics laureate, wrote about this in an article last year where you get a power law economy, which means that there's a small group of people that winner take most, that dominate the industry, that dominate the income, dominate the revenues, and a longer tail, like the one that Marshall showed earlier of people who are also contributing, but it's not nearly as clustered toward the middle as it is for other types of goods and services. And it's not just media and music, but Mark Andreessen has said that software is eating the world. It's coming to all those other industries, to manufacturing, finance, retailing, all the other ones I listed earlier, and that means that those economics of digitization are becoming more fundamental to every industry. It's transforming the economy. It is why we're studying the digital economy. There are many aspects of that, but on the theme of biotechnical change, let me point out that we have seen a real shift, not just in income between college and high school, but up to the 1%, and it turns out the 1% have their own 1%, there's the 1% of the 1%, that's what I'm charting here, the 0.01%, and the share of income in each of those groups has been going up recently to points that were last matched just before the Great Depression, if that's any source of comfort there. So that's another pattern we see in the economy. So let me just say a couple of thoughts about what's to be done, and then I wanna open up for questions and get your thoughts. We need to think about how we can use these technologies to create more shared prosperity, inclusive innovation, more abundance, but we emphatically do not think that the answer is to put the brakes on technology and stop the bounty that's being created, but it's to think about ways that we can have the rest of us catch up with those technologies. As we said in our earlier book, digital technologies change very rapidly, but organizations and skills don't tend to change nearly as fast, and because they're not changing as fast, there's kind of a mismatch there, leaving lots of individuals behind, and also lots of organizations behind. Jan Tinberger, a Nobel Prize winner in economics, put it this way, he said there's a race between technology and education, and for much of history, education was keeping up. We would broaden that, it's not just education and the skills of individuals, it's the knowledge and skills of organizations, or organizations structures aren't keeping up. Marshall was very good at illustrating some of the remarkable new types of organizational forms that are not just feasible, but that are optimal in a world of digitization, and they're not as well understood as they should be. Marshall's work is helping to pave the way in some categories of that, but there's much more work to be done, and even when they are understood, they don't diffuse broadly, most companies aren't really adapting to them very well. And taking it up a level further to society, we have to rethink our policies and our organizations and our institutions at that level as well. In the first machine age, we made some huge changes in the way society was organized. One example is mass public education, mandatory public education, that was introduced in the early 1800s. That was a radical idea at the time. Some called it socialist or other things that seemed kind of strange to do that, but of course, looking back, it was essential to that transition when we went from 90% of people working in farms to 40% around the turn of the 1900s, and then to less than 2% today in countries like the United States. All those people didn't simply become unemployed, they became redeployed into new industries in part because they had been reskilled, their education allowed them to do jobs that they couldn't have done previously, and there's also invention by entrepreneurs, people whether it's Henry Ford or Steve Jobs or Bill Gates or many others who helped invent entirely new industries that took advantage of this labor force. That's the scale and scope of changes we're gonna need to make going forward. So I would argue that the new grand challenge for us is not simply to make a self-driving car or a robot that can sit in a car and drive it, but it's to understand that digital technologies will continue to accelerate and our skills, our organizations, our institutions are lagging, and therefore simply doing business as usual, improving technology by itself isn't going to lead to the kind of shared prosperity that I think we want. We need to reinvent our economy and our society to keep up with this accelerating change. So that's very much what the initiative on the digital economy is all about is to take advantage of this huge opportunity that technology is providing us and think about how we can accelerate the rest of society to keep up with these amazing technologies. So let me stop there and see what kinds of questions and comments you may have on that. Thank you. So we got a question over here in the back. My understanding of the issues you've described around inequality particularly is there are two alternative hypotheses. One is the one you've described. I think we can also go back to Marshall's talk earlier about open platforms and people capturing value that way through power wars. The other is that value is being captured through rent seeking, whether through land, commodities, real estate, particularly in this country or patents. And maybe you can comment on that second bit, please. Yeah, I think there's a lot of truth to that and I've had some discussions. Joe Stiglitz in particular, we've talked a bit about that and he's emphasized that as well as the other ones. And I think that when I have discussions with him and others, I think we agree that there's multiple factors at work. There's those two that you mentioned. A lot of people would add globalization to the mix. Many people would add changes in unionization and tax policy. So there's a mixture of different things. I mean, my view of the evidence, I see specific jobs being automated and the demand for certain types of skills, especially routine information processing skills dropping quite precipitously and that's reflected in fewer jobs in those categories, but also lower wages for the people to continue to work. I see the ability of technology to scale what's happening in certain talent or luck or insight and be able to replicate that to billions of people and that's generating a lot of wealth for those people. So I see quite visibly the mechanism by which some of this is happening, but I also see some of the rent seeking and other factors and the patent system to pick one of the ones you describe, it's not something that was given to us by God or preordained by the laws of physics. It's something that we wrote down as a way to, according to the US Constitution, encourage the useful arts and sciences or something akin to that. And the purpose was to motivate people to invent and innovate more. If, as you suggest, it's having a counterproductive effect and hurting innovation or hurting the ability of society to gain prosperity, then it's not serving that purpose and we need to rethink and reorganize some of our intellectual property laws and I think it's quite likely that many of them are due for refresh. They were well designed perhaps for the era that they were invented in, but they're not necessarily appropriate, say, in the area of software, at least not the way they're currently defined and that's one example of how we might wanna reshape our organizations and institutions around privacy, around security, around other aspects of the economy so that we continue to benefit from the technology but we can't expect that we don't have to make changes anywhere else, that would be a miracle. If the cost of information processing falls by a million fold to not have to make changes anywhere else would be very unlikely and in fact, not true. Take another question now, here's one right here, Jonathan. I enjoyed the talk very much indeed. Can I just press you a little bit on the relationship between skills and all of this? Between skills? Between skills and all of this. One's reaction is, of course, in this kind of economy where you've got plenty of opportunity for skilled people, we need more people to get to school and so forth. On the other hand, some of these superstars, these rock stars and movie stars and this that and the other, I'm sure some of them are very well educated but not all of them have PhDs from MIT and all that kind of thing. That seems to be an exact counter example, whereby actually the very thing that they would not have wanted to have done was to go to school and this that and the other, they would go out and do their stuff. So can you just say a little bit more about how you see the skills and education issue given it's so nuanced, I think? No, this is very important and thanks for giving me a chance to elaborate a little bit on it. Many of those billionaires not only didn't get PhDs but a large number of them dropped out of their undergraduate institutions and still managed to do quite well and I think that underscores the fact that there's not just one kind of bias technical change, there are at least three and they're having different effects and different parts of the income distribution. I think what's going on with the billionaires, the top 1%, the top 100th of 1% isn't well described as skill bias technical change. There's something else going on there. The ability to replicate talent globally I think has more to do with that or luck or whatever it is that you want to describe that those people are bringing to the table. Something, some special quality and even on the skill bias technical change, I showed you the chart and one thing I didn't really emphasize was that while it had fanned out, the last 10 years or so it's kind of been level. I don't know if you remember how well you remember the chart. I won't go back to it as a few slides back but it was kind of leveling out and what that means is that skill bias technical change doesn't seem to be getting worse. It's still at a all time high as it was a few years ago. It's sort of been staying at that high level but that may not be the main story and furthermore, as one of my charts suggested, the nature of the skills that are automatable or substitutable versus complemented are constantly evolving and changing. When you hear things like machine learning and other things, I think Andy may get into some of the more recent developments. You can see that the types of things that were automated in the 1990s aren't the same things that are likely to be automated now. So skill bias technical change is evolving rapidly. There are other things going on especially at the top of the income distribution. It's not a simple one story solves everything. It's one that requires a research agenda to be frank. Do we have time for another question? David, are over here, yes? Hello, yeah. I was wondering if you've expanded your research to some international aspects. In particular, I was wondering about China and India where it seems to me that the educational segmentation is probably even greater than the US and maybe the social issues are many times more serious. I share your impression. And one of the things that we've been doing is expanding that agenda. And one of the reasons we're here outside the United States is to start getting some more international perspectives. It's partly data constrained. Thankfully, Roberto and others are giving us better data in other countries. To some extent, it's the old story of the drunk in the lamppost that we go where the light is and the illumination and we get a lot of data from the United States. We'd love to have better data from China and other countries so we could analyze this more in depth. But one kind of data you get from just visiting the places and seeing what's happening on the ground and we've done that and we've gone to those factories. And you may have also seen there are lots and lots of people working side by side, working on lots of tasks there. Far more labor intensive obviously than most American or British factories which are often essentially lights out, no labor. But also, I think, much more vulnerable to Baxter and that kind of automation because precisely because they're doing a lot of these simpler repetitive tasks. To be quite economic about it, the strategy of having lower wages is very effective for competing against Germany or Britain or the United States. If you've got workers that are essentially human and they can do the task more cheaply, you can do some labor arbitrage and beat your competitors that way. But that strategy of low wages that helped bring many Chinese out of poverty and get them a lot of the big share of the world's manufacturing, isn't a particularly good strategy for competing against Baxter or other robots that are getting twice as efficient every year or two. I wouldn't wanna be competing with Baxter based on lower wages. And that makes them in many ways much more vulnerable and Terry Gao, an entrepreneur, is seeing that kind of transition there. And ultimately, the last thing you said, I just wanna underscore, it may be that the second machine age has an even bigger impact on some of those countries, China and India or others, because they're gonna have to make that transition to the new kinds of skills that are needed in the second machine age even faster. They, in some ways, start at a different point than the more developed countries. And I think we could get one more question. Who are you pointing to? Right here, okay. Thank you. No, I see very well what you are explaining, new technology making certain job obsolescence, right? And then you need to really blow it. I see all of that. My question is, is there also something more profound going on due to interconnection? When you interconnect many agents' system together, you have typically the emergence of new behaviors at a different scale, and so I would like you to comment on that. Yeah, thank you. That's one of the most important phenomena that's going on. I touched a little bit on digitization. Another thing we talk a bit about is networks and the fact that we now have not millions or hundreds of millions, but literally billions of brains connected. And they not only can access, Wikipedia and all the other knowledge sources, but perhaps more importantly they can contribute to it. And they can share ideas with each other and create new kinds of organizational forms. Marshall mentioned Uber and there are a whole lot of other things with mobile phones, for instance, are enabling in the developing world. The word emergent behavior is a good one because it's a different kind of economic organization than the traditional market or traditional hierarchy. It's much more of a networked form of organization. And a big part of our research agenda going forward is to understand better these different kinds of organization. And we don't think there's just one. We think there's multiple different kinds that are being invented and evolving quite rapidly and it's a new kind of economics, but the network aspect of it is I think ultimately going to be at least as important as the digitization and as the artificial intelligence aspect of it. So I'm looking at the clock here and I'm afraid I'm out of time. It's been really good to have a chance to do this Q&A. I enjoy that very much. Another great pleasure at MIT is interacting with the incredibly smart colleagues and you've heard from a couple of them. The one who I've been spending the most time on, revealing my preference of who I've really been able to work quite well with, is my colleague and co-author and co-founder of the Initiative on the Digital Economy, Andy McAfee. He's not just one of the smartest guys that I know there, but also selfishly, he makes me feel smarter when I'm with him and that's always great to have in a colleague and I think he's gonna make you feel smarter when he shares with you some of his insights. He has the ability to cut to the core issues and really illuminate him in a way that even I can understand and I think you guys will be able to understand as well. So please join me in welcoming Andy McAfee.


Everybody forced here up in 2023 Bowman Chrome baseball hobby 12 box case number 28 pick your team style Alvin has the raise and on the twins Charles a nationals and Rangers David S. Phillies Gabriel C Dodgers Padres Giants and Mariners Jeffrey H the Orioles ashows Yankees Jeffrey two different Jeffrey H Jeffrey H a has Orioles ashows Yankees Jeffrey Okay, the last names are similar Jeffrey H 2300 23 Orioles ashows Yankees Jeffrey H 230 537 the A's Jim G Rocky's Brewers Kenny K Braves Dimebacks Kim has the Guardians Lynn with the Marlins Matt and tigers Peach AP Reds Raymond are Cardinals rich F Red Sox Robert L with the Mets Ryan Ryan B has the Blue J Scott F White Sox Steven you Pirates Thomas O Royals Tyler B has the Cubs and the Angels. Good luck here everybody Joe burrow and two You jaked out there wearing a Buck nears Jersey Moe is wearing his dolphin Jersey Not a real fan. I didn't think so. No one's not wearing any Patriots stuff today He's not a real fan Who is crashing hard early Herbert a bro. Herbert's playing well. Yeah, Herbert's playing great The chart the lease of the Chargers where he's is Justin Herbert first refractor is Espinoza. It's a 499 for the Cubs I'm each you guys first that's good There's plenty of football spots for the night including our 30 box mixer later 30 box mixer with the promo cam Collier came into the league First auto you on the Rivera to 499 Nationals first refractor auto blue refractor is Jeremy Mania number to 150 Astros Celestine base Well, are the Giants the worst team in the league?  They look like it this week They're down seventy nothing to the Cardinals. Yeah, they stink Got Diddy Gregorius number to 499. He's still free agent, right? Roll baseball classic flag DD. So he's a free agent. So that should go to the Yankees, right? Yeah to go to the Yankees Gregorius Yankees First base auto tour in Montgomery for the Marlins green shimmer to 99 Sir Mark Johnson you one of the pirates Pretty missed the wide open past there.   How's it going? So Nicholas? shimmer Jason choreo for the Guardians That's my son too I don't know Dylan seems fine to me said go Pats, you know, where he gets his fandom Good good good for Dylan good for her. She wouldn't let him get tainted with the Miami. Oh Jim you got I think you got that tonight. That sounds good Let's go Yeah, Steve's always out of his mind pink fuchsia vapor wave to 199 Spencer Jones Yankees Sorry for the the Pats, I don't know you're trying to say Invicta William Bergoglia Jordan Walker came to the league That's a shame Speckle first moment auto one corny L to 299 for the dime of X the fracter rookie Taj Bradley raised the 499 That's what they did last year it worked It went to the playoffs pages feds a pawn every single game Got a magenta Bryce Harper number to 299 for the Phillies Rikki refractor Nolan Gorman for the Cardinals number to 499 Speckle first woman auto Tigers Javier Osorio to 299 Osorio Tigers future shimmer first woman is Jose Gerardo number to 199 Marlins Fractor rookie not refractor. Sorry short print rookie Oswald Parazza Yankees It came to the league Corbin Carroll. What up Mitchell? It's not Mitchell first woman auto trey paltine for the Reds Gold refractor is O'Neill Cruz number to 50 for the Pirates now. I have box four Wait Steve like so the Giants Giants lose. Well, let's go Giants and get that comeback win Let's go Giants, baby feed sake one Feed the beast mean refractor now is a pojung wong Number to 299 believe that's gonna be random at the end. Got atomic first auto. You'll see galon the 100 nice galon Texas congrats on that we'll get that in a mag for you shimmer first is Camilo Diaz for the Astros Must be nobody about a score watch the red zone. They just keep showing like first-pounds second down nothing going on We were Fractor first bowman Jacob Reimer Mets to 150. I even saw see how you guys made me sad last week I'll go against all Eagles fans from now on got Kate Horton and Victor for the Cubs I was pretty upset that the Vikings lost on Thursday Yeah, she did it came to the league They probably won't Steve so you're good on that one first auto Joey Estes for the A's Yeah, I doubt the the Red Sox ever went a postseason game again gold refractor nice no line We're not over the Cardinals at a 50 Not over the Cardinals Check this card here nice Maybe they are At this point you may as well if you're gonna lose you're gonna be losing to the Cardinals you may as well There's gotta be a way to get out of that oh tip tip to pick Wasn't even Jones' fault at all at all Tough year, dude. I guess it kind of was though because it was kind of locked up.  They're gonna get immediately good Aqua ray wave is Mookie Betts to 199 Dodgers Invicta Henry Bolt for the A's Riley Green it came to the league Base first auto Jarrell or a tega for the twins now first refractor Ethan Salas 240 of 499 congrats to the Padres I'm gonna get that solace in a mag Like six or seven orders just came in Next break still at four tend to go in the choice case Yeah, still four to go in the next case 11 to go now on spectra Sunday fun day has passed halfway full that's at 15 to go now 30 box football mixer With the promos 15 to go Go Steve That's what I'm talking about That's what I'm talking about Yeah, isn't everybody Kyle you watching this giant scheme Stafford likes puka Baker may feel on that uh mvp path That's the fanpointed to the wall What are you doing doing there? What a mess They must like the way it feels outside Green refractor is Deedee Gregorius again the 99 for the Yankees What's up, Anthony? Oh, we're seeing Anthony reminds me that the trash list is tonight Oh, yeah First one with speckle auto to 299 is Osorio for the Tigers He's not on it All right, they didn't make it Oh, yeah, Anthony was pretty good this week actually Yeah Never once came in chat Bothered Sean No Got shimmers of holopadres No Matter of fact, I mean you got You got Steve you got kyle in there and uh And anthony Purple Jackson holiday to 250 for the Orioles Yo, Andrew Vargas Invicta of hour for the rays Let's go Sean Guy came to league Francisco Alvarez for the Mets Looks like the Lions could have used a uh are really really good running back today But uh that guy's on the Eagles now.  What? He dropped the pick right in his hands Refractor Angels Nelson Rada at a 4.99 We'll get acres and rams I don't know I don't Does anybody want to trade for a running back right now? Yellow to 75 auto Johnny Perron to 75 There we go raise Fuchsia shimmer to 2.99 Gavin Cross for the Royals The prospect Fuchsia shimmers to 1.99 Banana bandana Where do I get that Steve? Is that like a local beer to you? Very Wilson long touchdown Yellow Refractor first woman Davidson Gutierrez To 75 for the Mets Invicta Ideas For the Dodgers What is his camera doing? Solace Well they're just going to tackle it First base auto Christian Mena for the White Sox Marcus Strowman going to the Cubs Rolled baseball classic flag Number to 4.99 Marlon's gone on the next break Cubs gone So 3 to go on the next one 10 to go on the choice case Actually for that in just a moment here 22 to go on Luminance 5 to go on PD1 The NFC stinks Cowboys having trouble against Zach Wilson Giants can't win They can't even score on the Cardinals Commanders getting destroyed That's the worst division in the league Refractor Jordan Alvarez The 4.99 for the Astros Mets first woman auto Vincent Peroso Purple shimmer First Carter Young for the Orioles Equals best team in the league 250 on the Purple shimmer sorry So false That did make me laugh a bit First Refractor again Ethan Solace for the Padres 197 to 4.99 Get that one dusted off And get that one into a mag Brewers have a ton of guys Luis Lara What is the guy's name And who's the shortstop Why can't I think of him Checklist I got it though now it's killing me Yeah they probably have the most first autos in this Who is it Who is it They have more Lara Oregi Rodriguez Metsinger and Eribe Prospect autos Two of them are the twins have the most Saus just missed a big six Saus dropped it First Refractor Autos is Brennan Malone into 4.99 for the A's That is a Red Aqua Vaporwave Jonathan Mejia 105 for the Cardinals Yo Cardinals congrats on that one Celestin base Oh John's got it first down Yay Fuchsia rookie Tristan Casas for the Red Sox Number 2.99 First Bowman Aqua Anderson de la Santos Number 2 125 for the Orioles Base First Bowman Auto Rangers Daniel Mateo Shimmer First Bowman Elion Blue Refractor here It's going to be Alec Thomas Diamondbacks World Baseball Classic Flag to 150 Michael Harris First Bowman Auto Red Sox Luis Perales Lindor Refractor to 4.99 for the Mets David Taken the They got a choice spot Let's see where we're at here We've got Random Team's left Random Team's choice and then two teams left on the hobby case next Only Brewers and Rays Yeah they're blowing up your phone Here's to Atomic To 150 Fall League for the Orioles Invicta Crawford for the Giants Gold Speckle Here's a First Bowman Auto Hans Montero Number 19 out of 50 Great Haiti Yankees Now Ratsy Yankees will get that in the Mags for you Miguel Cabrera Tigers Refractor to 4.99 Rams looking decent Fuchsia Mookie Betts Number 2 3.99 for the Dodgers Purple Refractor is Tamar Johnson to 2.50 Pirates Base First Bowman Auto Gordon Graceffo for the Cardinals Fuchsia Shimmer Jackson Chorio for the Brewers Number 199 It's an interference, right? It has to be an interference He didn't turn around It was definitely an interference Ethan Salas I guess they didn't call it the First Bow of the War They're also going to be injured then Bucks 2-0 Baker undefeated Fuchsia Rookie Joey Ortiz Number 2.99 for the Orioles Atomic Jefferson Carol for the Brewers The 1-50 Fall League Insert Base First Bowman Auto Junior Franco for the Diamondbacks Juanipa Maella Another flag on him Shimmer first Shadaira Arinamo Brewers Blue Refractor is Keita Lanza to 1.50 for the Mets Invicta Cutter Coffee for the Red Sox Big ol' ding on the Invicta Boxin' Rapper going out to the Red Sox It came to League Corbin Carroll First Auto Ricardo Olivaar Going to the Twins now Olivaar Altoobay World Baseball Classic Flag Number to 4.99 Astros X-12 Pretty Sneak That was a good game Rams are decent this year it seems like Yeah, after slinging it man Slinging it doesn't have a cup yet do they? Still no cup so I mean But I mean hanging in there with the Niners Winning last week I mean right Rams are alright NFC that's good enough to make the playoffs If you're just alright you can make the playoffs Still just 2 to go in the next break everybody Again it's the Brewers And the Rays on sale Every Just about every game is road game for the Rams Just about but especially when the Niners are there First Refractor is Adon Sanchez for the Cubs Number to 4.99 Or if they were ever to play the Raiders at home Atomic First Audio Dario Laverde 77-100 Angels now Yeah unfortunately Somebody from the NFC North will in fact make the playoffs I don't probably be the Packers I would think I think they'll get better as the season goes along Shimmer first is Osorio for the Dodgers here I could see Lyons and Packers both making it Lyons offense is really good But defense don't need some work I think the Packers defense is solid Offense needs a little bit of work Sinvicta, Boxer 2-0 Saying what up Yeah man 2-0 let's go Aqua to 199, Xavier Isaac for the Rays Bowman Ascension Christopher Caro Nationals Walker it came to the league Purple Caro Yaniel Laboy to 250 It's Ya Boy Laboy And an Aqua Ray Wave on the back of the break Is Javier Baez to 199 Tigers Thank you Brock grabbing a Random Team Choice spot Okay so we have the Pojun Wong On the World Baseball Classic Flag to 99 Random to everybody Remember some top gets after 6 1 2, 3, 4, 5 And 6 Got the Reds No Steve you were talking mad trash last week No Reds up top So we had that by as the 199 The rest of the numbered cards here Was the non refractors So the refractors aren't in this So it's anything under It's like fuchsia and lower So to 299 was Ortiz Betz, Kostas, Harper 250 is Melton Young, Holiday Tamar Johnson to 199 Gavin Cross, Spencer Jones To 125, Isaac And to 199, Betz To 199, Chorio To 150, Reimer, Kirstad And Caro, Pitalonzo as well To 125, De Los Santos To 150, Peña to 99, Gregorius Tamar Johnson That is a Gutierrez to 75 To 75, Alec Thomas To 50, O'Neill Cruz Nolan Aronato, Tim Anderson Into five is Jonathan Mejia Autos now of Franco Graceffo Perales, Mateo Peroso That is Mena Elliott, Cortega, Sang Estes, Faltine Wisely, Montgomery To 499, Malone And Rivera To 299, Osorio Corniel and Osorio To 250, La Boy To 100, La Verde to 75, Peron Yeah, they're not good, Nate They're not good To 50 is Hans Montero For the Yankees Goldspeckle Auto Refractor Ethan Salas Twice the 499 And Atomic is Josie Galanado The 100 for the Rangers That does it for the break Thanks again everybody, moving on now to number 29