Wednesday, October 7, 2015

Building a car (from the perspective of a pragmatic software engineer)

The ideas for building a car from the perspective of a pragmatic software engineer came up during the series development of an Internet of Things (IoT) platform and control unit for a German automaker.
The automotive industry is currently undergoing a disruptive technology change mainly driven by software. The self-driving (electric) car will not only change the automotive industry but also society and urban life.
To enable this technology change, the automotive industry finally needs sustainable software platforms that allow for building reliable and scalable software systems in a very short time. (And I am definitely not talking about AUTOSAR here.)
Hence, the automotive industry needs to build software platforms including software development kits (SDK), great APIs, documentation and tools. The automakers must design for simplicity, modularity and security. This requires them to look at the car as a (classical) distributed system of computers and to find good abstractions to easily manage each system and the whole system of systems.
Today, I often see a lot of bad abstractions or no abstractions at all in automotive software systems. That results in slow development cycles, way to much (unreadable) code which is hard to maintain, and at worst, unhappy developers.
Having some good fundamental abstractions would allow for building applications from a set of reusable components that form the basic building blocks of this new software platforms. (And no, I don't have AUTOSAR Software Components in mind.)

So what kinds of software platforms do you need to build a car?
  • Deeply embedded (body) domain (engine, powertrain, light, etc.)
  • Infotainment domain (Head units, dashboards and head-up displays)
  • Internet of Things domain (secure cloud connectivity, eCall, WiFi hotspot, car sharing, over-the-air updates)
  • Driver assistance systems for self-driving cars (highly reliable, real-time number crunchers)
  • Private cloud (machine learning platforms to teach your self-driving cars, data mining to analyse customer behavior, online services like roadside assistance, etc.)
Deeply embedded (body) domain
Automotive real-time software systems of this domain run on embedded controllers with little CPU power and few memory. Therefore, application frameworks and libraries for these kinds of systems avoid or lack dynamic memory allocation and many other well-known abstractions. We have built our own application framework for this kind of deeply embedded systems, called Mindroid.ecpp.

Infotainment domain
Systems in this category are often the same as today's high-end smartphones. So why not use today's smartphone platforms for these kinds of system, or at least parts of it.

Internet of Things domain
Systems in this category are often the same size as today's mid-range smartphones. So why not use today's IoT application frameworks and smartphone platforms for these kinds of systems.

Driver assistance systems (for self-driving cars)
Hardware and software systems in this category are really big computing beasts. Due to its reliability and safety (ASIL) requirements, these are the most challenging and interesting automotive software platforms of the future.

Private clouds
Well, private clouds. Let's do it with Erlang technologies! (Or with any other well-known open source infrastructure.)
To be honest, building a machine learning and data mining platform really is some work to do. Today, only IT giants like Google, Facebook, Amazon, Apple, Microsoft, etc. own and operate such platforms. Hopefully, this will change.

Now let's bring the various car domains (yes, also the private cloud) together. As we learned from Alan Kay, the key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be. Hence, the connected car needs a simple, clean and robust messaging paradigm.
Choosing form the classical messaging paradigms of distributed systems, I think topic-based publish-subscribe messaging fits best for most use-cases. It allows us to build loosely coupled, flexible, scalable and easy to test (asynchronous) software systems. And it avoids dependencies, something that is really important when evolving systems. Furthermore, a publish-subscribe messaging infrastructure should support quality of service levels for at-most-once, at-least-once and exactly-once delivery to simplify the overall software architecture.
One can also add synchronous messaging systems, like Remote procedure calls. I always try to keep synchronous remote procedure calls to a minimum in order to avoid some of the dependencies (and shared state) they often introduce into distributed systems.
What data to exchange with the publish-subscribe messaging between the domains? Well, let's define a domain model for representing real-world (domain-specific) objects, processes and rules.
E.g. speed, mileage, door states, window states, sequences to safely open the car by a smartphone app, and so on.
Applications must only know about the application framework, the publish-subscribe mechanism and the domain model. They should definitely not know about which protocol to use for domain model exchange, serialization, model binding and many other things. This has to happen within the application framework and a publish-subscribe messaging service.
To evolve the domain model over time, protocols like Google's protocol buffers that allow for changing message formats without breaking backwards-compatibility provide great flexibility.
Using a publish-subscribe messaging mechanism abstracts away all the quirks of automotive bus systems and protocols like CAN, LIN, FlexRay, MOST or even Ethernet.
And of course you never have the "one size fits all" abstraction. E.g. besides publish-subscribe messaging you still need H.264 video streaming over RTP to connect your cameras to a driver assistance system, and others.

To sum up, the automotive industry should build application frameworks with a few simple building blocks for each domain and provide a variety of system services on top of it, e.g. logging, publish-subscribe messaging, power management (using wake locks and leases), etc. - just like Google did with the Open Handset Alliance.

Advice for ambitious system architects and application framework engineers
  • The most dangerous enemy of a better solution is an existing codebase that is just good enough - Eric S. Raymond.
  • Seriously think about threading and message-passing. Get the damn threading right by the application framework for the application developers. You are developing an application framework for developers, not against them!
  • Modularity is there for reusability of components and to ease refactorings within software systems while trying to minimize dependencies between the components. Try to minimize the number of control units within a car!
  • Keep an eye out for beauty and aesthetics in software systems. This has to do with software architecture and naming. Naming is so important!

This is my personal opinion about software engineering in the automotive sector, not that of the company I am working for, nor of anybody else in that company.