The State of Workflow

When talking about an relational database management systems (RDBMS) in a software development team most people will get the picture and shake their heads slightly up and down confirming they understand what you're saying. When using workflow terminology, the same crowds will shake their heads similarly but this time, every person will have a very different picture.


If database systems are like a respected, wise man telling a clear story, then workflow must be a buch of spoiled...

brats shouting their own truth at each other. With this statement I would like to point out that workflow management systems are at the very initial phase of the technology hype curve. We are heading for some exciting times in this area. To illustrate this, we can make the comparison with relational database management systems (RDBMS). When talking about an RDBMS in a software development team most people will get the picture and shake their heads slightly up and down confirming they understand what you're saying. When using workflow terminology, the same crowds will shake their heads similarly but this time, every person will have a very different picture.

Figure 1 : Workflow positioned in the hype-curve

One of the observations that form the basis of this statement is the overwhelming number of concepts used. None of the numerous specs and tools in this area is similar. Of course, they tend to overlap but it usually takes a dictionary and an extended concept mapper to compare any 2 of those.

A topic that certainly deserves to be included in an introduction about workflow is the relation between workflow and business process management (BPM). The term workflow is used to denote a series of related interactions between people and a computer system. Workflow is more often heard in developer communities. Sometimes, the meaning of workflow is restricted to the different screens of a user interface. Business process management (BPM) has a broader scope. Where workflow is more often used in a technical context, BPM has a broader scope. It implies also the non-technical issues like analysis, organizational impact, from the viewpoint of a manager.

First I start with explaining what workflow management systems are. Then, the advantages of business process management are covered. Next is an explanation of why the workflow market looks confusing at first sight. The main conclusion will be that selecting a workflow management system is the hardest task companies have to face. To give you a solid basis for this task, the core section of this article defines the four layers of a process definition. It describes the common workflow & BPM concepts in a neutral language. At the end, a guided tour along the specs and tools is added.

What is a WFMS?


A workflow management system (WFMS) is a software component that takes as input a formal description of business processes and maintains the state of processes executions, thereby delegating activities amongst people and applications.

To get started, let's define the basic terminology: a process definition and a process instance. A process definition is a formal description of a business process or procedure. A process instance is one execution of a process definition. So far so good. But when going one level deeper, we need to be very careful of the language we use. The way to express the steps within a process still hasn't found a common ground yet. That is the major difference between the specs and tools for workflow and BPM.

Why the term 'activity' should be banned...
Process definitions are usually expressed in terms of activities. I think *that* is the main reason of all the confusion in the workflow space and therefore a bad idea. I'll tell you why: the term activity blurs the distinction between a state and an action. A state (aka wait-state) in a process specifies a dependency upon an external actor. At process execution time, this means that the workflow engine has to wait until the external actor notifies the WFMS that the state is finished. E.g. waiting for an approval. An action is a piece of programming logic to be executed by the WFMS upon a specified event that occurs during process execution. The WFMS initiates the execution of the action on a specified event during process execution. E.g. sending an email when a state is assigned to an actor. As you can see, state and actions are really different so using the same name for these concepts is a bad habit. My proposal is to avoid the term 'activity' and replace it by either 'state' or 'action'.

Another important responsibility of a WFMS is maintaining context information for each process execution. A process context variable -or variable for short- is a variable associated with a process instance. E.g. the start-date of a holiday request, the key of a record in the database, a reference to a document in the document management system, ... Usually the process definition declares the process context variables. Then, at process instance creation time, the variables are instantiated. All mature WFMSs have customisable variable types.

Target usage

One option is to use a WFMS as an enterprise application integration (EAI) platform. Currently, in most enterprise environments, the IT infrastructure has various heterogeneous applications and databases running in an intranet. Typically, these applications have a clear purpose when they are introduced in the organization. Examples are customer management, document management, supply chain, order entry, billing, resource planning... Let's call them dedicated applications. Each of these dedicated applications contain knowledge about the business processes they have to support. The enterprise fits these automated processes of the software components into their overall non-automated business processes. Once such dedicated applications are installed and running, opportunities arise for new feature requirements that span over multiple applications. Enterprise application integration (EAI) is the discipline of implementing software requirements that involve multiple dedicated applications. Sometimes this is only data exchange that requires the implementation of a communication channel between two applications. Dedicated applications incorporate a number of business processes hard coded in the software. You could say that you buy a set of fixed, automated business processes. A WFMS on the other hand has got no prior knowledge of the domain. A WFMS takes a description of a business process as input and manages the executions of the process instances. That makes it much more flexible then dedicated applications but you have to do the effort to create a formal description of the business process. That is why WFMS and dedicated applications are complementary. A WFMS can be used to manage the overall process. As a rule of thumb, you could say use a dedicated application if it supports the processes you need. The first way to use a WFMS that was discussed here is to tie together all dedicated applications and create an EAI-platform with it.

A second option where a WFMS delivers high added value is for the development of workflow software that has a lot of people-related tasks. For this purpose, most WFMSs have a very convenient mechanism of creating forms for tasks. This target usage is especially productive in organizations focussing on ISO or CMM certification. Instead of documenting the procedures in a text format, a WFMS allows you to create automated support (e.g. by means of a web-application) of the procedures that are modelled as process definitions.

The third option is to embed a workflow engine inside another application. Remember from the first option that dedicated application incorporate a set of fixed business processes for a specific domain. The companies that develop the dedicated applications, can embed workflow engine software inside of their software. The workflow engine is in this case only used as an application component, hiding it from the application users. The main reason for embedding a workflow engine inside an application is for reuse (not reinventing wheels & hot water) and maintainability of the application software.

The case for workflow

Introducing workflow in an organization delivers benefits on the software development level as well as the business level.

Easy development

A workflow management system will ease development and even more, the maintenance of standard enterprise software.

  • Reduced development risk - the business analyst will talk the same language as the developer (in terms of states and actions). That implies that the developer will not have to make a translation from user requirements to a software design.
  • Centralized implementation - it's the business processes that change so the biggest advantage of using a workflow system is that the implementation is not a fuzzy combination of software pieces scattered over various systems.
  • Rapid application development - your software is freed from the task of keeping track of the participants in a process, leading to faster development and code that is better maintainable.

Business process management (BPM)

Before you can automate your business processes, the toughest, but mostly rewarding task is analyzing them and creating a formal description. has a nice description of the business benefits gained by this analysis.

  • Improved customer service - consistency in the processes leads to greater predictability in levels of response to customers
  • Flexibility - software control over processes enables their re-design in line with changing business needs
  • Business process improvement - a focus on business processes leads to their streamlining and simplification

I think they even missed the most important reason to start using a WFMS : reducing the risk of software development. If the business process part of a software system is not easy to change, organizations tend to spend a lot of effort in business process analysis and hope they'll "get it right the first time". Well, sadly enough, that is seldom the case as in any software development project. A WFMS makes it easy to deploy new versions of business processes. As a consequence, the use of a WFMS enables more efficient and risk-reduced development because the business process software can be developed in an iterative way.

Missing link

I really believe a WFMS is the missing link in enterprise development. First of all, I want to state that the default way of incorporating business process logic into enterprise software is a scattered approach. This means that the logic of the business process is spread over various systems like the EJB-container, database-triggers, message broker and the likes. No need to mention that this leads to unmaintainable software. As a consequence, these organizations will think of changing their business process software only as a last resort. They often prefer to change their processes to the software. This argument also applies to a larger extent to ERP software packages. In a second attempt, suppose we are aware of this problem and really want to centralize all code related to one process. This is fine for one business process, but if you implement multiple processes, you see duplication of code that manages state and process variables. Then, in a third approach, we could extract the duplicated code and put it in a central library. Well, that is actually a WFMS so don't bother implementing it yourself and choose one from the list below.

A workflow management system


A WFMS takes process definitions as input. For the moment, think of a process definition as a UML activity diagram, a UML state diagram or a finite state machine. Examples are submitting an expense note, asking for a holiday, requesting a quote, etc. Then, the WFMS maintains the state of the executions of these process definitions. For this, it needs to be informed of state-changes. The state changes of the process executions can be logged for monitoring purposes. The three main interfaces of a WFMS are :

  • definition The definition interface of a WFMS allows process developers to deploy process definitions. Note that the actor 'process developer' is a placeholder for the combination of a business analyst and a software developer.
  • Pitfall
    Many vendors of workflow management systems will let you believe that with their graphical process designer it only takes a business analyst to create process definitions. This vision originates from the fact that programming is hard. The vendors' sales people like to say "look, you don't have to write a single line of code". While not having to write code is a good thing, most vendors go overboard on this and don't foresee a mechanism to integrate code into a process definition in cases where this is appropriate. Remember from the target usage that when using a WFMS as an EAI-platform integrating code into a process is a must. Developing process definitions requires input from -and collaboration between- a business analyst and software developer. A good graphical process designer should support this collaboration.
  • execution The execution interface allows users and systems to act upon process instances. Process instances are executions of process definitions. The control flow of a process definition is a description of a state machine. The two main methods in the execution interface are starting a process instance and signalling the end of a state to the WFMS.
  • application The application interface denotes the WFMS-initiated-interaction between a WFMS and external systems. When a user or a system manages the execution of a process instance, events are generated (e.g. traversal of a transition). A process definition can specify that a piece of programming logic that has to be executed upon an event. The program logic can communicate with other systems in- or outside of the organization.
  • monitoring With monitoring interface, managers can gain insights through extracting statistics on the execution logs of the processes. Sometimes, the execution logs are also referred to as the audit trails.

These are 3 of the 5 interfaces as defined by the reference model of the WFMC.

The 4 layers of a process definition

In the next part, I try to answer the question "What is the content of a process definition?" This synthesis is based on principles and concepts that are used in the models of various specs & tools. It reflects the common ground between most of these models. The content of a process definition can be sliced into 4 different layers: state, context, programming logic and user interface.

The state layer

The state layer of a business process is all about expressing states and control flow. Control flow in standard programming languages is derived from Von Neuman architectures. There it defines the sequence of instructions that have to be executed. This control flow is determined by the order in which we write the instructions, if-statements, loop-statements, and so on. The control flow in a business process is basically the same. But instead of using an instruction as the basic element, the basic element in a business process is a state.

A state (also known as wait-state) in a process specifies a dependency upon an external actor. The meaning of a state is like "Now system X or person Y has to do something. Wait here until an external trigger of that actor signals the completion of the task." A state defines a dependency on a result provided by an external party. A typical example of a state is an approval step.

A state in a process definition also specifies which actor the execution depends on. The names given to these actors correspond with the notion of swimlanes in an activity diagram. With this information the WFMS can extract tasklists, which is the most common feature amongst WFMSs. As said before actors can be people or systems. For the states that require a human actor, the person has to be calculated at runtime by the WFMS. The dependency from a WFMS on the organizational data originates from this calculation. A very interesting article on this topic is 'Organizational Management in Workflow Applications' mentioned in the further reading section.

The control flow of a process definition is a set of states combined with the relations between the states. The logic between the states specifies which execution paths run concurrent and which are exclusive. Concurrent paths of execution are modelled with forks and joins, while exclusive paths of execution are modelled with decisions and merges. Note that in most models, every state has an implicit merge in front of it.

Often UML activity diagrams are used to model business processes. While it is an intuitive and widespread notation, a major problem poses in the diagram interpretation. In UML activity diagrams, no distinction is made between a state and an action (see 'Why the term 'activity' should be banned) They are both modelled as activities. The lack of this distinction (referred to as the lack of a state concept) is the most heard criticism by the academic community on UML activity diagrams. A second problem with UML activity diagrams was introduced in UML version 2.0. When multiple transitions arrive in an activity (read state), previous versions specified a default merge interpretation. The 2.0 version changed this to a default join, which requires synchronization. In my opinion, the graphical part of UML activity diagrams can still be used to model the state-layer of business processes on the condition that the semantics of these 2 constructs are changed as follows:

  1. in the diagram presentation of a business process, only model the state layer (states and control flow) and leave the actions out of the diagram. This means that every rectangle in the diagram should be a state instead of an activity
  2. if multiple transitions arrive in a state, define the default to be a default merge without synchronization

During process execution, a token is the pointer used by a WFMS to keep track of the current state of a process. This corresponds to the notion of program counter in Von Neuman architectures. When a token arrives in a state, the token is assigned to the external party for which the WFMS is waiting for. The external party can be a user, a group, or a computer system. Let's define an actor as a user or a system that acts upon the execution of a process. The assignment of an actor to a token is the only occasion where the WFMS needs to access the organizational data. This assignment allows the WFMS to extract task lists.

The context layer

A process context variable -or variable for short- is a variable associated with a process instance. Process variables allow a process developer to store data during the lifetime of the process instance. Some WFMS's have a fixed set of types, while in other systems, its possible to define your own custom types.

Note that variables can also be used to store references. A variable could reference e.g. a record in a database or a file on a network drive. When to use references typically depends on whether other applications use the referenced data.

Another interesting aspect related to process variables is how a WFMS turns data into information. Workflow is all about orchestrating the tasks and data between various heterogeneous systems across an organization. For the human tasks of a business process, the WFMS collects the relevant data-items from the other involved systems like e.g. SAP, databases, CMR-system, document management system. In each human step of the business process, only the relevant data-items are shown that are collected and calculated from these heterogeneous systems. By presenting only relevant data-items, data that resides on the disparate systems is transformed and presented as information.

The programming logic layer

Recall that an action is a piece of programming logic to be executed by the WFMS upon a specified event that occurs during process execution. With programming logic we mean a piece of software in binary or in source format, in any programming or scripting language. The programming logic layer combines all these pieces of software with the information that specifies on which events these pieces of software need to be executed. Examples of programming logic for integration are sending an email, sending a message on a message broker, fetching data from an ERP-system and updating a database.

The user interface layer

When an actor triggers the end of a state, usually that is the event where data is fed into the process variables. E.g. in the holiday example, when a boss signals to the WFMS that the state 'evaluation' is done, the boss provides the value 'approved' or 'disapproved' into the process. Some WFMSs allow to specify for each state what data can be fed into the process and how it should be stored into the process variables. From that information a user interface form can be generated to signal that requests information from a user. To see an example of a generic webapplication that allows actors to submit generated forms based upon a process definition, you can visit the jBpm online demo.

The workflow landscape

Executional processes versus a WFMS

A recent trend in the BPM community is the convergence of specifications about executional business processes. The approach promoted by XLANG, WSFL and BPML converged into BPEL. BPEL is based on interactions (message exchanges). BPEL is defined in the context of a service-oriented architecture. One of the prerequisites is that the services to be addressed are described in a WSDL declaration. BPEL then specifies an XML grammar that can be seen as a programming language to combine control flow with calls to the services defined in WSDL.

I see three main differences between the approach taken in executional business processes and state based workflow management systems:

  • state based versus message oriented: state based WFMSs are centred around the notion of state (or activity). The workflow engine maintains the state and calculates the transitions from one state to the next. On the other hand, executional business processes as BPEL are centred around the definition of reactions upon incoming message. A set of those reactions, along with some other bells and whistles, can be seen as a business process. That explains why BPEL is somewhat complementary to state based WFMSs. A BPEL onMessage event handler, which is a reaction upon an incoming message, could be executed e.g. on transitions between states.
  • process instance id versus message correlation: One of the complexities that are introduced with executional business processes is message correlation. Part of the process description must specify how the BPEL engine can detect the process instance identification from the incoming message. It has to be based on a data item in the message. On the other hand, state based WFMSs generate ids for each process instance that is created. The client can use this id in subsequent calls to the engine API.
  • central workflow engine API versus an abstract service endpoint: state based WFMSs have a central API, meaning that the client talks to one central API to manage executions of all process instances. In executional business processes, each process is exposed as a service. This means a different access point for every process definition.

Academic community

The academic community has the longest track record of involvement with workflow that dates back from the late seventies. The current trend in the research community considers petri nets as the mother of all process definition languages. For petri nets a wide variety of advanced analysis techniques have been described and last year I had the privilege of meeting professor Petri in person at the 2003 conference on Business Process Management. One of the best pieces of research that is made accessible and understandable for a large audience is workflow patterns. Workflow patterns compared a number of workflow management systems and expressed the common process modelling concepts in terms of petri nets.

Open source projects

Finally we are getting down to the links of real workflow management systems. This is where the difficult task starts of choosing one. Actually you should be happy: coping with making a choice is better then having no choice at all :-) One of the goals of this article was to explain the basic concepts of workflow so that you are better equipped to perform the selection process. But I realize that comparing workflow systems is one of the most challenging tasks at this moment for software architects.

The 3 sources for composing this list were my previous article, the list of Carlos E Perez, and another list by Topicus.

  • jBpm - jBpm is a flexible, extensible workflow management system written by the author of this article. Business processes, expressed in a simple and powerful language and packaged in process archives, serve as input for the jBpm runtime server. jBpm combines easy development of workflow-applications with excellent enterprise application integration (EAI) capabilities. jBpm includes a web-application and a scheduler. jBpm is a set of J2SE components that can also be deployed as a clustered J2EE application.
  • OpenEbXML - The Open ebXML project is working to providing a ebXML framework that primarily supports the v2.0 set of ebXML specifications soon to be released by UN/CEFACT and OASIS.
  • OSWorkflow - What makes OSWorkflow different is that it is extremely flexible.
  • wfmOpen - WfMOpen is a J2EE based implementation of a workflow facility (workflow engine) as proposed by the Workflow Management Coalition (WfMC) and the Object Management Group (OMG). Workflows are specified using WfMC's XML Process Definition Language (XPDL) with some extensions.
  • XFlow - XFlow runs within an EJB and servlet container.
  • PowerFolder - PowerFolder consists of a developer studio, administration environment, and a runtime engine.
  • Breeze - Breeze is a lightweight, cross-platform component-based workflow engine prototype.
  • Open Business Engine - Open Business Engine is an open source Java workflow engine which supports the Workflow Management Coalition's (WfMC) workflow specifications, including interface 1, also known as XPDL, interface 2/3 known as WAPI and interface 5 for auditing. OBE provides an environment for executing activities in a controlled, centralized environment. OBE supports both synchronous and asynchronous execution of workflows. The primary OBE implementation is based on J2EE.

Commercial vendors

Tool catalogs


Michael zur Muehlen has made this fine overview presentation about all the workflow related specifications.

I agree with John Pyke and Wil van der Aalst that the standards defined for workflow are still a work in progress. Another argument that points in that direction is the huge amount of overlapping specifications out there.

In my opinion, the multitude of specifications and the limited adoption of each of those specifications are because of limited common knowledge about the primitives. You see: workflow is a subject from which you get easily distracted. The distraction occurs when workflow related concepts get mixed up with complementary technologies or concepts. To give a very concrete example: workflow is completely complementary to web-services. You could expose the interface to access a WFMS as a web-service, but its definitely a bad idea to presume that a WFMS *always* has to be accessed through a web-service interface. Some specifications make this presumption. Apart from web-services, other often occurring distractions include email, inter-business communication, web-applications & organizational structure.

The first standardisation effort in workflow was the Workflow Management Coalition (WfMC). The WfMC started in 1993. A very interesting publication of the WfMC is the reference model. It defines the interfaces between a workflow management system and other actors. Another piece of work is the XPDL specification. XPDL defines an XML schema for specifying the declarative part of a workflow. In my opinion, the reference model and XPDL are the best specifications

  • JSR 207: Process Definition for Java - This is an initiative of the Java Community Process (JCP) that standardizes how to automate business processes on a J2EE server. The basic model of this JSR is to define a special type of ejb session bean that acts as the interface for a business process. The JSR wants to standardize a set of XML meta tags that should be specified as meta data (JSR175). The JSR207 compliant ejb-container then takes the session bean and the meta data as input and generates code that binds the methods together as specified in the meta data. This specification is still in an early phase, no publications have been made yet. The expert group was formed in March 2003.
  • WfMC's XPDL - The Workflow Management Coalition (WfMC) is a consortium of about 300 that defines a set of related standards based on an interesting reference model. The reference model describes the relation between a WFMS and other actors in a use case fashion. XPDL is the specification of WfMC that specifies an XML grammar the control flow of business processes.
  • ebXML's BPSS - ebXML is a set of related standards for business collaboration. Main focus is on communication between different companies. This can be seen as the successor of EDI. ebXML is a joint initiative of OASIS and UN/CEFACT. BPSS is the specification of ebXML that is closest to the concepts described in this article.
  • BPMI's BPML & WSCI - (Intalio, Sun, SAP, ...). BPMI also has defined a specification (BPMN) that describes how the 'executional' business processes should be visually represented.
  • BPEL - (Microsoft, BEA, IBM, SAP & Siebel) BPEL is the specification that results from a set of specifications based on message exchange: XLANG, WSFL, BPML. Also a proposal is being made that hooks this specification to Java: BPELJ. Instead of modelling the states of a process. This specification says how to create a reaction upon an incoming message. So this is not a specification for formal definition of business processes as laid out in this article. Simplified this can be seen as a programming language that is expressed in XML and that provides the control flow constructs for combining WSDL-services. As the name suggests, this specification is focussed (to be exact 'not limited to') to web services.
  • OMG's Workflow management facility - Is based upon the WfMC specifications and defines how this gets translated into CORBA.
  • UML - UML defines 9 types of diagrams for modelling and designing software systems. Each diagram type defines a visual representation and semantics for the things you can draw. One type of diagram, the activity diagram, has as a goal to specify how business processes should be visually represented. Note that with the 4 layers of a process definition, I tried to show that a process definition is more then just its visual representation. UML only specifies the visual part.
  • RosettaNet - RosettaNet mainly defines a set of Partner Interface Processes (PIP). A PIP specifies a process between 2 trading partners, including the message formats.
  • UBL - The Universal Business Language (UBL) defines a standard library of XML documents to be used for communication between different organizations. This can be seen as complementary to ebXML since ebXML only defines the infrastructure to set up inter-organization processes. This standard is competing with a subset of the RosettaNet standard. The RosettaNet PIP's define a set of processes between trading partners, including the message format. The message formats of the RosettaNet PIP's are at the same level as UBL.


I have shown in this article that the workflow market is still young and wild but that there are already solid tools out there:

  1. Only now, mature integration platforms such as J2EE and .NET are available. Running a WFMS on such a platform really boosts their added value. That is why only recently the workflow systems have been rediscovered.
  2. In 'The case for workflow' we showed actually how the introduction of workflow management systems delivers return on investment, both at the technical and the business level.
  3. The positioning of workflow in the technology hype-curve shows that the concepts used in BPM and workflow management still need to settle.
  4. The section 'open source projects' and commercial vendors lists a variety of tools that are eager to bring you all the benefits of workflow and business process management.

The conclusion we can extract from all this is that

  1. Specifications are not yet mature, no standards have been adopted on a broad scale
  2. Comparing workflow tools is the hardest challenge today for companies that embrace the BPM ideas
  3. Despite the fact that standardisation still might take a while, good workflow management systems are already available. That is a real incentive for organizations to dive already into the tool-selection process.

I hope that I have stimulated your interest in workflow and gave you on the right background for making an effective comparison.

Further reading


A special thanks for Gustavo Maciel Dias Vieira and Jef Vanbockryck for their valuable feedback on the draft versions of this article.

About the Author

Tom Baeyens leads the jBpm support organization, specialized in Java, workflow and business process management. His expertises are both at a technical, analysis and at a business level. Tom is the founder of, an open source workflow engine. Tom is also a member of the expertgroup of the JSR 207: Process Definition for Java. Tom Baeyens can be contacted at

Dig Deeper on Java web services and SOA

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.