DataStax Enterprise 4.0

Mixing workloads in a cluster


Creating a virtual data center


Workload segregation

The batch needs of Hadoop and the interactive needs of Solr are incompatible from a performance perspective, so these workloads need to be segregated. Cassandra real-time applications and DSE Search/Solr applications are also incompatible, but for a different reason--dramatically distinct access patterns:




In this diagram, nodes in data centers 1 and 2 (DC 1 and DC 2) can run a mix of:
  • Real-time queries (Cassandra and no other services)
  • Analytics (Cassandra and Hadoop)

In separate data centers, some DSE nodes handle search while others handle MapReduce, or just act as real-time Cassandra nodes. Cassandra ingests the data, Solr indexes the data, and you run MapReduce against that data, all in one cluster without having to do any manual extract, transform, and load (ETL) operations. Cassandra handles the replication and isolation of resources.

The Solr nodes run HTTP and hold the indexes for the Cassandra table data. If a Solr node goes down, the commit log replays the Cassandra inserts, which correspond to Solr inserts, and the node is restored automatically.

The diagram shows the different types of node, even Analytics and Cassandra nodes, in separate data centers. Separating Analytics and Cassandra nodes is not mandatory as it is between Solr and the other node types. Separating all workloads is typically, but not always, recommended by DataStax. Occasionally, there is a use case for keeping Analytics and Cassandra in the same data center. You do not have to have an one or more additional replication factors when nodes are in the same data center.



  • For production use or for use with mixed workloads, do not create the keyspace using SimpleStrategy. SimpleStrategy will not work in mixed workload clusters running Solr. NetworkTopologyStrategy is highly recommended for most deployments because it is much easier to expand to multiple data centers when required by future expansion.
  • From Hadoop and Cassandra real-time clusters in multiple data centers, do not attempt to insert data to be indexed by Solr using CQL or Thrift. Run the CQL or Thrift inserts on a Solr node in its own data center.

Replicating data across data centers

You set up replication for Solr nodes exactly as you do for other nodes in a Cassandra cluster, by creating a keyspace. You can change the replication of a keyspace after creating it.