Take the 2-minute tour ×

I'm simulating a discrete time Markov process on a finite state space corresponding to a linear interval $(x_1, x_2, ..., x_N) \in X$, where each time step $t_i$ involves updating the state of some number of agents labeled "Red" and "Blue". At any time point $t_i$ I have an array that keeps track of the positions and colors of the agents along the interval:

agentArray[[t]] = {
    {"Blue", 9}, {"Red", 8}, {"Blue", 9}, {"Blue", 9}, {"Blue", 8}, {"Blue", 8},
    {"Blue", 8}, {"Blue", 6}, {"Blue", 9}, {"Blue", 9}, {"Blue", 8}, {"Blue", 8},
    {"Red", 7}, {"Blue", 8}, {"Blue", 8}, {"Blue", 3}};

I'd like to first and foremost have a nice way in which to illustrate the occupancy levels (for agents of the types "Red" and "Blue") of the discrete states $1$ through $N$ in the Markov chain at some time point $t_i$. Given that Wolfram Research recently extensively updated their Markov processes toolkit in Mathematica, is there a nice way to do this automatically? Something like a heat map superimposed on a graph representation of the linear interval, with explicitly stated counts for the number of red and blue agents at each vertex?

Secondly, I was wondering if there was a way to make an animation showing the evolution of the Markov process? Something like the aforementioned graphic where we sample the state of the Markov chain at every $k$th time point?

Specifically: How do we use the aforementioned data structure to show a graph, with heat/color encoded "Red" and "Blue" occupancy levels at each vertex, with the occupancy counts also explicitly written in text above each vertex?

share|improve this question

1 Answer 1

up vote 5 down vote accepted

I present this for motivation. This is a small toy example,

The following image was created with Mathematica prior to the current implementations. (post here)

enter image description here

Using random walk on a graph as inspiration the following is presented:

  1. Transition probabilities:

    tp = {{0, 1/2, 1/2, 0}, {1/2, 0, 1/2, 0}, {1/3, 1/3, 0, 1/3}, {1, 0, 
    0, 0}}
  2. Create Markov objects and random function (in this case 10 steps)

    mkv = DiscreteMarkovProcess[{1, 0, 0, 0}, tp];
    rf = RandomFunction[mkv, {0, 10}, 25];
  3. Note that 25 paths are created to simulate. Now the paths are extracted and the number of elements in each state counted:

    paths = #[[All, 2]] & /@ rf["Paths"];
    cnt = Map[Function[x, Count[#, x]], Range[4]] & /@ Transpose@paths;
  4. Now to generate simulation from paths:

    bc = BarChart[#, ChartLabels -> Range[4], PlotRange -> {0, 25}, 
     LabelingFunction -> Above, PlotRangePadding -> {0, 5}] & /@ cnt;
    anim = Transpose@paths;
    arp = ArrayPlot[Partition[#, 5], 
     ColorRules -> {1 -> Red, 2 -> Blue, 3 -> Green, 4 -> Yellow}] & /@
    anex = MapThread[
        Column[{Style["Step " <> ToString[#3], 
           FontFamily -> "Calibri", 12], #2}]}], Graph[mkv]}, 
     Spacings -> 0] &, {arp, bc, Range[0, 10]}];

This was used to create the animated gif that steps through each path. Each element of array changes color to relevant state ( {1 -> Red, 2 -> Blue, 3 -> Green, 4 -> Yellow}). I should have put a legend but this is for illustration purposes:

enter image description here

This may not be exactly what you want but i hope it allows you to achieve it.

share|improve this answer

Your Answer


