<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 FlyingSquid for Video
In this notebook, we'll use FlyingSquid to train a label model for sequential video data. In this application, we'll be using broadcast tennis footage to detect tennis rallies (when the two players are continuously hitting the ball back and forth).
<table style="width
Step1:  In our L and Y matrices, each row represents a single frame in the video. Since rallies occur over contiguous frames, notice that the ground truth annotations have contiguous segments of the same label
Step2:  These labeling functions were written using Rekall queries and express heuristics like the number of people on court, the size of the people, and the number of near-white pixels. Notice that abstain rates tend to be much lower when using Rekall queries, since many queries automatically label the whole video
Step3:  Step 2
Step4:  Next, we'll use FlyingSquid to model this dependency structure
Step5:  Let's explain each argument that we just passed to LabelModel
Step6:  The above code will work for most video tasks!
To train the label model, all we need to do is pass L_train_seqs to the fit function
Step7:  Evaluating the label model
Now, let's use the dev set to evaluate the label model.
Since we are now modeling sequences, we want to use the function predict_proba_marginalized to get predictions for individual frames
Step8:  Step 3
Step9:  Bonus
<ASSISTANT_TASK:>
Python Code:
import numpy as np
from tutorial_helpers import *
L_train = np.load('L_train_video.npy')
L_dev = np.load('L_dev_video.npy')
Y_dev = np.load('Y_dev_video.npy')
print(L_train.shape)
print(L_dev.shape)
print(Y_dev.shape)
Explanation: FlyingSquid for Video
In this notebook, we'll use FlyingSquid to train a label model for sequential video data. In this application, we'll be using broadcast tennis footage to detect tennis rallies (when the two players are continuously hitting the ball back and forth).
<table style="width:100%">
    <tr style="background:none">
        <th><img src="https://raw.githubusercontent.com/HazyResearch/flyingsquid/master/figs/tennis_rally.png" width="100%"></th>
        <th><img src="https://raw.githubusercontent.com/HazyResearch/flyingsquid/master/figs/tennis_nonrally.png" width="100%"></th>
    </tr>
    <tr style="font-weight: bold; font-size:large; background:none">
        <td style="text-align: center">Rally Segment</td>
        <td style="text-align: center">Non-Rally Segment</td>
    </tr>
</table>
Step 1: Load Labeling Function Outputs
Again, we'll start by loading labeling function outputs. This time, we'll load in the outputs from actual labeling functions.
End of explanation
Y_dev[:100]
Explanation: In our L and Y matrices, each row represents a single frame in the video. Since rallies occur over contiguous frames, notice that the ground truth annotations have contiguous segments of the same label:
End of explanation
print_statistics(L_dev, Y_dev)
Explanation: These labeling functions were written using Rekall queries and express heuristics like the number of people on court, the size of the people, and the number of near-white pixels. Notice that abstain rates tend to be much lower when using Rekall queries, since many queries automatically label the whole video:
End of explanation
# We use v to denote the length of sequence we're modeling
v = 3
# Six labeling functions per frame
m_per_frame = 6
# Total number of labeling functions is m_per_frame * v
m = m_per_frame * v
# Figure out how many sequences we're going to have
n_frames_train = L_train.shape[0]
n_frames_dev = L_dev.shape[0]
n_seqs_train = n_frames_train // v
n_seqs_dev = n_frames_dev // v
# Resize and reshape matrices
L_train_seqs = L_train[:n_seqs_train * v].reshape((n_seqs_train, m))
L_dev_seqs = L_dev[:n_seqs_dev * v].reshape((n_seqs_dev, m))
Y_dev_seqs = Y_dev[:n_seqs_dev * v].reshape((n_seqs_dev, v))
Explanation: Step 2: Model the labeling functions (and temporal dependencies) with FlyingSquid
Next, we're going to use FlyingSquid to model the labeling functions. But we're going to end up doing something slightly more complicated in order to model temporal dependencies.
We'll model three frames at a time with three hidden variables, and model each labeling function labeling an individual frame in the sequence:
<div>
  <img src="https://raw.githubusercontent.com/HazyResearch/flyingsquid/master/figs/graphical_structure_video.png" style="padding: 20px 30px" width = '800px'>
</div>
In a given sequence of three frames, lambda_0, lambda_6, and lambda_12 model LF 0's outputs on the first, second, and third frames, respectively. Similarly, lambda_1, lambda_7, and lambda_13 model LF 1's outputs on the first, second, and third frames.
Our first step is resizing and reshaping our matrices:
End of explanation
from flyingsquid.label_model import LabelModel
label_model = LabelModel(
    m,
    v = v,
    y_edges = [
        (0, 1), (1, 2)
    ],
    lambda_y_edges = [
        (0, 0), (1, 0), (2, 0), (3, 0), (4, 0), (5, 0),
        (6, 1), (7, 1), (8, 1), (9, 1), (10, 1), (11, 1),
        (12, 2), (13, 2), (14, 2), (15, 2), (16, 2), (17, 2),
    ]
)
Explanation: Next, we'll use FlyingSquid to model this dependency structure:
End of explanation
# A simpler way to build the label model
label_model = LabelModel(
    m,
    v = v,
    y_edges = [ (i, i + 1) for i in range(v - 1) ],
    lambda_y_edges = [ (i, i // m_per_frame) for i in range(m) ]
)
Explanation: Let's explain each argument that we just passed to LabelModel:
* Our first argument is still m, the total number of labeling functions.
* The second argument, v, specifies how many sequences we're modeling.
* The third argument, y_edges, specifies (non-directional) edges between the hidden variables. Each pair in the array specifies an edge; in this case, we are specifying edges between Y_0 and Y_1, and between Y_1 and Y_2.
* The fourth argument, lambda_y_edges, specifies (non-directional) edges between observable variables and hidden variables. In this case, each pair in the array specifies an edge by using the first item to index into the observable varialbes, and using the second item to index into the hidden variables.
Now that we understand what's going on, we can actually express this in fewer lines of code:
End of explanation
label_model.fit(L_train_seqs)
Explanation: The above code will work for most video tasks!
To train the label model, all we need to do is pass L_train_seqs to the fit function:
End of explanation
probabilistic_labels = label_model.predict_proba_marginalized(L_dev_seqs)
preds = [ 1. if prob > 0.5 else -1. for prob in probabilistic_labels ]
accuracy = np.sum(preds == Y_dev[:n_seqs_dev * v]) / (n_seqs_dev * v)
print('Label model accuracy: {}%'.format(int(100 * accuracy)))
Explanation: Evaluating the label model
Now, let's use the dev set to evaluate the label model.
Since we are now modeling sequences, we want to use the function predict_proba_marginalized to get predictions for individual frames:
End of explanation
probabilistic_labels = label_model.predict_proba_marginalized(L_train_seqs)
print(probabilistic_labels.shape)
print(probabilistic_labels[:100])
Explanation: Step 3: Training an End Model
If necessary, we can use the probabilistic labels to train up an end model. All we need to do is call predict_proba_marginalized over L_train_seqs:
End of explanation
# Video Label model with dependencies between lambda_0 and lambda_1
label_model = LabelModel(
    m,
    v = v,
    y_edges = [ (i, i + 1) for i in range(v - 1) ],
    lambda_y_edges = [ (i, i // m_per_frame) for i in range(m) ],
    lambda_edges = [ (0, 1) ]
)
Explanation: Bonus: Dependencies between labeling functions
Now that we know how to specify dependencies manually, it's a simple step to specify dependencies between labeling functions:
<div>
  <img src="https://raw.githubusercontent.com/HazyResearch/flyingsquid/master/figs/graphical_structure_video_lambda_dep.png" style="padding: 20px 30px" width = '800px'>
</div>
All you have to do is pass in an extra argument, lambda_edges, that specifies edges between observable variables. For example, you can specify the above graph like this:
End of explanation
<END_TASK>