<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 This notebook provides a demo for the use of DeepSpeech Distances proposed in High Fidelity Speech Synthesis with Adversarial Networks as new evaluation metrics for neural speech synthesis.
The computation involves estimating Fréchet and Kernel distances between high-level features of the reference and the examined samples extracted from NVIDIA's DeepSpeech2 model.
We propose four distances
Step1:  Set up evaluation parameters and paths within the mounted Google Drive. Clone repository and download checkpoint.
Step2:  Do other necessary imports.
Step3:  Create random 2-second samples from source audio file. 
By default, we use our generated abstract ([1]) as a source of both clean reference samples and noise-corrupted samples for examination.
User can specify other audio files; in principle, DeepSpeech Distances require reference samples (e.g. real speech) and samples to be examined, all of the sample length. Although this demo uses a smaller number for the sake of fast computation, we suggest using at least 10,000 samples per each set, to obtain good estimates of the distances.
Step4:  Create evaluator object and load reference samples.
Step5:  Carry out distance estimation.
Step6:  Plot results.
Kernel distances usually give similar numbers (which, in case of small samples, might be negative due to unbiased estimator of MMD). Unconditional Frechet distance differs significantly from the conditional one for reference samples and always gives positive values due to biased estimator.
<ASSISTANT_TASK:>
Python Code:
!pip install python_speech_features
!pip install resampy
!pip install scipy
!pip install gdown
!pip install tqdm -U
Explanation: This notebook provides a demo for the use of DeepSpeech Distances proposed in High Fidelity Speech Synthesis with Adversarial Networks as new evaluation metrics for neural speech synthesis.
The computation involves estimating Fréchet and Kernel distances between high-level features of the reference and the examined samples extracted from NVIDIA's DeepSpeech2 model.
We propose four distances:
Fréchet DeepSpeech Distance (FDSD, based on FID, see [2])
Kernel DeepSpeech Distance (KDSD, based on KID, see [3])
conditional Fréchet DeepSpeech Distance (cFDSD),
conditional Kernel DeepSpeech Distance (cKDSD).
The conditional distances compare samples with the same conditioning (e.g. text) and asses conditional quality of the audio. The uncoditional ones compare random samples from two distributions and asses general quality of audio. For more details, see [1].
References
[1] Mikołaj Bińkowski, Jeff Donahue, Sander Dieleman, Aidan Clark, Erich Elsen, Norman Casagrande, Luis C. Cobo, Karen Simonyan, High Fidelity Speech Synthesis with Adversarial Networks, ICLR 2020.
[2] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Sepp Hochreiter, GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium, NeurIPS 2017.
[3] Mikołaj Bińkowski, Dougal J. Sutherland, Michael Arbel, Arthur Gretton, Demystifying MMD GANs, ICLR 2018.
Demo
Firsty, install dependencies and download the checkpoint of the pretrained DeepSpeech2 model.
End of explanation
PATH = '/content/drive/My Drive/DeepSpeechDistances'
SAMPLE_PATH = '/content/drive/My Drive/DeepSpeechDistances/abstract_samples'
NUM_SPLITS = 3  # number of data splits to comute std of DSD
SAMPLES_PER_SPLIT = 500  # number of samples in a single DSD run. 
# We recommend at least 10k samples for evaluation to get reasonable estimates.
AUDIO_LENGTH = 2  # length of individual sample, in seconds
NUM_NOISE_LEVELS = 3  # number of different noise levels for samples to evaluate
from google.colab import drive
drive.mount('/content/drive/', force_remount=True)
import sys, os
sh = lambda path: "'" + path + "'"
if not os.path.exists(PATH):
  !git clone https://github.com/mbinkowski/DeepSpeechDistances.git {sh(PATH)}
else:
  print('Found DeepSpeechDistances directory, skipping git clone.')
sys.path.append(PATH)
if not os.path.exists(os.path.join(PATH, 'checkpoint', 'ds2_large')):
  CKPT = sh(PATH + '/ds2.tar.gz')
  !gdown https://drive.google.com/uc?id=1EDvL9wMCO2vVE-ynBvpwkFTultbzLNQX -O {CKPT}
  !tar -C {sh(PATH +'/checkpoint/')} -xvf {CKPT}
  !rm {CKPT}
else:
  print('Found checkpoint directory, skipping download.')
Explanation: Set up evaluation parameters and paths within the mounted Google Drive. Clone repository and download checkpoint.
End of explanation
%tensorflow_version 1.x
import tensorflow.compat.v1 as tf
tf.disable_eager_execution()
import audio_distance
from sample_utils import subsample_audio
Explanation: Do other necessary imports.
End of explanation
subsample_audio(os.path.join(PATH, 'abstract.wav'), 
                SAMPLE_PATH,
                num_samples=NUM_SPLITS * SAMPLES_PER_SPLIT,
                length=AUDIO_LENGTH,
                num_noise_levels=NUM_NOISE_LEVELS)
Explanation: Create random 2-second samples from source audio file. 
By default, we use our generated abstract ([1]) as a source of both clean reference samples and noise-corrupted samples for examination.
User can specify other audio files; in principle, DeepSpeech Distances require reference samples (e.g. real speech) and samples to be examined, all of the sample length. Although this demo uses a smaller number for the sake of fast computation, we suggest using at least 10,000 samples per each set, to obtain good estimates of the distances.
End of explanation
reference_path = os.path.join(SAMPLE_PATH, 'ref', '*.wav')
eval_paths = [os.path.join(SAMPLE_PATH, f'noisy_{i+1}', '*.wav') for i
              in range(NUM_NOISE_LEVELS)]
evaluator = audio_distance.AudioDistance(
    load_path=os.path.join(PATH, 'checkpoint', 'ds2_large', 'model.ckpt-54800'),
    meta_path=os.path.join(PATH, 'checkpoint', 'collection-stripped-meta.meta'),
    required_sample_size=NUM_SPLITS * SAMPLES_PER_SPLIT,
    num_splits=NUM_SPLITS)
evaluator.load_real_data(reference_path)
Explanation: Create evaluator object and load reference samples.
End of explanation
dist_names = ['FDSD', 'KDSD', 'cFDSD', 'cKDSD']
def print_results(values):
  print('\n' + ', '.join(['%s = %.5f (%.5f)' % (n, v[0], v[1]) for n, v 
                          in zip(dist_names, values)]))
with tf.Session(config=evaluator.sess_config) as sess:
  print('Computing reference DeepSpeech distances.')
  values = evaluator.get_distance(sess=sess)
  print_results(values)
  distances = [values]
  for eval_path in eval_paths:
    print('\nComputing DeepSpeech distances for files in the directory:\n'
          + os.path.dirname(eval_path))
    values = evaluator.get_distance(sess=sess, files=eval_path)
    print_results(values)
    distances.append(values)
Explanation: Carry out distance estimation.
End of explanation
all_paths = [reference_path] + eval_paths
prefix_len = len(os.path.commonpath(all_paths))
sample_names = [path[prefix_len + 1:] for path in all_paths]
if all([os.path.basename(p) == '*.wav' for p in all_paths]):
  sample_names = [name[:-6] for name in sample_names]
import matplotlib.pyplot as plt
fig, ax = plt.subplots(1, 2, figsize=(12, 4))
x = range(NUM_NOISE_LEVELS + 1)
for i, kind in enumerate(['Frechet', 'kernel']):
  ax[i].set_title(kind + ' distances')
  ax[i].set_xticks(x)
  ax[i].set_xticklabels(sample_names)
  for j in [0, 2]:
    k = i + j
    ax[i].plot(x, [d[k][0] for d in distances], color='cmyk'[k],
               label=dist_names[k])
    ax[i].fill_between(x, [d[k][0] - d[k][1] for d in distances],
                       [d[k][0] + d[k][1] for d in distances], color='cmyk'[k],
                       alpha=.3)
  ax[i].legend()
  ax[i].grid()  
drive.flush_and_unmount()
Explanation: Plot results.
Kernel distances usually give similar numbers (which, in case of small samples, might be negative due to unbiased estimator of MMD). Unconditional Frechet distance differs significantly from the conditional one for reference samples and always gives positive values due to biased estimator.
End of explanation
<END_TASK>