<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Parse_data_to_tfrecord_library Test
This file consists several function test for the functions in the Parse_data_to_tfrecord_lib file.
Step1:  Test function img_to_example()
Step2:  Function Test
Step3:  Test batch_read_write_tfrecords()
This file reads input tfrecords in batches, and process the bboxes that meet the conditions. And write back the labeles, and cropped images to a new tfrecord file.
The batch_read_write_tfrecords also utilize read_tfrecord(), generate_tfexamples_from_detections(), write_tfexample_to_tfrecord(), parse_detection_confidences(), strip_top(all)_confidence_bbox(), img_to_example(), read_and_check_image()
NOTE
Step4:  Read back the generated tfrecords and check if the data stored inside the file meets the expectation.
<ASSISTANT_TASK:>
Python Code:
from parse_data_to_tfrecord_lib import img_to_example, read_tfrecord, generate_tfexamples_from_detections, batch_read_write_tfrecords
from PIL import Image  # used to read images from directory
import tensorflow as tf
import os
import io
import IPython.display as display
import numpy as np
tf.enable_eager_execution()
Explanation: Parse_data_to_tfrecord_library Test
This file consists several function test for the functions in the Parse_data_to_tfrecord_lib file.
End of explanation
IMG_PATH = './TC11/svt1/img/19_00.jpg'
features={'image': tf.FixedLenFeature([], tf.string),
          'label': tf.FixedLenFeature([], tf.int64)}
try:
    img = Image.open(IMG_PATH, "r")
except Exception as e:
    print(e)
    print(IMG_PATH + " is not valid")
example = img_to_example(img, label=0)
features = tf.io.parse_single_example(example.SerializeToString(), features)
# Testing
# The label feature should be value of 0
assert features['label'].numpy() == 0
# The pixel values of the original image and the stored image should be the same
decode_image = tf.image.decode_image(features['image']).numpy()
original_image = np.array(img.getdata())
assert decode_image.flatten().all() == original_image.flatten().all()
Explanation: Test function img_to_example():
The function reads in image files and outputs tf.examples. The output should store the information feeded into the function.
End of explanation
# Global constants
# Information from input tfrecord files
SOURCE_ID = 'image/source_id'
BBOX_CONFIDENCE = 'image/object/bbox/confidence'
BBOX_XMIN = 'image/object/bbox/xmin'
BBOX_YMIN = 'image/object/bbox/ymin'
BBOX_XMAX = 'image/object/bbox/xmax'
BBOX_YMAX = 'image/object/bbox/ymax'
INPUT_RECORD_DIR = './streetlearn-detections/'
file_name = "./streetlearn_detections_tfexample-00000-of-01000.tfrecord"
ID = "b'/cns/is-d/home/cityblock-streetsmart/yuxizhang/data/public/streetlearn/003419_2.jpg'"
CONFIDENCE = np.array([0.6700151, 0.45046127, 0.22411232, 0.09745394, 0.07810514, 0.06079888, 0.0587763, 0.05148118])
XMIN = np.array([9., 714.,  18., 703., 821., 420., 421., 370.])
YMIN = np.array([298., 441., 538., 613., 655., 649., 656., 637.])
XMAX = np.array([450., 823., 424., 844., 873., 445., 493., 435.])
YMAX = np.array([737., 735., 750., 740., 719., 737., 738., 741.])
parsed_image_dataset = read_tfrecord(os.path.join(INPUT_RECORD_DIR, file_name))
# Testing
# Check the data in the parsed_image_dataset
for example in parsed_image_dataset.take(1):
    confidence = example[BBOX_CONFIDENCE].values.numpy()
    xmin = example[BBOX_XMIN].values.numpy()
    ymin = example[BBOX_YMIN].values.numpy()
    xmax = example[BBOX_XMAX].values.numpy()
    ymax = example[BBOX_YMAX].values.numpy()
    
    assert str(example[SOURCE_ID].numpy()) == ID
    assert confidence.all() == CONFIDENCE.all()
    assert xmin.all() == XMIN.all()
    assert ymin.all() == YMIN.all()
    assert xmax.all() == XMAX.all()
    assert ymax.all() == YMAX.all()
Explanation: Function Test: read_tfrecord()
The function should read tfrecord files as input, and return DatasetV1Adapter storing a list of examples
End of explanation
INPUT_RECORD_DIR = './streetlearn-detections/'
INPUT_UCF_IMG_DIR = './UCF_Streetview_Dataset/raw/'
TF_FILE_DIR = './test_file.tfrecord'
writer = tf.io.TFRecordWriter(TF_FILE_DIR)
detection_property = {'include_top_camera':True, 'only_keep_top_confidence':True, 'balance':False}
file_range = [0, 1]
batch_read_write_tfrecords(file_range, INPUT_RECORD_DIR, INPUT_UCF_IMG_DIR, writer, detection_property)
writer.close()
Explanation: Test batch_read_write_tfrecords()
This file reads input tfrecords in batches, and process the bboxes that meet the conditions. And write back the labeles, and cropped images to a new tfrecord file.
The batch_read_write_tfrecords also utilize read_tfrecord(), generate_tfexamples_from_detections(), write_tfexample_to_tfrecord(), parse_detection_confidences(), strip_top(all)_confidence_bbox(), img_to_example(), read_and_check_image()
NOTE: This test is a functional test for all functions listed above. Also, this part is harder to compared to the ground truth. Therefore, a visulization of the results is performed here.
End of explanation
# Read the files back from the generated tfrecords
def parse_tf_records(file_dir):
    raw_image_dataset = tf.data.TFRecordDataset(file_dir)
    # Create a dictionary describing the features.
    image_feature_description = {
        'label': tf.io.FixedLenFeature([], tf.int64),
        'image': tf.io.FixedLenFeature([], tf.string),
    }
    def _parse_image_function(example_proto):
      # Parse the input tf.Example proto using the dictionary above.
      return tf.io.parse_single_example(example_proto, image_feature_description)
    parsed_image_dataset = raw_image_dataset.map(_parse_image_function)
    
    return parsed_image_dataset
parsed_image_dataset = parse_tf_records(TF_FILE_DIR)
for image_features in parsed_image_dataset:
    print(int(image_features['label']))
    image_raw = image_features['image'].numpy()
    display.display(display.Image(data=image_raw))
Explanation: Read back the generated tfrecords and check if the data stored inside the file meets the expectation.
End of explanation
<END_TASK>