<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Basic read and write operations
In this example we will explore how to create and read a simple Lightning Memory-Mapped Database (LMDB) using pyxis.
Writing data
Step1:  Let's start by creating some data that we can store. 2000 images with shape (254, 254, 3) along with 2000 targets are generated.
Step2:  The tenth image is set to be a completely red image.
Step3:  In the next step we instantiate a pyxis writer to write the data we created above to a directory called data.
Step4:  The map_size_limit is the size of the LMDB in MB. For file systems running ext4, there is no big cost associated with making this big. ram_size_limit is the limit on how many GB of data we can push per write operation.
When the LMDB writer is set up, we can run the pyxis.put_samples function to write data. Ideally you should send large data blobs at once. Here we will write all the 2000 samples generated above at once. If the size of the data you have is larger than the size of your RAM, then you can perform multiple calls to pyxis.put_samples.
Step5:  Data can be input in two different ways
Step6:  Reading data
Step7:  To read data from a LMDB we instantiate a pyxis reader using the directory of the database.
Step8:  All addressable keys can be found by calling the pyxis.get_data_keys function
Step9:  Python list indexing syntax is used to retrieve a whole sample from the LMDB. The pyxis.get_sample function can be used as well. The returned value is a Python dictionary, addressable by the keys used when writing the database.
Step10:  The code snippet above retrieves the tenth sample, i.e. the one with the red image.
Step11:  Slice objects are supported by the indexing method outlined above. For example
Step12:  Python 2.7 tend to favour __getslice__ over __getitem__ when slicing, however __getslice__ has been deprecated since Python 2.0. The functionality may vary when compared with Python 3. See Python 2 Data model documentation for more information.
Data within a sample can be retrieved directly by using the pyxis.get_data_value function. Here we ask the LMDB for the hundredth y value.
Step13:  Use len() to return the number of samples in the LMDB.
Step14:  Just like when writing, we should make sure to close the LMDB environment after we are done reading.
<ASSISTANT_TASK:>
Python Code:
from __future__ import print_function
import time
import numpy as np
import pyxis as px
np.random.seed(1234)
Explanation: Basic read and write operations
In this example we will explore how to create and read a simple Lightning Memory-Mapped Database (LMDB) using pyxis.
Writing data
End of explanation
nb_samples = 2000
X = np.zeros((nb_samples, 254, 254, 3), dtype=np.uint8)
y = np.arange(nb_samples, dtype=np.uint8)
Explanation: Let's start by creating some data that we can store. 2000 images with shape (254, 254, 3) along with 2000 targets are generated.
End of explanation
X[10, :, :, 0] = 255
Explanation: The tenth image is set to be a completely red image.
End of explanation
db = px.Writer(dirpath='data', map_size_limit=500, ram_gb_limit=2)
Explanation: In the next step we instantiate a pyxis writer to write the data we created above to a directory called data.
End of explanation
start = time.time()
db.put_samples('X', X, 'y', y)
print('Average time per image = {:.4f}s'.format((time.time() - start) / nb_samples))
Explanation: The map_size_limit is the size of the LMDB in MB. For file systems running ext4, there is no big cost associated with making this big. ram_size_limit is the limit on how many GB of data we can push per write operation.
When the LMDB writer is set up, we can run the pyxis.put_samples function to write data. Ideally you should send large data blobs at once. Here we will write all the 2000 samples generated above at once. If the size of the data you have is larger than the size of your RAM, then you can perform multiple calls to pyxis.put_samples.
End of explanation
db.close()
Explanation: Data can be input in two different ways:
pyxis.put_samples('key1', value1, 'key2', value2, ...)
pyxis.put_samples({'key1': value1, 'key2': value2, ...})
The code example above uses the first option with alternating keys and values. The second option is simply a Python dictionary.
There are two requirements of data that is put in a pyxis database:
All values must have the type numpy.ndarray.
The first axis in all values must represent the sample number.
Make sure to close the LMDB environment after writing. The close operation makes sure to store the number of samples that has been written.
End of explanation
try:
    %matplotlib inline
    import matplotlib.pyplot as plt
except ImportError:
    raise ImportError('Could not import the matplotlib library required to '
                      'plot images. Please refer to http://matplotlib.org/ '
                      'for installation instructions.')
Explanation: Reading data
End of explanation
db = px.Reader('data')
Explanation: To read data from a LMDB we instantiate a pyxis reader using the directory of the database.
End of explanation
db.get_data_keys()
Explanation: All addressable keys can be found by calling the pyxis.get_data_keys function:
End of explanation
sample = db[10]
print('X: ', sample['X'].shape, sample['X'].dtype)
print('y: ', sample['y'].shape, sample['y'].dtype)
Explanation: Python list indexing syntax is used to retrieve a whole sample from the LMDB. The pyxis.get_sample function can be used as well. The returned value is a Python dictionary, addressable by the keys used when writing the database.
End of explanation
plt.figure()
plt.imshow(sample['X'])
plt.axis('off')
plt.show()
Explanation: The code snippet above retrieves the tenth sample, i.e. the one with the red image.
End of explanation
samples = db[0:8:2]
for sample in samples:
    print(sample['y'])
Explanation: Slice objects are supported by the indexing method outlined above. For example:
End of explanation
db.get_data_value(100, 'y')
Explanation: Python 2.7 tend to favour __getslice__ over __getitem__ when slicing, however __getslice__ has been deprecated since Python 2.0. The functionality may vary when compared with Python 3. See Python 2 Data model documentation for more information.
Data within a sample can be retrieved directly by using the pyxis.get_data_value function. Here we ask the LMDB for the hundredth y value.
End of explanation
len(db)
Explanation: Use len() to return the number of samples in the LMDB.
End of explanation
db.close()
Explanation: Just like when writing, we should make sure to close the LMDB environment after we are done reading.
End of explanation
<END_TASK>