pyiqa.utils.lmdb_util
=====================

.. py:module:: pyiqa.utils.lmdb_util






Module Contents
---------------

.. py:function:: make_lmdb_from_imgs(data_path, lmdb_path, img_path_list, keys, batch=5000, compress_level=1, multiprocessing_read=False, n_thread=40, map_size=None)

   Make lmdb from images.

   Contents of lmdb. The file structure is:
   example.lmdb
   ├── data.mdb
   ├── lock.mdb
   ├── meta_info.txt

   The data.mdb and lock.mdb are standard lmdb files and you can refer to
   https://lmdb.readthedocs.io/en/release/ for more details.

   The meta_info.txt is a specified txt file to record the meta information
   of our datasets. It will be automatically created when preparing
   datasets by our provided dataset tools.
   Each line in the txt file records 1)image name (with extension),
   2)image shape, and 3)compression level, separated by a white space.

   For example, the meta information could be:
   `000_00000000.png (720,1280,3) 1`, which means:
   1) image name (with extension): 000_00000000.png;
   2) image shape: (720,1280,3);
   3) compression level: 1

   We use the image name without extension as the lmdb key.

   If `multiprocessing_read` is True, it will read all the images to memory
   using multiprocessing. Thus, your server needs to have enough memory.

   :param data_path: Data path for reading images.
   :type data_path: str
   :param lmdb_path: Lmdb save path.
   :type lmdb_path: str
   :param img_path_list: Image path list.
   :type img_path_list: str
   :param keys: Used for lmdb keys.
   :type keys: str
   :param batch: After processing batch images, lmdb commits.
                 Default: 5000.
   :type batch: int
   :param compress_level: Compress level when encoding images. Default: 1.
   :type compress_level: int
   :param multiprocessing_read: Whether use multiprocessing to read all
                                the images to memory. Default: False.
   :type multiprocessing_read: bool
   :param n_thread: For multiprocessing.
   :type n_thread: int
   :param map_size: Map size for lmdb env. If None, use the
                    estimated size from images. Default: None
   :type map_size: int | None


.. py:function:: read_img_worker(path, key, compress_level)

   Read image worker.

   :param path: Image path.
   :type path: str
   :param key: Image key.
   :type key: str
   :param compress_level: Compress level when encoding images.
   :type compress_level: int

   :returns: Image key.
             byte: Image byte.
             tuple[int]: Image shape.
   :rtype: str


.. py:class:: LmdbMaker(lmdb_path, map_size=1024**4, batch=5000, compress_level=1)

   LMDB Maker.

   :param lmdb_path: Lmdb save path.
   :type lmdb_path: str
   :param map_size: Map size for lmdb env. Default: 1024 ** 4, 1TB.
   :type map_size: int
   :param batch: After processing batch images, lmdb commits.
                 Default: 5000.
   :type batch: int
   :param compress_level: Compress level when encoding images. Default: 1.
   :type compress_level: int


   .. py:method:: put(img_byte, key, img_shape)


   .. py:method:: close()