In 1908, Lippmann has proved positively an important question: "Peut-on demander à la Photographie de nous rendre toute cette variété qu'ofre la vue directe des objets?". In the last century, researchers from several areas have struggled to put in practice what Lippmann has proved in theory. A century after Lippman's work, micro and nanophotonics technologies allow new characteristics and capabilities on digital imaging devices towards a more faithful representation of the real world. Key examples are High Dynamic Range (HDR) cameras and displays, light-field sensing devices, and holographic microscopes that enable new capturing, manipulation and visualization modalities and perspectives. HDR image technology is developed to record and display all the brightness information of the visible range in the scene. The use of light-field cameras (aka plenoptic cameras) allows obtaining simultaneously, with a single camera, multiple images on different focus planes as focus may be manipulated after acquisition and not at acquisition as usual. Holography works by capturing the interference pattern generated by the superimposition of a complex wavefront propagating through a real physical object with a coherent wavefront. Holographic data representations will carry even more information than light-field representations. In summary, after a century, the static planar-like images are giving place to richer volume-like images, closer to the visual experience that human can have when interacting with real world objects.

In terms of representation, HDR is at a more advanced stage as shown by the recent JPEG XT standard which already defines an efficient scalable coding methodology for this type of images. Plenoptic images are very promising considering also the increasing popularity of two cameras in the market, Lytro and Raytrix. However, there is still little literature dedicated to the efficient representation and quality assessment of light-field data. Holographic imaging is maturing the relevant underlying technologies, notably for microscope and even macroscopic holographic imaging systems; these technologies might lead this type of data to the imaging markets in a near future. As for plenoptic imaging, there is little literature dedicated to the representation and quality assessment of holographic data. Authors who tried compressing digital holograms or plenoptic images using coding standards like JPEG, MPEG-1, MPEG-2 and (MPEG-4) H.264/AVC ended up by concluding that they are not well-suited for their efficient compression as their information is inherently different from the conventional images.

These richer volume-like imaging modalities carry obviously more information than typical planar-like images. This information increase requires rethinking the (representation) modeling and coding of these new image modalities. For plenoptics and holographic images, there are no coding standards; however, the recent JPEG PLENO Executive Summary states the intention to "investigate how this evolution can be properly addressed while taking into account JPEG's legacy formats". Which means while keeping JPEG backward compatibility to allow new imaging experiences without disruption of the current JPEG ecosystem. It is thus expectable that the new coding standards for these new images modalities follow a similar scalable approach by encoding a simplified version using the conventional JPEG standard and further encoding extra information in enhancement layers targeting successively improved image experiences. While a common representation framework for these new data types would provide an interoperable framework that might enable new players and application developers, it is still not clear to what extent this may be achieved. The team proposing this project has already research experience in the representation of these new imaging modalities and it is deeply involved in the JPEG PLENO activities. Hence, they can be coded in a scalable way with a base layer only with standard information and keeping the JPEG legacy, and extended layers dedicated to code specific information to improve the quality or application possibilities (e.g. higher dynamic range, different perspectives, different focus, depth). Furthermore, the proposed research work will design solutions in the context of the JPEG PLENO emerging standardization activity, enlarging our previous international cooperation.

As it is not possible to think on image compression without thinking on image quality, this project is also dedicated to the quality assessment of these new and richer image modalities, trying as much as possible to define a common quality assessment framework. In summary, this project targets to design a common framework for the representation and quality assessment of new imaging modalities, such as light-field, HDR and holographic imaging, with international impact in terms of research and standardization.