UC-NGI-GR-I
From EGI Knowledge Base
Use Case title: Create a tele-immersion service. (Part of the AMI lab construction (AMbient Intelligence laboratory. Funded by the national program «ΑΡΙΣΤΕΙΑ»)
Short description: A single laboratory wishes to create a service that would be facilitated by the Grid’s computational resources and offer the computational infrastructure for visual communication services (e.g. tele-presence, tele-conferencing, tele-immersion, augmented & virtual reality).
Actors involved: VO coordinator, VO operator, NGI operator. In a possible future scenario and EGI operator could be required if the proposed service is to facilitate communication services beyond national borders.
Related Requirement: Offering the service on demand would enhance the value of the service. Most importantly, the bandwidth from our input sensors (cameras) to the processing infrastructure (CPUs) is required to be high
Pre-conditions: (optional) The end-users of the service that we wish to provide are regular users with no particular demographic attributes. However, in order to offer the service a multiple-camera environment is required as equipment. Therefore, the service is to be offered through a studio, laboratory, or theater. If a charge is applied to the usage of this service financial resources could be brought in the national infrastructure.
Steps: The actual steps required, so that the use case can be materialised.
- Customize software in order to be deployed
- Setup datapath from input sensors to CPUs
- Write application-dependent end-user visualization software
Description of service and answers to specific questions
We wish use the computational power of the grid in order to create a real-time application that would offer visual environment scanning, tele-immersion and tele-presence services. The service to be deployed has been already succesfully deployed and demonstrated in supercomputing facilities in the US [ref] and published [refs], by the author of this document. The software to facilitate this service is, however, an improvement of the software used in the above publications and has been published in [refs].
The core technology of our application works as follows. Assume a room or outdoors environment with a cluster of cameras situated around and /or inside the environment. The cameras image a large portion of the surface in the environment, including both stationary and non-stationary elements of the environment (e.g. humans).
Video is acquired from multiple cameras, with the individual frames of the multiple videos being synchronously acquired. Henceforth the term frame will refer to a set of images that are acquired by the above cameras at the same time.
Each such frame is passed as input in stereoscopic-reconstruction software that outputs a 3D model of the environment as imaged in the input frame. This process is performed multiple times per second, once for each acquired frame. Its result is a time-evolving 3D representation of the scene, or otherwise a “3D video” that was composed from the conventional (2D) videos acquired. If the processing of images is fast enough to cope with the video frame rate the process can be performed in real-time.
The grid is intended to be utilized in the parallelization of the workings of the above 3D reconstruction software, in order to meet real-time computational requirements. This parallelization is quite straightforward, because our software processes images in pairs, thus providing parallelization through data partitioning. The results produced from different image pairs are then combined into a final result, in a way that is also massively parallelizable. In fact if there is an abundance of CPUs, even the processing of a single pair of images can be furthermore parallelized, by tessellating the images into smaller parts and processing different parts independently. This potential of massive parallelization is feasible, because our reconstruction software processes the images locally. In other words, in order to reconstruct a piece of surface only the pixels at which this piece appears are required.
Depending on the type of application that the above technology will facilitate the end-user interface may differ. For example in the case of a tele-immersion application, a real-time virtual-reality 3D renderer is employed to provide the graphics for immersing the end-user in the dynamically reconstructed environment. In a surveillance type of application a simpler software module could be utilized, that e.g. pinpoints the locations of moving environment elements on an environment map. In general, the end-user interface is not going to require grid support and will be executed on a single personal computer that will be receiving the results of the grid-facilitated computation.
