UC-NGI-PL-application-II
From EGI Knowledge Base
Use Case title: LHC on-line filtering
Short description: This HEP application is a real-time application designed to filter events provided by one of the CERN's LHC experiments - ATLAS. The application will perform on-line analysis of events during the experiment run. ATLAS will produce very large amounts of data. - each event will contain about 1.5MB of data and be generated approximately every 25ns (40MHz). However, in most cases events can be dismissed, because they contain noise or are related to a known area of physics.
To extract interesting events from such an amount of data, three levels of filtering are applied. The first and the second filter levels are built using hardware filters and computer farms located in the experiment chamber. Both filter levels use simple algorithms and are characterized by very short decision times. The output data rate from the second filter (LVL2) is limited to approximately 4GB/s. The third filtering level (LVL3) consists of processor farms. LVL3 has to be able to process all events generated by LVL2.
The Grid environment can be used to reduce the cost of building the appropriate computation infrastructure used at the LVL3 filtering level. However, this is not a trivial task due to the lack of Quality of Service (QoS) in current Grid middleware. The idea is to create a specialized VO, which will ensure the desired QoS level.
The main problem with using the Grid for HEP applications is the job submission time, which is too long (and exhibits too much variance) to submit each event as a separate job. To achieve fast event processing, the ’pilot job’ concept can be used.
Actors involved: Experiment operators and supervisors, VO managers, RC operator
Related reguirment: Dedicated VO services, SLA between VO and sites
Steps:
- Development of dedicated services
- Local tests and validation
- Testbed deployment and testing
- Production phase
Post-conditions (optional): Support and maintenance
Project(s) involved (optional): int.eu.grid (March 2006 - April 2008)
