Mirror/FairMQ
1
0
Fork 0
mirror of https://github.com/FairRootGroup/FairMQ.git synced 2025-10-13 08:41:16 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update readme.md

Browse Source
This commit is contained in:
Mohammad Al-Turany 2021-12-09 10:20:30 +01:00 committed by Alexey Rybalchenko
parent ac49c55029
commit fa02234209
1 changed files with 6 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
readme.md
View File

@ -1,12 +1,13 @@
## FairMQ in a nutshell
Next-generation Particle Physics Experiments at [GSI](https://www.gsi.de)/[FAIR](https://www.gsi.de/forschungbeschleuniger/fair) and [CERN](https://home.web.cern.ch/) are facing [unprecedented data processing challenges](https://doi.org/10.1051/epjconf/201921405010). High detector data readout rates alone already require a non-trivial amount of distributed compute resources in the order of hundreds of servers per experiment. To achieve sufficient data compression rates in order to stay within the limits of economically feasible data storage capacity requirements, the complexity of tasks that need to be performed during the synchronous (online) data processing is significantly increased as well compared to previous generations of experiments.
Next-generation Particle Physics Experiments at [GSI](https://www.gsi.de)/[FAIR](https://www.gsi.de/forschungbeschleuniger/fair) and [CERN](https://home.web.cern.ch/) are facing [unprecedented data processing challenges](https://doi.org/10.1051/epjconf/201921405010). The expected data rates require a non-trivial amount of high performance compute (HPC) resources, i.e: Thausends of CPU/GPU cores per experiment. Online data processing/compression are crucial to fulfill the storage capacity requirements. The complexity of tasks that need to be performed during the online (synchronous) data processing is significantly higher than ever before. Complex tasks that usually runs offline (e.g: Calibration, Track finding, etc) have to run now online in a high performance and high throughput environment.
The [FairMQ C++ library](https://github.com/FairRootGroup/FairMQ/) is designed to aid the implementation of such large-scale online data processing workflows by
* providing an **asynchronous message passing abstraction** that integrates different existing data transport technologies (no need to re-invent the wheel),
* providing a **reasonably efficient data transport service** (zero-copy, high throughput - [TCP](https://zeromq.org/), [SHMEM](https://doi.org/10.1051/epjconf/201921405029), and [RDMA](https://doi.org/10.1051/epjconf/201921405022) implementations available),
* being **data format agnostic** (suitable data formats are usually experiment-specific), and
* providing other **basic building blocks** such as a simple state machine based execution framework and a plugin mechanism to integrate with external config/control systems.
- providing an **asynchronous message passing abstraction** that integrates different existing data transport technologies (no need to re-invent the wheel),
- providing a **reasonably efficient data transport service** (zero-copy, high throughput - [TCP](https://zeromq.org/), [SHMEM](https://doi.org/10.1051/epjconf/201921405029), and [RDMA](https://doi.org/10.1051/epjconf/201921405022) implementations available),
- being **data format agnostic** (suitable data formats are usually experiment-specific), and
- providing other **basic building blocks** such as a simple state machine based execution framework and a plugin mechanism to integrate with external config/control systems.
FairMQ is not an end-user application, but a library and framework used by experiment software experts to implement higher-level experiment-specific applications.