ϟ

Elizabeth Sexton-Kennedy

Here are all the papers by Elizabeth Sexton-Kennedy that you can download and read on OA.mg.
Elizabeth Sexton-Kennedy’s last known institution is . Download Elizabeth Sexton-Kennedy PDFs here.

Claim this Profile →
DOI: 10.1007/s41781-021-00055-1
2021
Cited 40 times
Challenges in Monte Carlo Event Generator Software for High-Luminosity LHC
Abstract We review the main software and computing challenges for the Monte Carlo physics event generators used by the LHC experiments, in view of the High-Luminosity LHC (HL-LHC) physics programme. This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group as an input to the LHCC review of HL-LHC computing, which has started in May 2020.
DOI: 10.1051/epjconf/202429508017
2024
Software Citation in HEP: Current State and Recommendations for the Future
In November 2022, the HEP Software Foundation and the Institute for Research and Innovation for Software in High-Energy Physics organized a workshop on the topic of Software Citation and Recognition in HEP. The goal of the workshop was to bring together different types of stakeholders whose roles relate to software citation, and the associated credit it provides, in order to engage the community in a discussion on: the ways HEP experiments handle citation of software, recognition for software efforts that enable physics results disseminated to the public, and how the scholarly publishing ecosystem supports these activities. Reports were given from the publication board leadership of the ATLAS, CMS, and LHCb experiments and HEP open source software community organizations (ROOT, Scikit-HEP, MCnet), and perspectives were given from publishers (Elsevier, JOSS) and related tool providers (INSPIRE, Zenodo). This paper summarizes key findings and recommendations from the workshop as presented at the 26th International Conference on Computing in High Energy and Nuclear Physics (CHEP 2023).
DOI: 10.1088/1742-6596/664/3/032012
2015
Cited 8 times
Fermilab Computing at the Intensity Frontier
The Intensity Frontier refers to a diverse set of particle physics experiments using high- intensity beams. In this paper I will focus the discussion on the computing requirements and solutions of a set of neutrino and muon experiments in progress or planned to take place at the Fermi National Accelerator Laboratory located near Chicago, Illinois. The experiments face unique challenges, but also have overlapping computational needs. In principle, by exploiting the commonality and utilizing centralized computing tools and resources, requirements can be satisfied efficiently and scientists of individual experiments can focus more on the science and less on the development of tools and infrastructure.
DOI: 10.1088/1742-6596/513/2/022034
2014
Cited 7 times
Stitched Together: Transitioning CMS to a Hierarchical Threaded Framework
Modern computing hardware is transitioning from using a single high frequency complicated computing core to many lower frequency simpler cores. As part of that transition, hardware manufacturers are urging developers to exploit concurrency in their programs via operating system threads. We will present CMS' effort to evolve our single threaded framework into a highly concurrent framework. We will outline the design of the new framework and how the design was constrained by the initial single threaded design. Then we will discuss the tools we have used to identify and correct thread unsafe user code. Finally we will end with a description of the coding patterns we found useful when converting code to being thread-safe.
DOI: 10.1088/1742-6596/608/1/012034
2015
Cited 5 times
Implementation of a Multi-threaded Framework for Large- scale Scientific Applications
The CMS experiment has recently completed the development of a multi-threaded capable application framework. In this paper, we will discuss the design, implementation and application of this framework to production applications in CMS. For the 2015 LHC run, this functionality is particularly critical for both our online and offline production applications, which depend on faster turn-around times and a reduced memory footprint relative to before. These applications are complex codes, each including a large number of physics-driven algorithms. While the framework is capable of running a mix of thread-safe and legacy modules, algorithms running in our production applications need to be thread-safe for optimal use of this multi-threaded framework at a large scale. Towards this end, we discuss the types of changes, which were necessary for our algorithms to achieve good performance of our multithreaded applications in a full-scale application. Finally performance numbers for what has been achieved for the 2015 run are presented.
DOI: 10.1088/1742-6596/1085/2/022006
2018
Cited 5 times
HEP Software Development in the Next Decade; the Views of the HSF Community
We have known for a while now that projections of computing needs for the experiments running in 10 years from now are unaffordable. Over the past year the HSF has convened a series of workshops aiming to find consensus on the needs, and produce proposals for research and development to address this challenge. At this time many of the software related drafts are far enough along to give a clear picture of what will result from this process. This talk will synthesize and report on some of the key elements to come out of this community work.
2015
Cited 4 times
An ontology design pattern for particle physics analysis
The detector final state is the core element of particle physics analysis as it defines the physical characteristics that form the basis of the measurement presented in a published paper. Although they are a crucial part of the research process, detector final states are not yet formally described, published in papers or searchable in a convenient way. This paper aims at providing an ontology pattern for the detector final state that can be used as a building block for an ontology covering the whole particle physics analysis life cycle.
DOI: 10.1109/tns.2007.910980
2008
Cited 5 times
The CMS High Level Trigger System
The CMS data acquisition (DAQ) system relies on a purely software driven high level trigger (HLT) to reduce the full Level 1 accept rate of 100 kHz to approximately 100 Hz for archiving and later offline analysis. The HLT operates on the full information of events assembled by an event builder collecting detector data from the CMS front-end systems. The HLT software consists of a sequence of reconstruction and filtering modules executed on a farm of O(1000) CPUs built from commodity hardware. This paper presents the architecture of the CMS HLT, which integrates the CMS reconstruction framework in the online environment. The mechanisms to configure, control, and monitor the filter farm and the procedures to validate the filtering code within the DAQ environment are described.
DOI: 10.1088/1742-6596/219/3/032011
2010
Cited 4 times
File level provenance tracking in CMS
The CMS off-line framework stores provenance information within CMS's standard ROOT event data files. The provenance information is used to track how each data product was constructed, including what other data products were read to do the construction. We will present how the framework gathers the provenance information, the efforts necessary to minimise the space used to store the provenance in the file and the tools that will be available to use the provenance.
DOI: 10.48550/arxiv.2008.13636
2020
Cited 4 times
HL-LHC Computing Review: Common Tools and Community Software
Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC's success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. In this document we address the issues for software that is used in multiple experiments (usually even more widely than ATLAS and CMS) and maintained by teams of developers who are either not linked to a particular experiment or who contribute to common software within the context of their experiment activity. We also give space to general considerations for future software and projects that tackle upcoming challenges, no matter who writes it, which is an area where community convergence on best practice is extremely useful.
DOI: 10.1088/1742-6596/664/3/032027
2015
Open access to high-level data and analysis tools in the CMS experiment at the LHC
The CMS experiment, in recognition of its commitment to data preservation and open access as well as to education and outreach, has made its first public release of high-level data under the CC0 waiver: up to half of the proton-proton collision data (by volume) at 7 TeV from 2010 in CMS Analysis Object Data format. CMS has prepared, in collaboration with CERN and the other LHC experiments, an open-data web portal based on Invenio. The portal provides access to CMS public data as well as to analysis tools and documentation for the public. The tools include an event display and histogram application that run in the browser. In addition a virtual machine containing a CMS software environment along with XRootD access to the data is available. Within the virtual machine the public can analyse CMS data; example code is provided. We describe the accompanying tools and documentation and discuss the first experiences of data use.
DOI: 10.48550/arxiv.2309.14571
2023
Software Citation in HEP: Current State and Recommendations for the Future
In November 2022, the HEP Software Foundation and the Institute for Research and Innovation for Software in High-Energy Physics organized a workshop on the topic of Software Citation and Recognition in HEP. The goal of the workshop was to bring together different types of stakeholders whose roles relate to software citation, and the associated credit it provides, in order to engage the community in a discussion on: the ways HEP experiments handle citation of software, recognition for software efforts that enable physics results disseminated to the public, and how the scholarly publishing ecosystem supports these activities. Reports were given from the publication board leadership of the ATLAS, CMS, and LHCb experiments and HEP open source software community organizations (ROOT, Scikit-HEP, MCnet), and perspectives were given from publishers (Elsevier, JOSS) and related tool providers (INSPIRE, Zenodo). This paper summarizes key findings and recommendations from the workshop as presented at the 26th International Conference on Computing in High Energy and Nuclear Physics (CHEP 2023).
DOI: 10.2172/1436702
2018
HEP Software Foundation Community White Paper Working Group - Data Analysis and Interpretation
At the heart of experimental high energy physics (HEP) is the development of facilities and instrumentation that provide sensitivity to new phenomena. Our understanding of nature at its most fundamental level is advanced through the analysis and interpretation of data from sophisticated detectors in HEP experiments. The goal of data analysis systems is to realize the maximum possible scientific potential of the data within the constraints of computing and human resources in the least time. To achieve this goal, future analysis systems should empower physicists to access the data with a high level of interactivity, reproducibility and throughput capability. As part of the HEP Software Foundation Community White Paper process, a working group on Data Analysis and Interpretation was formed to assess the challenges and opportunities in HEP data analysis and develop a roadmap for activities in this area over the next decade. In this report, the key findings and recommendations of the Data Analysis and Interpretation Working Group are presented.
DOI: 10.48550/arxiv.1812.00761
2018
HEP Software Foundation Community White Paper Working Group -- Data Organization, Management and Access (DOMA)
Without significant changes to data organization, management, and access (DOMA), HEP experiments will find scientific output limited by how fast data can be accessed and digested by computational resources. In this white paper we discuss challenges in DOMA that HEP experiments, such as the HL-LHC, will face as well as potential ways to address them. A research and development timeline to assess these changes is also proposed.
DOI: 10.48550/arxiv.2010.05102
2020
Software Sustainability & High Energy Physics
New facilities of the 2020s, such as the High Luminosity Large Hadron Collider (HL-LHC), will be relevant through at least the 2030s. This means that their software efforts and those that are used to analyze their data need to consider sustainability to enable their adaptability to new challenges, longevity, and efficiency, over at least this period. This will help ensure that this software will be easier to develop and maintain, that it remains available in the future on new platforms, that it meets new needs, and that it is as reusable as possible. This report discusses a virtual half-day workshop on "Software Sustainability and High Energy Physics" that aimed 1) to bring together experts from HEP as well as those from outside to share their experiences and practices, and 2) to articulate a vision that helps the Institute for Research and Innovation in Software for High Energy Physics (IRIS-HEP) to create a work plan to implement elements of software sustainability. Software sustainability practices could lead to new collaborations, including elements of HEP software being directly used outside the field, and, as has happened more frequently in recent years, to HEP developers contributing to software developed outside the field rather than reinventing it. A focus on and skills related to sustainable software will give HEP software developers an important skill that is essential to careers in the realm of software, inside or outside HEP. The report closes with recommendations to improve software sustainability in HEP, aimed at the HEP community via IRIS-HEP and the HEP Software Foundation (HSF).
DOI: 10.1051/epjconf/201921409010
2019
Quantum Computing
In recent years Quantum Computing has attracted a great deal of attention in the scientific and technical communities. Interest in the field has expanded to include the popular press and various funding agencies. We discuss the origins of the idea of using quantum systems for computing. We then give an overview in recent developments in quantum hardware and software, as well as some potential applications for high energy physics.
DOI: 10.48550/arxiv.1311.2567
2013
Snowmass Computing Frontier: Software Development, Staffing and Training
Report of the Snowmass CpF-I4 subgroup on Software Development, Staffing and Training
DOI: 10.1088/1742-6596/331/4/042023
2011
Release Strategies: The CMS approach for Development and Quality Assurance
Now that CMS has started data taking there is a balance to be struck between software release stability for operations and the need to improve the physics and technical performance of the code. In addition new code may need to be developed to correct for unforeseen data taking conditions, and has to be integrated into the mainstream releases with a minimum risk. To keep the process under control, CMS uses regular (twice a day) Integration Builds. A complex set of validation steps is used to verify the software at various stages, from the regular Integration Builds to running a full software and physics validation suite on the grid for major releases. CMS has adopted a development model that tries to strike the correct balance between the needs of stability and a constant improvement; this paper will describe our experience with this model, and tell the story of how the commissioning of the CMS offline has proceeded through the perspective of the past year's releases.
DOI: 10.1088/1742-6596/513/2/022026
2014
CMS Geometry Through 2020
CMS faces real challenges with upgrade of the CMS detector through 2020 and beyond. One of the challenges, from the software point of view, is managing upgrade simulations with the same software release as the 2013 scenario. We present the CMS geometry description software model, its integration with the CMS event setup and core software. The CMS geometry configuration and selection is implemented in Python. The tools collect the Python configuration fragments into a script used in CMS workflow. This flexible and automated geometry configuration allows choosing either transient or persistent version of the same scenario and specific version of the same scenario. We describe how the geometries are integrated and validated, and how we define and handle different geometry scenarios in simulation and reconstruction. We discuss how to transparently manage multiple incompatible geometries in the same software release. Several examples are shown based on current implementation assuring consistent choice of scenario conditions. The consequences and implications for multiple/different code algorithms are discussed.
DOI: 10.5281/zenodo.7003963
2022
HSF IRIS-HEP Second Analysis Ecosystem Workshop Report
DOI: 10.1088/1742-6596/396/3/032065
2012
Preparing for long-term data preservation and access in CMS
The data collected by the LHC experiments are unique and present an opportunity and a challenge for a long-term preservation and re-use. The CMS experiment has defined a policy for the data preservation and access to its data and is starting its implementation. This note describes the driving principles of the policy and summarises the actions and activities which are planned in the starting phase of the project.
DOI: 10.1088/1742-6596/898/4/042045
2017
Benchmarking high performance computing architectures with CMS’ skeleton framework
In 2012 CMS evaluated which underlying concurrency technology would be the best to use for its multi-threaded framework. The available technologies were evaluated on the high throughput computing systems dominating the resources in use at that time. A skeleton framework benchmarking suite that emulates the tasks performed within a CMSSW application was used to select Intel's Thread Building Block library, based on the measured overheads in both memory and CPU on the different technologies benchmarked. In 2016 CMS will get access to high performance computing resources that use new many core architectures; machines such as Cori Phase 1&2, Theta, Mira. Because of this we have revived the 2012 benchmark to test it's performance and conclusions on these new architectures. This talk will discuss the results of this exercise.
DOI: 10.48550/arxiv.1812.07861
2018
HEP Software Foundation Community White Paper Working Group - Data Processing Frameworks
Data processing frameworks are an essential part of HEP experiments' software stacks. Frameworks provide a means by which code developers can undertake the essential tasks of physics data processing, accessing relevant inputs and storing their outputs, in a coherent way without needing to know the details of other domains. Frameworks provide essential core services for developers and help deliver a configurable working application to the experiments' production systems. Modern HEP processing frameworks are in the process of adapting to a new computing landscape dominated by parallel processing and heterogeneity, which pose many questions regarding enhanced functionality and scaling that must be faced without compromising the maintainability of the code. In this paper we identify a program of work that can help further clarify the key concepts of frameworks for HEP and then spawn R&D activities that can focus the community's efforts in the most efficient manner to address the challenges of the upcoming experimental program.
DOI: 10.1109/rtc.2007.4382773
2007
The CMS High Level Trigger System
The CMS Data Acquisition (DAQ) System relies on a purely software driven High Level Trigger (HLT) to reduce the full Level-1 accept rate of 100 kHz to approximately 100 Hz for archiving and later offline analysis. The HLT operates on the full information of events assembled by an event builder collecting detector data from the CMS front-end systems. The HLT software consists of a sequence of reconstruction and filtering modules executed on a farm of 0(1000) CPUs built from commodity hardware. This paper presents the architecture of the CMS HLT, which integrates the CMS reconstruction framework in the online environment. The mechanisms to configure, control, and monitor the Filter Farm and the procedures to validate the filtering code within the DAQ environment are described.
DOI: 10.1051/epjconf/202024505035
2020
SpackDev: Multi-Package Development with Spack
High Energy Physics (HEP) software environments often involve ∼ hundreds of external packages and libraries, and similar numbers of experiment-specific, science-critical packages—many under continuous development. Managing coherent releases of the external software stack is challenging enough, but managing the highly-collaborative—and distributed—development of a large body of code against such a stack adds even more complexity and room for error. Spack is a popular Python-based package management tool with a specific focus on the needs of High Performance Computing (HPC) systems and system administrators whose strength is orchestrating the discrete download, build, testing, and installation of pre-packaged or tagged third-party software against similarly stable dependencies. As such it is becoming increasingly popular within HEP as that community makes increasing use of HPC facilities, and as efforts to develop future HPC systems utilize Spack to provide scientific software on those platforms [1]. SpackDev is a system to facilitate the simultaneous development of interconnected sets of packages. Intended to handle packages without restriction to one internal build system, SpackDev is integrated with Spack as a command extension in order to leverage features such as dependency calculations and build system configuration, and is generally applicable outside HEP. We describe SpackDev’s features and development over the last two years, initial experience using SpackDev in the context of the LArSoft liquid argon detector toolkit, and work remaining before it can be considered a fully-functional multi-package build system for HEP experiments utilizing Spack.
DOI: 10.1051/epjconf/202024509015
2020
Geant Exascale Pilot Project
The upcoming generation of exascale HPC machines will all have most of their computing power provided by GPGPU accelerators. In order to be able to take advantage of this class of machines for HEP Monte Carlo simulations, we started to develop a Geant pilot application as a collaboration between HEP and the Exascale Computing Project. We will use this pilot to study and characterize how the machines’ architecture affects performance. The pilot will encapsulate the minimum set of physics and software framework processes necessary to describe a representative HEP simulation problem. The pilot will then be used to exercise communication, computation, and data access patterns. The project’s main objective is to identify re-engineering opportunities that will increase event throughput by improving single node performance and being able to make efficient use of the next generation of accelerators available in Exascale facilities.
DOI: 10.1109/nssmic.2006.354216
2006
The CMS Simulation Software
In this paper we present the features and the expected performance of the re-designed CMS simulation software, as well as the experience from the migration process. Today, the CMS simulation suite is based on the two principal components - Geant4 detector simulation toolkit and the new CMS offline Framework and Event Data Model. The simulation chain includes event generation, detector simulation, and digitization steps. With Geant4, we employ the full set of electromagnetic and hadronic physics processes and detailed particle tracking in the 4 Tesla magnetic field. The Framework provides "action on demand" mechanisms, to allow users to load dynamically the desired modules and to configure and tune the final application at the run time. The simulation suite is used to model the complete central CMS detector (over 1 million of geometrical volumes) and the forward systems, such as Castor calorimeter and Zero Degree Calorimeter, the Totem telescopes, Roman Pots, and the Luminosity Monitor. The designs also previews the use of the electromagnetic and hadronic showers parametrization, instead of full modelling of high energy particles passage through a complex hierarchy of volumes and materials, allowing significant gain in speed while tuning the simulation to test beam and collider data. Physics simulation has been extensively validated by comparison with test beam data and previous simulation results. The redesigned and upgraded simulation software was exercised for performance and robustness tests. It went into Production in July 2006, running in the US and EU grids, and has since delivered about 60 millions of events.
DOI: 10.1088/1742-6596/664/3/032026
2015
A Review of Event Processing Frameworks used in HEP
Today there are many different experimental event processing frameworks in use by running or about to be running experiments. This talk will discuss the different components of these frameworks. In the past there have been attempts at shared framework projects for example the collaborations on the BaBar framework (between BaBar, CDF, and CLEO), on the Gaudi framework (between LHCb and ATLAS), on AliROOT/FairROOT (between Alice and GSI/Fair), and in some ways on art (Fermilab based experiments) and CMS' framework. However, for reasons that will be discussed, these collaborations did not result in common frameworks shared among the intended experiments. Though importantly, two of the resulting projects have succeeded in providing frameworks that are shared among many customer experiments: Fermilab's art framework and GSI/Fair's FairROOT. Interestingly, several projects are considering remerging their frameworks after many years apart. I'll report on an investigation and analysis of these realities. With the advent of the need for multi-threaded frameworks and the scarce available manpower, it is important to collaborate in the future; however it is also important to understand why previous attempts at multi-experiment frameworks either worked or didn't work.
DOI: 10.1088/1742-6596/219/4/042051
2010
CMS partial releases: Model, tools, and applications online and framework-light releases
With the integration of all CMS software packages into one release, the CMS software release management team faced the problem that for some applications a big distribution size and a large number of unused packages have become a real issue. TWe describe a solution to this problem. Based on functionality requirements and dependency analysis, we define a self-contained subset of the full CMS software release and create a Partial Release for such applications. We describe a high level architecture for this model, and tools that are used to automate the release preparation. Finally we discuss the two most important use cases for which this approach is currently implemented.
DOI: 10.2172/1874309
2017
HEP Analysis Ecosystem Workshop Report
Over the past 20 years the HEP community has developed and gravitated around an analysis ecosystem centered on ROOT. ROOT and its ecosystem both dominate HEP analysis and impact the full event processing chain, providing foundation libraries, I/O services etc. that have prevalence in the field. The analysis tools landscape is however evolving in ways that can have a durable impact on the analysis ecosystem and a strong influence on the analysis and core software landscape a decade from now, a timescale currently in an intensive planning round with the HEP Software Foundation(HSF)Community White Paper process. Data intensive analysis is growing in importance in other sciences and in the wider world. Powerful tools and new development initiatives, both within our field and in the wider open source community, have emerged. Creative developers have an ever more powerful open source toolkit available and are applying it towards innovations that leverage both open source and the ROOT ecosystem. ROOT itself is approaching a major re-engineering with ROOT 7, leveraging the powerful evolution of C++, with a major overhaul in its interfaces as seen both by its users and by satellite tools in the ROOT ecosystem.
DOI: 10.5281/zenodo.6599290
2017
HEP Analysis Ecosystem Workshop Report
DOI: 10.48550/arxiv.2205.08193
2022
The HEP Software Foundation Community
The HEP Software Foundation was founded in 2014 to tackle common problems of software development and sustainability for high-energy physics. In this paper we outline the motivation for the founding of the organisation and give a brief history of its development. We describe how the organisation functions today and what challenges remain to be faced in the future.
DOI: 10.48550/arxiv.2212.04889
2022
Second Analysis Ecosystem Workshop Report
The second workshop on the HEP Analysis Ecosystem took place 23-25 May 2022 at IJCLab in Orsay, to look at progress and continuing challenges in scaling up HEP analysis to meet the needs of HL-LHC and DUNE, as well as the very pressing needs of LHC Run 3 analysis. The workshop was themed around six particular topics, which were felt to capture key questions, opportunities and challenges. Each topic arranged a plenary session introduction, often with speakers summarising the state-of-the art and the next steps for analysis. This was then followed by parallel sessions, which were much more discussion focused, and where attendees could grapple with the challenges and propose solutions that could be tried. Where there was significant overlap between topics, a joint discussion between them was arranged. In the weeks following the workshop the session conveners wrote this document, which is a summary of the main discussions, the key points raised and the conclusions and outcomes. The document was circulated amongst the participants for comments before being finalised here.
DOI: 10.5281/zenodo.7418264
2022
HSF IRIS-HEP Second Analysis Ecosystem Workshop Report
DOI: 10.2172/1488605
2018
HEP Software Foundation Community White Paper Working Group - Data and Software Preservation to Enable Reuse
In this chapter of the High Energy Physics Software Foundation Community Whitepaper, we discuss the current state of infrastructure, best practices, and ongoing developments in the area of data and software preservation in high energy physics. A re-framing of the motivation for preservation to enable re-use is presented. A series of research and development goals in software and other cyber-infrastructure that will aid in the enabling of reuse of particle physics analyses and production software are presented and discussed.
DOI: 10.1109/escience.2018.00090
2018
Strategies for Modeling Extreme Luminosities in the CMS Simulation
The LHC simulation frameworks are already confronting the High Luminosity LHC (HL-LHC) era. In order to design and evaluate the performance of the HL-LHC detector upgrades, realistic simulations of the future detectors and the extreme luminosity conditions they may encounter have to be simulated now. The use of many individual minimum-bias interactions to model the pileup poses several challenges to the CMS Simulation framework, including huge memory consumption, increased computation time, and the necessary handling of large numbers of event files during Monte Carlo production. Simulating a single hard scatter at an instantaneous luminosity corresponding to 200 pileup interactions per crossing can involve the input of thousands of individual minimum-bias events. Brute-force Monte Carlo production requires the overlay of these events for each hard-scatter event simulated.
DOI: 10.22323/1.340.0704
2019
The Future of Software and Computing for HEP - Pushing the Boundaries of the Possible
A Sampling of Data NeedsInternational science requires international data movement and storage.The LHC computing grid is both an example and a foundation on which to build an exascale data facility.This will be an expensive enterprise meaning it will have to be shared with all of the international data intensive sciences.This trend has already started with both DUNE and Bell II using LHC networks and co-located computing facilities.In order to get a sense of the scale of needs, I'd like to sample from a number of experiments in particle and astro-particle physics planned to be in operation in the mid-2020s.CMS and ATLAS project that in order to support the number of existing collaborators with data formats similar to the ones in use today they will need 5 exabytes of disk storage each.Work is ongoing to reduce the size of data analysis formats while maintaining their usefulness for physics analysis.However, no matter what, the raw data from the HL-LHC will need to be stored and archived.In 2026 that is projected to require 600PB per year.One of the major neutrino experiments operating on that time scale is DUNE.DUNE has the capability of generating an impossibly large amount of data.If all channels are completely read out (no zero suppression), at the full rate of the DAQ bandwidth limit, continuously over a year, DUNE would collect 150exabytes / year.Fortunately that level of detail for the full detector is only need for recording super nova events.Suppression of 39Ar decay, cold electronics noise, space charge effects, and argon impurities in these new liquid Ar TPC detectors, need to be studied and understood well enough to allow for effective zero suppression strategies in the readout.The target for DUNE is to produce 30PB per year.LSST will conduct a deep survey with a frequency that results in taking repeat images of every part of the sky every few nights in multiple bands for ten years.They plan on collecting
DOI: 10.2172/1477989
2018
The Future of Software and Computing for HEP: Pushing the Boundaries of the Possible
• Introduction • A Data Centric Vision for the long-term future • Community White Paper - Software and Computing tools • The changing landscape of computing even quantum computing • How much has been reflected in this conference… my observations • Summary
DOI: 10.2172/1477986
2019
Quantum Computing
• DISCLAIMER: This is Jim’s talk and unfortunately he could not be here, so I will do my best to present it. • At the last CHEP we heard a talk about Quantum Computing from a hardware perspective. • Jim is a software algorithm person and he will concentrate more on that. • 20months is a long time in Quantum Information Science, QIS • The Josephson Junction technology we heard about then has evolved as predicted and we now have “mid-range” devices available on the cloud. Introduction 6/20/
DOI: 10.48550/arxiv.1804.03983
2018
HEP Software Foundation Community White Paper Working Group - Data Analysis and Interpretation
At the heart of experimental high energy physics (HEP) is the development of facilities and instrumentation that provide sensitivity to new phenomena. Our understanding of nature at its most fundamental level is advanced through the analysis and interpretation of data from sophisticated detectors in HEP experiments. The goal of data analysis systems is to realize the maximum possible scientific potential of the data within the constraints of computing and human resources in the least time. To achieve this goal, future analysis systems should empower physicists to access the data with a high level of interactivity, reproducibility and throughput capability. As part of the HEP Software Foundation Community White Paper process, a working group on Data Analysis and Interpretation was formed to assess the challenges and opportunities in HEP data analysis and develop a roadmap for activities in this area over the next decade. In this report, the key findings and recommendations of the Data Analysis and Interpretation Working Group are presented.
DOI: 10.1109/nssmic.2005.1596421
2006
The CMS Object-Oriented Simulation
The CMS object oriented Geant4-based program is used to simulate the complete central CMS detector (over 1 million geometrical volumes) and the forward systems such as the Totem telescopes, Castor calorimeter, zero degree calorimeter, Roman pots, and the luminosity monitor. The simulation utilizes the full set of electromagnetic and hadronic physics processes provided by Geant4 and detailed particle tracking in the 4 tesla magnetic field. Electromagnetic shower parameterization can be used instead of full tracking of high-energy electrons and positrons, allowing significant gains in speed without detrimental precision losses. The simulation physics has been validated by comparisons with test beam data and previous simulation results. The system has been in production for almost two years and has delivered over 100 million events for various LHC physics channels. Productions are run on the US and EU grids at a rate of 3-5 million events per month. At the same time, the simulation has evolved to fulfill emerging requirements for new physics simulations, including very large heavy ion events and a variety of SUSY scenarios. The software has also undergone major technical upgrades. The framework and core services have been ported to the new CMS offline software architecture and event data model. In parallel, the program is subjected to ever more stringent quality assurance procedures, including a recently commissioned automated physics validation suite
DOI: 10.48550/arxiv.physics/0306112
2003
Commissioning the CDF Offline Software
CDF II is one of the two large collider experiments at Fermilab's Tevatron. Over the past two years we have commissioned the offline computing system. A task that has involved bringing up hundreds of computers and millions of lines of C++ software. This paper reports on this experience, concentrating on the software aspects of the project. We will highlight some of the successes as well as describe some of the work still to do.
2021
HL-LHC Computing Review Stage-2, Common Software Projects: Event Generators
This paper has been prepared by the HEP Software Foundation (HSF) Physics Event Generator Working Group (WG), as an input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which is due to take place in November 2021. It complements previous documents prepared by the WG in the context of the first phase of the LHCC review in 2020, including in particular the WG paper on the specific challenges in Monte Carlo event generator software for HL-LHC, which has since been updated and published, and which we are also submitting to the November 2021 review as an integral part of our contribution.
2000
The CDF Run II Data Catalog and Data Access Modules
The data access components of the CDF experiment Data Handling systems, together with the Data File Catalog, provide both the physicists running an analysis job and the production system running reconstruction jobs with a logical view of their input and output data, and transparent interaction with the storage management components.
1997
The Fermilab physics class library
The Fermilab Physics Class Library Task Force has been formed to supply classes and utilities, primarily in support of efforts by CDF and D0 toward using C++. A collection of libraries and tools will be assembled via development by the task force, collaboration with other HEP developers, and acquisition of existing modules. The main emphasis is on a kit of resources which physics coders can incorporate into their programs, with confidence in robustness and correct behavior. The task force is drawn from CDF, DO and the FNAL Computing and Beams Divisions. Modules-containers, linear algebra, histograms, etc.-have been assigned priority, based on immediate Run II coding activity, and will be available at times ranging from now to late May.