Backfilling a cloud with grid jobs

oleh: Fayer Simon, Whitehouse Dan

Format: Article
Diterbitkan: EDP Sciences 2024-01-01

Deskripsi

Imperial College London hosts a large Tier-2 WLCG grid site based around a HTCondor batch system; additionally it provides cloud computing facilities using OpenStack to non-WLCG activities. These cloud resources are open to opportunistic usage provided the impact on the primary cloud users remains low. In common with most Tier-2 sites we see constant job pressure from the WLCG VOs, while the usage of our cloud is much more intermittent. To allocate grid jobs to the available opportunistic cloud resources we implemented a lightweight backfill system based on the OpenStack Python API.