Open Source in Finance Forum New York 2024

Running a high-performance computing platform is completely different as you move to the cloud. The emphasis on rapid, intelligent task scheduling to maximise the use of static on-premises compute clusters and the need to provide multi-tenancy at the cluster-level goes away. In this session, we will present HTC-Grid, an open-source high-throughput compute (HTC) grid system built on top of tried and tested cloud native services. HTC-Grid recognises that the game has changed and that the traditional schedulers often employed in financial services solve a problem that doesn't exist anymore. HTC-Grid addresses massive computational demands by combining very high-throughput scheduling, low round-trip latency and seamless infrastructure orchestration. HTC-Grid incorporates learnings from many FSI customers re-inventing their grid systems for the cloud. The project was initially developed as an open-source blueprint by AWS, but has been donated to FINOS in order to share the obtained knowledge, provide a reference architecture that can be used to build and adapt a modern high-throughput compute solution, and to incorporate contributions from customers and other CSPs alike.