Open Source in Finance Forum New York 2024

There is little doubt that GenAI will have an impact on almost every aspect of our business and personal lives. However, we are at an interesting juncture: models are becoming ever more powerful, with prototypes showing ever greater promise, but there remain significant challenges when it comes to the reality of putting this technology into practice. How do we take these astonishing experiments and turn them into something safe and reliable that can ultimately be trusted in the wild? Financial services has a fairly long history with AI and machine learning. However, its place within the industry is shifting. It is no longer the exclusive tool of Data Scientists. Rather it is a tool we are placing in the hands of our engineers, product owners, and, if we are to maximise the benefits, we should be putting it in everyone's hands. But to do-so safely, we need to educate people regarding the strengths and weaknesses, the risks and opportunities this technology presents. Within FINOS, we are forming an AI readiness group that tackles some of these challenges. This talk will take a deep dive, exploring the changes we need to make as AI rapidly becomes a standard tool in our armoury

Change Management team implemented GenAI based framework for reviewing quality of change implementation requests (CIR) for making production software or infrastructure changes, that ensures a certain standard and guidelines are adhered potentially preventing the production impact, and possibility of Change Control Failure resulting in an audit or regulatory findings. For an ecosystem that completes thousands of changes each year, this framework provides an efficient method to review change quality, ensure SDLC compliance and implement change controls. GenAI Assisted Quality check: Introduction of LLM based evaluation of CIR: •Every CIR will be scanned through LLM using pre defined rubrics, in case it is not up to a certain standard, or missing any information, the AI assistant will prompt back to user to provide the information to create a complete CR. •This is implemented using zero shot prompt and completion inference of LLM model. Graph based RAG Status: •CIRs are stored in a graph data structure, through which relations are captured •LLM is used for generating cypher queries to traverse the knowledge graph to determine relevant relation between CIRs and flag potential risk

Large financial institutions typically deploy their IT services across a number of private and public cloud providers. This asks for a platform-agnostic design of software applications to minimize the amount of rework required to migrate services between platforms. This applies to AI/ML models deployed for inference as much as any other software. In this presentation, we present an innovative framework ("inference-server") to streamline the deployment of an AI/ML model to the cloud. The framework is published by J.P. Morgan Chase at https://github.com/jpmorganchase/inference-server The framework abstracts the cloud infrastructure and connectivity aspects through a simple plugin system. This requires an ML software engineer to define an ML model as 2 functions only: 1) to load the model in memory and 2) to invoke that loaded model for a given set of input features. These 2 functions are then added to a Docker container image along the inference-server framework. This approach leads to a very modular design where an AI/ML model can be trivially tested in a development environment. If required the model could be deployed and accessed from any cloud computing platform.