Open Source in Finance Forum New York 2024

There is little doubt that GenAI will have an impact on almost every aspect of our business and personal lives. However, we are at an interesting juncture: models are becoming ever more powerful, with prototypes showing ever greater promise, but there remain significant challenges when it comes to the reality of putting this technology into practice. How do we take these astonishing experiments and turn them into something safe and reliable that can ultimately be trusted in the wild? Financial services has a fairly long history with AI and machine learning. However, its place within the industry is shifting. It is no longer the exclusive tool of Data Scientists. Rather it is a tool we are placing in the hands of our engineers, product owners, and, if we are to maximise the benefits, we should be putting it in everyone's hands. But to do-so safely, we need to educate people regarding the strengths and weaknesses, the risks and opportunities this technology presents. Within FINOS, we are forming an AI readiness group that tackles some of these challenges. This talk will take a deep dive, exploring the changes we need to make as AI rapidly becomes a standard tool in our armoury

Change Management team implemented GenAI based framework for reviewing quality of change implementation requests (CIR) for making production software or infrastructure changes, that ensures a certain standard and guidelines are adhered potentially preventing the production impact, and possibility of Change Control Failure resulting in an audit or regulatory findings. For an ecosystem that completes thousands of changes each year, this framework provides an efficient method to review change quality, ensure SDLC compliance and implement change controls. GenAI Assisted Quality check: Introduction of LLM based evaluation of CIR: •Every CIR will be scanned through LLM using pre defined rubrics, in case it is not up to a certain standard, or missing any information, the AI assistant will prompt back to user to provide the information to create a complete CR. •This is implemented using zero shot prompt and completion inference of LLM model. Graph based RAG Status: •CIRs are stored in a graph data structure, through which relations are captured •LLM is used for generating cypher queries to traverse the knowledge graph to determine relevant relation between CIRs and flag potential risk

Large financial institutions typically deploy their IT services across a number of private and public cloud providers. This asks for a platform-agnostic design of software applications to minimize the amount of rework required to migrate services between platforms. This applies to AI/ML models deployed for inference as much as any other software. In this presentation, we present an innovative framework ("inference-server") to streamline the deployment of an AI/ML model to the cloud. The framework is published by J.P. Morgan Chase at https://github.com/jpmorganchase/inference-server The framework abstracts the cloud infrastructure and connectivity aspects through a simple plugin system. This requires an ML software engineer to define an ML model as 2 functions only: 1) to load the model in memory and 2) to invoke that loaded model for a given set of input features. These 2 functions are then added to a Docker container image along the inference-server framework. This approach leads to a very modular design where an AI/ML model can be trivially tested in a development environment. If required the model could be deployed and accessed from any cloud computing platform.

In this thought-provoking presentation, we explore the profound impact of artificial intelligence (AI) on software development within the financial services industry. Drawing parallels with historical technological revolutions that reshaped economies, such as the Roman water mills, the steam engine, and the early 2000s push for automation, we demonstrate how AI is poised to be the ultimate general-purpose technology of our time. We delve into the economic and technological implications of AI adoption, highlighting how it drives increased productivity, cost reductions, and an unprecedented expansion of demand for innovative financial products and services. The presentation underscores the emerging productivity polarity, where those who embrace AI will gain a significant competitive advantage over those who resist its integration. Recognizing the pivotal role of open source in driving innovation within the finance sector, we emphasize the urgency for technologists to seamlessly incorporate AI into their daily workflows. By doing so, developers can position themselves as vanguards, fostering collaboration and unlocking new frontiers in open finance software development.

When leveraging AI in financial services, there are new technologies and security challenges to tackle in addition to the usual DevOps and scaling considerations. As now exists several sources of information for AI security best practices, we are not left on our own. But practically applying them in your project is not so well described. In this talk, Rowan Baker and Vicente Herrera will explain how to start with a security guideline recommendation like the “Simple Governance Framework” from the FINOS AI Readiness Group and apply it to a real project They will show how to transform the recommendations into real actions and how specific controls or mitigations have been chosen and implemented in a practical way. Some approaches may be conventional but important, like those regarding supply chain security, others will be more AI specific and novel. They will also show how the guidelines may not cover specific implementation, how to extend the coverage and contribute back so others benefit from the knowledge you have gained.

Generative AI has plenty of use cases in Finance. From the customer support on the front office, to research, business analysis and optimization of investment cases. However most companies get stuck at the "Proof of concept" stage. While it is relatively quick to create a demo with a "Wow effect", building the right pipelines, governance, security and controls and in general get the buy in and approvals from legal and compliance teams is extremely hard. In this presentation we will show a detailed runbook on how to leverage Generative AI cases at scale in the organization, from the perspective of the first company that delivered a production application in the market. We will review how to build an open innovation system that promotes collaboration in what we called "The era of the 14.000 innovators" where every employee became a prompt engineer. And how we harnessed that power to put dozens of use cases in production (internal and external) as well as the edge research on agents and beyond.

we delve into the critical intersection RAG systems and robust data governance practices. As organizations increasingly leverage RAG to enhance their AI capabilities, the need for strong data management becomes paramount. Our discussion will cover: Data Discovery and Routing: Learn how to leverage enterprise data catalogs to enhance RAG's query processing precision and efficiency. Ownership and Explainability: Explore techniques for incorporating data ownership information and rich metadata to improve transparency and decision-making. Data Security and Access Control: Understand the implementation of Role-Based and Policy-Based Access Controls, including innovative approaches using GenAI for dynamic permission assessment. Data Quality and Bias Mitigation: Discover strategies for enriching RAG systems with data quality metrics and methods for detecting and mitigating biased data. Knowledge Graphs and Master Data Management: Uncover how these tools can dramatically improve the accuracy of LLM-based responses Data Lineage and Root Cause Analysis: integrate data lineage capabilities for enhanced explainability and efficient problem-solving. Advanced Prompting Techniques: