LLM evaluation framework for Lendwell
Automated evaluation of prompts and models so every change is measured before release. A case study by Rahul Mehta, Machine Learning Engineer (LLMs & RAG).
Challenge
Lendwell, an SME lending platform, changed prompts based on a few manual tests. Quality regressions reached customers twice in one month.
Solution
I built an evaluation framework with curated test sets, model-graded and rule-based metrics, cost and latency tracking, and a CI job that blocks releases that reduce quality.