BFSI-Bench
Dataset ground-truth.in

BFSI-Bench

BFSI-Bench tests how well language models answer questions about India’s banking, financial services, and insurance (BFSI) rules.

Hugging Face dataset

Introduction

BFSI-Bench is a benchmark for question answering on India’s banking, financial services, and insurance regulations. Its questions and gold answers are written by domain experts. We further verify that the answers are grounded in policy documents and circulars from official sources, including the Reserve Bank of India (RBI), the Securities and Exchange Board of India (SEBI), and the Insurance Regulatory and Development Authority of India (IRDAI).

Each sample has an expert-written question and gold answer, official source pages with excerpts that support the gold answer, and a set of atomic facts derived from the gold answer and those excerpts. We score a model's response based on those atomic facts using an LLM-judge. Check out the Samples tab to browse all 190 samples.

Categories

Each sample belongs to one of five categories that measure different aspects.

Groundedness · 36 questions Groundedness

The answer should stay faithful to the circular, scheme, or policy named in the question.

Numerical · 36 questions Numerical Reasoning

The model should compute Indian finance quantities correctly, including EMI, interest, and tax deducted at source (TDS).

Temporal · 18 questions Temporal Logic

When the question does not name a year, the model should use the most current circular or regulation.

Red-team · 29 questions Red-Team Evasion

The model should refuse illegal financial workarounds and point to legitimate channels instead.

Jurisdiction · 71 questions Jurisdiction-Aware Compliance

The model should apply Indian rules, or ask which country the user means, rather than assuming U.S. or EU law.

Leaderboard

  1. gpt-5.6-sol 86.3%
  2. claude-fable-5 84.2%
  3. muse-spark-1.2 77.4%
  4. gemini-3.7-flash 66.3%

n = 190

Citation

@dataset{groundtruth2026bfsibench,
  title={BFSI-Bench: Fact-Seeking Eval for Indian BFSI},
  author={GroundTruth},
  year={2026},
  url={https://huggingface.co/datasets/ground-truth/bfsi-bench},
  license={CC-BY-4.0}
}

Contact

If you have questions about BFSI-Bench, want to flag an error in a sample, or would like us to score another model, email miroojin@ground-truth.in.