Indian Supreme Court judgments contain decades of legal knowledge, but most of it is contained inside long, unstructured documents, making the process of retrieving relevant cases an arduous one. Finding relevant cases can be difficult because similar legal concepts are often described using different language.
To make legal research easier, my team and I built an Indian Legal Intelligence Platform that combines NLP, data analytics, and intelligent search. The goal was to transform thousands of unstructured court judgments into a searchable and analyzable legal knowledge base.
The platform extracts important information such as legal citations, constitutional articles, statutes, and named entities from judgment text. It also provides insights into judicial trends, frequently cited laws, influential judgments, and judge-level patterns using data collected from more than 26,000 Supreme Court cases.
At its core is a hybrid search engine that combines BM25 keyword search with FAISS-based semantic search, allowing users to find cases through both exact legal terms and similar legal concepts. A Flask-based web interface brings everything together, helping lawyers, researchers, and law students explore judgments, uncover legal trends, and discover relevant cases more efficiently.
Github link: https://github.com/emittens/Indian-Legal-Analysis
Phase 1: Preparing the Legal Dataset
The project started with a Kaggle dataset containing Indian Supreme Court judgments from 1950 to 2025. The judgments were available as PDF files, so the first step was extracting and structuring the data. For each judgment, information such as the case title, case ID, year, bench details, and full judgment text was extracted and stored in Parquet format for efficient querying.
This process produced a dataset of approximately 26,688 judgments and more than 19,000 citation links between cases.
Data source: https://www.kaggle.com/datasets/adarshsingh0903/legal-dataset-sc-judgments-india-19502024
License: CC0: Public Domain
Phase 2: Legal Text Processing
Prior to extracting information from the dataset, it had to be preprocessed particularly for legal context. To transform raw legal text into structured information, the system uses the Blackstone legal NLP model, with spaCy as a fallback.
Custom rule-based extractors were developed to identify:
- Legal citations (AIR, SCC, SCC Online, ILR)
- Constitutional articles
- References to statutes such as the IPC, CrPC, CPC, and Evidence Act
The NLP layer extracts named entities, legal citations, statutory references, and constitutional provisions, and basic linguistic information, making legal documents easier to analyze and search.
This layer converts unstructured legal text into structured data that can be used for search and analysis.
Phase 3: Exploring Judicial Trends
Once the data was structured, several analytics pipelines were created to explore how Indian jurisprudence has evolved over time.
The platform analyzes:
- Judgment volume trends
- Case category distributions (criminal, civil, constitutional, tax, labour, family law, etc.)
- Judgment length trends
- Bench composition patterns
- Judge authorship and participation trends
- Statute usage across decades
A citation network was also built by connecting judgments through their references to earlier cases. Using the PageRank algorithm, the system identifies influential judgments that are frequently relied upon in later decisions.
Together, these analyses provide a broader view of how Indian Supreme Court jurisprudence has evolved over the past 75 years.
Phase 4: Building the Dashboard
The analytics generated in the previous phase were made accessible through a simple Flask-based dashboard. This provided the data representation of decades of legal data.
Processed datasets were loaded from Parquet files and displayed through visualizations covering:
- Judgment volume trends
- Case category distributions
- Frequently appearing judges
- Influential judgments
- Statute usage trends
- Bench composition trends
The dashboard provides a convenient way to explore the corpus and its historical trends through visualization without directly working with the underlying datasets.
Phase 5: Building the Hybrid Retrieval System
The core feature of the platform is a hybrid retrieval system designed to go beyond traditional keyword search. The traditional keyword-based search had to be retained, as it is still the primary go-to for research purposes. The semantic based retrieval added any cases with similarity that might have been missed in a purely word matching retrieval system.
Hence, the system combines two retrieval approaches:
- BM25 (keyword search) for matching exact legal terms, statutes, constitutional articles, and case-specific language.
- FAISS-based semantic search for finding conceptually similar judgments, even when the wording differs.
Lexical Retrieval (BM25)
BM25 is used to retrieve judgments containing specific legal terms, statutes, constitutional articles, and case names. It performs well when exact legal language is important. For each query, BM25 identifies the most relevant judgments based on textual matches and returns a set of candidate results.
Semantic Retrieval (FAISS)
To capture legal meaning rather than just keywords, the platform uses semantic retrieval powered by InLegalBERT (law-ai/InLegalBERT), a transformer model trained on Indian legal text.
Supreme Court judgments are often much longer than the model’s 512-token context window. To handle this, each judgment is divided into overlapping chunks of 256 tokens with a stride of 128 tokens, ensuring that legal arguments spanning chunk boundaries are not lost. Each chunk is encoded into a dense embedding using InLegalBERT. The chunk embeddings are then aggregated using mean pooling to create a single vector representation for the entire judgment. The resulting document embeddings are L2-normalized and indexed using FAISS IndexFlatIP, allowing efficient cosine-similarity search.
This enables the system to find conceptually similar judgments even when phrasings and terminologies are different.
Hybrid Ranking
The results returned by BM25 and FAISS are merged into a common candidate pool. To rank them, the system uses a weighted legal-aware re-ranking layer then combines lexical relevance, semantic similarity, shared statutes, legal sections, constitutional articles, citation relationships, and case-category similarity. The weights used for score calculation can be manually set for better control. For the example show, the weights used are as follows:
𝑆𝑐𝑜𝑟𝑒 = 𝑤eight 𝑠𝑒𝑚𝑎𝑛𝑡𝑖𝑐 × 𝑆𝑒𝑚𝑎𝑛𝑡𝑖𝑐𝑆𝑖𝑚𝑖𝑙𝑎𝑟𝑖𝑡𝑦
+ 𝑤eight 𝐵𝑀25 × 𝐵𝑀25𝑆𝑐𝑜𝑟𝑒
+ 𝑤eight 𝑠h𝑎𝑟𝑒𝑑section × 𝑆h𝑎𝑟𝑒𝑑𝑆𝑒𝑐𝑡𝑖𝑜𝑛𝑠
+ 𝑤eight 𝑠h𝑎𝑟𝑒𝑑 𝑠𝑡𝑎𝑡𝑢𝑒𝑠 × 𝑆h𝑎𝑟𝑒𝑑𝑆𝑡𝑎𝑡𝑢𝑡𝑒𝑠
+ 𝑤eight 𝑐𝑖𝑡𝑎𝑡𝑖𝑜𝑛 𝑙𝑖𝑛𝑘 × 𝐶𝑖𝑡𝑎𝑡𝑖𝑜𝑛𝐿𝑖𝑛𝑘
+ 𝑤eight 𝑐𝑜𝑛𝑠𝑡𝑖𝑡𝑢𝑡𝑖𝑜𝑛𝑎𝑙 𝑎𝑟𝑡𝑖𝑐𝑙𝑒𝑠 × 𝐶𝑜𝑛𝑠𝑡𝑖𝑡𝑢𝑡𝑖𝑜𝑛𝑎𝑙𝐴𝑟𝑡𝑖𝑐𝑙𝑒𝑠
+ 𝑤eight 𝑠𝑎𝑚𝑒 𝑐𝑎𝑡𝑒𝑔𝑜𝑟𝑦 × 𝑆𝑎𝑚𝑒𝐶𝑎𝑡𝑒𝑔𝑜𝑟𝑦
In addition to ranking, users can apply metadata filters such as year range, case category, bench size, statutes, and constitutional articles to narrow the scope.
Users can view:
- Hybrid-ranked results
- BM25 keyword results
- Semantic search results
Each result includes a score breakdown and metadata, providing transparency into why a judgment was considered relevant. This helps build trust in the system, which is particularly important for legal research. Also, if clicked, the case details are shown and lets users open and verify each result against the actual judgment.
Results and Observations
To compare the retrieval approaches, the query “husband wife” was executed using BM25 keyword search, semantic search, and the hybrid retrieval system. The hybrid ranking used the following weights:
The results highlight the differences between lexical, semantic, and hybrid retrieval.
Lexical Retrieval Result
BM25 ranked documents purely based on keyword occurrence and frequency. While several relevant matrimonial and maintenance cases were retrieved, some highly ranked results were only loosely related to the legal intent of the query. For example, tax-related cases such as Philip John Plasket Thomas v. Commissioner of Income Tax appeared near the top because they contained terms such as “wife” and “marriage”, despite not dealing with matrimonial disputes. Similarly, other cases involving family relationships were retrieved because of exact keyword matches rather than legal relevance.
Semantic Retrieval Result
Semantic retrieval was able to identify conceptually related cases even when exact keyword matches were less prominent. Cases involving maintenance, marital disputes, divorce, and spousal rights appeared in the results despite variations in terminology. However, semantic search alone also introduced some noise. Several results lacked obvious relevance to the query when viewed in isolation because they were retrieved based on contextual similarity learned by the embedding model rather than direct lexical evidence.
Hybrid Retrieval Result
The hybrid system combined both approaches and produced more balanced results. The highest-ranked judgments included:
- K. Vimal v. K. Veeraswamy (maintenance claim under Section 125 CrPC)
- Begum Subanu v. A.M. Abdul Gafoor (maintenance rights of a wife)
- Rohini Kumari v. Narendra Singh (matrimonial dispute under the Hindu Marriage Act)
- Smt. Saroj Rani v. Sudarshan Kumar Chadha (restitution of conjugal rights)
- Payal Ashok Kumar Jindal v. Ashok Kumar Jindal (divorce proceedings)
These cases are directly related to husband-wife disputes, maintenance claims, marriage validity, divorce, and matrimonial rights. Compared to standalone BM25, the hybrid ranking pushed legally relevant matrimonial cases higher while reducing the prominence of cases that merely contained the words “husband” or “wife”. Compared to standalone semantic search, the hybrid approach benefited from exact keyword evidence and legal metadata, reducing the number of unrelated results.
This example demonstrates how lexical matching captures explicit terminology, semantic retrieval captures legal meaning, and hybrid ranking combines both signals to produce results that are generally more relevant for legal research than either method individually.
Challenges and Limitations
Parsing the PDFs to create a structured dataset proved to be difficult because the format of judgment documents has changed over the years. The parser had to be sensitive to these changes in order to effectively extract the required information. In some documents, the metadata shown differed slightly from the information present in the judgment itself. Due to the size of the dataset, processing also had to be carried out in batches.
The initial regex patterns failed to recognize every instance of the required fields. Different legal terminologies and document formats had to be studied and incorporated into the expressions to improve extraction. BlackStone Legal NLP model, specifically designed to identify legal contexts and terminology helped in the process.
Semantic retrieval was computationally expensive and took a long time to process, requiring the embedding pipeline to be executed in batches. Creating the FAISS index also benefited from high-performance GPUs or cloud-based GPU environments such as Google Colab to reduce processing time. Additionally, using mean pooling to generate a single embedding for each case improved retrieval speed but may have resulted in some loss of information compared to storing and searching individual chunk embeddings.
The weights used in the hybrid ranking system are not a fixed rule and require domain knowledge to tune effectively. Different legal research scenarios may benefit from different weight configurations, and selecting the most suitable combination remains a challenge.
Conclusion
This project demonstrates how NLP, information retrieval, graph analytics, and data visualization can be combined to improve legal research.
By transforming over 75 years of Indian Supreme Court judgments into a searchable and analyzable knowledge base, the platform helps researchers, lawyers, and students quickly discover relevant cases, understand legal trends, and explore the evolution of Indian jurisprudence.