HierLegalBERT Hierarchical Legal BERT for Indian Supreme Court Judgment Analysis Multi Task Learning · Hierarchical Attention · Indian Legal NLP · OpenNyaya Corpus Abstract Existing legal NLP models treat court judgments as flat token sequences, truncating or ignoring the vast majority of a typical 5–15 page Indian Supreme Court judgment. HierLegalBERT is a hierarchical BERT variant that explicitly models the structural hierarchy of Indian Supreme Court judgments — preamble, facts, arguments, analysis, and order — through a two level encoder architecture. A sentence level BERT encoder with novel script type embeddings encodes each sentence independently. A document level transformer with section type positional encoding then reasons across all sentences with awareness of their structural role in the judgment. Four task heads — NER, clause classification, judgment prediction, and contradiction detection — are trained jointly using Kendall et al. uncertainty weighted multi task learning , eliminating manual loss tuning. Trained on 19,000+ Indian Supreme Court judgments from the OpenNyaya corpus (1950–2023), HierLegalBERT provides a reproducible, publicly available baseline for Indian…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy