Samanvaya

Progress in multilingual large language models, machine translation, and cross-lingual NLP has created a pressing need for high-quality Indian language resources. Initiatives such as Universal Dependencies, the IIIT‑Hyderabad/LTRC annotation framework, and language-specific corpora have made considerable progress — yet several characteristic features of Indian languages remain difficult to capture uniformly, and their underlying unity is often overlooked.

Indian languages exhibit rich morphology, agglutinative structure, productive participial formations, complex verbal systems, evidential and reportative markers, honorific distinctions, relative‑correlative constructions, and grammar-driven semantic relations described in detail within the Indian grammatical tradition. These phenomena are frequently simplified or normalized during corpus creation, tokenization, annotation, and translation — leading to information loss and reduced linguistic fidelity.

The Samanvaya initiative addresses these concerns through a unified grammar-aware framework for creating and annotating multilingual corpora in Indian languages. It adds a layer that preserves language-specific grammatical information for computational use. This tutorial explains the Samanvaya tagging system for grammar-aware resource creation and its relevance to future Indian language NLP systems.

ICON has long been the leading forum for NLP research on Indian languages. As the field moves toward multilingual foundation models, agentic systems, and LLM-driven NLP, consistent, grammar-aware linguistic resources are increasingly important.

This tutorial addresses questions central to that shift: how parallel corpora can preserve grammatical features across Indian languages; whether tokenization can be standardized across agglutinative and non-agglutinative languages; how annotation frameworks can better capture phenomena unique to Indian languages; what role traditional Indian grammatical knowledge can play in enhancing AI models; and how linguistically informed resources can advance machine translation, parsing, and multilingual LLMs. Its goal is to foster collaboration among linguists, NLP researchers, corpus developers, and AI practitioners toward shared community standards for grammar-aware Indian language resources.