Development and Evaluation of the ModernUzBERT Semantic Embedding Model with Extended Context for the Uzbek Language

Хужаяров, И.Ш., Очилов, М.М., Холматов, О.А., Жуманов, В.И.

Рақамли технологияларнинг назарий ва амалий масалалари · 2026-yil

Annotatsiya

This research is dedicated to the development of ModernUzBERT, an advanced embedding model for the Uzbek language based on contemporary Transformer architecture with an extended context window of 8192 tokens. In low-resource and agglutinative languages such as Uzbek, the limited capacity for processing comprehensive context significantly hampers model performance. To address this challenge, a large-scale text corpus in the Uzbek Latin script comprising over 125 million tokens was curated, and an optimized vocabulary of 52,000 sub-word units was developed. The model training process was executed in two primary stages: initially, the model was pre-trained from scratch using the Masked Language Modeling (MLM) objective; subsequently, it was adapted for semantic search via Supervised Fine-Tuning (SFT) using a dataset of 30,000 question-answer pairs. Through the implementation of Flash Attention 2 and Unpadding technologies, GPU resource utilization efficiency was substantially enhanced. The 8192-token context window enables the analysis of large-scale documents while preserving semantic integrity. Upon conclusion of the study, the developed model and all its components were released for open-access to the scientific community on the Hugging Face platform.

Maqola ma’lumotlari
MualliflarХужаяров, И.Ш., Очилов, М.М., Холматов, О.А., Жуманов, В.И.
JurnalРақамли технологияларнинг назарий ва амалий масалалари
NashriyotТАТУ Самарқанд филиали
ISSN2181-3086, 2181-3094
Nashr sanasi2026-05-15
Jild9
Son2
Betlar45-53
TilIngliz
Ilmiy sohaTexnika
DOI10.62132/ijdt.v9i2.375
Indeksga qo‘shilgan2026-08-26

DOI: 10.62132/ijdt.v9i2.375 · Maqolaning asl sahifasi

Kalit so‘zlar

NLP, узбекский язык, ModernBERT, семантические эмбеддинги, информационный поиск, длинный контекст, NLP, uzbek language, ModernBERT, semantic embedding, information retrieval, long context, flash attention

Ilmiy soha

Texnika

Iqtibos keltirish

APAХужаяров, И.Ш., Очилов, М.М., Холматов, О.А., & Жуманов, В.И. (2026). Development and Evaluation of the ModernUzBERT Semantic Embedding Model with Extended Context for the Uzbek Language. Рақамли технологияларнинг назарий ва амалий масалалари, 9(2), 45-53. https://doi.org/10.62132/ijdt.v9i2.375
GOST (OAK)Хужаяров, И.Ш. va b. Development and Evaluation of the ModernUzBERT Semantic Embedding Model with Extended Context for the Uzbek Language // Рақамли технологияларнинг назарий ва амалий масалалари. – 2026. – T. 9, № 2. – P. 45-53. – DOI: 10.62132/ijdt.v9i2.375.
MLAХужаяров, И.Ш., Очилов, М.М., Холматов, О.А., and Жуманов, В.И. “Development and Evaluation of the ModernUzBERT Semantic Embedding Model with Extended Context for the Uzbek Language.” Рақамли технологияларнинг назарий ва амалий масалалари, 9(2), 45-53, 2026. https://doi.org/10.62132/ijdt.v9i2.375
ChicagoХужаяров, И.Ш., Очилов, М.М., Холматов, О.А., and Жуманов, В.И. “Development and Evaluation of the ModernUzBERT Semantic Embedding Model with Extended Context for the Uzbek Language.” Рақамли технологияларнинг назарий ва амалий масалалари, 9(2), 45-53 (2026). https://doi.org/10.62132/ijdt.v9i2.375
HarvardХужаяров, И.Ш., Очилов, М.М., Холматов, О.А., and Жуманов, В.И (2026) ‘Development and Evaluation of the ModernUzBERT Semantic Embedding Model with Extended Context for the Uzbek Language’, Рақамли технологияларнинг назарий ва амалий масалалари, 9(2), 45-53. https://doi.org/10.62132/ijdt.v9i2.375
BibTeX
@article{article2026,
  title = {Development and Evaluation of the ModernUzBERT Semantic Embedding Model with Extended Context for the Uzbek Language},
  author = {Хужаяров, И.Ш. and Очилов, М.М. and Холматов, О.А. and Жуманов, В.И.},
  journal = {Рақамли технологияларнинг назарий ва амалий масалалари},
  year = {2026},
  volume = {9},
  number = {2},
  pages = {45-53},
  doi = {10.62132/ijdt.v9i2.375},
  url = {https://doi.org/10.62132/ijdt.v9i2.375}
}

Boshqa maqola uchun iqtibos tayyorlash

Рақамли технологияларнинг назарий ва амалий масалалари jurnalidan boshqa maqolalar

Рақамли технологияларнинг назарий ва амалий масалалари — barcha maqolalar · 2026-yil arxivi

Texnika sohasidagi boshqa maqolalar

Texnika sohasidagi OAK jurnallari