The article deals with the topic of tokenization of text corpora of the Uzbek language, which should take into account the linguistic features of the spelling of the language. The types of words are analyzed and a mathematical model of word formation for complex, paired, repeated, and compound words is proposed. And also based on the models developed a finite state machine to demonstrate the spelling rules of the language, and the rules are implemented by regular expressions
| Mualliflar | Bakaev, I.I. |
|---|---|
| Jurnal | ТАТУ хабарлари |
| Nashriyot | Тошкент ахборот технологиялари университети |
| ISSN | 2010-9857, 2181-1083 |
| Nashr sanasi | 2021-04-03 |
| Til | Ingliz |
| Ilmiy soha | Texnika |
| DOI | 10.51348/tuitmct431 |
| Indeksga qo‘shilgan | 2026-09-02 |
DOI: 10.51348/tuitmct431 · Maqolaning asl sahifasi
| APA | Bakaev, I.I. (2021). Linguistic features tokenization of text corpora of the Uzbek language. ТАТУ хабарлари. https://doi.org/10.51348/tuitmct431 |
|---|---|
| GOST (OAK) | Bakaev, I.I. Linguistic features tokenization of text corpora of the Uzbek language // ТАТУ хабарлари. – 2021. – DOI: 10.51348/tuitmct431. |
| MLA | Bakaev, I.I. “Linguistic features tokenization of text corpora of the Uzbek language.” ТАТУ хабарлари, 2021. https://doi.org/10.51348/tuitmct431 |
| Chicago | Bakaev, I.I. “Linguistic features tokenization of text corpora of the Uzbek language.” ТАТУ хабарлари (2021). https://doi.org/10.51348/tuitmct431 |
| Harvard | Bakaev, I.I (2021) ‘Linguistic features tokenization of text corpora of the Uzbek language’, ТАТУ хабарлари. https://doi.org/10.51348/tuitmct431 |
| BibTeX | @article{bakaevii2021,
title = {Linguistic features tokenization of text corpora of the Uzbek language},
author = {Bakaev, I.I.},
journal = {ТАТУ хабарлари},
year = {2021},
doi = {10.51348/tuitmct431},
url = {https://doi.org/10.51348/tuitmct431}
} |
Boshqa maqola uchun iqtibos tayyorlash
In this study, the reliability of mobile system base stations (BTS) is assessed by analyzing data obtained on faults in about 200 BTS over a six-month period. Five BTSs with the highest number of failures and duration…
In this paper, a mathematical model of the operation process of the ZigBee standard wireless network for remote monitoring systems based on the Markov chain apparatus was developed. Using the model was evaluated of the…
The article describes an implementing a real time speaker identification system by voice for embedded and general purpose computers. A review and analysis of existing speaker identification algorithms are made. The…
The article discusses the features and areas of application of multi-axis electromagnetic mechatronic modules in modules of mechatronic and robotic systems that allow not only to increase productivity and improve the…
In recent years, the world's population growth has been intensifying, resulting in specific problems related to the depletion of natural resources, food shortages, declining fertile lands, and changing weather…
Fundamental changes have occurred in today’s economy. These changes alter the relationship we have with our customers, our suppliers, our business partners and our colleagues. IT developments have presented companies…
The article examines the morphology of the Karakalpak language, which belongs to the Kipchak group of the Turkic language family. The forms of word formation in the Karakalpak language, their sequences and the affixes…
The article deals with the problems of determining the main quality indicators of a telecommunications network with different types of communication channels and the time-varying structure of their functioning to ensure…
The article discusses the approach to creating projects of organizational management systems that allow to use the theoretical and practical results of research in the field of artificial intelligence in their design…
ТАТУ хабарлари — barcha maqolalar · 2021-yil arxivi
This article examines compression as a meaning-oriented strategy in sign language translation and interpreting. Compression is understood not as arbitrary shortening, but as the selective reformulation of a source…
This study investigates the impact of Edge Computing on bandwidth utilization in Internet of Things (IoT)–driven Nigerian telecom networks. The rapid growth of IoT devices has significantly increased data traffic…
This article analyzes the scientific, pedagogical, and methodological foundations for the effective integration of national upbringing factors with modern global competencies in the context of continuous education…
This article addresses the problem of improving an adaptive traffic signal control algorithm at urban intersections by taking into account the dynamic characteristics of traffic flow. Fixed-time control under conditions…
Maqolada paxta xomashyosini yirik iflosliklardan tozalash jarayonida arrali baraban va kolosnikli panjara o‘rtasidagi o‘zaro ta’sirni takomillashtirish masalasi ko‘rib chiqilgan. Mavjud tozalagichlarda qo‘llaniladigan…
Maqolada paxta xomashyosini yirik va mayda iflosliklardan tozalash samaradorligini oshirishga qaratilgan ilmiy tadqiqotlar tizimlashtirilib, konstruktiv va texnologik omillarning o‘zaro ta’siri qiyosiy tahlil qilindi…