Text segmentation
From Wikipedia, the free encyclopedia
Written text segmentation is the process of dividing written text into words or other similar meaningful units. The term applies to mental processes used by humans when reading text, and to artificial processes implemented in computers, which are the subject of natural language processing.
The problem is relatively trivial for written languages that have explicit word boundary markers, such as the word spaces of written English or the distinctive initial, medial and final letter shapes of Arabic. When such clues are not consistently available, the task often requires fairly non-trivial techniques, such as statistical decision-making, large dictionaries, as well as consideration of syntactic and semantic constraints.
[edit] See also
- Speech segmentation
- Hyphenation
- Japanese text processing
 |
This writing system-related article is a stub. You can help Wikipedia by expanding it. |
