Swift · Unicode Segmentation

Unicode Segmentation

6.0.0-rc.1 · 91a27bdd

These methods return byte-accurate [Range<Index>] arrays you can subscript back into the source string.

utf8Words() splits into UAX-29 words that tile the input, so every byte belongs to exactly one word.

for range in "Hello, 世界!".utf8Words() { // tiles the Latin run and the CJK run
    print("Hello, 世界!"[range])
}

utf8Lines(skipEmpty:) splits on the Unicode line-break characters and CRLF, and utf8Tokens(skipEmpty:) splits on the 25 Unicode White_Space characters. Both keep empty segments by default under the cross-language KEEP policy; pass skipEmpty: true to drop them. With N delimiters you get N+1 segments, so "a\n\nb\n".utf8Lines() yields four ranges: "a", "", "b", and "".

let lines = "a\n\nb\n".utf8Lines()                      // 4 ranges, including empties
assert(lines.count == 4)
let words = "  hi  there  ".utf8Tokens(skipEmpty: true) // ["hi", "there"]
assert(words.count == 2)