3 Commits

Author SHA1 Message Date
Kevin Sawicki
3d1ca5abb1 Add specs for partial surrogate pairs 2013-06-04 13:43:19 -07:00
Kevin Sawicki
f63866b2a2 Rename hasSurrogatePairs() to hasSurrogatePair() 2013-06-04 13:43:19 -07:00
Kevin Sawicki
a109a3317e Add support for surrogate pairs
Surrogate pairs, meaning characters outside the Basic
Multilingual Plane, are now broken out as atomic tokens.

Closes #567
2013-06-04 13:43:19 -07:00