Inherits from NSObject
Declared in RKStringTokenizer.h

Overview

The RKStringTokenizer class provides an interface for tokenizing input text into a set of searchable words. Diacritics are removed and the input text is tokenized case insensitively. A set of stop words can be optionally trimmed from the result token set.

Tasks

Configuring Tokenization

  •   stopWords

    The set of stop words that are to be removed from the token set.

    property

Tokenizing a String of Text

  • – tokenize:

    Tokenizes the given string by folding it case and diacritic insensitively and then splitting it apart using the the word unit delimiters for the current locale. If a set of stop words has been provided, the resulting token set will have the stop words subtracted.

Properties

stopWords

@property (nonatomic, strong) NSSet *stopWords
Discussion

The set of stop words that are to be removed from the token set.

Defaults to nil.

Declared In

RKStringTokenizer.h

Instance Methods

tokenize:

- (NSSet *)tokenize:(NSString *)string
Discussion

Tokenizes the given string by folding it case and diacritic insensitively and then splitting it apart using the the word unit delimiters for the current locale. If a set of stop words has been provided, the resulting token set will have the stop words subtracted.

Parameters

string

A string of text you wish to tokenize.

Return Value

A set of searchable text tokens extracted from the given string.

Declared In

RKStringTokenizer.h