My code base primarily leverages the quanteda R package (reference: https://quanteda.io/).
- Take in a string of words (for example, don’t forget to buy bread and a case of)
- Take last 4 words of the above string of words (or less if less than 4 words are provided)
- Start with 4-gram (or trigram or bigram or unigram, if less than 4 words are provided)
- Check to see if result exists
- If no results, then stupid back off to n-1 gram, n-2 gram until there’s only one word
– If nothing if found, not even for one word, it just return a period (.)