Python NLTK nltk.tokenize.ConditionalFreqDist() Function

Python / By

NLTK in Python:

NLTK is a Python toolkit for working with natural language processing (NLP). It provides us with a large number of test datasets for various text processing libraries. NLTK can be used to perform a variety of tasks such as tokenizing, parse tree visualization, and so on.

Tokenization

Tokenization is the process of dividing a large amount of text into smaller pieces known as tokens. These tokens are extremely valuable for detecting patterns and are regarded as the first stage in stemming and lemmatization. Tokenization also aids in the replacement of sensitive data elements with non-sensitive data elements.

Natural language processing is utilized in the development of applications such as text classification, intelligent chatbots, sentiment analysis, language translation, and so on. To attain the above target, it is essential to consider the pattern in the text.

Natural Language Toolkit features an important module called NLTK tokenize sentences, which is further divided into sub-modules.

  • word tokenize
  • sentence tokenize

nltk.tokenize.ConditionalFreqDist() Function:

Using nltk.tokenize.ConditionalFreqDist() function, we can count the frequency of words in a sentence.

Syntax:

tokenize.ConditionalFreqDist()

Parameters: This method doesn’t accept any parameters

Return Value:

The frequency of words in a sentence as a dictionary is returned by the ConditionalFreqDist() function

NLTK nltk.tokenize.ConditionalFreqDist() Function in Python

Method #1: Using ConditionalFreqDist() Function(Static Input)

Approach:

  • Import ConditionalFreqDist() function from the probability of nltk module using the import keyword
  • Import word_tokenize from tokenize of nltk module using the import keyword
  • Creating a reference/Instance variable(Object) for the ConditionalFreqDist Class
  • Give the string as static input and store it in a variable.
  • Pass the above-given string to the word_tokenize() function and loop in each word of it using the for loop
  • Here the word_tokenize function splits the given string into tokens i.e words
  • Get the length of the word using the len() function and store it in a variable
  • Increment the above condition(length of word) and word count by 1
  • Here index of the dictionary is the above condition(Length of the word)
  • Printing all the Conditional Freq Dictionary values.
  • The Exit of the Program.

Below is the implementation:

# Import ConditionalFreqDist() function from probability of nltk module using the import keyword
from nltk.probability import ConditionalFreqDist
# Import word_tokenize from tokenize of nltk module using the import keyword
from nltk.tokenize import word_tokenize
    
# Creating a reference/Instance variable(Object) for the ConditionalFreqDist Class
tkn = ConditionalFreqDist()
    
# Give the string as static input and store it in a variable.
gvn_str = "Python Programs Sample Codes in Python Codes"
# Pass the above given string to the word_tokenize() function and loop in each word of it using the for loop
# (Here the word_tokenize function splits the given string into tokens i.e words)
for wrd in word_tokenize(gvn_str):
  # Get the length of the word using the len() function and store it in a variable
  condition = len(wrd)
  # Increment the above condition(length of word) and word count by 1
  # Here index of dictionary is the above condition(Length of word)
  tkn[condition][wrd] += 1
  
# Printing all the Conditional Freq Dictionary values  
tkn

Output:

ConditionalFreqDist(nltk.probability.FreqDist,
{2: FreqDist({'in': 1}),
5: FreqDist({'Codes': 2}),
6: FreqDist({'Python': 2, 'Sample': 1}),
8: FreqDist({'Programs': 1})})
# Getting the Frequency dictionary of words which are having length =6
tkn[6]

Output:

FreqDist({'Python': 2, 'Sample': 1})

Method #2: Using ConditionalFreqDist() Function (User Input)

Approach:

  • Import ConditionalFreqDist() function from the probability of nltk module using the import keyword
  • Import word_tokenize from tokenize of nltk module using the import keyword
  • Creating a reference/Instance variable(Object) for the ConditionalFreqDist Class
  • Give the string as user input using the input() function and store it in a variable.
  • Pass the above-given string to the word_tokenize() function and loop in each word of it using the for loop
  • Here the word_tokenize function splits the given string into tokens i.e words
  • Get the length of the word using the len() function and store it in a variable
  • Increment the above condition(length of word) and word count by 1
  • Here index of the dictionary is the above condition(Length of the word)
  • Printing all the Conditional Freq Dictionary values.
  • The Exit of the Program.

Below is the implementation:

# Import ConditionalFreqDist() function from probability of nltk module using the import keyword
from nltk.probability import ConditionalFreqDist
# Import word_tokenize from tokenize of nltk module using the import keyword
from nltk.tokenize import word_tokenize
    
# Creating a reference/Instance variable(Object) for the ConditionalFreqDist Class
tkn = ConditionalFreqDist()
    
# Give the string as user input using the input() function and store it in a variable.
gvn_str = input("Enter some random string = ")
# Pass the above given string to the word_tokenize() function and loop in each word of it using the for loop
# (Here the word_tokenize function splits the given string into tokens i.e words)
for wrd in word_tokenize(gvn_str):
  # Get the length of the word using the len() function and store it in a variable
  condition = len(wrd)
  # Increment the above condition(length of word) and word count by 1
  # Here index of dictionary is the above condition(Length of word)
  tkn[condition][wrd] += 1

# Printing all the Conditional Freq Dictionary values  
tkn

Output:

Enter some random string = good morning all good good morning hello hi hi
ConditionalFreqDist(nltk.probability.FreqDist,
{2: FreqDist({'hi': 2}),
3: FreqDist({'all': 1}),
4: FreqDist({'good': 3}),
5: FreqDist({'hello': 1}),
7: FreqDist({'morning': 2})})