It is common to feel frustrated when you are not able to reach the minimum count of words to complete an essay assignment. You might have included everything that you think is relevant for the topic. But somehow you feel your creative prowess is not gaining momentum to expand on the written essay.
A few students try to rewrite sentences; give more examples to the description to increase the word count. However, these might work quantitively but they don’t necessarily make any changes to the essay qualitatively. Let’s look into the details on essay sentence generation using nlp algorithms.
How textify.ai developed a unique model for essay sentence generation
There are chances that you might run out of ideas while writing an essay. Yet, there are one or two keywords that linger in your mind. It might be hard to compose them into a sentence keeping the essay context.
To solve this , NLP engineers at textify.ai came up with a text-to-text transformer model where the input and the output are text strings. The model was specifically fine-tuned on college essays and experiments were carried out to make sure that the generated output has some kind of context associated with the essay.
The text-to-text transfer T5 model
Google’s state of the art, T5 — Text-to-Text Transfer Transformer Model attempts to combine all the downstream tasks into a text-to-text format.
The text-to-text framework where the inputs are modeled in such a way that the model shall recognize a task, and the output is simply the text version of the expected outcome. It is quite different from the BERT-style models that can only output either a class label or a span of the input. The T5 allows us to use the same model along with the loss function and hyperparameters on any NLP task, namely text summarization, question answering and text generation.
Fine-tuning the model for essay sentence generation
The modelling approach proceeded by casting the data to text in the structured format, one attribute contained the keywords extracted using keybert and another attribute with the corresponding sentences.
The T5 model was then fine-tuned for 10 epochs with a learning rate of 0.00005. Each sentence from an essay was extracted using pandas. The attributes of the dataset include keywords and the corresponding sentences. As in machine translation tasks, we input the model with the text and target text (language to be translated). A similar pattern was followed here as well.
We started to experiment with the base model having just the keywords and the sentences. Although the model was performing well here, it was not taking in any context associated with the sentences. Also, some of the keywords were missing in the generated text. It was also evident that the hierarchy of the keywords given to the model, had a great effect on the final predicted text.
Improvising the model for better prediction
Before feeding the model with keyword sentence pairs, classification of sentences based on a few parameters like compound, complex and simple type was carried out. On average, the length of complex and compound sentences were higher than simple ones. They also had more keywords present in them.
Model Predictions
Here you can have a look at the predicted text by the model based on the provided keywords. While generating the text, we specify the type of sentence.
Putting it All Together
This is an overall walkthrough of the procedure followed to generate essay sentences with a few keywords. The current model is satisfactory but the experimentation to build a better model is ongoing. Here we have parameters like the context of the essay, keywords, type of essay sentence that makes it challenging to build a robust model that suits the requirement of each individual user. Nonetheless, you can try a few keywords and get an essay sentence here!
Check out more AI tools.
Sign up for Textify AI membership.
