Generating passages similar in style to a set of 9 examples (Question)
Hello everyone
I hope I can find some guidance here for a project in generative AI.
I have a set of 9 short passages from a TOEFL-like English test. I need to generate more passages that match the style of the examples set. The passages are 50 – 100 words, and are cut at the end in the middle of a sentence, and the examinees’ task is to choose the correct answer that completes the text correctly, out of 4 options.
Here’s what I considered:
- Ask ChatGPT to generate a similar passage using few-shot prompting.
- Build a scoring / distance method to measure the distance between the generated passage and the examples set.
- Ask ChatGPT to alter the passage until I’m satisfied with the score.
Some questions:
1. For the scoring method, I’m considering TFIDF of POS (part of speech) and function words. Is that a good idea? Any other suggestions? I did consider embeddings, but wouldn’t that lead to passages similar in content rather than in style? 2. How would you generate 3 wrong answers that also fit the style of the wrong answers in the examples? I thought I’d cluster the examples’ wrong answers into 3 categories using k-means, figure out what distinguishes each class from the others, and ask ChatGPT to generate one wrong answer from each category (e.g. bad grammar / contradictory information / etc.). 3. Any other approaches that you’d suggest? Could i build a generative model that takes in an article (e.g. Wikipedia article) and modifies it so the format and style matches the examples’, or is the examples set too small for that?
submitted by /u/RunOrDieTrying
[link] [comments]
/u/RunOrDieTrying
Go to original source