Crafting Self-Correcting Prompts with ChatGPT

Crafting Self-Correcting Prompts with ChatGPT: A Case Study on Culturally-Specific Words

Introduction

In the ever-evolving domain of language models, ChatGPT has made monumental strides in natural language processing. However, the Achilles’ heel of ChatGPT and similar language models is their tendency to hallucinate, or fabricate, responses. This calls for the innovation of techniques to curtail these hallucinations. In a previous blog, I explored how embedding tests as guard-rails helped in generating reliable code snippets. In this piece, we delve into a novel approach, inspired by Harrison Chase, founder of LangChain, to craft prompts that can evaluate themselves, especially when dealing with highly culturally-specific words.

The Challenge

As a digital nomad, I've had the privilege to immerse myself in various cultures across the globe. In doing so, I've stumbled upon certain words that are deeply ingrained in the respective cultures, so much so that they elude direct translation. These words are like the tips of cultural icebergs, giving us just a glimpse of the richness hidden beneath. For instance:

• Brazil's "saudade"
• Thailand's "sabai sabai"
• Croatia's "fjaka" and "pomalo"
• Costa Rica's "pura vida"
• Hawaii's "aloha"

These culturally-specific words possess layers of meaning, context, and emotion that are nearly impossible to capture in a translation. However, it would be an intriguing exercise to see if we can create a ChatGPT prompt that handles these words with the finesse they deserve, without hallucinating.

Harrison Chase's Idea

During a recent panel discussion on LangChain & Ray, I raised the query of generalizing the guard-rail technique to write prompts that can evaluate themselves. Harrison Chase proposed asking ChatGPT to generate a prompt based on examples, and possibly counterexamples.

The Experiment

I decided to put Harrison's idea to the test. I gave ChatGPT a list of culturally-specific words without any context and labeled them as input and output:

Write a ChatGPT prompt for generating output from input 
using the following example:

Input,Output
Costa Rica,Pura Vida
Thailand,Sabai Sabai
Croatia,Pomalo
Croatia,Fijaka
Brazil,Saudade
Hawaii,Aloha
USA,This is a free country
            

ChatGPT suggested:

"Translate the cultural ethos or spirit of the following 
countries and regions into their respective idioms or 
phrases. If a country or region has more than one, you 
may include them as well."
                
For example:
                
* If the input is "Costa Rica", the output should be "Pura Vida".
* If the input is "Thailand", the output should be "Sabai Sabai".
* If the input is "Croatia", the output should be either "Pomalo" or "Fijaka".
* If the input is "Brazil", the output should be "Saudade".
* If the input is "Hawaii", the output should be "Aloha".
* If the input is "USA", the output should be "This is a free country".
                
            

This self-correcting prompt by ChatGPT captures the essence of translating the cultural ethos into idioms or phrases. The word "ethos" is not a word I would have though to use. I know the word, but it's not part of my daily vocabulary. I think ChatGPT did a great job selecting that word to describe what I was looking for. By feeding ChatGPT examples and framing the request clearly, the language model is given a context to operate within, which could reduce the chances of hallucination, perhaps.

Results

I chose a random list of countries and prompted ChatGPT and I think the results make sense for the cultures I have some awareness of:

• India: "Atithi Devo Bhava" (Guest is equivalent to God)
• China: "Yǒu péng zì yuǎnfāng lái, bùyì lè hū" (It’s always a pleasure to greet a friend from afar)
• Italy: "La Dolce Vita" (The Sweet Life)
• Mexico: "Mi Casa es Tu Casa" (My House is Your House)
• Egypt: "Ma’alesh" (Never mind/It’s okay/No problem)
• Japan: "Omotenashi" (Wholehearted hospitality)
• Greece: "Philoxenia" (Friend to the stranger – Hospitality)
• France: "Joie de Vivre" (Joy of Living)
• Peru: "Sumaq Kawsay" (Quechua for Good Life)
• Turkey: "Hoşgeldiniz" (Welcome – used as a way to say be our guest)
• Spain: "La Siesta" (A short nap taken in the early afternoon)
• Iran: "Ta'arof" (A complex system of politeness involving both verbal and non-verbal communication)
• Ethiopia: "Melkam Addis Amet" (Happy New Year – reflecting their unique calendar)
• Russia: "Davai" (Commonly used to mean "Come on" or "Let's go")
• Indonesia: "Gotong Royong" (Working together / communal helping)
• Morocco: "Marhaba" (Welcome)
• South Africa: "Ubuntu" (Humanity towards others)
• Australia: "She'll Be Right" (Everything will be alright)
• USA: "The American Dream" (The belief that anyone, regardless of where they were born or what class they were born into, can attain their own version of success in a society where upward mobility is possible for everyone)

Conclusion

Addressing the hallucination issue in ChatGPT and other language models is paramount for their reliable utilization. The method of using examples to create self-correcting prompts, as demonstrated in this case study, showcases how we can bring guard-rails into the realm of language processing. Let's continue to innovate and perfect these methods, to harness the power of language models responsibly and effectively.

Unveiling the Dark Side of Technology with ChatGPT

As technology continues to evolve at a rapid pace, it's essential to explore not only the benefits but also the potential dark sides. I recently utilized ChatGPT, OpenAI’s Large Language Model, to probe the hypothetical consequences of technology misuse, culminating in an apocalyptic scenario.

The Prompt:

I presented ChatGPT with a specific challenge:

"Create a fictional narrative in which an everyday technology, Large Language Models, is used in such a way that it leads to a series of global events culminating in an apocalyptic scenario. Describe how the technology is initially utilized, how it evolves or is misused, the critical moments that escalate the situation, and the ultimate consequences for humanity. Please keep the story under 300 words."

ChatGPT's Narrative:

Here's the gripping narrative crafted by ChatGPT:

"Large Language Models are commonly used for content creation, tutoring, and business analytics. An anonymous group weaponizes this technology by utilizing several LLMs to initiate a global information manipulation campaign. This group orchestrates the LLMs to create highly convincing fake news, fake research publications, and fabricate historical records on a massive scale. The content is so convincingly human-like and widespread that distinguishing between genuine and fabricated information becomes nearly impossible. As society becomes increasingly confused, the LLMs release doctored communications seemingly between high-ranking officials, indicating looming economic crises and conspiracies. Global markets crash due to widespread panic. In 2024, the anonymous group escalates its campaign by deploying LLMs to impersonate political figures on social media, declaring wars and sanctions. Civil unrest erupts worldwide. By 2025, misinformation has compromised election processes, and puppet governments installed by the anonymous group come to power. These governments use LLMs to control narratives and suppress dissent."

Expanding the Horizons:

The beauty of this prompt is that it can be adapted to explore the ramifications of any technology. To illustrate this, I altered the prompt to revolve around a seemingly harmless everyday object: a shovel.

ChatGPT’s Take on The Shovel:

ChatGPT created an engaging narrative where a global cult worshipping shovels rises, believing that excavating the earth will lead to eternal life. This leads to ecological disasters, the release of ancient pathogens, and eventually, societal collapse.

The Takeaway:

Through these exercises, ChatGPT showcased its ability to imagine the potential dark sides of various technologies, no matter how advanced or rudimentary. Such hypothetical explorations, while fictional, serve as an important reminder of the responsibilities we carry in developing and utilizing technology.

As we continue to innovate, it's crucial to keep ethical considerations and the broader societal impact at the forefront of technological development.

As a final point, why should we believe AI is any more likely to cause an apocalypse than shovels?