Prompt Engineering

🎓 Complete all tutorials to earn your Free LLMs & Transformers Certificate
Shareable on LinkedIn • Verified by AITutorials.site • No signup fee

🎯 What is Prompt Engineering?

LLMs are incredibly powerful, but they're surprisingly sensitive to how you ask questions. The same query phrased slightly differently can yield drastically different results—from brilliant insights to complete nonsense.

Prompt engineering is the practice of crafting, refining, and optimizing inputs (prompts) to get better, more reliable, and more consistent outputs from large language models. It's both an art (creativity, intuition) and a science (systematic testing, measurement).

The Economics of Prompting

Consider the options for improving LLM performance:

• Fine-tuning: $10,000-$100,000+ in compute, weeks of work, requires ML expertise
• Prompt engineering: $0 in compute, hours to days of work, anyone can learn it

For most applications, prompt engineering should be your first approach. Only fine-tune when prompting reaches its limits.

A Simple Example of Prompt Impact

❌ Vague Prompt:
"Tell me about machine learning"

Output: Generic 3-paragraph overview, not very useful

✅ Engineered Prompt:
"Explain machine learning to a software engineer with 5 years of experience. Focus on practical applications and compare supervised vs unsupervised learning with code examples in Python. Keep it under 300 words."

Output: Targeted, technical explanation with concrete examples—exactly what's needed!

The Evolution of Prompting

Prompt engineering has evolved rapidly:

• 2018-2019 (GPT-2): Basic prompts, mostly text completion
• 2020 (GPT-3): Few-shot learning discovered, prompts become powerful
• 2022 (ChatGPT): Instruction-following, conversational prompts
• 2023-Present: Advanced techniques (CoT, ReAct, Tree-of-Thought), prompt optimization frameworks

🔧 Core Prompt Engineering Techniques

1. Zero-Shot Prompting

Ask the model to perform a task without any examples. Modern large models (GPT-4, Claude, Llama 2 70B) are surprisingly capable at zero-shot tasks.

Task: Sentiment Classification
Prompt: "Classify the sentiment of this review as positive, negative, or neutral: 'This product exceeded my expectations!'"
Output: "Positive"

---

Task: Translation
Prompt: "Translate to French: 'Hello, how are you?'"
Output: "Bonjour, comment allez-vous?"

---

Task: Code Generation
Prompt: "Write a Python function to calculate factorial recursively"
Output: [complete working code]

When to use zero-shot:

• Simple, well-known tasks (translation, basic classification)
• Using large, instruction-tuned models (GPT-4, ChatGPT)
• When you don't have examples readily available
• For quick prototyping and exploration

Limitations:

• Less reliable for domain-specific or complex tasks
• Output format may vary without examples
• Smaller models struggle with zero-shot

2. One-Shot Prompting

Provide a single example to demonstrate the task and desired format.

Prompt:
"Extract company name and ticker symbol.

Text: 'Apple Inc. reported strong earnings.'
Extraction: Company: Apple Inc., Ticker: AAPL

Text: 'Microsoft announced a new AI product.'
Extraction:"

3. Few-Shot Prompting (In-Context Learning)

Provide multiple examples (typically 2-5) to teach the model the task pattern. This is one of the most powerful techniques—discovered with GPT-3 in 2020.

Prompt:
"Classify product reviews by urgency (urgent, moderate, low):

Review: 'Product broken, need refund ASAP!'
Urgency: urgent

Review: 'Great product, works well'
Urgency: low

Review: 'Missing a part, can you send it?'
Urgency: moderate

Review: 'Love it but instructions unclear'
Urgency: low

Review: 'Received wrong item, need correct one immediately'
Urgency:"

Few-shot best practices:

• 2-5 examples: More isn't always better (diminishing returns)
• Diverse examples: Cover different cases, edge cases
• Consistent format: Keep input/output structure identical
• Order matters: Later examples have more influence
• Quality over quantity: 3 good examples beat 10 mediocre ones

4. Chain-of-Thought (CoT) Prompting ⭐

Ask the model to reason step-by-step before giving the final answer. This dramatically improves performance on complex reasoning tasks.

Prompt: "If John has 10 apples and gives 3 to Mary, then buys 5 more, how many does he have?"
Output: "12"  ❌ (often wrong on multi-step problems)

Prompt: "If John has 10 apples and gives 3 to Mary, then buys 5 more, how many does he have? Let's think step by step."

Output: "Let's solve this step by step:
1. John starts with 10 apples
2. He gives 3 to Mary: 10 - 3 = 7 apples
3. He buys 5 more: 7 + 5 = 12 apples
Final answer: John has 12 apples."  ✅

When to use CoT:

• Math word problems
• Multi-step reasoning
• Logical deduction
• Commonsense reasoning
• Planning and problem-solving

Zero-Shot CoT (Magic Phrase)

Simply adding "Let's think step by step" or "Let's solve this step by step" often triggers reasoning!

Few-Shot CoT

Show examples with reasoning chains included:

Q: Roger has 5 tennis balls. He buys 2 more cans with 3 balls each. How many balls does he have?
A: Roger starts with 5 balls. 2 cans × 3 balls = 6 balls. 5 + 6 = 11 balls. Answer: 11

Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many do they have?
A: Cafeteria starts with 23. They use 20: 23 - 20 = 3. They buy 6: 3 + 6 = 9 apples. Answer: 9

Q: [Your actual question]

5. Role Prompting

Frame the model as an expert in a specific domain to get more targeted, authoritative responses.

Generic: "Explain neural networks"
Output: Basic, generic explanation

Role-based: "You are a machine learning professor teaching a graduate-level course. Explain neural networks to PhD students, focusing on backpropagation mathematics and optimization challenges."
Output: Technical, sophisticated explanation with mathematical depth

Effective roles:

• "You are an expert software engineer..."
• "You are a helpful teaching assistant..."
• "You are a professional copywriter specializing in..."
• "Act as a Python debugging expert..."
• "You are a business analyst with 10 years experience..."

6. Instruction Prompting

Be explicit about what you want, how you want it, and any constraints.

Task: [What to do]
Context: [Background information]
Format: [How to structure output]
Constraints: [Limitations, things to avoid]
Tone: [Formal, casual, technical, etc.]

Example:
Task: Write a product description
Context: For a noise-canceling headphone targeting commuters
Format: 3 bullet points followed by a call-to-action
Constraints: Under 100 words, avoid technical jargon
Tone: Friendly and persuasive

7. Structured Output Prompting

Request specific output formats for easy parsing and integration.

Prompt: "Extract information from this text and return as JSON:
{
  'name': string,
  'company': string,
  'email': string,
  'phone': string
}

Text: 'Contact John Smith at Apple Inc., email john@apple.com or call 555-0123'"

Output:
{
  "name": "John Smith",
  "company": "Apple Inc.",
  "email": "john@apple.com",
  "phone": "555-0123"
}

Comparison of Techniques

Parameter Tuning

💡 Advanced Prompt Engineering Techniques

1. Self-Consistency

Generate multiple responses to the same prompt and select the most common answer. This reduces hallucinations and improves reliability on reasoning tasks.

import openai
from collections import Counter

prompt = "If a train travels 120 km in 2 hours, what's its average speed? Think step by step."

# Generate 5 responses
responses = []
for i in range(5):
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.7  # Some randomness for diversity
    )
    answer = extract_answer(response.choices[0].message.content)
    responses.append(answer)

# Take majority vote
final_answer = Counter(responses).most_common(1)[0][0]
print(f"Responses: {responses}")
print(f"Final answer: {final_answer}")

# Output: Responses: ['60 km/h', '60 km/h', '60 km/h', '60 km/h', '60 km/h']
#         Final answer: 60 km/h

When to use: Critical decisions, math problems, complex reasoning where accuracy matters more than cost.

2. Tree-of-Thought (ToT)

Explore multiple reasoning paths like a search tree, evaluate intermediate steps, and select the best path. This is like beam search for reasoning.

Problem: Plan a 3-day trip to Paris with $500 budget

Step 1: Generate multiple high-level plans
  Path A: Focus on museums (Louvre, Orsay, etc.)
  Path B: Focus on landmarks (Eiffel, Notre-Dame, etc.)
  Path C: Mix of culture + food experiences

Step 2: For each path, generate detailed itineraries
  Path A Day 1: [detailed plan]
  Path A Day 2: [detailed plan]
  ...

Step 3: Evaluate each complete plan
  Path A: Budget check, feasibility, user preferences
  Path B: Budget check, feasibility, user preferences
  Path C: Budget check, feasibility, user preferences

Step 4: Select best path based on evaluation
  Winner: Path C (balanced, within budget, diverse experiences)

Use cases: Planning, strategic decision-making, complex problem-solving with multiple constraints.

3. ReAct (Reasoning + Acting)

Interleave reasoning with actions (tool use, searches, API calls). The model thinks, acts, observes results, and thinks again.

Question: What's the current population of Tokyo?

Thought 1: I need to find current data. My training cutoff is 2023, so I should search.
Action 1: Search["Tokyo population 2024"]
Observation 1: Tokyo metropolitan area has approximately 37.4 million people as of 2024

Thought 2: I found the answer. Let me verify this is the metropolitan area, not just Tokyo proper.
Action 2: Search["Tokyo city proper population"]
Observation 2: Tokyo city proper (23 wards) has about 14 million people

Thought 3: The question likely refers to the metro area, which is more commonly cited.
Answer: Tokyo's metropolitan area has approximately 37.4 million people as of 2024.

Implementation requires: Tool integration (search APIs, calculators, databases) and orchestration framework (LangChain, AutoGPT).

4. Prompt Chaining

Break complex tasks into sequential, simpler steps. Each step's output becomes the next step's input.

# Step 1: Extract key information
prompt_1 = f"Extract main points from this article: {article_text}"
summary = llm(prompt_1)

# Step 2: Generate questions
prompt_2 = f"Generate 3 questions answered by this summary: {summary}"
questions = llm(prompt_2)

# Step 3: Create social media posts
prompt_3 = f\"\"\"Create 3 tweet-length posts promoting this content.
Summary: {summary}
Questions: {questions}
Format: Engaging, under 280 chars each\"\"\"
tweets = llm(prompt_3)

# Step 4: Generate hashtags
prompt_4 = f"Generate 5 relevant hashtags for: {tweets}"
hashtags = llm(prompt_4)

Benefits:

• Simpler prompts for each step (easier to debug)
• Can use different models for different steps (cost optimization)
• Intermediate outputs can be cached/reused
• Better error handling and retry logic

5. Retrieval Augmented Generation (RAG)

Ground LLM responses in external knowledge by retrieving relevant documents and including them in the prompt. This reduces hallucinations and enables up-to-date information.

1. User Query: "What's our company's return policy for electronics?"

2. Embedding: Convert query to vector using embedding model

3. Retrieval: Search vector database for similar documents
   Retrieved: ["Electronics Return Policy v2.3", "Warranty Terms", ...]

4. Augmented Prompt:
   \"\"\"Use the following context to answer the question.
   
   Context:
   [Document 1: Electronics Return Policy v2.3]
   Electronics can be returned within 30 days with receipt...
   
   [Document 2: Warranty Terms]
   Extended warranty available for purchase...
   
   Question: What's our company's return policy for electronics?
   
   Answer based only on the context provided. If the context doesn't contain 
   the answer, say "I don't have that information."
   \"\"\"

5. LLM Generation: Model generates answer grounded in retrieved docs

Code Example:

from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import FAISS
from langchain.llms import OpenAI
from langchain.chains import RetrievalQA

# 1. Create vector store from documents
embeddings = OpenAIEmbeddings()
docs = load_documents()  # Your knowledge base
vectorstore = FAISS.from_documents(docs, embeddings)

# 2. Create RAG chain
qa_chain = RetrievalQA.from_chain_type(
    llm=OpenAI(temperature=0),
    chain_type="stuff",
    retriever=vectorstore.as_retriever(search_kwargs={"k": 3})
)

# 3. Query with automatic retrieval
query = "What's the return policy for electronics?"
answer = qa_chain.run(query)
print(answer)

6. Prompt Decomposition

Break complex questions into sub-questions, answer each, then synthesize.

Complex Question: "Compare the economies of Japan and Germany"

Decomposition:
1. What's Japan's GDP, growth rate, and key industries?
2. What's Germany's GDP, growth rate, and key industries?
3. What are the major economic challenges each faces?
4. How do their trade relationships differ?

[Answer each sub-question]

Synthesis:
Combine all answers into comprehensive comparison

7. Constrained Generation

Force the model to follow specific formats, constraints, or rules.

Prompt: \"Write a haiku about AI. 
Strict requirements:
- Line 1: exactly 5 syllables
- Line 2: exactly 7 syllables  
- Line 3: exactly 5 syllables
- Theme: artificial intelligence
- Tone: contemplative

Count syllables before submitting your answer.\"

Output:
Thinking machines learn (5)
From vast oceans of data (7)
Mirror of our minds (5)

8. Negative Prompting

Tell the model what NOT to do, avoiding common failure modes.

Prompt: \"Explain quantum computing to a 10-year-old.

DO NOT:
- Use technical jargon (qubits, superposition, entanglement)
- Give mathematical formulas
- Make it longer than 100 words
- Use analogies involving cats in boxes

DO:
- Use simple everyday examples
- Keep it fun and engaging
- Focus on what it can do, not how it works\"

9. Iterative Refinement

Generate draft, critique it, refine it. Repeat until quality is acceptable.

Iteration 1: \"Write a product description for wireless earbuds\"
Output 1: [Generic description]

Iteration 2: \"Improve this description. Make it more compelling, add specific features (battery life, noise cancellation), and include a clear benefit statement: [Output 1]\"
Output 2: [Better description]

Iteration 3: \"Refine further. Add social proof element and create urgency: [Output 2]\"
Output 3: [Polished description]

10. Meta-Prompting (Constitutional AI)

Have the model critique and improve its own outputs based on principles.

Step 1 - Generate:
\"Write a news article about climate change\"
[Article generated]

Step 2 - Critique:
\"Critique this article for:
1. Bias or one-sided viewpoints
2. Missing important perspectives
3. Factual accuracy concerns
4. Emotional manipulation
[Article text]\"

Step 3 - Revise:
\"Revise the article addressing these critiques: [critique]
Original: [article]\"

🎓 Best Practices & Guidelines

The 5 C's of Effective Prompting

1. Clear

Be explicit about what you want. Ambiguity leads to unpredictable outputs.

❌ Vague: "Write code"
✅ Clear: "Write a Python function that reads a CSV file, calculates the average of the 'price' column, and returns the result as a float"

2. Contextual

Provide relevant background information.

❌ No context: "How do I fix this?"
✅ With context: "I'm a Python developer working with pandas. My DataFrame has missing values in the 'age' column. How do I fill them with the median age?"

3. Constrained

Set boundaries on length, format, style, and scope.

✅ "Explain neural networks in exactly 3 sentences, suitable for a software engineer, avoiding mathematical notation"

4. Complete

Include all necessary information in the prompt.

✅ "Translate this product description to Spanish. Target audience: Mexican consumers. Tone: Friendly and professional. Length: match original."

5. Consistent

Maintain consistent formatting and terminology across examples.

Prompt Engineering Checklist

• ✅ Clarity: Is the task unambiguous?
• ✅ Format: Did you specify output format (JSON, list, paragraph)?
• ✅ Length: Did you constrain output length?
• ✅ Examples: For complex tasks, did you provide 2-5 examples?
• ✅ Tone/Style: Did you specify desired tone (formal, casual, technical)?
• ✅ Edge Cases: Did you address potential ambiguities?
• ✅ Success Criteria: Is it clear what "good" output looks like?
• ✅ Reasoning: For complex tasks, did you ask for step-by-step thinking?

Common Mistakes to Avoid

Prompt Templates for Common Tasks

Text Summarization

\"\"\"Summarize the following text in [X] sentences/words.
Focus on [key aspects].
Target audience: [audience].
Tone: [tone].

Text:
[content]

Summary:\"\"\"

Code Generation

\"\"\"Write a [language] function that:
- [requirement 1]
- [requirement 2]
- [requirement 3]

Include:
- Docstring explaining the function
- Type hints (if applicable)
- Error handling for edge cases
- Example usage

Function name: [name]\"\"\"

Data Extraction

\"\"\"Extract the following information from the text below.
Return as JSON with these exact keys: [list keys]
If information is missing, use null.

Text:
[content]

JSON:\"\"\"

Creative Writing

\"\"\"Write a [type] about [topic].
Length: [X] words
Tone: [tone]
Target audience: [audience]
Include: [specific elements]
Avoid: [things to exclude]

[Optional: Opening line to continue from]\"\"\"

Optimizing for Cost and Speed

• Use cheaper models for simple tasks: GPT-3.5 for classification, GPT-4 for complex reasoning
• Constrain max_tokens: Don't let model ramble, saves money
• Cache frequent prompts: Store and reuse common responses
• Batch similar requests: Process multiple items in one prompt when possible
• Use stop sequences: Terminate generation early when done

# Instead of 100 separate API calls:
for review in reviews:
    sentiment = llm(f"Classify sentiment: {review}")  # $$$

# Batch process in one call:
prompt = \"\"\"Classify sentiment for each review (positive/negative/neutral):

1. "Great product!" → 
2. "Terrible service" → 
3. "It's okay" → 
...
100. "Amazing!" → 
\"\"\"
results = llm(prompt)  # $$ (much cheaper!)

Safety and Ethics

Prompt Versioning & Management

For production applications, treat prompts like code:

• Version control: Store prompts in git with clear versioning
• A/B testing: Compare prompt variants on real data
• Performance tracking: Monitor accuracy, latency, cost per prompt
• Documentation: Document why specific phrasing was chosen
• Rollback capability: Keep previous versions when updating prompts

# Example prompt management
PROMPTS = {
    "sentiment_v1": "Classify sentiment: {text}",
    "sentiment_v2": "Analyze the sentiment (positive/negative/neutral): {text}",
    "sentiment_v3": \"\"\"You are a sentiment analysis expert.
                      Classify this review's sentiment.
                      Review: {text}
                      Sentiment:\"\"\"
}

# Track performance
METRICS = {
    "sentiment_v1": {"accuracy": 0.82, "avg_cost": 0.0003},
    "sentiment_v2": {"accuracy": 0.87, "avg_cost": 0.0004},
    "sentiment_v3": {"accuracy": 0.91, "avg_cost": 0.0006}
}

# Use best prompt for production
current_prompt = "sentiment_v2"  # Best balance of accuracy and cost

📊 Prompt Engineering at Scale

Building a Prompt Engineering Pipeline

For production systems, systematic prompt development is crucial:

1. Define Evaluation Metrics

from sklearn.metrics import accuracy_score, f1_score

# Metrics depend on task type
METRICS = {
    'classification': ['accuracy', 'f1', 'precision', 'recall'],
    'generation': ['bleu', 'rouge', 'human_eval'],
    'extraction': ['exact_match', 'f1'],
    'summarization': ['rouge', 'coherence', 'faithfulness']
}

2. Create Test Dataset

# Create diverse test set with ground truth
test_data = [
    {
        'input': 'This product is amazing! Love it!',
        'expected': 'positive',
        'category': 'clear_positive'
    },
    {
        'input': 'Not bad, but could be better',
        'expected': 'neutral',
        'category': 'ambiguous'
    },
    # Include edge cases, ambiguous examples, etc.
]

3. Test Multiple Prompt Variants

import openai
import pandas as pd

prompt_variants = {
    'v1_simple': \"Classify sentiment: {text}\",
    'v2_explicit': \"Classify this review as positive, negative, or neutral: {text}\",
    'v3_role': \"\"\"You are a sentiment analysis expert. 
                   Classify sentiment (positive/negative/neutral): {text}
                   Sentiment:\"\"\",
    'v4_cot': \"\"\"Classify sentiment with reasoning:
                 Review: {text}
                 Analysis: [Think about tone, keywords, context]
                 Sentiment:\"\"\",
    'v5_fewshot': \"\"\"Classify sentiment:
                     
                     'Great product!' → positive
                     'Terrible!' → negative
                     'It's okay' → neutral
                     
                     '{text}' → \"\"\"
}

results = []
for variant_name, template in prompt_variants.items():
    correct = 0
    total_cost = 0
    latencies = []
    
    for test_case in test_data:
        prompt = template.format(text=test_case['input'])
        
        # Measure latency
        import time
        start = time.time()
        response = openai.ChatCompletion.create(
            model=\"gpt-3.5-turbo\",
            messages=[{\"role\": \"user\", \"content\": prompt}],
            temperature=0
        )
        latency = time.time() - start
        
        # Extract answer and evaluate
        answer = response.choices[0].message.content.strip().lower()
        correct += (answer == test_case['expected'])
        
        # Calculate cost (example: $0.002 per 1K tokens)
        tokens_used = response.usage.total_tokens
        total_cost += (tokens_used / 1000) * 0.002
        latencies.append(latency)
    
    results.append({
        'variant': variant_name,
        'accuracy': correct / len(test_data),
        'avg_cost': total_cost / len(test_data),
        'avg_latency': sum(latencies) / len(latencies),
        'total_cost': total_cost
    })

# Analyze results
df = pd.DataFrame(results)
df = df.sort_values('accuracy', ascending=False)
print(df)

variant        accuracy  avg_cost  avg_latency  total_cost
v5_fewshot        0.94     0.0006       1.2s        0.06
v4_cot            0.91     0.0008       1.5s        0.08
v3_role           0.87     0.0004       0.9s        0.04
v2_explicit       0.83     0.0003       0.8s        0.03
v1_simple         0.78     0.0002       0.7s        0.02

Decision: Use v5_fewshot for production (best accuracy)
Alternative: v3_role if cost is primary concern (good balance)

4. Continuous Monitoring

# Production monitoring
class PromptMonitor:
    def __init__(self):
        self.metrics = {
            'success_rate': [],
            'latency': [],
            'cost': [],
            'user_feedback': []
        }
    
    def log_call(self, prompt_version, latency, cost, success):
        # Log all metrics
        self.metrics['latency'].append(latency)
        self.metrics['cost'].append(cost)
        self.metrics['success_rate'].append(1 if success else 0)
        
        # Alert if performance degrades
        if self.get_recent_success_rate() < 0.85:
            self.alert_team(\"Prompt performance degraded!\")
    
    def get_recent_success_rate(self, window=100):
        recent = self.metrics['success_rate'][-window:]
        return sum(recent) / len(recent) if recent else 0

monitor = PromptMonitor()

# In production
response = llm(prompt)
monitor.log_call(\"v5_fewshot\", latency, cost, success)

Tools & Frameworks

📚 Summary

Key Concepts Mastered

Technique Comparison

What You Can Build Now

• ✅ Customer support chatbots with consistent, helpful responses
• ✅ Content generation systems with quality control
• ✅ Data extraction pipelines with structured output
• ✅ Research assistants using RAG for up-to-date information
• ✅ Code generation tools with iterative refinement
• ✅ Multi-agent systems with prompt chaining
• ✅ Production-ready LLM applications with monitoring

Next Steps

Now that you've mastered prompt engineering, you're ready to dive deeper into:

• In-Context Learning & RAG: Build sophisticated retrieval systems
• Fine-tuning: Customize models for specific domains
• LLM Applications: Deploy production-ready AI systems