Open Source LLM Prompting

Introduction

Open-source LLMs (Llama, Mistral, Qwen, Phi) let you self-host, fine-tune and control everything. Prompting style varies more than with hosted models.

Beginner analogy: Like running your own server vs cloud — more control, more work.

Understanding the topic

Core concepts to understand:

• Each family has its own chat template (Llama-3, ChatML, etc.).
• Smaller models need more explicit examples.
• Use Ollama, vLLM, TGI to host.
• Match prompt format to training format (or accuracy drops).
• Great for privacy & cost at scale.

Syntax reference

Visual workflow / architecture:

Llama-3 chat template:
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
You are helpful.
<|eot_id|><|start_header_id|>user<|end_header_id|>
Hello!
<|eot_id|>

Real-world use

Used by enterprises with strict data residency, and any team running high-volume inference where API cost is prohibitive.

Best practices

• Match the official chat template exactly.
• Add more examples — smaller models need more guidance.
• Quantise (Q4/Q8) for cheaper inference.

Hands-on exercise

Interview preparation — practice these questions:

• Q1. Pros & cons of open-source LLMs?
• Q2. What is a chat template?
• Q3. Hosting options?
• Q4. Why prompt format matters more here?