🌡️ Temperature ⭐
Output randomness control করে। 0 = deterministic, 1 = creative, 2 = wild।
| মান | ব্যবহার |
|---|---|
| 0.0 | Math, code, factual extraction |
| 0.3-0.5 | Analysis, summary, structured task |
| 0.7-0.8 | Creative writing, brainstorm |
| 1.0+ | Poetry, wild idea generation |
🎯 Top-p (Nucleus Sampling)
Top probability mass থেকে sample নেয়। 0.9 = top ৯০% সম্ভাব্য token-এর মধ্য থেকে।
top_p=0.1 → খুব focused top_p=0.9 → diverse কিন্তু reasonable
⚠️ Best Practice
Temperature আর top-p একসাথে tweak করবেন না। একটা change করুন, অন্যটা default রাখুন।
🎛️ অন্যান্য Parameter
- max_tokens — output length cap
- stop — যেই sequence-এ থামবে ("\n\n", "###")
- frequency_penalty — repetition কমায় (0-2)
- presence_penalty — নতুন topic encourage
- seed — reproducibility (একই seed + temp 0 = একই output)
🔌 API Example
{
"model": "gpt-4o",
"messages": [...],
"temperature": 0.2,
"max_tokens": 500,
"top_p": 1.0,
"frequency_penalty": 0.5,
"stop": ["\n\n###"]
}
📡 Streaming
Token-by-token output → perceived response দ্রুত।
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0].delta.content || "");
}
📊 Structured Output ⭐
JSON Mode (basic)
{
"response_format": { "type": "json_object" }
}
// Prompt-এ "Respond in JSON" বলতে হবে
JSON Schema (strict) ⭐
{
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "user_extract",
"schema": {
"type": "object",
"properties": {
"name": { "type": "string" },
"age": { "type": "integer" },
"skills": { "type": "array", "items": { "type": "string" } }
},
"required": ["name", "age"]
},
"strict": true
}
}
}
// Guaranteed valid JSON, schema-এ match করবে
🔧 Function / Tool Calling
{
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {
"city": { "type": "string" },
"unit": { "enum": ["C", "F"] }
},
"required": ["city"]
}
}
}]
}
// Model নিজেই সিদ্ধান্ত নেবে কখন function call করবে
✅ Structured Output Best Practice
- JSON Schema > "JSON only" prompt
- Output validate করুন (Zod, Pydantic)
- Invalid হলে retry
- Structured task-এ temperature কম (0-0.3)
- Example output দিন
- Field description clear হওয়া দরকার
🚨 Common Pitfall
- Temperature 0 ≠ hundred percent identical (distributed system-এ সামান্য variation)
- max_tokens কম → উত্তর truncate হয়
- JSON চাইলে schema ছাড়া markdown code block (```json) চলে আসে
- Stop sequence issue (HTML tag, code embedded)
- Tool hallucination — model বানানো function call করতে পারে
💡 Production Tips
- Timeout set করুন (request hang করতে পারে)
- Retry with exponential backoff
- Rate limit handle (429 response)
- Cost log per request
- Token usage monitor
- Failed output শিখতে save করুন