6 LLM Engineering Patterns You Can Actually Use in Production

Most "AI integration" tutorials show you how to call the OpenAI API and print the response. That is fine for learning, but it is not production. Real applications need streaming, memory, structured data, cost control, and explainability.

I built all six patterns into a live banking application. Here is exactly what I did and why each one matters.

// Backend — Express SSE endpoint
app.post('/api/chat-stream', async (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.flushHeaders();

  const stream = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    stream: true,
    messages: req.body.messages,
  });

  for await (const chunk of stream) {
    const text = chunk.choices[0]?.delta?.content || '';
    if (text) res.write(`data: ${JSON.stringify({ text })}\n\n`);
  }
  res.write('data: [DONE]\n\n');
  res.end();
});

// Frontend — read the stream
const reader = response.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  for (const line of decoder.decode(value).split('\n')) {
    if (!line.startsWith('data:')) continue;
    const data = line.slice(5).trim();
    if (data === '[DONE]') break;
    const { text } = JSON.parse(data);
    setStreamingText(prev => prev + text);
  }
}

// Send full history on every request
const response = await fetch('/api/chat-stream', {
  method: 'POST',
  body: JSON.stringify({
    message: userMessage,
    history: messages.map(m => ({
      role: m.role,
      content: m.content
    }))
  })
});

// Backend builds context from history
const messages = [
  { role: 'system', content: 'You are a financial advisor...' },
  ...history,  // full conversation
  { role: 'user', content: message }
];

const completion = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  response_format: { type: 'json_object' },
  messages: [{
    role: 'system',
    content: 'Return ONLY valid JSON. No markdown, no explanation.'
  }, {
    role: 'user',
    content: `Recommend 3 banking products for user with
              avg spend $${avgSpend}/month.
              Return: { "products": [{name, tagline, benefit, icon}] }`
  }]
});

const { products } = JSON.parse(completion.choices[0].message.content);

const completion = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  n: 5,  // 5 responses, 1 API call
  messages: [{ role: 'user', content: prompt }],
});

// Cache all 5 — serve randomly to avoid repetition
const options = completion.choices.map(c => c.message.content);
fs.writeFileSync('cache.json', JSON.stringify({ options }));

// ML model predicts the number
const predicted = regression.predict(months.length + 1);

// GPT explains it in human language
const advice = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{
    role: 'user',
    content: `Predicted spending: $${predicted.toFixed(2)}.
              History: ${spending.join(', ')}.
              Give one specific saving tip.`
  }]
});

The live demo of all 6 patterns is at kirti.github.io/ai-banking-poc-frontend. Open DevTools → Network tab → filter by "stream" to watch the SSE events arrive in real time.