My First LLM API Call: Streaming, Retries, and a Key Out of Git

My first Gemini call streamed a one-sentence answer in the terminal. The next run hit 503 high demand — the same request worked a moment later. Here is the TypeScript I use for streaming, for retrying only when a failure might clear (503 and 429), and for stopping when it will not (400, 401, or a model name that does not work on your account).

This is part 2 of Learning AI Engineering in Public.

One-sentence answer: ChatGPT is an app. The API is how a program calls the same kind of model.

The app and the API

ChatGPT gives you a text box. An API gives your code a function. I called Gemini from TypeScript with @google/genai, and I ran the file with tsx.

The call uses generateContentStream, so tokens print as they arrive instead of waiting for the full reply. Temperature is 0.2 — low enough to stay steady. The model is gemini-3.8-flash. Wrap the API call in withBackoff, not the for await loop — retries happen before streaming starts.

import { GoogleGenAI } from "@google/genai";
 
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });
 
const RETRY_DELAYS_MS = [2_000, 4_000, 8_000];
 
function httpStatus(error: unknown): number | undefined {
  if (typeof error === "object" && error !== null && "status" in error) {
    const status = (error as { status?: unknown }).status;
    return typeof status === "number" ? status : undefined;
  }
 
  return undefined;
}
 
async function withBackoff<T>(task: () => Promise<T>): Promise<T> {
  let attempt = 0;
 
  while (true) {
    try {
      return await task();
    } catch (error) {
      const delay = RETRY_DELAYS_MS[attempt];
      const status = httpStatus(error);
      const retryable = status === 503 || status === 429;
 
      if (!retryable || delay === undefined) throw error;
 
      await new Promise((resolve) => setTimeout(resolve, delay));
      attempt += 1;
    }
  }
}
 
async function main() {
  const stream = await withBackoff(() =>
    ai.models.generateContentStream({
      model: "gemini-3.8-flash",
      contents: "Explain a token in one sentence.",
      config: { temperature: 0.2 },
    }),
  );
 
  for await (const chunk of stream) {
    if (chunk.text) process.stdout.write(chunk.text);
  }
 
  process.stdout.write("\n");
}
 
main();

What this block does:

GoogleGenAI is the client. It reads the key from process.env.GEMINI_API_KEY so the secret never sits in the source file.
withBackoff wraps generateContentStream only. If the call fails with 503 or 429, it waits and tries again before any tokens arrive.
Each chunk is a piece of the answer as it is produced. Writing chunk.text to stdout is why the terminal fills in live — same feel as ChatGPT typing.
temperature: 0.2 keeps wording fairly consistent. That matters when you want a factual one-liner, not a creative remix.

Retry 503 and 429

I hit a 503 high-demand error on a repeat run. The same request can work a moment later, so I retry it. The waits are 2s, then 4s, then 8s. I use that same backoff for 429.

When I tried gemini-2.5-flash, it was not available on my account. A wrong model name is not fixed by waiting — read the error, check the current docs, then change the string. Guessing model names from memory is how you burn an afternoon. The SDK puts the HTTP status on the error as status. 400 means the request itself is wrong. 401 means the key was rejected. Retrying those sends the same broken request again. 429 means you are being rate limited; 503 means the service is under high demand — both can clear after a short wait. Retry only when the problem might be temporary.

What the helper does:

RETRY_DELAYS_MS is the schedule: wait longer each time. That is exponential backoff in three steps.
httpStatus only looks for a numeric status field. It does not invent a code.
withBackoff runs your task once per attempt. On failure it checks whether the status is retryable. If yes, it sleeps and tries again. If no — or if the delays are used up — it throws.

The same selective retry for 429 and 503 appears in the production embedding path on /lab. When retry is not enough, Part 5: A Mini RAG Pipeline adds model fallback.

Keep the key out of git

The key lives in .env, not in the source file. I start the script with --env-file, and .env is listed in .gitignore. If the key leaks, rotate it: delete that key and create a new one.

GEMINI_API_KEY=your-key

tsx --env-file=.env first-call.ts

What these blocks do:

The .env line is the secret store. Your code reads process.env.GEMINI_API_KEY.
--env-file=.env loads that file into the process before tsx runs.
Gitignoring .env keeps the key out of commits. Rotation is the recovery plan when something still leaks.

Takeaways

ChatGPT is the app; @google/genai is how TypeScript calls the model — stream with generateContentStream, temperature 0.2, model gemini-3.8-flash, run with tsx.
Wrap the stream call in withBackoff, not the read loop; retry 503 and 429 at 2s, 4s, and 8s — not 400, 401, or a model name that fails every time.
gemini-2.5-flash was not available on my account when I tried it; read the error and the current docs before you swap model strings.
Store the key in .env, load with --env-file, gitignore that file, and rotate if it leaks.
Selective retry on /lab matches this script; Part 5 covers fallback when backoff alone is not enough.

The app and the API

Retry 503 and 429

Keep the key out of git

Takeaways

Related posts