Harden and deploy the RAG service¶
When I run the system on my laptop, I can restart a command or wait for a slow model call. A deployed service has to handle those problems without me watching it.
I use hardening to mean adding the limits and failure handling that make the service safe to run for other people. A database can go offline, a model provider can be slow, and a request can contain bad input. We need to decide what happens in each case.
Put a deadline around dependencies¶
Set separate timeouts for PostgreSQL, embedding, reranking, and generation. A request-wide timeout alone tells you too late which dependency consumed the budget.
Retry temporary failures such as a short provider outage or connection reset. Wait a little longer between each attempt, which is called exponential backoff, and stop after a small number of attempts. Invalid input, access denial, and malformed model output will not improve because you tried them five more times.
If vector search fails and word search is still available, you may return a word-search-only result. Record that fallback in the trace and debugging details. Do not present the reduced result as if the full pipeline ran.
Timeouts describe one request. The system running your service also needs a quick way to decide whether the process should stay alive and receive new traffic.
Expose two kinds of health¶
A liveness check answers, “Is the program running?” A readiness check answers, “Can this program reach the database and other services needed to answer a Query?”
Keep them separate. Whatever system runs your service can restart a dead process without sending user traffic to one that is running but cannot reach PostgreSQL.
With failure and health behavior decided, package the exact code and dependencies you tested. Configuration and secrets should remain outside that package so each deployment can supply its own values.
Package configuration, not secrets¶
Package the FastAPI service in a container, which bundles the code, Python runtime, and dependencies into one repeatable unit. Pass database URLs, model names, provider keys, timeouts, and feature switches through environment variables. Commit .env.example, never a real .env file containing secrets.
Use your application's existing controlled process for changing the PostgreSQL table structure. This Course teaches the required tables and queries, not a general database migration tool.
The package is ready to run, but I still want one final stop before release. The same tests and evaluation that guided development should decide whether this version can ship.
Run the gates before release¶
A release should pass chunking and ingestion tests, retrieval evaluation, Citation and abstention checks, malicious-Source tests, and the API contract tests. Pin Python and dependency versions so the container and evaluation run use the same code.
Emit structured logs and OpenTelemetry traces with request IDs. Keep raw protected content out of normal telemetry.
The Course stays provider-neutral because free tiers and hosting products change. The deployment Reference lists current inexpensive options, while the service depends only on a Python runtime, PostgreSQL with pgvector, environment-injected secrets, and outbound model access.
A static documentation website can live on Cloudflare Pages. Deploy the RAG API inside the access controls and infrastructure your application already uses.
You have now followed one system from discovering a filename to operating a cited RAG service. The Advanced lessons explore techniques you can add when a measured problem calls for them.