Parse CSV files as records¶
A CSV file is a table stored as text. Each line is usually a row, and the first line names the columns.
I do not turn a table into paragraphs by default. Rows and columns already give us a precise way to retrieve values.
Read the records¶
Python's standard library handles the example pricing table:
import csv
from pathlib import Path
path = Path("data/plans.csv")
with path.open(newline="", encoding="utf-8") as handle:
plans = list(csv.DictReader(handle))
print(plans[1]["included_build_minutes"])
DictReader uses each column name as a dictionary key. The output keeps plan, monthly_price_usd, and included_build_minutes as separate fields.
CSV does not carry strong data types. The value 5000 arrives as text, so convert and validate numbers, dates, and booleans before calculations.
Use exact lookup for exact questions¶
For How many build minutes come with Pro?, I look up the Pro row and read the named field. SQL becomes useful when you need filters, sorting, joins, counts, or totals.
Keep the file path, row number or stable row key, and selected fields with the result. A generated Answer should be able to cite the record it used.
Add text only for meaning-based search¶
Some tables contain long descriptions or notes. A user may ask with words that do not appear exactly in one field. In that case, you can build a readable text representation for each row:
def plan_text(row: dict[str, str]) -> str:
return (
f"Plan: {row['plan']}\n"
f"Monthly price: ${row['monthly_price_usd']}\n"
f"Included build minutes: {row['included_build_minutes']}"
)
Embed that text only if your Evaluation Queries show a need for meaning-based retrieval. Keep the original fields beside it so the model does not have to extract exact numbers from generated prose.
Watch for CSV edge cases¶
Real CSV files can contain quoted commas, line breaks inside cells, duplicate headers, missing values, and unexpected encodings. Use a CSV parser instead of splitting lines on commas, and reject rows that do not match the expected schema.
For small CSV files, the standard library is enough. For large analytical data, load the records into PostgreSQL and let SQL do the work.