One file per asset schema

Layout

One .parquet file per asset. Each file has one row per timestamp and one column per field.

Use it when you process assets one at a time or in parallel, or when you only need a few assets from a large universe. Each file is loaded independently.

A download by universe gives one file per listed asset, each with rows only for the time the pair was listed. If a single asset is left, you get that file alone rather than a zip.

One file per asset, all with the same columns. Read a single file directly, or read the whole folder and keep track of which file each row came from.

import pandas as pd
from pathlib import Path

btc = pd.read_parquet("data/btc_usdt_1h.parquet")

df = pd.concat(
    {p.stem.split("_")[0].upper(): pd.read_parquet(p) for p in Path("data").glob("*.parquet")},
    names=["base"],
)

import polars as pl

btc = pl.read_parquet("data/btc_usdt_1h.parquet")

df = pl.read_parquet("data/*.parquet", include_file_paths="file")

import polars as pl

lf = pl.scan_parquet("data/*.parquet", include_file_paths="file")

# Nothing is read yet: do whatever you want on lf
# (filter, select, group_by, with_columns, ...)

df = lf.collect()