Wide schema

Layout

One row per timestamp, with one column per asset and field. The asset and the field are joined in the column name: BTC_close, ETH_open… close_ts is the first column.

Use it for cross-asset work at each timestamp: spreads, correlations, portfolio returns. All assets are aligned on the same rows, and adding an asset adds columns. Columns are plain strings, so it works the same in pandas, Polars, SQL engines, or feature pipelines for machine learning.

Wide is offered for fixed bases only, not for a download by universe: with hundreds of assets it would have thousands of mostly empty columns. To mask a wide file of your own bases by the universe, use the wide universe schema.

One column per asset and field, named BTC_close, ETH_close… There is no index to handle: select columns by name in pandas or in Polars.

import pandas as pd

df = pd.read_parquet(
    "ohlcv_usdt_1h.parquet",
    columns=["close_ts", "BTC_close", "ETH_close"],
)

import polars as pl

df = pl.read_parquet("ohlcv_usdt_1h.parquet")
closes = df.select("close_ts", pl.col("^.*_close$"))

import polars as pl

lf = pl.scan_parquet("ohlcv_usdt_1h.parquet")

# Nothing is read yet: do whatever you want on lf
# (filter, select, group_by, with_columns, ...)

df = lf.collect()