Download / Loading data

Loading the wide schema

One column per asset and field, named BTC_close, ETH_close… There is no index to handle: select columns by name in pandas or in Polars.

pandas

Pass columns to read only the series you need.

import pandas as pd

df = pd.read_parquet(
    "ohlcv_usdt_1h.parquet",
    columns=["close_ts", "BTC_close", "ETH_close"],
)

Polars

Select by name, or by regex across all assets.

import polars as pl

df = pl.read_parquet("ohlcv_usdt_1h.parquet")
closes = df.select("close_ts", pl.col("^.*_close$"))

Polars (lazy)

Chain your operations, then call collect() last. Only the selected columns are read from disk when it runs.

import polars as pl

lf = pl.scan_parquet("ohlcv_usdt_1h.parquet")

# Nothing is read yet: do whatever you want on lf
# (filter, select, group_by, with_columns, ...)

df = lf.collect()