Backfilling API¶

Auto-generated API documentation for StreamForge backfilling.

BinanceBackfilling¶

BinanceBackfilling ¶

BinanceBackfilling(symbol: str, timeframe: str, from_date: str, to_date: str = 'now', file_path: Optional[str] = None, data_type: str = 'klines', market_type: str = 'DEFAULT', transformer: Optional[Callable[[Dict[str, Any]], Dict[str, Any]]] = None)

run ¶

run()

register_emitter ¶

register_emitter(emitter: DataEmitter, model: Optional[Union[Base, BaseModel]] = None, map_object: Optional[Dict] = None)

set_transformer ¶

set_transformer(transformer_func: Callable[[Dict[str, Any]], Dict[str, Any]], inplace: bool = False)

OkxBackfilling¶

OkxBackfilling ¶

OkxBackfilling(symbol: str, timeframe: str, from_date: str, to_date: str = 'now', file_path: Optional[str] = None, data_type: str = 'klines', transformer: Optional[Callable[[Dict[str, Any]], Dict[str, Any]]] = None)

run ¶

run()

register_emitter ¶

register_emitter(emitter: DataEmitter, model: Optional[Union[Base, BaseModel]] = None, map_object: Optional[Dict] = None)

set_transformer ¶

set_transformer(transformer_func: Callable[[Dict[str, Any]], Dict[str, Any]], inplace: bool = False)

BybitBackfilling¶

BybitBackfilling ¶

BybitBackfilling(symbol: str, timeframe: str, from_date: str, to_date: str = 'now', file_path: Optional[str] = None, market_type: str = 'DEFAULT', transformer: Optional[Callable[[Dict[str, Any]], Dict[str, Any]]] = None)

Bybit backfilling class for fetching historical kline/candlestick data.

Uses Bybit REST API to fetch historical data for backfilling purposes. Supports Spot, Linear (USDT/USDC perpetuals), and Inverse futures markets.

Parameters:

Name	Type	Description	Default
`symbol`	`str`	Trading pair symbol (e.g., 'BTCUSDT')	required
`timeframe`	`str`	Candle timeframe (e.g., '1m', '5m', '1h', '1d')	required
`from_date`	`str`	Start date in '%Y-%m-%d' format	required
`to_date`	`str`	End date in '%Y-%m-%d' format or 'now' (default: 'now')	`'now'`
`file_path`	`Optional[str]`	Optional custom file path for CSV output	`None`
`market_type`	`str`	Market type - SPOT, LINEAR, or INVERSE (default: 'DEFAULT')	`'DEFAULT'`
`transformer`	`Optional[Callable[[Dict[str, Any]], Dict[str, Any]]]`	Optional data transformation function	`None`

Examples:

Basic usage:

>>> backfiller = BybitBackfilling(
...     symbol="BTCUSDT",
...     timeframe="1m",
...     from_date="2024-01-01",
...     to_date="2024-01-31"
... )
>>> backfiller.run()

With market type:

>>> backfiller = BybitBackfilling(
...     symbol="BTCUSDT",
...     timeframe="1h",
...     from_date="2024-01-01",
...     market_type="LINEAR"
... )
>>> backfiller.run()

run ¶

run()

Execute backfilling process.

Creates default CSV emitter if none registered, then fetches and emits historical data to registered emitters.

register_emitter ¶

register_emitter(emitter: DataEmitter, model: Optional[Union[Base, BaseModel]] = None, map_object: Optional[Dict] = None)

Register output emitter (CSV, Postgres, etc.).

set_transformer ¶

set_transformer(transformer_func: Callable[[Dict[str, Any]], Dict[str, Any]], inplace: bool = False)

Set data transformation function.

Common Usage¶

Basic Backfill¶

from streamforge import BinanceBackfilling

backfiller = BinanceBackfilling(
    symbol="BTCUSDT",
    timeframe="1h",
    from_date="2024-01-01",
    to_date="2024-12-31"
)

backfiller.run()  # Sync method

With Emitter¶

backfiller = BinanceBackfilling(
    symbol="BTCUSDT",
    timeframe="1h",
    from_date="2024-01-01",
    to_date="2024-12-31"
)

backfiller.register_emitter(postgres_emitter)
backfiller.run()

With Transformer¶

def my_transformer(data: dict) -> dict:
    return {...data, "custom_field": value}

backfiller.set_transformer(my_transformer)
backfiller.run()

Bybit with Market Type¶

from streamforge import BybitBackfilling

# Spot market (default)
backfiller = BybitBackfilling(
    symbol="BTCUSDT",
    timeframe="1h",
    from_date="2024-01-01",
    to_date="2024-12-31",
    market_type="SPOT"
)

# Linear futures
backfiller = BybitBackfilling(
    symbol="BTCUSDT",
    timeframe="1h",
    from_date="2024-01-01",
    to_date="2024-12-31",
    market_type="LINEAR"
)

# Inverse futures
backfiller = BybitBackfilling(
    symbol="BTCUSD",  # Note: USD for inverse
    timeframe="1h",
    from_date="2024-01-01",
    to_date="2024-12-31",
    market_type="INVERSE"
)

backfiller.run()

Back to API Reference →