Skip to content

Getting started

The shortest path through the Aetelier SDK. By the end of this page you will have a Rust project that connects to Bybit, streams a few minutes of live order-book and trade data, and writes a small Parquet file you can read back in any analytics tool.

If you want the full runnable variant with logging, error handling, and a multi-worker manifest, jump to Tutorial 1: Bybit → Parquet.

Prerequisites

  • Rust 1.88+ (the workspace uses edition 2024).
  • A Bybit account is not required — we'll use the public WebSocket endpoint, which doesn't need authentication for market data.

1. Create a new crate

cargo new --bin aetelier-quickstart
cd aetelier-quickstart

2. Add aetelier-sdk to Cargo.toml

The SDK is distributed from GitHub (the core crates are not published on crates.io):

[package]
name = "aetelier-quickstart"
version = "0.1.0"
edition = "2024"

[dependencies]
aetelier-sdk = { git = "https://github.com/aetelierxyz/aetelier-sdk", tag = "v0.1.0", features = ["parquet"] }
tokio       = { version = "1", features = ["full"] }
anyhow      = "1"
clap        = { version = "4", features = ["derive"] }
tracing     = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }

aetelier-sdk is a facade crate — adding it pulls in the workspace crates you'll touch (aetelier-types, aetelier-connect, aetelier-io) so you don't have to list each one.

The parquet feature opts into the Parquet sink and the Parquet reader/writer functions in aetelier-io.

3. Describe the worker in a TOML manifest

Collection is manifest-driven: a MarketWorker is configured from a TOML file that says which exchange, which symbols, which data types, how to synchronize, and where output goes. Save this as worker.toml:

[collect]
exchange = "bybit"
gap_threshold_secs = 30

[collect.datatypes.orderbook]
enabled = true
depth = 50

[collect.datatypes.trades]
enabled = true

[collect.sync]
sync_mode = "on_time"
flush_threshold = 3000

[collect.sync.update_frequency]
value = 200
unit = "Millis"

[[collect.output]]
type = "parquet"
dir = "data/snapshots/"

[[collect.output]]
type = "terminal"

[[workers]]
symbol = "BTCUSDT"

Two outputs fan out at once — Parquet for persistence, terminal for visibility. The 200 ms grid with flush_threshold = 3000 means one Parquet flush every 10 minutes of snapshots.

4. The minimum viable runner

use std::path::PathBuf;

use aetelier_sdk::aetelier_connect::{
    config::workers::MarketWorkerManifest, workers::market_worker::MarketWorker,
};
use clap::Parser;
use tokio::sync::watch;

#[derive(Parser)]
struct Cli {
    #[arg(short, long, default_value = "worker.toml")]
    config: PathBuf,
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    tracing_subscriber::fmt::init();
    let cli = Cli::parse();

    let manifest = MarketWorkerManifest::from_toml(&cli.config)?;
    let configs = manifest.resolve_all();
    anyhow::ensure!(!configs.is_empty(), "manifest is empty");

    let (shutdown_tx, shutdown_rx) = watch::channel(false);
    tokio::spawn(async move {
        tokio::signal::ctrl_c().await.expect("ctrl-c handler");
        let _ = shutdown_tx.send(true);
    });

    let worker = MarketWorker::from_config(configs.into_iter().next().unwrap())?;
    let report = worker.run(shutdown_rx).await?;

    println!(
        "collected {} events into {} snapshots ({} flushes)",
        report.ingestion.total_events, report.snapshots_produced, report.flushes,
    );
    Ok(())
}

Run it:

cargo run

You'll see one terminal line per 200 ms snapshot. Press Ctrl-C to stop; the worker shuts down gracefully and reports what it collected. After a few minutes you'll have a tree like:

data/snapshots/
├── orderbooks/bybit_BTCUSDT_ob_sync_<timestamp>.parquet
└── trades/bybit_BTCUSDT_trades_sync_<timestamp>.parquet

The filename convention ({symbol}_{datatype}_{mode}_{ts}) is documented on the aetelier-io page.

What you just learned

Concept Where it lives
Exchange clients with reconnect aetelier-connect
Workers that synchronize feeds aetelier-connect MarketWorker
TOML worker manifests aetelier-connect MarketWorkerManifest
Output fan-out (parquet, terminal) [[collect.output]] in the manifest
Parquet readers / writers aetelier-io
The data types this all flows through aetelier-types

Where to go next