Getting started¶
The shortest path through the Aetelier SDK. By the end of this page you will have a Rust project that connects to Bybit, streams a few minutes of live order-book and trade data, and writes a small Parquet file you can read back in any analytics tool.
If you want the full runnable variant with logging, error handling, and a multi-worker manifest, jump to Tutorial 1: Bybit → Parquet.
Prerequisites¶
- Rust 1.88+ (the workspace uses edition 2024).
- A Bybit account is not required — we'll use the public WebSocket endpoint, which doesn't need authentication for market data.
1. Create a new crate¶
2. Add aetelier-sdk to Cargo.toml¶
The SDK is distributed from GitHub (the core crates are not published on crates.io):
[package]
name = "aetelier-quickstart"
version = "0.1.0"
edition = "2024"
[dependencies]
aetelier-sdk = { git = "https://github.com/aetelierxyz/aetelier-sdk", tag = "v0.1.0", features = ["parquet"] }
tokio = { version = "1", features = ["full"] }
anyhow = "1"
clap = { version = "4", features = ["derive"] }
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
aetelier-sdk is a facade crate — adding it pulls in the workspace
crates you'll touch (aetelier-types, aetelier-connect, aetelier-io)
so you don't have to list each one.
The parquet feature opts into the Parquet sink and the Parquet
reader/writer functions in aetelier-io.
3. Describe the worker in a TOML manifest¶
Collection is manifest-driven: a MarketWorker is configured from a
TOML file that says which exchange, which symbols, which data types,
how to synchronize, and where output goes. Save this as
worker.toml:
[collect]
exchange = "bybit"
gap_threshold_secs = 30
[collect.datatypes.orderbook]
enabled = true
depth = 50
[collect.datatypes.trades]
enabled = true
[collect.sync]
sync_mode = "on_time"
flush_threshold = 3000
[collect.sync.update_frequency]
value = 200
unit = "Millis"
[[collect.output]]
type = "parquet"
dir = "data/snapshots/"
[[collect.output]]
type = "terminal"
[[workers]]
symbol = "BTCUSDT"
Two outputs fan out at once — Parquet for persistence, terminal for
visibility. The 200 ms grid with flush_threshold = 3000 means one
Parquet flush every 10 minutes of snapshots.
4. The minimum viable runner¶
use std::path::PathBuf;
use aetelier_sdk::aetelier_connect::{
config::workers::MarketWorkerManifest, workers::market_worker::MarketWorker,
};
use clap::Parser;
use tokio::sync::watch;
#[derive(Parser)]
struct Cli {
#[arg(short, long, default_value = "worker.toml")]
config: PathBuf,
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
tracing_subscriber::fmt::init();
let cli = Cli::parse();
let manifest = MarketWorkerManifest::from_toml(&cli.config)?;
let configs = manifest.resolve_all();
anyhow::ensure!(!configs.is_empty(), "manifest is empty");
let (shutdown_tx, shutdown_rx) = watch::channel(false);
tokio::spawn(async move {
tokio::signal::ctrl_c().await.expect("ctrl-c handler");
let _ = shutdown_tx.send(true);
});
let worker = MarketWorker::from_config(configs.into_iter().next().unwrap())?;
let report = worker.run(shutdown_rx).await?;
println!(
"collected {} events into {} snapshots ({} flushes)",
report.ingestion.total_events, report.snapshots_produced, report.flushes,
);
Ok(())
}
Run it:
You'll see one terminal line per 200 ms snapshot. Press Ctrl-C to stop; the worker shuts down gracefully and reports what it collected. After a few minutes you'll have a tree like:
data/snapshots/
├── orderbooks/bybit_BTCUSDT_ob_sync_<timestamp>.parquet
└── trades/bybit_BTCUSDT_trades_sync_<timestamp>.parquet
The filename convention ({symbol}_{datatype}_{mode}_{ts}) is
documented on the aetelier-io
page.
What you just learned¶
| Concept | Where it lives |
|---|---|
| Exchange clients with reconnect | aetelier-connect |
| Workers that synchronize feeds | aetelier-connect MarketWorker |
| TOML worker manifests | aetelier-connect MarketWorkerManifest |
| Output fan-out (parquet, terminal) | [[collect.output]] in the manifest |
| Parquet readers / writers | aetelier-io |
| The data types this all flows through | aetelier-types |
Where to go next¶
- Tutorial 1: Bybit → Parquet — the same flow with multi-symbol manifests, proper error handling, and graceful shutdown.
- Tutorial 2: multi-exchange sync — fanning out across Bybit, Coinbase, Binance, Kraken with a single synchronizer.
- Architecture — how the crates fit together.