Explore the code Parallel replay
History is processed in parallel rather than one block after another.
Historical data
Replay the whole chain.
Without the archive node.
From genesis to the current block, into your own database, on a path you can stop and resume.
- Genesis to head
- Resumable
- No archive node
- Any supported chain
Ethereum → Firehose → Substreams → PostgreSQL
Interactive illustration
StreamingFast
ecosystem
01 Historical data
Looks familiar?
The request returned 200, and you still do not know whether the dataset covers the right contracts.
The backfill has been running for days, and nobody can say which block it is on.
It died at 70 percent, and the only option on the table is to start again.
Want to see this on the chain you actually work on?
Get an API key
RUST → WEBASSEMBLYA replay path, not a faster loop.
Following the head and replaying years of blocks are two different jobs. Most teams use one code path for both, which is why the backfill is slow and still has holes. Firehose reads flat files instead of asking an endpoint for one block at a time, so history is replayed in parallel and the live stream is left to do its own job.
Read the guideFilterMapAggregateDeliver
▧ substreams.yamlPACKAGE STRUCTURE
01 specVersion: v0.1.0
02 package:
03 name: token_transfers
04 version: v0.1.0
05
06 modules:
07 - name: map_transfers
08 kind: map
09 inputs:
10 - source: sf.ethereum.type.v2.Block
11 output:
12 type: proto:transfers.Transfers Illustrative excerpt Build a complete package ↗
What you get
A cursor you own
Persist where you are. A restart resumes there, not at the beginning.
Decoded on the way through
The output is the entity your product uses, not a log you still have to interpret.
Coverage you can state
A record of what the dataset contains, so the next person does not have to guess.
The same path for live
When history is caught up, the stream continues. No second implementation.
02 Historical data
How it works
FROM HISTORY TO THE CHAIN HEAD
Historical blocksNew blocks
TRANSACTIONS
TRACES & LOGS
STATE CHANGES
Rich block data. One continuous stream.
Define the range
Name the network, the contracts and the period the dataset has to answer for.
Replay it
Run the backfill, parallel, resumable, into flat files or straight into your database.
Verify the edges
Check the first block, the last block and a sample inside, then record the coverage.
Want to see this on the chain you actually work on?
Get an API key03 Historical data
What you stop owning
[01]
An archive node to run and keep synced
[02]
A separate backfill path to build and maintain
[03]
The code that unwinds a reorganization in your database
Historical data
Replay your chain, not a slide.
Read the guide ↗Retrieve complete blockchain history
Define coverage, plan the backfill and verify the dataset before you rely on it.