Firehose
  • Firehose
  • Introduction
    • Firehose Overview
    • Prerequisites
  • Architecture
    • Components
      • Firehose-enabled Node
      • Reader
      • Merger
      • Relayer
      • gRPC Server
      • High Availability
    • Data Flow
    • Data Storage
  • Firehose Setup
    • Overview
    • Ethereum
      • Installation
      • Single-Machine Deployment
      • Reprocessing history
      • Synchronization
    • Cosmos
      • Single-Machine Deployment
    • NEAR
      • Single-Machine Deployment
    • System Requirements
  • Community Integrations
    • Starknet
      • Networks and nodes
      • Local deployment with Docker
      • Local deployment without Docker
  • Integrate New Chains
    • Benefits
    • Integration overview
    • Design Principles
    • Firehose Acme
  • References
    • Supported Protocols
    • Naming Conventions
    • Schemas
    • Indexing
    • FAQ
  • Release Notes
    • Change logs
      • Nov 8th 2023 Polygon Update
Powered by GitBook
On this page
  • Reader Component in Detail
  • Firehose Depends on the Reader
  1. Architecture
  2. Components

Reader

StreamingFast Firehose reader component

PreviousFirehose-enabled NodeNextMerger

Last updated 1 year ago

Reader Component in Detail

The Reader component is responsible for extracting data from instrumented blockchain nodes.

The Reader component utilizes the StreamingFast library to run a blockchain node instance as a sub-process. Alternatively, the Reader component can consume the stdout of the process where reader-stdin is implemented.

Once the process has been started, the Reader component:

  • reads the data being generated by the node,

  • forwards the data downstream to other connected components including the Relayer, Firehose gRPC Server, etc.

  • flushes the data to Object Storage for durability, and for the Merger to pick up the data.

Firehose Depends on the Reader

The data consumed by Firehose is provided by the Reader component.

Tip: The Reader component is the initial and deterministic data producer for Firehose and all of its components.

The Blockchain node underlying and managed by the Reader can be considered simplistic. They don't have archiving capabilities or any additional features.

Note: after Firehose has been instrumented on a node it will begin returning substantial amounts of data.

node-manager