Single-Machine Deployment

StreamingFast Firehose local deployment

Ethereum Local Deployment

Local Deployments Intro

Firehose setups that are created on a computer directly, as opposed to cloud-based setups, are considered "local deployments." Local deployments remain on the computer the operators have direct access to and management over.

Use the target computer's home directory to begin If a dedicated directory hasn't yet been identified or selected for the Firehose setup. Create a directory named “firehome”, or something similar that's reflective of the target setup being created.

Firehose Configuration

Config File in Detail

The configuration file for Firehose is stored in the main Firehose directory. The name of the file can be anything, however, using the name of the specific chain and version of Firehose is recommended for clarity.

The example below assumes a default Ethereum-based Firehose configuration is being created. For Binance use a filename similar to “bnb-firehose-config.yaml.”

Use the content provided below for the configuration file. The settings provided are ready-to-go for default Ethereum setups. Additional information is provided for the other Ethereum-compatible chains.

start:
  args:
  # Define Firehose components that will be used for this setup
  - merger
  - firehose
  - reader-node
  - relayer
  - combined-index-builder
  flags:
    # Prevent creation of file for logging (logs from previous sessions are not visible with this setting)
    log-to-file: false

    # Ethereum is the default chain, its id is 1. Update these flags for different chains such as Binance
    common-chain-id: "1"
    common-network-id: "1"
    # Update the reader-node-path to reference the geth binary for the chain and OS being targeted
    reader-node-path: ./geth_linux
    # Update the reader-code-arguments for the chain being targeted
    # Find specific values in the Firehose Ethereum Setup Documentation
    reader-node-arguments: "+--cache 8192 --maxpeers 100 --metrics --metrics.port 6061 --port=30303 --http.port=8545 --snapshot=true --txlookuplimit=1000"
    reader-node-log-to-zap: false

Geth Binary Path

Firehose needs to know the location of the Geth binary. The configuration file contains a reader-node-path flag specifically designed to point Firehose to the Geth binary. The binary name will be the same for the different blockchains.

Note: the names of the Geth binaries differ depending on the target operating system. The differences are indicated by “_linux” or “_mac” in the binaries file name. Geth can be used directly on the command line without specifying a path if it has been installed globally on the system in the user/local/bin directory.

The updated flag in the configuration file for a macOS-based Firehose setup should reflect the following.

reader-node-path: ./geth_mac

Unique chain and network ID

Each blockchain has a unique, numeric ID for the chain and network. Ethereum is the default blockchain implementation; the chain and network ID for it are the numeric value of 1. The numeric value is different for the other Ethereum-compatible chains.

Use the table below to locate the chain and network ID for the blockchain used in the Firehose setup. Both of the following flags need to be updated in the configuration file.

common-chain-id

common-network-id

Blockchain
chainID
networkID

Ethereum

1

1

Binance

56

56

Polygon

137

137

Goerli

5

5

The numbers for the network and chain need to be enclosed in quotes. The update for Ethereum, for example, will read as the following. Note, the space between the colon and the starting quote.

common-chain-id: “1”

common-network-id: “1”

Note: The chain and network ID will be different for each of the different Ethereum-compatible chains.

Default Ethereum Firehose Setups

Ethereum Mainnet is the default Geth version provided by StreamingFast. For Firehose setups targeting Ethereum Mainnet the genesis file and reader-node-arguments steps below are not applicable. Proceed to the step for running Firehose.

For Firehose setups targeting other Ethereum-compatible blockchains, such as Polygon, further action is required. Additional information is provided to explain the genesis file creation process and reader-node-arguments modifications.

Firehose setups targeting Goerli do not require genesis files however further modifications to the reader-node-arguments value are necessary. Additional information specific to Goerli setups is provided below.

Download genesis JSON File

Firehose requires genesis block information to begin indexing for Ethereum-compatible blockchains. The genesis block information is provided in JSON files available for download. Find the JSON file relevant to the blockchain for the Firehose setup being created.

Binance genesis JSON file

Right-click and save the JSON file to the main Firehose directory created in the first step of the setup process.

https://raw.githubusercontent.com/streamingfast/firehose-docs/master/configs/binance/genesis.json

Alternatively, the JSON file can be downloaded with curl using a terminal window. Issue the following command to the terminal, making sure the shell session is in the main Firehose setup directory.

Polygon genesis JSON file

Right-click and save the JSON file to the main Firehose directory created in the first step of the setup process.

https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVgApTPXzd7Z9BUUosaeF%2Fuploads%2FyshZBel3YKwFIOYYvyGf%2Fgenesis.json

Alternatively, the JSON file can be downloaded with curl using a terminal window. Issue the following command to the terminal, making sure the shell session is in the main Firehose setup directory.

Add Genesis File Reference to Config File

After the JSON files have been downloaded the configuration file needs to be updated to inform Firehose where the file is located. Add the following flag to the configuration file.

reader-node-bootstrap-data-url: ./genesis.json

Update reader-node-arguments in Config File

The reader-node-arguments flag provides Firehose with information it needs for startup. Use the flags provided below specific to the chain being targeted.

Binance reader-node-arguments

The reader-node-arguments flag for Binance need to be updated to reflect the following.

Polygon reader-node-arguments

The reader-node-arguments flag for Polygon need to be updated to reflect the following.

Goerli reader-node-arguments

As previously mentioned, Goerli does not require specific JSON files. The config file does however require modifications. The reader-node-arguments flag should be updated to reflect the value provided below. Note, Goreli setups do not need to pass values for bootsnotes as with the Binance and Polygon setups.

Important: The cache argument passed to fireeth should be updated to 1024. This is due to the smaller size of the Goreli test network.

Test Installation

The following command is used to start Firehose. Run the command from the directory that Firehose was installed to if other than usr/local/bin.

Successful Installation Logging

After issuing the start command to the terminal it can take up to thirty seconds for Firehose to begin connecting to peers and processing block data.

Logging will be rapidly printed to the terminal window. See the log below for example logging.

Press and hold the Control key and then press the C key three times to terminate Firehose and end all processes.

After Firehose has been set up it needs to be synchronized with the target blockchain. Full sync is a data and time-intensive process and can take several days or longer to complete for full archive nodes. Further documentation is provided for synchronizing Firehose.

Fireeth Tools

Firehose also provides tools for operators to inspect many facets of the application. For example, the tools check merged-blocks command can be used to view and investigate the block data produced by Firehose.

The merged blocks tool will print to the terminal window. Look for messaging similar to what's shown below to determine if the installation was successful.

Problems

It may be necessary to remove the quarantine attribute on the Geth and Firehose binary files to resolve signing issues on macOS.

Note: this step will only need to be completed once.

PreviousInstallationNextReprocessing history

Last updated