Systems

A system is any software that maintains your application's state. It could be a database, cache, file system, SaaS, etc.

The primary function of a system is to allow rngo to seamlessly deliver simulated data to the appropriate destination. In order to do so, the rngo config must:

1. specify the connection paramters for each system
2. associate streams with systems

The rngo CLI can also connect to and inspect systems to automatically infer and update your stream config.

Config

Here's an example of how to configure a Postgres system in .rngo/config.yml:

systems:
  db:
    type: postgres
    parameters:
      host:
        default: localhost
      port:
        value: 5439

Systems live under the top-level systems key - this system is named "db" and its type is of course Postgres. It also has customized host and port parameters. Parameters are how rngo connects to the system both during import and inference.

Parameters

If you're able to follow rngo's enviroment variable conventions, you won't need to configure system parameters. By default, rngo will look for parameters at:

$RNGO_{system}_{parameter}

So, rngo will expect to find the db system's host parameter at the RNGO_DB_HOST environment variable.

When importing more complex simulations, you can also disambiguate systems with the same name in different namespaces:

$RNGO_{namespace}_{system}_{parameter}

So RNGO_ORDERS_DB_HOST is the host parameter for the db system in the orders namespace.

Custom Environment Variables

To customize the environment variable rngo looks for, set the env key under the parameter key:

systems:
  db:
    type: postgres
    parameters:
      host:
        env: PGHOST

rngo will first look for the host value at PGHOST and fall back to RNGO_DB_HOST if it's not found.

Default Value

You can also specify a default that rngo will fall back to if the environment variable is missing:

systems:
  db:
    type: postgres
    parameters:
      host:
        default: localhost

This means that if the RNGO_DB_HOST environment variable is not set, rngo will use the value localhost for the host. This can be used in combination with the env key.

Harcoded Value

You can also specify a value that rngo will always use as the parameter:

systems:
  db:
    type: postgres
    parameters:
      port:
        value: 5438

In this case, rngo will always use 5438 as the port for the db system, and ignore any environment variables and default values.

System Outputs

You associate streams with systems via the stream config, for example:

streams:
  users:
    outputs:
      db:
        paramters:
          table: USER
    schema:
    #...

With this configuration, rngo wil load the data generated by the users stream into the USER table of the db Postgres system. See stream system outputs for more details.

Stream Inference

rngo infer will connect to each configured system, infer stream configs based upon the state (schema and / or data) and add them to .rngo/config.yml.

By default, this command is append-only, meaning that it will only add new streams and not touch any potentially-edited existing streams. Inferred streams will always be associated with the systems from which they came.

Types

postgres

Outputs data as CSV and supports both inference and import.

Parameters