Streams

A stream defines the schema, content and volume of one data set within a simulation. At a minimum, a stream must have:

1. a unique name (within its namespace)
2. a JSON Schema

Currently, you can only add and update streams via your project config.

Schema

All streams define a JSON schema - in the config, it is specified under the schema keyword. When a stream is included in a simulation, rngo guarantees that the data it generates for that stream will be valid against its schema.

rngo will eventually fully support the 2020-12 draft of JSON Schema. For now, only a subset of the vocabularies and keywords are supported.

Custom Vocabulary

rngo extends JSON schema with a custom vocabulary to support the generation of realistic data. All extension keywords are nested under the rngo keyword and do not change the validation semantics of the schema.

rngo.value

The rngo.value keyword specifies an expression that returns either a Set or a single Value. For example:

type: object
properties:
  name:
    type: string
    rngo:
      value: enums.fullNames

In this case, a value will be randomly selected from enum.fullName set when generating a value for the name property.

Upon stream creation or update, rngo will validate that all rngo.value expressions returns a value or set of the correct type.

See Expressions for more information.

Rate

Use the rate keyword to specify the rate at which the stream should produce new events, expressed in hertz. For example, this stream will produce events at a rate of roughly 1 event per 10 seconds:

streams:
  users:
    rate: 0.1
    schema:
      #...

rngo builds in variance, so the observed rate over any sub-interval of the simulation may be higher or lower than the configured one.

The value is an expression, so to make the rate increase over time, you could do something like this:

streams:
  users:
    rate: 0.1 + (0.0001 * sim.offset)
    schema:
      #...

The expression is sampled periodically over the course of the simulation, so the rate will change in steps.

Rates will always be adjusted to be greater than or equal to zero and less than 1000 events / second.

Systems

Streams may be associated with one or more systems. See the Systems reference for details.

Outputs

You can also customize the output of a stream's data. In the config, this looks like this:

streams:
  users:
    outputs:
      - format: csv
    schema:
    #...