Tzopilotl
Docs
GitHub

Tzopilotl — Foreign Function Interface

Comprehensive guide to embedding Tzopilotl and extending it with native functions.

1. Overview

The Tzopilotl FFI enables two-way communication between host C/C++ code and the Tzopilotl interpreter:

The FFI also provides compile-time real-time safety enforcement: when compiling for a real-time VM, the type checker rejects any call to a function marked as non-RT-safe, guaranteeing that compiled RT code never calls a blocking function.

Both a C++ API (via tzpl.hpp) and a C API (via tzpl.h) are provided. The C API wraps the C++ implementation with opaque handle types, making it suitable for use from C, Objective-C, Swift, and any language with a C FFI.

2. Architecture

2.1 Object Model

Tzopilotl has a three-object architecture:

Object Thread Cardinality Purpose
TypeUniverse Any (init only) 1 per process Shared type interning. All Type* pointers come from here.
Compiler Non-real-time 1 per process Compiles source code. Creates shared compilation targets. Foreign functions are registered here.
VM Any (including RT) 1 per thread Executes compiled code. Owns a TLSF memory pool and incremental GC. Designed to run on a real-time audio thread.
  Host Application
  +-------------------------------------------------+
  |                                                 |
  |   TypeUniverse                                  |
  |       |                                         |
  |   Compiler  <-- registerForeignFunction(...)    |
  |       |                                         |
  |       | compile(source, file, target)           |
  |       v                                         |
  |   CompileResult                                 |
  |       |                                         |
  |       |--- install --->  VM (RT thread)         |
  |       |--- execute -->  VM                      |
  |       |                                         |
  |       |  Host can call vm.callFunction(...)     |
  |       |  VM calls foreign functions via CFun    |
  |                                                 |
  +-------------------------------------------------+

2.2 Compilation Pipeline

  1. Register foreign functions on the Compiler (before any compilation).
  2. Create a compilation target via createTarget(), specifying whether it is RT-restricted.
  3. Compile source code against the target. The type checker validates all function calls, including RT safety. Foreign functions are available to the source code as ordinary functions.
  4. Install the CompileResult into the VM (copies globals, including Primitive and CodeBlock objects).
  5. Execute the main block, or call individual functions by name.

2.3 Word Layout

All values in the VM are stored in a 64-bit Word union. Foreign functions interact with values through this type:

// C++ (vm.hpp)
union Word {
    i64       i;   // Integer, Bool (0 or 1)
    f64       f;   // Float
    SymbolPtr s;   // Symbol (interned string pointer)
    void*     p;   // Generic pointer (e.g. CodeBlock*)
    Obj*      o;   // GC-managed object pointer
};

// C (tzpl.h)
typedef union { int64_t i; double f; void* p; } tzpl_word;

3. C++ Embedding API

Include <tzpl.hpp> for the complete C++ embedding interface. All symbols are in the ts namespace.

3.1 Setup & Lifecycle

#include <tzpl.hpp>

// 1. Create shared type universe (once per process)
ts::TypeUniverse types;

// 2. Create compiler (once per process, non-RT thread)
ts::Compiler compiler(types);

// 3. Create a shared compilation target
ts::VMTarget target = compiler.createTarget();

// 4. Create VM bound to the target (one per thread, can live on RT thread)
ts::VM vm(64 * 1024 * 1024, types, target);  // 64 MB pool

The VM constructor takes a pool size in bytes. This pool is managed by a TLSF allocator—the system allocator is never called after construction.

3.2 Registering Foreign Functions

Foreign functions use the same CFun signature as built-in functions:

using CFun = void (*)(VM&, u16 resultReg, u16 argc, u16 argBase);
ParameterDescription
vmThe executing VM. Use vm.reg(argBase + i) to read argument i.
resultRegRegister to write the return value into: vm.reg(resultReg).
argcNumber of arguments passed by the caller.
argBaseFirst argument register. Arguments are at argBase, argBase+1, …, argBase+argc-1.

Example: A simple addTen function

// Native implementation: addTen(x: Int) -> Int
void addTen(ts::VM& vm, u16 dst, u16 argc, u16 argBase) {
    i64 x = vm.reg(argBase).i;
    vm.reg(dst).i = x + 10;
}

// Register with the compiler
compiler.registerForeignFunction(
    "addTen",                           // name in Tzopilotl
    types.types().intType,               // return type: Int
    { types.types().intType },            // param types: (Int)
    addTen,                              // function pointer
    true,                                // pure (can be constant-folded)
    true                                 // RT-safe
);

After registration, Tzopilotl code can call addTen as an ordinary function:

-- Tzopilotl source
let result = addTen(32);   -- result = 42

registerForeignFunction signature

void Compiler::registerForeignFunction(
    const std::string& name,
    Type* returnType,
    std::vector<Type*> paramTypes,
    CFun cfun,
    bool pure = false,     // pure functions can be constant-folded by the compiler
    bool rtSafe = false    // RT-safe functions can be called from RT-restricted VMs
);
Note: Foreign functions must be registered before compiling any source code that calls them. The type checker resolves foreign function names during compilation.

Reporting errors from a foreign function

A recoverable failure should be returned as a value — the stdlib convention is Option<T> for builtins and Result<T, String> for APIs that carry an error message. For a failure the program must not continue past, print a diagnostic to vm.printOutput() and call vm.haltWithError(): execution stops at the current builtin-call boundary (statements after the call never run), and the host’s script run exits nonzero. This is the same trap behind the language-level panic(message) builtin and unwrap on a none. Plain vm.setHalted(true) is not an error signal — it doubles as the internal “nested execution unit finished” marker used by coroutine resumption, and on its own it does not stop the caller.

// divideExact(a: Int, b: Int) -> Int, halting on a zero divisor
void divideExact(ts::VM& vm, u16 dst, u16 argc, u16 argBase) {
    i64 b = vm.reg(argBase + 1).i;
    if (b == 0) {
        fprintf(vm.printOutput(), "Error: divideExact by zero\n");
        vm.haltWithError();
        return;
    }
    vm.reg(dst).i = vm.reg(argBase).i / b;
}

Example: A function with Float parameters

// lerp(a: Float, b: Float, t: Float) -> Float
void lerp(ts::VM& vm, u16 dst, u16 argc, u16 argBase) {
    f64 a = vm.reg(argBase).f;
    f64 b = vm.reg(argBase + 1).f;
    f64 t = vm.reg(argBase + 2).f;
    vm.reg(dst).f = a + (b - a) * t;
}

compiler.registerForeignFunction(
    "lerp",
    types.types().floatType,
    { types.types().floatType,
      types.types().floatType,
      types.types().floatType },
    lerp,
    true,    // pure
    true     // RT-safe
);

3.3 Compiling & Running

// Compile source code
ts::CompileResult result = compiler.compile(source, "script.x", target);

if (!result.success) {
    // Handle errors
    for (auto& err : result.errors) {
        std::cerr << err.message << std::endl;
    }
    return;
}

// Install globals and execute the main block
vm.makeCurrent();
vm.install(result);
vm.execute(result.mainBlock);

3.4 Calling Language Functions from the Host

After compiling and installing, the host can call Tzopilotl functions by name using VM::callFunction(). The CompileResult contains metadata about all exported functions.

Step 1: Find the function

// Look up a function by name
const auto* ef = ts::findExportedFunction(result, "myHandler");

// Or with overload resolution by parameter types
const auto* ef = ts::findExportedFunction(result, "process",
    { types.types().floatType, types.types().intType });

Step 2: Call it

if (ef && ef->isCodeBlock) {
    // Get the CodeBlock from the VM's globals
    auto* block = static_cast<ts::CodeBlock*>(vm.global(ef->globalIndex).p);

    // Prepare arguments
    ts::Word args[2];
    args[0].f = 440.0;   // Float argument
    args[1].i = 48000;   // Int argument

    // Call — returns value in register 0
    vm.makeCurrent();
    ts::Word result = vm.callFunction(block, args, 2);

    f64 output = result.f;
}

ExportedFunc structure

struct CompileResult::ExportedFunc {
    std::string       name;
    u32              globalIndex;    // index into VM's global array
    std::vector<Type*> paramTypes;
    Type*            returnType;
    bool             isCodeBlock;    // true = user-defined (CodeBlock*)
                                     // false = built-in (Primitive*)
};

callFunction signature

// Call a compiled function. Args are copied into registers.
// Returns the value in register 0.
Word VM::callFunction(CodeBlock* block, const Word* args, u16 argc);
Important: callFunction resets the VM's frame stack and register file. It is designed to be called from the host between events, not reentrantly from within a foreign function.

3.5 User Data

The VM carries an opaque void* pointer that foreign functions can use to access host state (e.g., an audio engine, a game world, a database connection):

struct AudioEngine { /* ... */ };

AudioEngine engine;
vm.setUserData(&engine);

// Inside a foreign function:
void getAmplitude(ts::VM& vm, u16 dst, u16 argc, u16 argBase) {
    auto* engine = static_cast<AudioEngine*>(vm.userData());
    vm.reg(dst).f = engine->getAmplitude();
}
Note: When using the C API, tzpl_vm_create() automatically stores the wrapper pointer in userData. If you call tzpl_vm_set_user_data(), you will overwrite this internal pointer. The C API's foreign function trampoline uses userData to recover the tzpl_vm* wrapper, so setting user data from C requires care. For C API usage, prefer storing host state in a struct and passing a pointer to it via the tzpl_vm_set_user_data() mechanism (the trampoline is designed to work even when user data is replaced, since it recovers the C callback pointer from ffiData_ on the Primitive).

3.6 Full C++ Example

#include <tzpl.hpp>
#include <iostream>

// Foreign function: clamp(x: Float, lo: Float, hi: Float) -> Float
void ffi_clamp(ts::VM& vm, u16 dst, u16 argc, u16 argBase) {
    f64 x  = vm.reg(argBase).f;
    f64 lo = vm.reg(argBase + 1).f;
    f64 hi = vm.reg(argBase + 2).f;
    vm.reg(dst).f = (x < lo) ? lo : (x > hi) ? hi : x;
}

int main() {
    // Setup
    ts::TypeUniverse types;
    ts::Compiler compiler(types);

    auto* Float = types.types().floatType;

    // Register foreign function
    compiler.registerForeignFunction(
        "clamp", Float, { Float, Float, Float },
        ffi_clamp, true, true);

    // Create target and compile
    ts::VMTarget target = compiler.createTarget();
    ts::CompileResult result = compiler.compile(R"(
        fn process(x: Float) -> Float {
            clamp(x * 2.0, -1.0, 1.0)
        }
    )", "dsp.x", target);

    if (!result.success) {
        std::cerr << "Compile failed" << std::endl;
        return 1;
    }

    // Create VM bound to the target, install, and call
    ts::VM vm(64 * 1024 * 1024, types, target);
    vm.makeCurrent();
    vm.install(result);

    auto* ef = ts::findExportedFunction(result, "process");
    if (ef && ef->isCodeBlock) {
        auto* block = static_cast<ts::CodeBlock*>(vm.global(ef->globalIndex).p);
        ts::Word args[1];
        args[0].f = 0.8;
        ts::Word r = vm.callFunction(block, args, 1);
        std::cout << "process(0.8) = " << r.f << std::endl;
        // Output: process(0.8) = 1.0  (0.8 * 2.0 = 1.6, clamped to 1.0)
    }

    return 0;
}

3.7 Object Accessors

When a foreign function receives an object-typed argument (String, Fraction, Complex, Array, Tuple, or Struct), the register contains an Obj* pointer. The accessor API in <tzpl.hpp> provides type-safe functions to read data from these objects without needing to include internal headers:

String

Obj* obj = vm.reg(argBase).o;
const char* data = ts::stringData(obj);   // pointer to UTF-8 bytes
size_t      len  = ts::stringSize(obj);   // byte length

Fraction

int64_t n = ts::fractionNumer(obj);   // numerator
int64_t d = ts::fractionDenom(obj);   // denominator

Complex

double re = ts::complexReal(obj);   // real part
double im = ts::complexImag(obj);   // imaginary part

Array

size_t len = ts::arraySize(obj);

// Use the getter matching the element type:
int64_t val = ts::arrayGetInt(obj, i);     // for [Int] or [Bool]
double  val = ts::arrayGetFloat(obj, i);   // for [Float]
Obj*    val = ts::arrayGetObj(obj, i);     // for [String], [T] where T is object type
Note: Out-of-bounds access returns 0, 0.0, or nullptr respectively.

Tuple

size_t len = ts::tupleSize(obj);
ts::Word  w = ts::tupleGet(obj, i);   // element as Word (use .i, .f, or .o)

Struct

size_t n = ts::structFieldCount(obj);
ts::Word w = ts::structGetField(obj, i);   // field value by index

Full accessor signatures

namespace ts {
// String
const char* stringData(Obj* obj);
size_t      stringSize(Obj* obj);

// Fraction
int64_t fractionNumer(Obj* obj);
int64_t fractionDenom(Obj* obj);

// Complex
double complexReal(Obj* obj);
double complexImag(Obj* obj);

// Array
size_t  arraySize(Obj* obj);
int64_t arrayGetInt(Obj* obj, size_t index);
double  arrayGetFloat(Obj* obj, size_t index);
Obj*    arrayGetObj(Obj* obj, size_t index);

// Tuple
size_t tupleSize(Obj* obj);
Word   tupleGet(Obj* obj, size_t index);

// Struct
size_t structFieldCount(Obj* obj);
Word   structGetField(Obj* obj, size_t index);
}

4. C Embedding API

Include <tzpl.h> for the C embedding interface. All functions are prefixed with tzpl_.

4.1 Opaque Types & Handles

typedef struct tzpl_type_universe tzpl_type_universe;
typedef struct tzpl_compiler      tzpl_compiler;
typedef struct tzpl_vm            tzpl_vm;
typedef struct tzpl_compile_result tzpl_compile_result;
typedef struct tzpl_vm_target     tzpl_vm_target;
typedef struct { void* ptr; }       tzpl_obj_handle;   // opaque object handle
typedef struct { void* ptr; }       tzpl_type_handle;
typedef union  { int64_t i; double f; void* p; } tzpl_word;

These are opaque wrappers around the C++ implementation. The host never accesses internal fields directly.

4.2 Lifecycle

/* Create */
tzpl_type_universe* types    = tzpl_type_universe_create();
tzpl_compiler*      compiler = tzpl_compiler_create(types);
tzpl_vm_target*     target   = tzpl_create_target(compiler, 0);
tzpl_vm*            vm       = tzpl_vm_create(64 * 1024 * 1024, types, target);

/* Compile */
tzpl_compile_result* result = tzpl_compile(compiler, source, "file.x", target);

if (tzpl_compile_result_success(result)) {
    tzpl_vm_install(vm, result);
    tzpl_vm_execute(vm, result);
}

/* Cleanup (reverse order) */
tzpl_compile_result_destroy(result);
tzpl_vm_destroy(vm);
tzpl_target_destroy(target);
tzpl_compiler_destroy(compiler);
tzpl_type_universe_destroy(types);

4.3 Registering Foreign Functions

C foreign functions receive a tzpl_vm* (not a C++ reference) and use accessor functions to read/write registers:

/* C foreign function signature */
typedef void (*tzpl_cfun)(tzpl_vm* vm, uint16_t result_reg,
                            uint16_t argc, uint16_t arg_base);

Example

/* addTen(x: Int) -> Int */
void my_add_ten(tzpl_vm* vm, uint16_t dst,
                 uint16_t argc, uint16_t arg_base) {
    int64_t x = tzpl_vm_reg_int(vm, arg_base, arg_base);
    tzpl_vm_set_reg_int(vm, dst, x + 10);
}

/* Register it */
tzpl_type_handle int_type = tzpl_type_int(types);

tzpl_register_foreign_function(
    compiler,
    "addTen",       /* name */
    int_type,        /* return type */
    &int_type,       /* param types array */
    1,               /* param count */
    my_add_ten,      /* function pointer */
    1,               /* pure */
    1                /* RT-safe */
);

4.4 Type Handles

Use these functions to obtain type handles for foreign function registration:

FunctionLanguage TypeWord Field
tzpl_type_int(types)Int.i
tzpl_type_float(types)Float.f
tzpl_type_bool(types)Bool.i (0 or 1)
tzpl_type_string(types)String.o
tzpl_type_symbol(types)Symbol.s
tzpl_type_void(types)Void(none)

4.5 Register Access

Within a C foreign function, use these to read arguments and write results:

/* Read argument registers */
int64_t          tzpl_vm_reg_int  (tzpl_vm* vm, uint16_t base, uint16_t reg);
double           tzpl_vm_reg_float(tzpl_vm* vm, uint16_t base, uint16_t reg);
tzpl_obj_handle tzpl_vm_reg_obj  (tzpl_vm* vm, uint16_t base, uint16_t reg);

/* Write result register */
void tzpl_vm_set_reg_int  (tzpl_vm* vm, uint16_t reg, int64_t val);
void tzpl_vm_set_reg_float(tzpl_vm* vm, uint16_t reg, double val);
void tzpl_vm_set_reg_obj  (tzpl_vm* vm, uint16_t reg, tzpl_obj_handle obj);
Tip: The base parameter in the read functions is currently unused (reserved for future stack frame support). Pass arg_base for consistency.

Reading arguments pattern

/* For a function f(a: Int, b: Float, c: Int): */
int64_t a = tzpl_vm_reg_int  (vm, arg_base, arg_base + 0);
double  b = tzpl_vm_reg_float(vm, arg_base, arg_base + 1);
int64_t c = tzpl_vm_reg_int  (vm, arg_base, arg_base + 2);

4.6 Object Accessors

The C API provides accessor functions for reading data from object-typed values. First obtain a tzpl_obj_handle from a register, then use the appropriate accessor:

Obtaining an object handle

/* Read an object argument */
tzpl_obj_handle obj = tzpl_vm_reg_obj(vm, arg_base, arg_base + 0);

String

const char* data = tzpl_string_data(obj);   /* pointer to UTF-8 bytes */
size_t      len  = tzpl_string_size(obj);   /* byte length */

Fraction

int64_t n = tzpl_fraction_numer(obj);   /* numerator */
int64_t d = tzpl_fraction_denom(obj);   /* denominator */

Complex

double re = tzpl_complex_real(obj);   /* real part */
double im = tzpl_complex_imag(obj);   /* imaginary part */

Array

size_t len = tzpl_array_size(obj);

/* Use the getter matching the element type: */
int64_t          val = tzpl_array_get_int(obj, i);     /* for [Int] or [Bool] */
double           val = tzpl_array_get_float(obj, i);   /* for [Float] */
tzpl_obj_handle val = tzpl_array_get_obj(obj, i);     /* for [String] etc. */
Note: Out-of-bounds access returns 0, 0.0, or a null handle respectively.

Tuple

size_t     len = tzpl_tuple_size(obj);
tzpl_word w   = tzpl_tuple_get(obj, i);   /* element as tzpl_word */

Struct

size_t     n = tzpl_struct_field_count(obj);
tzpl_word w = tzpl_struct_get_field(obj, i);   /* field value by index */

4.7 Calling Language Functions from C

/* Prepare arguments */
tzpl_word args[2];
args[0].f = 440.0;
args[1].i = 48000;

/* Call by name */
tzpl_word result = tzpl_vm_call(vm, compile_result,
                                    "process", args, 2);

double output = result.f;

tzpl_vm_call looks up the named function in the CompileResult's exported function list, matches by name and argument count, then dispatches to either callFunction (for user-defined functions) or evalPrimitive (for built-in/foreign functions). Returns a zero word if the function is not found.

Argument limit: tzpl_vm_call supports up to 16 arguments. This is a compile-time limit in the implementation.

4.8 User Data

void  tzpl_vm_set_user_data(tzpl_vm* vm, void* data);
void* tzpl_vm_get_user_data(tzpl_vm* vm);

Use these to attach arbitrary host state to the VM. Access it from within foreign functions via tzpl_vm_get_user_data(vm).

4.9 Full C Example

#include "tzpl.h"
#include <stdio.h>

/* Foreign function: square(x: Float) -> Float */
void ffi_square(tzpl_vm* vm, uint16_t dst,
                 uint16_t argc, uint16_t arg_base) {
    double x = tzpl_vm_reg_float(vm, arg_base, arg_base);
    tzpl_vm_set_reg_float(vm, dst, x * x);
}

int main(void) {
    /* Setup */
    tzpl_type_universe* types    = tzpl_type_universe_create();
    tzpl_compiler*      compiler = tzpl_compiler_create(types);

    /* Register foreign function */
    tzpl_type_handle float_t = tzpl_type_float(types);
    tzpl_register_foreign_function(
        compiler, "square", float_t, &float_t, 1,
        ffi_square, 1, 1);

    /* Create target and compile */
    tzpl_vm_target* target = tzpl_create_target(compiler, 0);

    const char* source =
        "fn hypotenuse(a: Float, b: Float) -> Float {\n"
        "    sqrt(square(a) + square(b))\n"
        "}\n";

    tzpl_compile_result* result = tzpl_compile(
        compiler, source, "math.x", target);

    if (!tzpl_compile_result_success(result)) {
        for (int i = 0; i < tzpl_compile_result_error_count(result); i++) {
            fprintf(stderr, "%s\n", tzpl_compile_result_error(result, i));
        }
        return 1;
    }

    /* Create VM bound to target, install, and call */
    tzpl_vm* vm = tzpl_vm_create(64 * 1024 * 1024, types, target);
    tzpl_vm_install(vm, result);

    tzpl_word args[2];
    args[0].f = 3.0;
    args[1].f = 4.0;
    tzpl_word r = tzpl_vm_call(vm, result, "hypotenuse", args, 2);
    printf("hypotenuse(3, 4) = %f\n", r.f);
    /* Output: hypotenuse(3, 4) = 5.000000 */

    /* Cleanup */
    tzpl_compile_result_destroy(result);
    tzpl_vm_destroy(vm);
    tzpl_target_destroy(target);
    tzpl_compiler_destroy(compiler);
    tzpl_type_universe_destroy(types);

    return 0;
}

5. Real-Time Safety

5.1 Safety Model

Tzopilotl is designed to run on real-time audio threads where blocking system calls are forbidden. The FFI extends this with a per-function rtSafe flag and compile-time enforcement:

The rtSafe flag is a contract, not an enforcement mechanism at the C++ level. If you mark a foreign function as RT-safe, it is your responsibility to ensure it does not call malloc, printf, acquire mutexes, perform file I/O, or make any other potentially blocking system call. The flag controls only compile-time access control.

5.2 Compile-Time Enforcement

When a target is created with rtRestricted = true, the type checker rejects all calls to non-RT-safe functions:

// C++: create an RT-restricted target
ts::VMTarget target = compiler.createTarget(true);

// C: create an RT-restricted target
tzpl_vm_target* target = tzpl_create_target(compiler, 1);

With RT restriction enabled, compiling this code:

gc();   -- gc is not RT-safe

Produces a compile error:

Type error [script.x:1:1]
  |
1 | gc();
  | ^^
  Function 'gc' is not real-time safe and cannot be called
  when compiling for a real-time VM

Meanwhile, most built-in functions (including print and println for debugging) and all user-defined functions that don't transitively call non-RT-safe functions compile without error.

5.3 Built-in Function RT Classification

CategoryFunctionsRT-Safe
Math sin, cos, tan, asin, acos, atan, atan2, sqrt, cbrt, exp, log, log2, log10, pow, abs, floor, ceil, round, trunc, min, max, clamp, hypot, etc. Yes
Arithmetic add, sub, mul, div, mod, negate, etc. Yes
Collections len, head, tail, append, reverse, sort, map, filter, fold, etc. Yes
Conversions toInt, toFloat, toString, etc. Yes
Random urand, brand, irand, rand, xrand, pick, picks Yes
I/O print, println, fmt Yes
Debug / GC gc, disassemble No

All user-defined functions inherit RT safety transitively. If a user function only calls RT-safe functions, it is itself RT-safe.

5.4 CLI Flag

The interpreter's command-line interface supports an --rt flag for testing RT safety enforcement:

# This will produce a compile error:
$ tzpl --rt script_with_println.x

# This will succeed (only RT-safe functions used):
$ tzpl --rt dsp_code.x

Test files can use the -- @rt marker on the first line to automatically enable RT mode in the test runner:

-- @rt
let x = sin(3.14);
let y = cos(0.0);
let z = x + y;

6. Type Mapping Reference

This table shows how Tzopilotl types correspond to C/C++ access patterns within foreign functions:

Language Type Word Field C++ Access (in CFun) C Access (in tzpl_cfun)
Bool .i vm.reg(argBase + i).i (0 or 1) tzpl_vm_reg_int(vm, base, reg)
Int .i vm.reg(argBase + i).i tzpl_vm_reg_int(vm, base, reg)
Float .f vm.reg(argBase + i).f tzpl_vm_reg_float(vm, base, reg)
Symbol .s vm.reg(argBase + i).s (use C++ API)
String .o ts::stringData(obj), ts::stringSize(obj) tzpl_string_data(h), tzpl_string_size(h)
Fraction .o ts::fractionNumer(obj), ts::fractionDenom(obj) tzpl_fraction_numer(h), tzpl_fraction_denom(h)
Complex .o ts::complexReal(obj), ts::complexImag(obj) tzpl_complex_real(h), tzpl_complex_imag(h)
[T] .o ts::arraySize(obj), ts::arrayGetInt/Float/Obj(obj, i) tzpl_array_size(h), tzpl_array_get_int/float/obj(h, i)
Tuple .o ts::tupleSize(obj), ts::tupleGet(obj, i) tzpl_tuple_size(h), tzpl_tuple_get(h, i)
Struct .o ts::structFieldCount(obj), ts::structGetField(obj, i) tzpl_struct_field_count(h), tzpl_struct_get_field(h, i)
Void (none) No return value; resultReg is ignored No return value
Note: For the C++ API, obj is an Obj* read from vm.reg(argBase + i).o. For the C API, h is a tzpl_obj_handle obtained via tzpl_vm_reg_obj(vm, base, reg).

Writing return values

// C++ — write into the result register
vm.reg(resultReg).i = 42;       // Int or Bool
vm.reg(resultReg).f = 3.14;     // Float
vm.reg(resultReg).o = strObj;   // String or other Obj*

// C — write into the result register
tzpl_vm_set_reg_int(vm, dst, 42);
tzpl_vm_set_reg_float(vm, dst, 3.14);
tzpl_vm_set_reg_obj(vm, dst, obj_handle);

7. Internals

This section describes the internal implementation for contributors and advanced users.

7.1 Registration Flow

When registerForeignFunction() is called on the Compiler, it stores a ForeignFuncEntry:

struct ForeignFuncEntry {
    std::string          name;
    Type*               returnType;
    std::vector<Type*>  paramTypes;
    CFun                cfun;
    bool                pure;
    bool                rtSafe;
    void*               ffiData = nullptr;   // stored on Primitive for trampolines
};

During compilation, the TypeChecker calls registerBuiltins(), which iterates the compiler's foreign function list and installs each one as a Primitive global:

// In TypeChecker::registerBuiltins()
for (auto& entry : compiler_.foreignFunctions()) {
    u32 idx = compiler_.addGlobal(true);
    auto* prim = new Primitive(compiler_.voidType());
    prim->cfun_    = entry.cfun;
    prim->pure_    = entry.pure;
    prim->rtSafe_  = entry.rtSafe;
    prim->ffiData_ = entry.ffiData;
    compiler_.global(idx).o = prim;

    FuncInfo info;
    info.returnType  = entry.returnType;
    info.paramTypes  = entry.paramTypes;
    info.globalIndex = idx;
    info.bodyChecked = true;
    info.isBuiltin   = true;
    info.rtSafe      = entry.rtSafe;
    functions_[entry.name].push_back(info);
}

From this point on, foreign functions are indistinguishable from built-in functions. The overload resolver, auto-mapper, and pipeline operator all work with them.

7.2 C API Trampoline

The C API cannot use ts::VM& references directly. A trampoline bridges the gap:

Tzopilotl code calls foreign function
        |
        v
  VM dispatches via Primitive::cfun_
        |
        v
  c_ffi_trampoline(VM&, dst, argc, argBase)
        |
        |-- Recovers C callback from Primitive::ffiData_
        |-- Recovers tzpl_vm* from VM::userData()
        |
        v
  user's tzpl_cfun(tzpl_vm*, dst, argc, argBase)
        |
        v
  User reads args via tzpl_vm_reg_int/float()
  User writes result via tzpl_vm_set_reg_int/float()

The trampoline implementation:

// Recover the C wrapper from the VM's userData
static tzpl_vm* to_tzpl_vm(ts::VM* vm) {
    return static_cast<tzpl_vm*>(vm->userData());
}

// Trampoline: VM calls this, it forwards to the C callback
static void c_ffi_trampoline(ts::VM& vm, u16 dst, u16 argc, u16 argBase) {
    auto* prim = vm.currentPrimitive();
    auto cfun = reinterpret_cast<tzpl_cfun>(prim->ffiData_);
    auto* wrapper = to_tzpl_vm(&vm);
    cfun(wrapper, dst, argc, argBase);
}

Key points:

7.3 Exported Functions

At the end of Compiler::compile(), the function table is iterated to populate CompileResult::exportedFunctions:

for (auto& [name, overloads] : typeChecker.functions()) {
    for (auto& fi : overloads) {
        if (fi.isTemplate) continue;  // skip unresolved templates
        CompileResult::ExportedFunc ef;
        ef.name        = name;
        ef.globalIndex = fi.globalIndex;
        ef.paramTypes  = fi.paramTypes;
        ef.returnType  = fi.returnType;
        ef.isCodeBlock = !fi.isBuiltin;
        result.exportedFunctions.push_back(std::move(ef));
    }
}

The isCodeBlock field distinguishes user-defined functions (stored as CodeBlock* in the global) from built-in/foreign functions (stored as Primitive*). The host uses this to choose between VM::callFunction() and VM::evalPrimitive().

8. API Reference

8.1 C++ API Reference

Compiler

MethodDescription
VMTarget createTarget(bool rtRestricted = false) Create a shared compilation target. Set rtRestricted to true to enable RT safety checking. Lifetime is managed by shared_ptr.
bool isRTRestricted() const Query whether the current compilation target is RT-restricted.
void registerForeignFunction(name, returnType, paramTypes, cfun, pure, rtSafe) Register a host-provided function. See Section 3.2.
const vector<ForeignFuncEntry>& foreignFunctions() const Access the registered foreign function list.
CompileResult compile(source, filename, target) Compile source code against a target. Returns success/failure, errors, and exported function metadata.

VM

MethodDescription
void setUserData(void* data) Store an opaque pointer accessible from foreign functions.
void* userData() const Retrieve the stored user data pointer.
Word callFunction(CodeBlock* block, const Word* args, u16 argc) Call a compiled function. Resets the frame stack. Returns register 0.
Word evalPrimitive(Primitive* prim, const Word* args, u16 argc) Evaluate a Primitive with given arguments. Used for calling built-in/foreign functions directly.
Primitive* currentPrimitive() const The Primitive currently being executed (set by the opcode dispatcher).
Word& reg(u16 r) Access register r in the current frame.
Word& global(u32 index) Access global variable at index.

CompileResult

FieldTypeDescription
success bool Whether compilation succeeded.
errors vector<Error> Compile errors (if any).
mainBlock CodeBlock* The compiled top-level code.
exportedFunctions vector<ExportedFunc> Metadata for all callable functions. See Section 7.3.

Free Functions

FunctionDescription
const ExportedFunc* findExportedFunction(result, name, paramTypes = {}) Look up an exported function by name (and optionally by parameter types for overload resolution). Returns nullptr if not found.

Object Accessors

FunctionDescription
const char* stringData(Obj* obj)Pointer to the String's UTF-8 bytes.
size_t stringSize(Obj* obj)Byte length of the String.
int64_t fractionNumer(Obj* obj)Fraction numerator.
int64_t fractionDenom(Obj* obj)Fraction denominator.
double complexReal(Obj* obj)Complex number real part.
double complexImag(Obj* obj)Complex number imaginary part.
size_t arraySize(Obj* obj)Number of elements in the Array.
int64_t arrayGetInt(Obj* obj, size_t index)Get Int element from [Int] or [Bool]. Returns 0 if out of bounds.
double arrayGetFloat(Obj* obj, size_t index)Get Float element from [Float]. Returns 0.0 if out of bounds.
Obj* arrayGetObj(Obj* obj, size_t index)Get object element from [T] where T is an object type. Returns nullptr if out of bounds.
size_t tupleSize(Obj* obj)Number of elements in the Tuple.
Word tupleGet(Obj* obj, size_t index)Get Tuple element as Word. Returns zero Word if out of bounds.
size_t structFieldCount(Obj* obj)Number of fields in the Struct.
Word structGetField(Obj* obj, size_t index)Get Struct field value by index. Returns zero Word if out of bounds.

8.2 C API Reference

TypeUniverse

FunctionDescription
tzpl_type_universe* tzpl_type_universe_create(void)Create a shared type universe.
void tzpl_type_universe_destroy(tzpl_type_universe*)Destroy a type universe.

Compiler

FunctionDescription
tzpl_compiler* tzpl_compiler_create(tzpl_type_universe*)Create a compiler.
void tzpl_compiler_destroy(tzpl_compiler*)Destroy a compiler.
tzpl_vm_target* tzpl_create_target(tzpl_compiler*, int rt_restricted)Create a shared compilation target. Pass non-zero for RT safety enforcement.
void tzpl_target_destroy(tzpl_vm_target*)Destroy a compilation target.
tzpl_compile_result* tzpl_compile(tzpl_compiler*, const char* source, const char* filename, tzpl_vm_target*)Compile source code. Caller owns the result.

CompileResult

FunctionDescription
int tzpl_compile_result_success(const tzpl_compile_result*)Returns 1 on success, 0 on failure.
int tzpl_compile_result_error_count(const tzpl_compile_result*)Number of compile errors.
const char* tzpl_compile_result_error(const tzpl_compile_result*, int index)Get error message at index. Valid until result is destroyed.
void tzpl_compile_result_destroy(tzpl_compile_result*)Destroy a compile result.

Type Handles

FunctionReturns
tzpl_type_handle tzpl_type_int(tzpl_type_universe*)Handle for Int type
tzpl_type_handle tzpl_type_float(tzpl_type_universe*)Handle for Float type
tzpl_type_handle tzpl_type_bool(tzpl_type_universe*)Handle for Bool type
tzpl_type_handle tzpl_type_string(tzpl_type_universe*)Handle for String type
tzpl_type_handle tzpl_type_symbol(tzpl_type_universe*)Handle for Symbol type
tzpl_type_handle tzpl_type_void(tzpl_type_universe*)Handle for Void type

Foreign Function Registration

FunctionDescription
int tzpl_register_foreign_function(tzpl_compiler*, const char* name, tzpl_type_handle return_type, const tzpl_type_handle* param_types, int param_count, tzpl_cfun cfun, int pure, int rt_safe) Register a foreign function. Returns 1 on success. pure: function can be constant-folded. rt_safe: function is safe for RT VMs.

VM

FunctionDescription
tzpl_vm* tzpl_vm_create(size_t pool_size, tzpl_type_universe*, tzpl_vm_target*)Create a VM bound to a target, with given pool size.
void tzpl_vm_destroy(tzpl_vm*)Destroy a VM.
void tzpl_vm_make_current(tzpl_vm*)Set this VM as current for the calling thread.
void tzpl_vm_install(tzpl_vm*, const tzpl_compile_result*)Install compiled code into the VM.
void tzpl_vm_execute(tzpl_vm*, const tzpl_compile_result*)Execute the main block.
void tzpl_vm_gc_heartbeat(tzpl_vm*)Drive one GC heartbeat. Call on timer or audio callback.
void tzpl_vm_set_user_data(tzpl_vm*, void*)Set user data pointer.
void* tzpl_vm_get_user_data(tzpl_vm*)Get user data pointer.
tzpl_word tzpl_vm_call(tzpl_vm*, const tzpl_compile_result*, const char* name, const tzpl_word* args, int argc)Call a language function by name. Returns result word (zero if not found).

Register Access

FunctionDescription
int64_t tzpl_vm_reg_int(tzpl_vm*, uint16_t base, uint16_t reg)Read integer value from register.
double tzpl_vm_reg_float(tzpl_vm*, uint16_t base, uint16_t reg)Read float value from register.
tzpl_obj_handle tzpl_vm_reg_obj(tzpl_vm*, uint16_t base, uint16_t reg)Read object handle from register.
void tzpl_vm_set_reg_int(tzpl_vm*, uint16_t reg, int64_t val)Write integer value to register.
void tzpl_vm_set_reg_float(tzpl_vm*, uint16_t reg, double val)Write float value to register.
void tzpl_vm_set_reg_obj(tzpl_vm*, uint16_t reg, tzpl_obj_handle obj)Write object handle to register.

Object Accessors

FunctionDescription
const char* tzpl_string_data(tzpl_obj_handle obj)Pointer to the String's UTF-8 bytes.
size_t tzpl_string_size(tzpl_obj_handle obj)Byte length of the String.
int64_t tzpl_fraction_numer(tzpl_obj_handle obj)Fraction numerator.
int64_t tzpl_fraction_denom(tzpl_obj_handle obj)Fraction denominator.
double tzpl_complex_real(tzpl_obj_handle obj)Complex number real part.
double tzpl_complex_imag(tzpl_obj_handle obj)Complex number imaginary part.
size_t tzpl_array_size(tzpl_obj_handle obj)Number of elements in the Array.
int64_t tzpl_array_get_int(tzpl_obj_handle obj, size_t index)Get Int element. Returns 0 if out of bounds.
double tzpl_array_get_float(tzpl_obj_handle obj, size_t index)Get Float element. Returns 0.0 if out of bounds.
tzpl_obj_handle tzpl_array_get_obj(tzpl_obj_handle obj, size_t index)Get object element. Returns null handle if out of bounds.
size_t tzpl_tuple_size(tzpl_obj_handle obj)Number of elements in the Tuple.
tzpl_word tzpl_tuple_get(tzpl_obj_handle obj, size_t index)Get Tuple element as tzpl_word. Returns zero if out of bounds.
size_t tzpl_struct_field_count(tzpl_obj_handle obj)Number of fields in the Struct.
tzpl_word tzpl_struct_get_field(tzpl_obj_handle obj, size_t index)Get Struct field by index. Returns zero if out of bounds.

Stats

FunctionDescription
uint32_t tzpl_vm_num_globals(const tzpl_vm*)Number of globals in the VM.
size_t tzpl_vm_allocated(const tzpl_vm*)Bytes currently allocated.
size_t tzpl_vm_pool_size(const tzpl_vm*)Total pool size.
uint32_t tzpl_vm_num_live_objects(const tzpl_vm*)Number of live GC objects.
uint32_t tzpl_vm_num_live_words(const tzpl_vm*)Number of live GC words.
void tzpl_vm_set_print_output(tzpl_vm*, FILE*)Redirect print output.

9. OSC Integration

9.1 Overview

Tzopilotl includes built-in Open Sound Control (OSC) support for controlling the audio engine and sending messages over the network. OSC functions are available when the application is built with TZPL_BUILD_OSC=ON.

The OSC system has three components:

All OSC functions are registered under the osc foreign module. Import them with:

import osc.*;

9.2 Setup & Configuration

The OSC server can be started in two ways:

CLI flag

tzpl --osc-port 57120 myscript.x

Project config file

In a project directory's config file, add:

oscPort = 57120

Setting the port to 0 disables the OSC server. The server starts automatically after the audio engine is initialized and shuts down cleanly on exit.

From Tzopilotl code

The server can also be started and stopped programmatically:

import osc.*;

oscServerStart(57120);  -- start listening on port 57120
oscServerPort() println;  -- prints 57120
oscServerStop();           -- stop the server

9.3 Remote Sending

Send OSC messages to a remote host over UDP. Each variant sends a different argument type:

FunctionDescription
oscSend(host String, port Int, address String) VoidSend a message with no arguments.
oscSendI(host String, port Int, address String, value Int) VoidSend a message with one Int argument.
oscSendF(host String, port Int, address String, value Float) VoidSend a message with one Float argument.
oscSendS(host String, port Int, address String, value String) VoidSend a message with one String argument.
oscSendArgs(host String, port Int, address String, args [Float]) VoidSend a message with an array of Float arguments.

Example: sending to another application

import osc.*;

-- Send a note-on to a remote synth
oscSendArgs("192.168.1.50", 57120, "/note/on", [60.0, 0.8]);

-- Send a control change
oscSendF("localhost", 9000, "/control/cutoff", 1200.0);

9.4 Local Sending

Local send functions dispatch OSC messages directly to the in-process dispatcher, bypassing the network entirely. This is the recommended way to control the audio engine from Tzopilotl code via OSC addresses.

FunctionDescription
oscSendLocal(address String) VoidDispatch a message with no arguments.
oscSendLocalI(address String, value Int) VoidDispatch a message with one Int argument.
oscSendLocalF(address String, value Float) VoidDispatch a message with one Float argument.
oscSendLocalS(address String, value String) VoidDispatch a message with one String argument.
oscSendLocalArgs(address String, args [Float]) VoidDispatch a message with an array of Float arguments.

Example: controlling the engine locally

import osc.*;

-- Start audio via local OSC dispatch
oscSendLocal("/engine/startAudio");

-- Set master gain to 0.5
oscSendLocalF("/engine/masterGain", 0.5);

-- Create a node and set a control
oscSendLocalArgs("/engine/newNode", [1.0]);  -- not ideal for mixed types; prefer the FFI bridge for this
Tip: For engine operations with mixed argument types (String + Int, etc.), the direct FFI functions in the audio_engine module (newNode, connect, etc.) are more convenient. Local OSC dispatch is most useful for simple commands and for testing OSC address routing.

9.5 Server Control

FunctionDescription
oscServerStart(port Int) BoolStart the OSC server on the given UDP port. Returns true on success.
oscServerStop() VoidStop the OSC server.
oscServerPort() IntReturn the port the server is listening on, or 0 if stopped.
Note: All OSC functions are registered as non-RT-safe. They must not be called from a real-time context.

9.6 Handler Registration

Register Tzopilotl functions as handlers for incoming OSC messages. When a message arrives at the registered address, the handler is called with the parsed OSC arguments via the NRTVM (mutex-serialized, non-RT).

FunctionDescription
onMessage(address String, handler fn() Void) VoidRegister a no-argument handler for the given OSC address.
onMessageI(address String, handler fn(Int) Void) VoidRegister a handler that receives one integer argument.
onMessageF(address String, handler fn(Float) Void) VoidRegister a handler that receives one float argument.
onMessageS(address String, handler fn(String) Void) VoidRegister a handler that receives one string argument.
onMessageArgs(address String, handler fn([Float]) Void) VoidRegister a handler that receives all OSC arguments as a float array.
removeHandler(address String) VoidRemove a previously registered handler.
import osc;

-- respond to /trigger by printing a message
osc.onMessage("/trigger", fn() {
    println("triggered!");
});

-- respond to /volume with a float value
osc.onMessageF("/volume", fn(level Float) {
    println("volume: " $ level toString);
});

-- respond to /note with an integer (MIDI note number)
osc.onMessageI("/note", fn(note Int) {
    println("note: " $ note toString);
});

-- later, remove the handler
osc.removeHandler("/trigger");
Note: Handlers stay alive as long as they are in the runtime's handler table — the table is a GC root, so a handler with no other reference (e.g. a top-level lambda passed straight to osc.onMessage) is kept reachable for the collector. Registering a new handler for the same address replaces the previous one; removeHandler drops the entry, after which the prior handler becomes collectable if nothing else holds it. All handler invocations are serialized through the NRTVM mutex, so handlers never run concurrently.

9.7 VM OSC Addresses

The following built-in OSC addresses allow external applications to interact with the Tzopilotl VM:

AddressArgumentsDescription
/tzpl/evals sourceCompile and execute Tzopilotl source code. Replies with /tzpl/eval/ok on success or /tzpl/eval/error <message> on failure.
/tzpl/calls address, [i|f|s]... argsInvoke a user-registered handler by its OSC address, passing optional typed arguments.
-- From an external OSC client (e.g. Python, Max/MSP, SuperCollider):
--
-- Send to /tzpl/eval with a string argument:
--   /tzpl/eval "println(42 * 2);"
--
-- The VM compiles and runs the code, then replies:
--   /tzpl/eval/ok          (on success)
--   /tzpl/eval/error "..." (on failure)
--
-- Invoke a user-registered handler:
--   /tzpl/call "/volume" 0.75
Tip: /tzpl/eval is the primary mechanism for interactive coding over the network. Combined with osc.onMessage*(), it enables workflows where code and control messages are sent from external editors, controllers, or other applications.

9.8 Engine OSC Addresses

When the OSC server is running, the following addresses control the audio engine. These are the same commands available through the audio engine FFI, but accessible from any OSC client (external applications, hardware controllers, other machines on the network).

Lifecycle & Configuration

AddressArgumentsDescription
/engine/startAudiononeStart audio output.
/engine/stopAudiononeStop audio output.
/engine/masterGainf gainSet master gain.
/engine/safetyLimiteri on/offEnable (1) or disable (0) the safety limiter.
/engine/loadDefss pathLoad all plugin definitions from a directory.
/engine/loadDefs path, s nameLoad a single plugin definition.

Queries

Query commands send a reply back to the sender's address and port:

AddressReply AddressReply Arguments
/engine/getStreamTime/reply/getStreamTimef time
/engine/listNodeDefs/reply/listNodeDefss... names
/engine/isAudioRunning/reply/isAudioRunningi 0 or 1

Graph Manipulation

Standalone graph commands are automatically wrapped in a begin()/go(0) transaction. Inside a bundle, they are batched together.

AddressArgumentsDescription
/engine/newNodes defName, i nodeIDCreate a new node.
/engine/freeNodei nodeIDFree a node.
/engine/freeAllNodesnoneFree all nodes.
/engine/replaceNodei old, i new, f xfade, i curveReplace a node with crossfade.
/engine/connecti srcNode, i srcPort, i dstNode, i dstPortConnect two nodes.
/engine/connectXi i i i f iConnect with crossfade and curve.
/engine/disconnectInputi nodeID, i portDisconnect an input.
/engine/disconnectInputXi i f iDisconnect input with crossfade.
/engine/disconnectOutputi nodeID, i portDisconnect an output.
/engine/disconnectNodei nodeIDDisconnect all connections of a node.
/engine/reconnectOutputi i i i f iReconnect an output to a new source with crossfade.
/engine/setInputi nodeID, i port, f valueSet an input to a constant value.
/engine/setInputXi i f f iSet input with crossfade.
/engine/setControli nodeID, i controlID or s name, f valueSet a control parameter, addressed by controlID or by name (a string argument is resolved against the node's def at submit).

Notes

AddressArgumentsDescription
/engine/noteOni nodeID, i noteID, f... paramsTrigger a note with parameters.
/engine/noteOffi nodeID, i noteIDRelease a note.
/engine/allNotesOffi nodeIDRelease all notes on a node.
/engine/noteSetParamsi nodeID, i noteID, i firstParam, f... valuesUpdate parameters on a playing note.

9.9 Bundles & Scheduling

OSC bundles group multiple messages into an atomic transaction with an optional timetag for scheduling:

Silo selection

By default, bundle commands target silo 0. To target a different silo, include /engine/silo <int> as the first element of the bundle:

-- OSC bundle (pseudo-notation):
-- #bundle timetag
--   /engine/silo 1
--   /engine/newNode "sine" 100
--   /engine/connect 100 0 0 0

Auto-wrapping

Graph manipulation commands sent as standalone messages (outside a bundle) are automatically wrapped in a begin()/go(0) transaction so they execute atomically. Inside a bundle, this auto-wrapping is suppressed and the bundle provides the transaction boundary; the /engine/silo element selects the silo passed at submit.

Note: Bundles can be nested. Inner bundles inherit the outer bundle's transaction context.

10. NATS Integration

10.1 Overview

Tzopilotl includes built-in NATS messaging support for networked control and distributed communication. NATS functions are available when the application is built with TZPL_BUILD_NATS=ON.

NATS is a high-performance publish/subscribe messaging system. Compared to OSC (UDP-based, connectionless), NATS provides:

The NATS system has two components:

All NATS functions are registered under the nats foreign module. Import them with:

import nats.*;
Note: NATS requires a running NATS server. Install with brew install nats-server and start with nats-server. The C client library is also required: brew install cnats.

10.2 Setup & Configuration

The NATS client can be connected in two ways:

CLI flag

tzpl --nats-url nats://127.0.0.1:4222 myscript.x
tzpl --nats-url nats://127.0.0.1:4222 --engine-name piano myscript.x

Project config file

In a project directory's config file, add:

natsUrl = "nats://127.0.0.1:4222"
engineName = "piano"

Leaving natsUrl empty or omitting it disables NATS. When a URL is provided, the client connects after all subsystems are initialized. Engine command handlers and VM handlers are subscribed automatically.

From Tzopilotl code

The connection can also be managed programmatically:

import nats.*;

natsConnect("nats://127.0.0.1:4222");  -- connect to server
natsIsConnected() println;              -- prints true
natsUrl() println;                      -- prints nats://127.0.0.1:4222
natsDisconnect();                        -- disconnect

10.3 Connection Management

FunctionDescription
natsConnect(url String) BoolConnect to a NATS server. Returns true on success. Automatically subscribes all registered handlers.
natsDisconnect() VoidDisconnect from the NATS server. Unsubscribes all active subscriptions.
natsIsConnected() BoolReturn true if the client is currently connected.
natsUrl() StringReturn the URL of the current connection, or "" if not connected.
natsEngineName() StringReturn the engine name set via --engine-name or config, or "" if unset.
Note: All NATS functions are registered as non-RT-safe. They must not be called from a real-time context.

10.4 Publishing

Publish messages to NATS subjects. The payload is a string (for typed variants, the value is converted to its string representation).

FunctionDescription
natsPub(subject String, data String) VoidPublish a string payload to a subject.
natsPubI(subject String, value Int) VoidPublish an integer (as its string representation).
natsPubF(subject String, value Float) VoidPublish a float (as its string representation).
natsRequest(subject String, data String, timeoutMs Int, handler fn(String) Void) VoidAsync request/reply: publishes a request and invokes the handler with the reply payload when it arrives. Returns immediately. Uses the nats.c delivery thread internally (no extra threads spawned). The timeoutMs parameter is reserved for future use.
natsPubMsg(subject String, msg Bytes) VoidPublish a binary message — the exact bytes of a Bytes buffer, including embedded NULs (e.g. an encoded Msg message). Length-preserving, unlike the string variants.
natsRequestMsg(subject String, msg Bytes, timeoutMs Int, handler fn(Bytes) Void) VoidBinary request/reply: the reply payload is delivered to the handler as Bytes.

Example: publishing messages

import nats.*;

-- Send a control message
natsPub("synth.cutoff", "1200.0");

-- Send typed values
natsPubF("synth.volume", 0.8);
natsPubI("synth.note", 60);

-- Async request/reply (1 second timeout)
natsRequest("engine.isAudioRunning", "", 1000, fn(reply String) {
    println("audio running: " $ reply);
});

10.5 Handler Registration

Register Tzopilotl functions as handlers for incoming NATS messages. When a message arrives on the subscribed subject, the handler is called via the NRTVM (mutex-serialized, non-RT). The message payload is passed as the argument.

FunctionDescription
onMessage(subject String, handler fn() Void) VoidRegister a no-argument handler (payload ignored).
onMessageS(subject String, handler fn(String) Void) VoidRegister a handler that receives the payload as a string.
onMessageI(subject String, handler fn(Int) Void) VoidRegister a handler that parses the payload as an integer.
onMessageF(subject String, handler fn(Float) Void) VoidRegister a handler that parses the payload as a float.
onMessageMsg(subject String, handler fn(Bytes) Void) VoidRegister a handler that receives the raw payload as a Bytes buffer (length-preserving, embedded NULs intact). The handler typically decodes it to an Msg or reads it via a Reader.
removeHandler(subject String) VoidRemove a previously registered handler and unsubscribe from the subject.
import nats;

-- respond to a trigger subject
nats.onMessage("trigger", fn() {
    println("triggered!");
});

-- respond to a string message
nats.onMessageS("chat", fn(msg String) {
    println("received: " $ msg);
});

-- respond to a float value
nats.onMessageF("volume", fn(level Float) {
    println("volume: " $ level toString);
});

-- later, remove the handler
nats.removeHandler("trigger");
Note: Handlers stay alive as long as they are in the runtime's handler table — the table is a GC root, so a handler with no other reference (e.g. a top-level lambda passed straight to nats.onMessage) is kept reachable for the collector. Registering a new handler for the same subject replaces the previous one; removeHandler drops the entry, after which the prior handler becomes collectable if nothing else holds it. All handler invocations are serialized through the NRTVM mutex. If the client is already connected when a handler is registered, the subscription is created immediately.

10.6 VM NATS Subjects

The following built-in NATS subjects allow external applications to interact with the Tzopilotl VM:

SubjectPayloadDescription
tzpl.evalTzopilotl source codeCompile and execute source code. If a NATS reply subject is provided, replies with "ok" on success or "error: <message>" on failure.
tzpl.callsubject arg1 arg2 ...Invoke a user-registered handler by its NATS subject, passing space-separated arguments.

Example: remote eval via NATS

-- From an external NATS client (e.g. nats-cli, Python, Go):
--
-- Publish source code to tzpl.eval:
--   nats pub tzpl.eval 'println(42 * 2);'
--
-- Request/reply (with response):
--   nats request tzpl.eval 'println(42 * 2);'
--   -> "ok"
--
-- Invoke a user-registered handler:
--   nats pub tzpl.call 'volume 0.75'
Tip: tzpl.eval is the primary mechanism for interactive coding over NATS. Combined with nats.onMessage*(), it enables workflows across machines on a network, with NATS providing reliable delivery and automatic reconnection.

10.7 Engine NATS Subjects

When connected to a NATS server, the following subjects control the audio engine. These mirror the OSC engine addresses but use dot-separated NATS subject names and space-separated text payloads.

Lifecycle & Configuration

SubjectPayloadDescription
engine.startAudiononeStart audio output.
engine.stopAudiononeStop audio output.
engine.masterGaingainSet master gain (float).
engine.safetyLimiter0|1Enable (1) or disable (0) the safety limiter.
engine.loadDefspathLoad all plugin definitions from a directory.
engine.loadDefpath nameLoad a single plugin definition.

Queries

Query commands reply via the NATS reply subject (use request/reply pattern):

SubjectReply Payload
engine.getStreamTimeStream time as a decimal number.
engine.listNodeDefsSpace-separated list of definition names.
engine.isAudioRunning0 or 1.

Graph Manipulation

Each graph command is automatically wrapped in a begin()/go(0) transaction. Payloads are space-separated values.

SubjectPayloadDescription
engine.newNodedefName nodeIDCreate a new node.
engine.freeNodenodeIDFree a node.
engine.freeAllNodesnoneFree all nodes.
engine.replaceNodeoldID newID xfade curveReplace a node with crossfade.
engine.connectsrcNode srcPort dstNode dstPortConnect two nodes.
engine.connectXsrcNode srcPort dstNode dstPort xfade curveConnect with crossfade and curve.
engine.disconnectInputnodeID portDisconnect an input.
engine.disconnectOutputnodeID portDisconnect an output.
engine.disconnectNodenodeIDDisconnect all connections of a node.
engine.setInputnodeID port valueSet an input to a constant value.
engine.setControlnodeID controlID|name valueSet a control parameter. The second argument is a controlID if it is all digits, otherwise a control name (resolved against the node's def at submit).

Notes

SubjectPayloadDescription
engine.noteOnnodeID noteID [param...]Trigger a note with optional float parameters.
engine.noteOffnodeID noteIDRelease a note.
engine.allNotesOffnodeIDRelease all notes on a node.

Example: controlling the engine from the command line

-- Using the nats CLI tool:
--
-- Start audio
--   nats pub engine.startAudio ''
--
-- Create a node
--   nats pub engine.newNode 'sine 100'
--
-- Connect it to the output
--   nats pub engine.connect '100 0 0 0'
--
-- Query if audio is running
--   nats request engine.isAudioRunning ''
--   -> 1
--
-- Set master gain
--   nats pub engine.masterGain '0.5'

10.8 Distributed Engines

When running multiple Tzopilotl instances on different machines (or in separate processes), NATS provides a natural coordination layer. Each instance can be given a unique engine name so that commands and messages can be addressed to specific engines.

Engine naming

Set the engine name via CLI or config file:

tzpl --nats-url nats://server:4222 --engine-name piano piano_script.x
tzpl --nats-url nats://server:4222 --engine-name drums drums_script.x

When an engine name is set, all engine command handlers and VM handlers are registered under three subject prefixes:

PrefixExampleReaches
engine.*engine.newNodeThe local engine (always registered, backward-compatible).
engines.{name}.*engines.piano.engine.newNodeOnly the engine named "piano".
engines.all.*engines.all.engine.newNodeAll named engines (broadcast).

Sending commands to a specific engine

import nats.*;

-- Send a command to the "drums" engine
natsPub("engines.drums.engine.newNode", "kick 100");

-- Eval code on the "piano" engine
natsPub("engines.piano.tzpl.eval", "println(42);");

-- Broadcast: stop audio on all engines
natsPub("engines.all.engine.stopAudio", "");

-- Query your own engine name
let name = natsEngineName();
println("I am: " $ name);

Language-to-language messaging

Tzopilotl code on different machines can communicate freely using natsPub / onMessageS with any subject convention. A recommended pattern:

import nats.*;

-- On the "piano" engine: listen for chord changes
nats.onMessageS("nodes.piano.chords", fn(chord String) {
    println("chord change: " $ chord);
});

-- On the "drums" engine: send a chord change to "piano"
natsPub("nodes.piano.chords", "Cmaj7");
Note: Each engine maintains its own independent tempo. There is no automatic tempo synchronization across engines. This is by design, as the system supports multi-tempo music. If you need synchronized timing, coordinate explicitly via NATS messages.

10.9 NATS vs. OSC

Both protocols are supported simultaneously and share the same engine command set. Choose based on your needs:

FeatureOSCNATS
TransportUDP (connectionless)TCP (connection-oriented)
ReliabilityFire-and-forget; messages can be lostReliable delivery with auto-reconnection
Server requiredNo (direct peer-to-peer)Yes (NATS server)
Payload formatBinary (typed OSC arguments)Text (space-separated values)
Request/replyManual (reply to sender address)Built-in async with timeout and callback
BundlingYes (atomic transactions with timetag)No (each message is independent)
Multi-machineRequires explicit host/port addressingTransparent via NATS server clustering
Local dispatchYes (oscSendLocal*)No (always goes through server)
Subject namingSlash-separated (/engine/newNode)Dot-separated (engine.newNode)
EcosystemMature in music/audio (SuperCollider, Max/MSP)Strong in cloud/microservices, growing in creative coding
Tip: Use OSC when working with existing music software that speaks OSC, or when low-latency local communication is essential. Use NATS when building distributed systems across multiple machines, or when you need reliable delivery and request/reply semantics.

11. Clock / Tempo Scheduling

11.1 Overview

Tzopilotl schedules musical events by beat position rather than wall-clock time, converting beats to time with a TempoRamp that supports smooth tempo transitions. There are two cooperating scheduling layers:

In both layers, a reschedulable handler that returns a positive Float is automatically run again that many beats later, which is the idiom for repeating patterns. Tempo can be changed instantly or ramped linearly over a number of beats.

11.2 NRT Clock Module — Scheduling Functions

All NRT clock functions are registered under the clock foreign module:

import clock.*;

Engine clock sync. Every scheduling function below also has a form taking an engine TempoClock slot as its FIRST argument (matching the audio_engine convention). While audio is running, all beat scheduling — both forms — is slaved to the engine's TempoClock slots (the clocks the silos schedule against, synchronized across silos): a handler fires, latency early, when the actual engine slot reaches the beat, following tempo changes and ramps made from anywhere — the clock module, the audio_engine FFI (setTempo(clock, bpm) / schedTempoChange), or silo code. The slotless forms target slot 0. When audio is not running, scheduling falls back to the module's internal timeline.

Reschedulable handlers

These take a Fn() Float handler. If the handler returns a positive finite number, it is rescheduled that many beats later. Return 0.0 or a negative number to stop.

FunctionDescription
sched(deltaBeats Float, handler Fn() Float) IntSchedule relative to the current logical beat of slot 0. Returns a timer ID.
sched(clock Int, deltaBeats Float, handler Fn() Float) IntSame, on engine TempoClock slot clock.
schedAbs(beat Float, handler Fn() Float) IntSchedule at an absolute beat position on slot 0. Returns a timer ID.
schedAbs(clock Int, beat Float, handler Fn() Float) IntSame, on engine TempoClock slot clock.

One-shot handlers

These take a Fn() Void handler and never reschedule.

FunctionDescription
after(deltaBeats Float, handler Fn() Void) IntSchedule relative to the current logical beat of slot 0. Returns a timer ID.
at(beat Float, handler Fn() Void) IntSchedule at an absolute beat position on slot 0. Returns a timer ID.
after(clock Int, deltaBeats Float, handler Fn() Void) Int / at(clock Int, beat Float, handler Fn() Void) IntThe same, on engine TempoClock slot clock.

Coroutine driver

FunctionDescription
go(clock Int, c Coroutine<Float>) IntRun a coroutine on engine TempoClock slot clock (below: the slot 0 form).
go(c Coroutine<Float>) IntRun a coroutine on the clock. Each yielded Float is a beat delta: after yielding, the coroutine is resumed that many beats later; when it finishes, scheduling stops. Returns a timer ID. (Defined in clock.x on top of sched.)

Cancellation

FunctionDescription
cancel(timerID Int) VoidCancel a scheduled event by its timer ID.

11.3 Tempo Control

Tempo is specified in beats per minute (BPM). The default tempo is 60 BPM.

FunctionDescription
setTempo(bpm Float) VoidSet the tempo immediately.
schedTempoChange(beat Float, targetBPM Float, rampBeats Float) IntSchedule a smooth tempo ramp starting at the given beat, reaching targetBPM over rampBeats beats. The ramp is linear in beats (exponential in seconds). Returns a timer ID.

A tempo ramp with rampBeats = 0 is an instantaneous change at the specified beat.

Note: Internally, tempo is stored as beats per second (BPS); the FFI converts BPS = BPM / 60. After a ramp completes, the ending tempo is held until a new ramp is installed. These two functions also mirror their change onto engine TempoClock slot 0, so engine bundles scheduled on slot 0 stay aligned with NRT clock callbacks. To drive other slots, use the engine setTempo(clock, bpm) / schedTempoChange(clock, …) overloads in §11.6.

11.4 Beat & Tempo Queries

FunctionDescription
getTempo() FloatCurrent tempo in BPM.
getBeats() FloatCurrent beat position. Inside a handler callback, returns the handler's logical beat; outside, returns the wall-clock beat.
getBeatDur() FloatDuration of one beat in seconds at the current tempo.

11.5 Latency

The NRT clock fires its callbacks slightly ahead of their beat time by a configurable latency. This gives engine commands time to travel through the lock-free FIFO and arrive at the real-time thread before they must execute. The default latency is 50 ms.

Note: These latency functions apply to the NRT clock module only. Engine TempoClock slots (§11.6) fire beat-scheduled bundles sample-accurately on the audio thread with no latency offset — a bundle bound to (clock, beat) executes at the exact sample the slot reaches that beat.
FunctionDescription
setLatency(seconds Float) VoidSet the latency compensation in seconds.
getLatency() FloatGet the current latency in seconds.

11.6 Engine TempoClock Slots

The engine owns N beat-based TempoClock slots (the count is fixed at engine startup; the default is a single slot, index 0). Slot k is synchronized across every silo: all silos share the same beat origin and tempo for that slot, and a tempo change to slot k is broadcast to all silos at once. This lets independent parts on different silos be scheduled against a common musical grid while still allowing several independent grids (one per slot).

These functions live in the audio_engine module and are RT-safe:

import audio_engine.*;

Beat-scheduled command bundles

Open a command bundle with begin(), add graph/note commands, then commit it to a silo. Committing either fires immediately or defers until a clock slot reaches a beat:

FunctionDescription
sched(silo Int) IntCommit the open bundle to silo for immediate execution. Returns an engine error code (0 = ok).
sched(silo Int, clock Int, beat Float) IntCommit the open bundle to silo, to fire when slot clock reaches beat (late-bound: the binding is to the beat, so tempo changes before the beat arrives still land the commands at the right musical position).
schedPolicy(silo Int, clock Int, beat Float, policy Int) IntLike sched(silo, clock, beat), but with an explicit SchedPolicy for late delivery (see below).

The SchedPolicy enum (from audio_engine) controls what happens if the target beat has already passed by the time the bundle reaches the audio thread:

PolicyBehavior when late
schedImmediateFire as soon as it arrives, ignoring the beat.
schedBetterLateThanNeverFire at the next sample even though the beat has passed.
schedOnTimeOnlyDrop the bundle if its beat has already passed.

Per-slot tempo (synchronized across silos)

FunctionDescription
setTempo(clock Int, bpm Float) IntSet slot clock's tempo, on every silo, immediately.
schedTempoChange(clock Int, atBeat Float, targetBPM Float, rampBeats Float) IntRamp slot clock to targetBPM over rampBeats beats, starting at atBeat, on every silo.

Per-slot queries

FunctionDescription
clockBeats(clock Int) FloatCurrent beat position of slot clock.
clockTempo(clock Int) FloatCurrent tempo (BPM) of slot clock.
Note: While audio runs, the NRT clock module (§11.2–11.5) is slaved directly to the engine's TempoClock slots — its callbacks fire as the actual engine slot reaches the beat, whichever way the tempo was changed (clock.setTempo, audio_engine.setTempo(clock, bpm), schedTempoChange ramps). Use the clock module to run code on the beat (NRT callbacks, latency early); use the engine slot functions to schedule engine commands on the beat, sample-accurately. getBeats(clock) / getTempo(clock) read the engine slot itself.

11.7 Asynchronous delay

Inside an async fn, await delay(beats) suspends the async task for a number of beats on the VM's asynchronous event loop, then resumes after the statement. delay is a built-in (no import required):

FunctionDescription
delay(beats Float) Future<Void>Return a pending future that resolves after beats have elapsed on the async event loop's virtual-beat timeline.
delayReal(seconds Float) Future<Void>(clock module) Return a pending future resolved by the live scheduler thread after seconds of wall-clock time, independent of tempo. In a render context it resolves at logical render time instead.
delayBeats(beats Float) Future<Void>(clock module) Return a pending future resolved when the live tempo clock reaches now + beats — tracking tempo changes and ramps, and firing latency-early, exactly like a sched() handler. From within a clock handler on the same slot, relative to the handler's beat.
delayBeats(clock Int, beats Float) Future<Void>Same, on engine TempoClock slot clock.
async fn part() Void {
    println("start");
    await delay(4.0);   -- yield for 4 beats, then continue
    println("4 beats later");
}
Note: delay drives the async/await event loop (the orchestration layer on the NRT VM), which is distinct from the engine TempoClock slots that schedule audio-thread commands. See the “Async and Await” section of Tzopilotl by Example for the full async model (async fn, await, Future<T>, awaitAll / gather). Timelines: on a SILO VM the per-sample tick fires delay timers against the audio tempo clock, so await delay there is sample-accurate real time. On the NRT VM the timeline is logical: a top-level await pumps the loop by jumping the virtual beat to the earliest timer, so await delay(4.0) in a script or notebook cell resolves without waiting any wall-clock time (this is what lets NRT renders run faster than real time). When a script really must pace itself against the outside world, use await delayReal(seconds) from the clock module: the future is resolved by the live scheduler’s wall-clock thread, and the wait survives the clear-schedulers panic (which drops handlers, not pending awaits). To await musical time on the same tempo clock that drives sched/go — ramps and all — use await delayBeats(beats), the awaitable sibling of a sched() callback.

11.8 Silo Task Scheduling

Code running on a silo's own VM (a task module installed with siloLoad) can schedule beat-driven tasks on that silo's TempoClock slots. Because the slots are synchronized across silos (§11.6), parts loaded onto different silos can be started on a common downbeat. The silo-side primitives are:

FunctionDescription
spawn(clock Int, c Coroutine<Float>) IntRun a coroutine as a task on slot clock of the current silo. Each yielded Float is a beat delta; finishing the coroutine stops the task. Returns a task ID.
scheduleTask(clock Int, handler Fn() Float) IntLower-level form: schedule a reschedulable handler (positive return = beats until next run; ≤ 0 stops) on slot clock.

The orchestration side (main / NRT VM) loads parts and starts them together via attachVM, siloLoad (an awaitable load barrier returning Future<String>), and siloStartAt(beat, silos). See §13 (Silo VM) for the full lifecycle.

Silo-only enforcement. The silo-side primitives — spawn(clock, c), scheduleTask, playNote, releaseNote — need the real-time silo context and are rejected at compile time if called from code compiled for the main/NRT VM (a script, the REPL, a notebook cell): “Function … is silo-only”. This is the mirror of the rule that keeps non-RT-safe functions out of silo code. Defining a wrapper around one in a shared module is fine; the wrapper itself becomes silo-only.

11.9 Examples

Repeating pattern (NRT clock)

import clock.*;
import audio_engine.*;

-- Play a note every beat using the reschedule convention.
-- The handler returns 1.0, so it fires again 1 beat later.
sched(0.0, fn() Float {
    begin();
    noteOn(100, 60, [440.0, 0.5]);
    sched(0);
    1.0;  -- reschedule 1 beat later
});

One-shot delayed event

import clock.*;
import audio_engine.*;

-- Turn off all notes after 8 beats
after(8.0, fn() Void {
    begin();
    allNotesOff(100);
    sched(0);
});

Tempo ramp

import clock.*;

setTempo(120.0);  -- start at 120 BPM

-- At beat 16, begin ramping to 140 BPM over 8 beats
schedTempoChange(16.0, 140.0, 8.0);

-- At beat 32, instantly drop to 90 BPM
schedTempoChange(32.0, 90.0, 0.0);

Beat-scheduled engine bundle (slot 0)

import audio_engine.*;

-- Set the grid tempo on clock slot 0 (all silos).
setTempo(0, 120.0);

-- Build a note bundle and bind it to beat 8 on slot 0.
-- It fires sample-accurately when the slot reaches beat 8,
-- even if the tempo changes before then.
begin();
noteOn(100, 67, [587.33, 0.7]);
sched(0, 0, 8.0);

Scheduling from within a handler

import clock.*;
import audio_engine.*;

-- Schedule two notes a half-beat apart, starting at beat 4
at(4.0, fn() Void {
    begin();
    noteOn(100, 60, [440.0, 0.8]);
    sched(0);

    -- sched() is relative to the current logical beat (4.0)
    after(0.5, fn() Void {
        begin();
        noteOn(100, 67, [587.33, 0.6]);
        sched(0);
    });
});

12. SynthDef Compiler

12.1 Overview

The synthdef foreign module provides functions for compiling synth definitions from s-expression descriptions. The compilation pipeline parses the s-expression, performs graph analysis, generates C++ code, compiles it into a dynamic library (.dylib), and optionally loads it into the running audio engine.

All functions are registered under the synthdef foreign module. Import them with:

import synthdef.*;

Both functions are marked as non-RT-safe (they invoke the system compiler and perform file I/O), so they cannot be called from a real-time compiled context.

12.2 Functions

FunctionDescription
compileSynthDef(sexpr String) StringCompile a synth definition from an s-expression string. Generates C++ code and compiles it to a .dylib, but does not load it into the engine. Returns "" on success, or an error message on failure. The result is cached for subsequent calls.
compileSynthDefAndLoad(sexpr String) StringCompile a synth definition and load it into the running audio engine. Performs the same compilation as compileSynthDef, then loads the resulting .dylib and registers the definition with the engine. Returns "" on success, or an error message on failure. Requires an engine to be attached to the VM.
compileSynthDefAndLoadAsync(sexpr String) Future<String>Async variant of compileSynthDefAndLoad: the compile runs on the VM's background I/O worker, so scheduled music keeps playing while clang runs. The future resolves to "" on success or an error message; when it resolves, the def has been loaded and registered. This is the backend of the synthdef module's defSynth. Inside an NRT render the job runs synchronously and the future is returned already resolved.
writeCompileAndLoadAsync(name String, cppSource String) Future<String>Async backend of synthc's defSynthX: writes already-generated C++ source, compiles and links it on the background worker, then loads and registers the def. Same resolution semantics as compileSynthDefAndLoadAsync.

The sexpr argument must be in the format (Synth <name> (Graph ...)). The synth name is extracted from the s-expression.

Note: compileSynthDefAndLoad requires an audio engine to be attached to the VM via the AppContext user data. If no engine is present, it returns the error "no engine attached to VM".

12.3 Compilation Caching

Both functions maintain an in-memory cache keyed by the s-expression content. When compileSynthDefAndLoad is called with an s-expression that has already been compiled, it skips recompilation and reuses the cached .dylib path, then loads and registers the definition directly. This makes repeated calls with the same definition fast.

12.4 Examples

Low-level: compiling from an s-expression string

import synthdef.*;
import audio_engine.*;

let sexpr = "(Synth mySine (Graph 1 ((0 Constant 1 12 (440.0)) (1 Outlet \"out\" 0))))";
let err = compileSynthDefAndLoad(sexpr);
if (err length > 0) {
    println("ERROR: " $ err);
};

-- The "mySine" definition is now available in the engine.
-- Create a node using it:
newNode("mySine", 1);

High-level: using the synthdef module's defSynth

The synthdef Tzopilotl module provides defSynth, which builds the graph from a function, converts it to an s-expression, and calls compileSynthDefAndLoadAsync automatically. It is an async function; await it when the following code needs the def to be loaded:

import synthdef.*;

defSynth(fn() { 440.0 sinosc outlet }, "mySine") await;

Compile only (without loading)

import synthdef.*;

let sexpr = "(Synth offlineDef (Graph 1 ((0 Constant 1 12 (440.0)) (1 Outlet \"out\" 0))))";
let err = compileSynthDef(sexpr);
if (err length > 0) {
    println("Compile failed: " $ err);
} else {
    println("Compiled successfully (not loaded into engine)");
};

13. Silo VM

13.1 Overview

Each engine silo can optionally have a Tzopilotl VM attached for event-driven scripting on the real-time audio thread. The silo VM is separate from the main NRT VM—it has its own TLSF memory pool, its own global variables, and its own compilation target with rtRestricted = true.

The silo VM is event-driven: it does not run per-sample or per-buffer. Code sent via siloEval is compiled on an NRT thread and then executed on the silo's RT thread. Between events, the VM is idle. A per-buffer GC heartbeat drives a bounded slice of the incremental tracing collector (mark/sweep work) to reclaim dead objects without overrunning the audio block.

All functions are registered under the audio_engine foreign module:

import audio_engine.*;

13.2 Functions

FunctionDescription
attachVM(siloIndex Int) Int Create a new VM (16 MB TLSF pool) and attach it to the specified silo. The VM is compiled with rtRestricted = true. Returns 0 on success, or an error code if the silo index is out of range or a VM is already attached.
detachVM(siloIndex Int) Int Detach the VM from the specified silo and destroy it. The detach happens on the RT thread (the silo stops using the VM), and deletion happens on the engine's NRT command thread. Returns 0 on success.
siloEval(siloIndex Int, code String) Int Compile code with the silo's RT-restricted target on the calling NRT thread, then send the compiled result to the silo for installation and execution on the RT thread. Module imports (e.g., import audio_engine.*;) are supported. Returns 0 on success; compile errors are printed to stderr.

All three functions are NRT-only—they must be called from the main NRT VM (e.g., from the REPL, a script, or an OSC/NATS handler). They cannot be called from within RT code.

13.3 Lifecycle

  1. Attach: attachVM(siloIndex) creates a fresh VM with its own memory pool and an rtRestricted compilation target. A separate module compiler cache is created so that modules compiled for the RT target don't collide with the NRT target's cache.
  2. Evaluate: siloEval(siloIndex, code) compiles on the NRT thread, then sends two commands through the silo's lock-free FIFO:
    • A code install command that extends the VM's globals.
    • An execute command that runs the top-level block on the RT thread.
    Multiple siloEval calls accumulate globals incrementally, like the REPL.
  3. Detach: detachVM(siloIndex) nulls the silo's VM pointer on the RT thread, then deletes the VM on the NRT command thread. This two-phase approach ensures the RT thread never accesses a deleted VM.
Note: siloEval bypasses the command bundling API (begin/go(silo)), so it can be called safely inside or outside of a command bundle without conflict.

13.4 RT Restrictions

Code compiled for the silo VM uses rtRestricted = true. This means:

13.5 Examples

Basic: attach, evaluate, detach

import audio_engine.*;

attachVM(0);

-- This runs on silo 0's RT thread
siloEval(0, "
    println(42);
");

detachVM(0);

Building an audio graph from the silo VM

import audio_engine.*;

engineStart();
attachVM(0);

-- Compile and execute RT code on silo 0
siloEval(0, "
    import audio_engine.*;
    begin();
    newNode(\"SinOsc\", 100);
    connect(100, 0, 0, 0);
    go(0);
");

-- Later: clean up
detachVM(0);

Incremental evaluation (globals persist)

import audio_engine.*;

attachVM(0);

-- First eval: define a variable
siloEval(0, "var counter = 0;");

-- Second eval: use it (globals persist across evals)
siloEval(0, "
    counter = counter + 1;
    println(counter);
");

detachVM(0);

14. Non-Real-Time Rendering

14.1 Overview

Non-real-time (NRT) rendering runs the audio engine offline: no audio device is opened, the render loop runs as fast as the host can compute, and audio is written to a 32-bit float WAV file instead of the speakers. NRT mode is intended for bouncing/mixdown of long pieces, batch generation of audio assets, golden-file regression tests for synthdefs, and CI environments without a working audio device.

Key properties of the NRT model:

14.2 FFI Functions

All NRT-render FFI lives in the audio_engine module. Every call is non-blocking; the language never waits on a render. Multiple renders may be in flight simultaneously, and renders coexist with live audio.

-- Start a render. setup runs once on the render's background thread under
-- the NRTVM lock. Inside setup, FFI calls (ae.begin/newNode/sched, clock.go,
-- etc.) are routed to this render's engine + scheduler rather than the live
-- engine. Returns a handle.
fn renderNRT(path String, setup Fn() Void) Int
fn renderNRT(path String, durationSeconds Float, setup Fn() Void) Int

-- Non-blocking status query. Returns true if the render has finished or if
-- the handle is invalid.
fn isRenderDone(handle Int) Bool

-- One-shot completion callback. If the render has already finished, the
-- callback fires immediately. Otherwise it fires from the live tempo
-- scheduler thread when the render completes.
fn onRenderDone(handle Int, callback Fn() Void) Void

-- Ask a specific render to stop. Runs the renderer's default tail
-- (or tailSeconds, if provided) before closing the WAV file.
fn stopRender(handle Int) Void
fn stopRender(handle Int, tailSeconds Float) Void

-- Ask THE CURRENT render to stop. Only meaningful when called from inside
-- a render's setup or from a handler running on that render's scheduler.
fn endRender() Void
fn endRender(tailSeconds Float) Void

Stop conditions

A render stops when any of these fires first:

After the stop signal, the renderer continues for a tail (default 1.0 s) so reverbs, comb filters, release envelopes, and crossfades can decay naturally before truncation. Override per-render via endRender(tail) / stopRender(h, tail), or globally via the CLI --tail flag.

Polling vs. callback

Polling via a coroutine:

import clock.*;
import audio_engine.*;

let h = renderNRT("out.wav", fn() { -- ... -- });

go(coro fn() Float {
    while (!isRenderDone(h)) {
        yield 0.1;
    }
    println("render done");
}());

Callback:

let h = renderNRT("out.wav", fn() { -- ... -- });
onRenderDone(h, fn() { println("render done"); });

14.3 CLI Invocation

The --nrt CLI flag is a convenience wrapper that runs a script as the setup of a single render:

tzpl --nrt out.wav script.x

Equivalent to calling renderNRT(out.wav, ...) with the entire script as the setup. Implies --nogui; no audio device is opened. Waits (at the C++ level, not in the language) for the render to complete before exiting.

FlagArgumentDescription
--nrt<path>Output WAV path. Float32 IEEE WAV; sample rate and channel count come from the engine config (defaults: 48000 Hz, 2 channels).
--duration<seconds>Optional hard cap. If absent, the render runs until script-driven stop or scheduler idle.
--tail<seconds>Default tail rendered after the stop signal (default 1.0).
--nrt-safety-cap<seconds>Upper bound (default 3600). Render aborts with a stderr warning if exceeded.
--sample-rate<hz>Engine sample rate.
--buffer-frames<n>Render block size.
--channels<n>Output channel count.
Wrapping an existing script: scripts that already use go(coro fn() Float { ... }()) for sequencing work with tzpl --nrt out.wav script.x unchanged. The tempo scheduler detects when the top-level coroutine finishes and stops automatically. For patches with long reverb tails, add --tail 5 (or call endRender(5.0) from the script) so the decay isn't truncated.

14.4 Behavior & Determinism

Node IDs 0 and 1 are reserved by each engine for the output and input nodes respectively. User synthdefs should use IDs of 2 or above (conventionally 100+); attempting newNode("...", 1) returns tzpl_errNodeIDAlreadyTaken.

14.5 C++ Embedding

// bridge/include/tzpl_nrt_render.hpp
namespace bridge {
    struct RenderJobOpts {
        std::string path;
        double sampleRate       = 48000.0;
        int    channels         = 2;
        int    bufferFrames     = 512;
        double durationSeconds  = 0.0;   // 0 = open-ended
        double tailSeconds      = 1.0;
        double safetyCapSeconds = 3600.0;
        int    numSilos         = 4;
    };

    // Start a render. Returns a handle. setup runs on the render thread,
    // under the NRTVM mutex, with the render's context installed.
    int64_t renderNRTAsync(RenderJobOpts const& opts,
                              AppContext* appCtx,
                              std::function<void()> setup);

    bool isRenderDone(int64_t handle);
    void onRenderDone(int64_t handle, std::function<void()> callback);
    void stopRender(int64_t handle);
    void stopRender(int64_t handle, double tailSeconds);

    // Host shutdown: joins every render thread. Must be called BEFORE the
    // live NRTVM and AppContext are destroyed.
    void shutdownRenderRegistry();
    // Polls until all renders are done; C++-only wait, never exposed to
    // Tzopilotl.
    void joinAllRenders(int pollMs = 50);

    // Current thread's render context (nullptr when not inside a render).
    RenderContext const* currentRenderContext();
}

The host (app) is responsible for populating AppContext::initEngine:

appCtx.initEngine = [](engine::Engine* e) {
    // register built-in test plugins
    engine::createSineNode(e);
    engine::createAddOpNode(e);
    // ...
    // optionally load synthdef .dylibs from the project's dylib directory
    engine::loadDefs(e, projectDylibDir.c_str());
};

The bridge invokes this hook on every per-render engine immediately after construction, so renders see the same defs as the live engine.

Lower-level engine entry points (used internally by renderNRTAsync; hosts rarely need them directly):

// engine/src/tzpl_client_interface.hpp
namespace engine {
    Engine* newEngineNRT(EngineConfig const& config,
                          AudioStreamParameters& streamParams);
    void    renderNRTBlock(Engine* e, f32* outBuffer);
}

14.6 Examples

CLI: fixed-duration render

-- sine.x: define a synth and queue it for play
import audio_engine.*;

begin();
newNode("sinosc", 100);
setInput(100, 0, 440.0);  -- frequency
setInput(100, 1, 0.2);    -- amplitude
connect(100, 0, 0, 0);
sched(0);
tzpl --nrt sine.wav --duration 1 sine.x

CLI: open-ended render of a coroutine-driven script

Scripts that already use go(coro fn() Float { ... }()) for sequencing work as-is. The renderer stops automatically when the coroutine completes (plus the default 1 s tail).

-- sequence.x
import clock.*;
import audio_engine.*;

go(coro fn() Float {
    begin();
    newNode("sinosc", 100);
    setInput(100, 0, 220.0);
    setInput(100, 1, 0.15);
    connect(100, 0, 0, 0);
    sched(0);
    yield 1.0;
    begin();
    freeNode(100);
    sched(0);
}());
tzpl --nrt sequence.wav sequence.x

Script-driven async render with completion callback

import clock.*;
import audio_engine.*;

let h = renderNRT("/tmp/take.wav", fn() Void {
    go(coro fn() Float {
        begin();
        newNode("sinosc", 100);
        setInput(100, 0, 220.0);
        setInput(100, 1, 0.15);
        connect(100, 0, 0, 0);
        sched(0);
        yield 2.0;
        endRender();
    }());
});

onRenderDone(h, fn() Void { println("render finished"); });

Multiple concurrent renders

import clock.*;
import audio_engine.*;

let a = renderNRT("a.wav", fn() Void {
    go(coro fn() Float {
        begin();
        newNode("sinosc", 100);
        setInput(100, 0, 220.0);
        setInput(100, 1, 0.15);
        connect(100, 0, 0, 0);
        sched(0);
        yield 1.0;
        endRender();
    }());
});

let b = renderNRT("b.wav", fn() Void {
    go(coro fn() Float {
        begin();
        newNode("sinosc", 100);
        setInput(100, 0, 440.0);
        setInput(100, 1, 0.15);
        connect(100, 0, 0, 0);
        sched(0);
        yield 0.5;
        endRender();
    }());
});

onRenderDone(a, fn() Void { println("a done"); });
onRenderDone(b, fn() Void { println("b done"); });

Live audio while rendering

import clock.*;
import audio_engine.*;

-- live audio
engineStart();
go(coro fn() Float {
    -- ... live pattern ...
}());

-- a render running in the background; the live engine is unaffected
let h = renderNRT("bounce.wav", fn() Void {
    -- ... render setup ...
});

15. Binary Messages (Msg)

15.1 Overview

The messageEncoding module encodes a Msg value to and from a compact binary buffer — the “TZB” format. It is the serialization substrate for moving structured data between threads (silos, each of which has its own VM and heap) and between processes (e.g. over NATS), where heap pointers cannot be shared.

The value type it serializes is the Msg enum (the message module):

enum Msg {
    bool   Bool,
    int    Int,
    float  Float,
    symbol Symbol,
    string String,
    vec    [Msg],
}

Two ways to read an encoded message are provided: a full decode into a Msg tree, and a zero-copy Reader that reads fields and walks children directly off the buffer with no tree allocation. List children are stored columnarly, so reaching the i-th child is O(1).

import std.messageEncoding.*;
import std.message.*;
Note: symbol and string are distinct cases that both carry text: a symbol is an interned Symbol (renders bare, e.g. note); a string is text (renders quoted, e.g. "note"). The format preserves the distinction with separate tags, and round-trips a symbol back to an interned Symbol.

15.2 The Bytes Type

Bytes is a growable, length-based binary buffer (like String, but with no UTF-8 assumptions — it holds arbitrary bytes including embedded NULs). It is the type encode returns and decode consumes, and what natsPubMsg / onMessageMsg carry. These low-level builders and readers are built-ins (no import needed) and underlie messageEncoding.x; most code only needs encode / decode / reader.

FunctionDescription
bytes() BytesA new empty buffer.
byteLength(b Bytes) IntCurrent length in bytes.
putU8! / putU32! / putU64!(b Bytes, v Int) VoidAppend 1 / 4 / 8 bytes, little-endian.
putF64!(b Bytes, v Float) VoidAppend an IEEE-754 double (8 bytes).
putUtf8!(b Bytes, s String) VoidAppend the raw bytes of s (no length prefix).
setU32At!(b Bytes, off Int, v Int) VoidOverwrite 4 bytes at off (back-patch); no-op if out of range.
u8At / u32At / i64At(b Bytes, off Int) IntRead 1 / 4 / 8 bytes LE. Bounds-checked: an out-of-range read returns 0.
f64At(b Bytes, off Int) FloatRead a double; 0.0 if out of range.
utf8At(b Bytes, off Int, len Int) StringCopy len bytes as a String, clamped to the buffer.
toSymbol(s String) SymbolIntern a String into a Symbol (the inverse of Symbol.toString); used by decode.

15.3 encode & decode

FunctionDescription
encode(o Msg) BytesSerialize a Msg to a fresh TZB buffer.
decode(b Bytes) MsgDeserialize a TZB buffer back to a Msg. Returns an empty vec if b is not a valid message.
isMessage(b Bytes) BoolTrue if b carries a valid TZB header.
import std.messageEncoding.*;
import std.message.*;

let msg = encode(Msg.vec([
    Msg.symbol(toSymbol("note")),
    Msg.int(60),
    Msg.float(0.8),
    Msg.string("hello"),
]));

byteLength(msg) println;          -- size in bytes
decode(msg) toString println;     -- [note, 60, 0.8, "hello"]
Untrusted input: decode is total and crash-safe. All reads are bounds-checked, the header is validated, child counts are clamped to the buffer length, and a per-message node budget plus a recursion-depth cap bound the work, so a truncated or corrupt buffer yields a small/empty Msg rather than reading out of bounds, looping, or overflowing the stack. Still, validate foreign payloads with isMessage before trusting them.

15.4 Zero-Copy Reader

A Reader is a cursor into the buffer. It reads tags, scalars, and string text, and navigates a vec's children, without building a Msg tree. tag returns a MsgTag (bool, int, float, symbol, string, vec).

FunctionDescription
reader(b Bytes) ReaderA cursor at the root value.
tag(r Reader) MsgTagThe kind of this node.
asBool / asInt / asFloat(r Reader)Read a scalar value.
asStr(r Reader) StringRead the text of a symbol or string node.
childCount(r Reader) IntNumber of children (vec nodes).
child(r Reader, i Int) ReaderA cursor for child i — O(1).
children(r Reader) [Reader]Cursors for all children.
let r = reader(msg);
if (tag(r) == MsgTag.vec) {
    childCount(r) println;               -- 4
    asStr(child(r, 0)) println;          -- note
    asInt(child(r, 1)) println;          -- 60

    for (c : children(r)) {
        match (tag(c)) {
            MsgTag.int:    use(asInt(c));
            MsgTag.string: use(asStr(c));
            _: skip();
        }
    }
}

15.5 Wire Format

All integers are little-endian. A message begins with a 4-byte header ('T' 'Z' 'B' + version 2) and a u32 offset to the root value slot. Each node is a value slot — a tag byte plus an 8-byte payload — interpreted by tag:

TagCase8-byte payload
0bool0 / 1
1inti64
2floatf64
3symboloffset to a string blob
4stringoffset to a string blob
5vecoffset to a list node

The format is defined once in shared/tzpl_sexpr_bin.hpp and mirrored byte-for-byte by lang/modules/std/messageEncoding.x. This section is the reference documentation for TZB; integration-tests/scripts/sexpr_bin_interop.sh checks that the two implementations still agree byte for byte.

15.6 Sending over NATS

NATS carries these buffers verbatim. Publish with natsPubMsg and receive with onMessageMsg, whose handler gets the raw payload as Bytes:

import nats.*;
import std.messageEncoding.*;
import std.message.*;

-- receive: decode each incoming message
nats.onMessageMsg("part.cmd", fn(m Bytes) {
    if (isMessage(m)) {
        println("got " $ (decode(m) toString));
    }
});

-- send: encode a Msg and publish it
natsPubMsg("part.cmd", encode(Msg.vec([
    Msg.symbol(toSymbol("play")),
    Msg.int(60),
])));

15.7 C++ Access

The C++ header shared/tzpl_sexpr_bin.hpp (namespace tzpl::sbin) is the canonical layout definition and offers a matching zero-copy Reader (returning string_views — no allocation) plus an encode over a small Value variant. This is what the engine / silo side uses to inspect a message without a round-trip through the language.

// shared/tzpl_sexpr_bin.hpp
using namespace tzpl::sbin;

// build a message
std::vector<uint8_t> buf = encode(Value::Vec({
    Value::Symbol("note"), Value::Int(60),
}));

// read it back, zero-copy
if (Reader::valid(buf.data(), buf.size())) {
    Reader r = Reader::root(buf.data(), buf.size());
    std::string_view name = r.child(0).asStr();   // "note"
    int64_t n             = r.child(1).asInt();   // 60
}

16. Actors

16.1 Overview

An actor is an async fn coroutine with a private mailbox. It receives Msg messages one at a time, runs to its next await receive, then parks until more mail arrives. Actors share no mutable state — they communicate only by message — so the model is real-time safe and the same code runs at NRT (the main VM) or inside a silo (the audio thread). Messages are values of the actor's message type M; in practice M is Msg, which the silo and NATS bridges serialize with encode/decode.

The primitives are built-ins (no import needed) and all real-time safe, so they work unchanged inside a silo. Cross-VM addressing uses an interned Symbol name: an actor registers a process-global name, and senders elsewhere reach it by that name (see Silo Messaging).

16.2 Primitives

-- Spawn an actor from a behavior and an initial message. The behavior is an
-- async fn taking (self, init); the init value's type fixes M. Returns the handle.
fn spawn(behavior async fn(self Actor<M>, init M) Void, init M) Actor<M>

-- Enqueue msg into actor's mailbox (handed to a parked receiver, else queued).
fn send(actor Actor<M>, msg M) Void

-- Await the next message; parks the actor until one arrives. Used as
--   let m = await receive(self);
fn receive(self Actor<M>) Future<M>

-- Give the actor a process-global name (Symbol or String). Returns the actor,
-- so it chains off spawn. The SPAWNER names the actor, not the actor itself.
fn register(actor Actor<M>, name Symbol) Actor<M>
fn register(actor Actor<M>, name String) Actor<M>

-- Send by registered name to an actor in THIS VM. Cross-VM delivery goes through
-- the silo/NATS bridges, which call this on the destination side.
fn sendByName(name Symbol, msg M) Void
Note: register takes the name as its second argument so it reads naturally chained off spawn: worker spawn(Msg.int(0)) register('worker). Binding the result rebinds the name to the Actor (shadowing the behavior fn), so use a fresh name or leave it as a statement.

16.3 Driving the Loop

Spawned actors do not run until the event loop drives them. Two drivers, both on the main (NRT) thread:

-- Drive all ready actors until the system is quiescent, then return.
fn runActors() Void

-- Long-lived: drive actors, then park the thread until an external event (a NATS
-- delivery, a silo outbox message) wakes it. For servers.
fn serveActors() Void

Use runActors for a batch that drains to completion; serveActors for a process that stays alive reacting to outside traffic. The silo router runActorServer wraps runActors with outbox draining.

16.4 Example

Two NRT actors addressed by name (from lang/tests/actors/nrt_byname.x):

import std.message.*;

async fn relay(self Actor<Msg>, init Msg) Void {
    while (true) {
        let m = await receive(self);
        match (m) {
            Msg.int(n): { println("relay " $ (n toString));
                            if (n < 4) { sendByName('collector, Msg.int(n + 1)); } }
            _: {}
        }
    }
}
-- ... a 'collector actor that bounces n+1 back to 'relay ...

relay spawn(Msg.int(0)) register('relay);
collector spawn(Msg.int(0)) register('collector);
sendByName('relay, Msg.int(0));
runActors();

17. Silo Messaging

17.1 Overview

Silo messaging carries actor messages across the VM boundary: from NRT to a silo, from a silo to NRT, and silo to silo. Because a silo runs on the audio thread it cannot allocate or block, so the transport is lock-free and the encoded Msg bytes — never an Obj* — cross the boundary:

Every message targets an actor by registered Symbol name; the destination side calls sendByName to enqueue it. The low-level FFIs below carry raw Bytes; the actors and silo_actors modules wrap them to take a Msg directly.

17.2 NRT → Silo

-- Deliver an encoded message to the actor `name` in silo `silo`, on the next
-- audio block. Returns an error code. (audio_engine)
fn siloDeliverBytes(silo Int, name Symbol, b Bytes) Int

-- Beat-scheduled variant: lands sample-accurately when TempoClock `clock`
-- reaches `beat`. (audio_engine)
fn siloDeliverBytesAt(silo Int, clock Int, beat Float, name Symbol, b Bytes) Int

The actors module wraps these to take a Msg:

-- actors.x (NRT side)
fn siloSend(silo Int, name Symbol, msg Msg) Void
fn siloSendAt(silo Int, clock Int, beat Float, name Symbol, msg Msg) Void

17.3 Silo → NRT / Silo

From inside a silo actor or task, push a message into the silo's outbox. Real-time safe (a by-value push into a lock-free ring). target is the destination silo index, or -1 for an actor at NRT.

-- Low-level: push pre-encoded bytes. Returns errInternal (without enqueuing) if
-- the payload exceeds the per-message cap or the outbox is full, else errNone.
-- (audio_engine, rt-safe)
fn siloOutbox(target Int, name Symbol, b Bytes) Int

-- silo_actors.x: encodes the Msg and posts it -- the silo-side counterpart to
-- siloSend. Importable into silo code (rt-safe).
fn siloPost(target Int, name Symbol, msg Msg) Int
Note: The outbox is a fixed ring — up to 256 messages in flight, each up to 4096 encoded bytes. A larger message is rejected (not truncated); check the return code if your payloads can be large.

17.4 The NRT Router

The main thread must drain every silo's outbox and route the traffic: silo→silo messages are forwarded to the destination silo; silo→NRT messages are decoded and delivered to the named NRT actor. The primitives:

-- Drain every silo outbox. Forwards silo->silo to the destination silo; stashes
-- silo->NRT messages in an inbox. Returns the count routed. (audio_engine)
fn pumpSiloOutboxes() Int

-- The silo->NRT inbox. The language drains it and does decode + sendByName
-- itself (both must run as lang on the main VM).
fn nrtActorMsgCount() Int      -- messages waiting
fn nrtActorMsgName() Symbol    -- peek the next message's target actor
fn nrtActorMsgTake() Bytes     -- pop the next message's encoded payload

The actors module assembles these into one router loop — run it on the main thread instead of runActors when silos route traffic:

-- actors.x
fn runActorServer() Void {
    while (true) {
        pumpSiloOutboxes();                 -- forward silo->silo; stash silo->NRT
        var k = nrtActorMsgCount();
        while (k > 0) {
            let nm = nrtActorMsgName();
            let b  = nrtActorMsgTake();
            sendByName(nm, decode(b));        -- deliver to the NRT actor
            k = k - 1;
        }
        runActors();                         -- drive NRT actors that got mail
        sleepMs(2);
    }
}

17.5 Silo Node Commands

A silo actor typically reacts to a message by triggering a synth node in its own graph. These run immediately on the silo's RT thread (no bundle, no allocation) and are real-time safe:

-- Trigger note `noteID` on `node` now, with a Float parameter array. (audio_engine, rt-safe)
fn playNote(node Int, noteID Int, params [Float]) Int

-- Release a sounding note on `node`. (audio_engine, rt-safe)
fn releaseNote(node Int, noteID Int) Int
Note: These are the in-task counterparts to the NRT noteOn/noteOff bundle commands (§13): same effect, but issued from inside the silo rather than queued from NRT.

17.6 NATS Bridge

Cross-process delivery reuses the same name-addressed actors over NATS. natsBridgeActor (from the nats module) wires a subject to a local actor — every message published to the subject is decoded and delivered to the named actor's mailbox:

-- nats.x
fn natsBridgeActor(subject String, name Symbol) Void

-- equivalent to the hand-written:
onMessageMsg(subject, fn(b Bytes) {
    if (isMessage(b)) { sendByName(name, decode(b)); }
});

Drive the bridged actor with serveActors() so the process parks until a NATS delivery wakes it. See §10 for connection setup and §15.6 for the publish side.

17.7 Example

A silo sender emitting a line to an NRT conductor (from integration-tests/scripts/silo_to_nrt.x):

-- inside the silo module
import silo_actors.*;
import std.message.*;

async fn sender(self Actor<Msg>, init Msg) Void {
    let scale = [60.0, 64.0, 67.0, 72.0];
    var i = 0;
    while (i < 4) {
        await delay(1.0);
        siloPost(-1, 'conductor, Msg.float(scale[i % 4]));
        i = i + 1;
    }
}
spawn(sender, Msg.int(0));

The NRT side spawns a conductor actor, registers it, and runs runActorServer() to route the silo→NRT traffic. See integration-tests/scripts/silo_to_nrt.x for the full program.

18. Audio Engine

18.1 Overview

The audio engine bindings, registered by bridge::registerAudioEngineFFI(), control the live audio graph: starting the stream, loading plugins, creating and connecting nodes, setting parameters, and triggering voices. All functions live in the audio_engine module:

import audio_engine.*;

Conventions used throughout this section:

Related engine functionality documented elsewhere in this guide:

18.2 Engine Lifecycle

FunctionDescription
engineStart() VoidStart the audio stream. Must be called before any audio is produced.
engineStop() VoidStop the audio stream.
isAudioRunning() BoolReturns true if the audio stream is currently running.
getStreamTime() FloatCurrent audio stream time in seconds.
masterGain(gain Float) VoidSet the master output gain. 1.0 = unity, 0.0 = silence. When the safety limiter is enabled, the master gain can reduce the limiter's gain but never increase it — it will not fight the limiter. When audio is below the limit, the master gain applies freely. When the limiter is disabled, the master gain is a simple multiply.
safetyLimiter(on Bool) VoidEnable or disable the safety limiter on the master output. When enabled, output is hard-limited to prevent clipping/damage.
inputChannels() IntNumber of active hardware input channels (0 if audio input is disabled).

18.3 Plugin Loading

Plugins are shared libraries (.dylib/.so) conforming to the C plugin ABI (shared/tzpl_plugin_abi.h). Any conforming library can be loaded, not just those produced by the synthdef compiler (§12). Both functions are NRT-only (file I/O + dlopen).

FunctionDescription
loadPlugins(path String) BoolLoad all plugin definitions found in the directory at path. Returns true on success.
loadPlugin(path String, name String) BoolLoad a single plugin definition from the shared library at path under the given name. Returns true on success.

18.4 Command Bundling

Commands that modify the audio graph are not executed immediately. They are collected into a bundle and submitted atomically. The target silo (audio worker thread) is chosen when the bundle is submitted — either for the next audio callback (go(silo) / sched(silo)) or bound to a beat on an engine TempoClock slot (sched(silo, clock, beat)).

FunctionDescription
begin() IntBegin a new command bundle. Returns an error code.
go(silo Int) IntSubmit the current bundle to the given silo for immediate execution on the next audio callback. Silo indices start at 0. Returns an error code.
sched(silo Int) IntSame as go(silo) — immediate submission.
sched(silo Int, clock Int, beat Float) IntSubmit the current bundle to the given silo, to fire when TempoClock slot clock reaches beat. Late-bound to the beat, sample-accurate. See §11.6.
schedPolicy(silo Int, clock Int, beat Float, policy Int) IntLike sched(silo, clock, beat), but with an explicit SchedPolicy for late delivery. See §11.6.

Validation and atomicity

Because the silo is unknown until submit, the command builders (newNode, connect, setInput, …) only record the request; the only error a builder itself can return is errNoActiveBundle. All validation happens at go()/sched(), against the submitting thread's (NRT) view of the chosen silo's graph, and the bundle is atomic: on the first invalid command (unknown def name, duplicate node ID, type/rate/channel mismatch, bad port index, out-of-range silo or clock) the entire bundle is discarded, nothing is applied, and the submit call returns that error. Submit always closes the bundle, success or failure.

Command ordering rules:

Note: Calling begin() while a previous bundle is still open (queued but never submitted) returns errCommandsQueuedButNotSent; issuing graph commands with no open bundle returns errNoActiveBundle.
First-class bundles: the bundles module (import bundles.*;) layers a data-structure representation on top of this API — an EngineCmd enum for the bundleable commands and a Bundle struct holding an ordered sequence of them. A Bundle is an ordinary value that can be built anywhere, stored, inspected, and submitted repeatedly with b go(silo) / b sched(silo, clock, beat) / b schedPolicy(…), which replay it through begin()/sched().

18.5 Node Operations

These must be called within a begin()/go(silo) bundle.

FunctionDescription
newNode(defName String, nodeID Int) IntCreate a node instance from the plugin definition named defName, assigning it nodeID. The node ID must be unique within the silo.
freeNode(nodeID Int) IntRemove and free the node with the given ID.
freeAllNodes() IntRemove and free all nodes on the current silo.
channelOffset(offset Int) IntSet the channel offset for the current silo's output in the hardware buffer. With an offset of 2 on a 4-channel output, the silo writes to channels 2–3 instead of 0–1, letting different silos target different hardware channels (surround, multi-speaker). Clamped to the hardware channel count. Default 0.

18.6 Connections

These must be called within a begin()/go(silo) bundle. Node 0 is the hardware output; connecting to (0, 0) sends audio to the output. Node 1 is the hardware input; connecting from (1, 0) receives live audio (requires audio input to be enabled).

FunctionDescription
connect(srcNode Int, srcPort Int, dstNode Int, dstPort Int) IntConnect an output port to an input port. Takes effect instantly (no crossfade).
connectX(srcNode Int, srcPort Int, dstNode Int, dstPort Int, xfade Float, curve Int) IntConnect with a crossfade. xfade is the fade duration in seconds; curve is a FadeCurve.
disconnectInput(dstNode Int, dstPort Int) IntDisconnect whatever is connected to the given input port. Takes effect instantly.
disconnectInputX(dstNode Int, dstPort Int, xfade Float, curve Int) IntDisconnect an input port with a crossfade to silence.
disconnectSource(srcNode Int, srcPort Int, dstNode Int, dstPort Int) IntDisconnect the specific connection from the given output port to the given input port. Takes effect instantly.
disconnectSourceX(srcNode Int, srcPort Int, dstNode Int, dstPort Int, xfade Float, curve Int) IntSame, with a crossfade to silence.
disconnectOutput(srcNode Int, srcPort Int) IntDisconnect all connections from the given output port.
disconnectNode(nodeID Int) IntDisconnect all inputs and outputs of the given node.
reconnectOutput(oldSrcNode Int, oldSrcPort Int, newSrcNode Int, newSrcPort Int, xfade Float, curve Int) IntMove all connections from one output port to another, crossfading over xfade seconds.
replaceNode(oldNodeID Int, newNodeID Int, xfade Float, curve Int) IntReplace one node with another, reconnecting all inputs and outputs and crossfading over xfade seconds. The old node is freed after the fade completes.
Note: Connections are validated on the audio thread; a connection that would create a cycle, or whose ports disagree in element type, signal rate, or channel count, is rejected with the corresponding Err code (§18.10).

18.7 Parameter Control

These must be called within a begin()/go(silo) bundle.

FunctionDescription
setInput(nodeID Int, portIndex Int, value Float) IntSet an input port to a constant value (no crossfade). Overrides any existing connection on that port.
setInputX(nodeID Int, portIndex Int, value Float, xfade Float, curve Int) IntSet an input port to a constant value with a crossfade from the current value.
setControl(nodeID Int, controlID Int, value Float) IntSet a control parameter on a node. Controls are node-specific named parameters distinct from audio-rate input ports.
setControl(nodeID Int, name String, value Float) IntBy-name overload: the engine resolves the control name against the node's def when the bundle is submitted. An unknown name aborts the bundle with errControlNotFound.

18.8 Note / Voice Management

For nodes that support polyphonic voice allocation (e.g., voicer nodes). These must be called within a begin()/go(silo) bundle.

FunctionDescription
noteOn(nodeID Int, noteID Int, params [Float]) IntStart a new voice on the given voicer node. noteID identifies the voice for later noteOff/noteSetParams calls; params holds the voice's initial parameter values.
noteOff(nodeID Int, noteID Int) IntRelease the voice identified by noteID.
allNotesOff(nodeID Int) IntRelease all active voices on the given voicer node.
noteSetParams(nodeID Int, noteID Int, firstParam Int, values [Float]) IntUpdate parameters on an active voice: sets values length consecutive parameters starting at index firstParam.
Note: Code running inside a silo (a silo actor or task) triggers voices directly with playNote/releaseNote instead of a bundle — see §17.5.

18.9 Signal Taps

A tap reads a running signal back out of the engine: a node outlet, or the master output bus. It is the engine-level primitive behind the app's meters and scopes, and it needs no GUI — headless scripts, tests, and silo code all use the same functions.

FunctionDescription
allocTapID() IntA process-unique tap id. Use this rather than inventing ids: the same allocator hands out ids to the ui widgets and the graph view, so drawing from it makes collisions impossible.
tapOutlet(node Int, outlet Int, tapID Int, mode Int) IntBundled. Tap an outlet. mode is a TapMode. The outlet must carry audio (f32); anything else is rejected with errTypeMismatch.
tapMaster(tapID Int, mode Int) IntBundled. Tap the master output bus — post safety-limiter, post master gain, i.e. what the device plays. Must be submitted to silo 0. Node 0 (Audio Out) has no outlets, so this is the only way to reach the final mix.
untap(tapID Int) IntBundled. Remove a tap and free its slot. Must be submitted to the same silo the tap was installed on (master taps: silo 0); anything else is rejected with errSiloOutOfRange, because the removal has to run on the thread that owns the table holding it.
freeVmTaps() IntRemove every tap this VM created, and return how many. A silo VM sweeps its own silo's script taps; the main VM sweeps its own. Taps behind the app's own meters and scopes are never touched.
freeAllTaps() IntRemove every tap regardless of owner — including the app's. A reset-the-world escape hatch, not routine cleanup.
tapExists(tapID Int) BoolWhether the tap is live. Unknown ids read false rather than failing.
tapPeak(tapID Int) Float / tapRms(tapID Int) FloatLevel over the last publish window (512 samples), collapsed across channels. Unknown ids read 0.0.
tapChans(tapID Int) IntChannel count captured by the tap.
tapSamples(tapID Int, max Int) [Float]Drain captured samples (tapScope mode only): interleaved frames of tapChans channels. The count is clamped to whole frames and to 4096 samples; call repeatedly for more.
let id = allocTapID();
begin();
tapOutlet(700, 0, id, TapMode.tapMeter);
go(0);

-- ... later ...
if (tapPeak(id) > 0.99) { println("clipping"); }

begin(); untap(id); go(0);

One consumer per tap. A tap's samples come from a single-producer/single-consumer queue. Two readers draining the same tapID — a silo script and a GUI scope, say — is a data race, not merely a split stream. Give each consumer its own tap; that is what the ui widgets do.

Taps are a finite resource. Each silo's table holds a bounded number (128), shared with any ui meters and graph-view meters. Past that, tapOutlet fails at submit with errResourceLimit — reported immediately rather than handing back an id that would silently read silence. Untap what you no longer need; getEngineStats reports each silo's live tap count.

Bulk cleanup

Every tap records who made it, so a script can clean up after itself without tracking ids:

-- at the end of a piece, or before reloading silo code
freeVmTaps();

freeVmTaps() removes only the calling VM's taps. The app's own taps — those behind ui.meter/ui.scope widgets and the graph view's node meters — are tagged to the host and are never swept by it; being sweepable is opt-in, so anything that does not ask is safe. freeAllTaps() ignores ownership entirely and takes those too, which leaves the app's meters reading silence until the code that created them runs again — use it to recover from a leak, not as routine teardown.

A silo VM's taps are keyed by silo index, not by VM identity. siloLoad replaces a silo's VM, and keying on the VM would strand the previous one's taps with nothing able to reclaim them; the trade is that a reloaded VM inherits, and can free, its predecessor's taps on that silo — usually what you want when reloading.

Both are housekeeping calls, not per-block ones: they walk the tap registry and submit one bundle per silo. Neither joins a bundle you have open — called between begin() and go() they remove nothing and return 0, rather than clobbering it.

Reading from a silo VM

All the read functions, and the bundled create/remove commands, are RT-safe, so silo code (§13) can meter its own signal:

-- inside code loaded onto a silo with siloLoad
fn watch(id Int) Void {
    if (tapRms(id) < 0.001) { -- the voice has died away }
}

From a silo, a read resolves through that silo's own tap table: no lock and no shared map, because the thread doing the reading is the thread that publishes the values. The cost is scope — a silo sees only its own taps (silo 0 additionally sees master taps). Cross-silo reads are not supported: a tap belonging to another silo reads exactly as an unknown id does, false / 0.0 / no samples. Read another silo's taps from the main VM instead, where the locking path sees every tap.

18.10 Introspection & Utilities

FunctionDescription
listSynthDefs() [String]Names of all registered node definitions (synth plugins). NRT-only.
readFile(path String) StringRead a file's contents into a String, or "" if it cannot be opened. NRT-only host utility — handy for loading silo task code: await siloLoad(0, readFile("tasks/bass.x")).
sleepMs(ms Int) VoidBlock the calling thread for ms milliseconds. NRT-only; used by polling loops such as the actor server (§17.4). For musical timing, prefer the clock module (§11) over sleeping.

18.11 Enums

Defined in the wrapper module bridge/modules/audio_engine.x; import audio_engine.*; brings them in together with the foreign functions. Pass the enum case directly wherever a function takes the corresponding Int parameter.

FadeCurve

CaseDescription
fadeLinearLinear interpolation
fadeExponentialExponential curve (perceptually even volume change)
fadeSmoothstepHermite smoothstep (S-curve)
fadeEqualPowerEqual-power crossfade (constant energy)
fadeOutInFade out then fade in (dip in the middle)
fadeEaseInCubicCubic ease-in (slow start, fast end)
fadeEaseOutCubicCubic ease-out (fast start, slow end)

SchedPolicy

Controls what happens when a beat-scheduled bundle arrives after its beat has passed; see §11.6 for the case-by-case behavior. Cases: schedImmediate, schedBetterLateThanNever, schedOnTimeOnly.

Err

Error codes returned by the engine FFI functions. 0 (errNone) is success.

CaseDescription
errNoneSuccess (0)
errInternalInternal engine error
errNodeIDAlreadyTakenNode ID already in use on this silo
errNodeDefNotFoundNo plugin definition with the given name
errNodeNotFoundNo node with the given ID
errNoteNotFoundNo active note with the given ID
errControlNotFoundNo control with the given ID
errDeviceNotFoundAudio device not found
errAlreadyAddedNode was already added to the silo
errAlreadyRemovedNode was already removed from the silo
errSiloOutOfRangeSilo index out of range
errInputOutOfRangeInput port index out of range
errOutputOutOfRangeOutput port index out of range
errNoAudioDevicesNo audio devices available
errAudioNotInitializedAudio stream not initialized
errCommandsQueuedButNotSentCommands queued but go(silo)/sched(silo, …) not called before the next begin()
errNoActiveBundleNo begin() was called before commands — the only error the command builders themselves return; all other bundle-related codes come from the submit call
errEngineInUseEngine is already in use
errCyclicConnectionConnection would create a cycle
errTypeMismatchPort element types do not match
errRateMismatchPort signal rates do not match
errChanMismatchPort channel counts do not match (connect adapts them instead; this is for setInput/replaceNode, which do not)
errNumPortsMismatchNumber of ports does not match
errNotImplementedFeature not yet implemented
errTooLateScheduled event missed its deadline
errClockOutOfRangeTempoClock slot index out of range

Enable

CaseDescription
kOffDisabled (0)
kOnEnabled (1)

18.12 Example

Create a 440 Hz sine oscillator, fade it in, hold it, then fade it out:

import audio_engine.*;

engineStart();

begin();
newNode("sinosc", 101);
setInput(101, 0, 440.0);                                -- frequency
setInputX(101, 1, 0.2, 0.5, FadeCurve.fadeLinear);      -- fade amp to 0.2 over 0.5 s
connect(101, 0, 0, 0);                                  -- node output -> hardware out
go(0);

sleepMs(3000);

begin();
setInputX(101, 1, 0.0, 1.0, FadeCurve.fadeEaseOutCubic);
go(0);

sleepMs(1500);
engineStop();

For beat-accurate scheduling of bundles instead of sleeping, see the examples in §11.9.