jpmml-evaluator library (used for PMML model
scoring). The nixpkgs fetch source hash has been updated to restore
local build compatibility.Fix for https://github.com/b-rodrigues/tlang/issues/434
jpmm-statsmodels PR is not yet merged into NixOS/nixpkgs master branch, so it’s included in the rstats-on-nix fork. It seems like the Maven dependencies got cahnged after release so compilation was failing. This also means that previous releases of T are likely not installable anymore.
with_tz() and force_tz() now correctly apply
timezone offsets via POSIX localtime_r, supporting full
IANA timezone names (e.g. "Europe/Paris",
"America/New_York").to_factor on all-NA input now preserves factor metadata
(levels, ordered) through the Arrow round-trip, keeping the native Arrow
path active across data cleaning pipelines.VNA variants (NAInt, NAFloat,
NADate, NADatetime, …) now set the
corresponding type flag in values_to_column, so all-NA
columns route to the correct typed Arrow column
(e.g. IntColumn, FloatColumn) instead of
always falling back to NAColumn. Added
NADatetime variant to disambiguate datetime NAs from date
NAs.lag and
lead now inspect the input vector’s element type and use
the appropriate typed NA variant for fill positions
(e.g. NA(Int) for integer vectors, NA(Float)
for float vectors).group_by + summarize and typed NA for
Date/Datetime null cells in pivot_wider and
expand.get() data-mask awareness:
get("name") inside NSE data verbs (mutate,
filter, summarize, …) now checks the data mask
before the global environment, enabling dynamic column name
resolution.This release:
t init --project and t init --package now
accept --include-atelier (CLI flag) to configure the IDE in
the project environment. When running inside an active Atelier session,
the REPL automatically updates variables, writes diagrams/plots to the
_atelier/ directory, and runs a protected variables watcher
via the new tui_update builtin. Atelier can also be
declared as a project dependency via [additional-tools] in
tproject.toml.node_when(condition, node_value) and
node_fork(...) allow conditional node inclusion evaluated
at pipeline construction time, preserving Nix’s static DAG
requirement.map_pattern(dependency): Maps a node over each element
of a List, Vector, or DataFrame dependency.cross_pattern(sub_pattern1, sub_pattern2, ...):
Generates a Cartesian product of multiple map_pattern
sub-patterns.slice_pattern(dependency, indices): Creates branches
selecting specific element indices.head_pattern(dependency, n) /
tail_pattern(dependency, n): Restricts branches to the
first or last n elements.sample_pattern(dependency, n): Randomly samples
n elements from a dependency (with deterministic seed
behavior).expand_pipeline: Adds the built-in
function expand_pipeline(p) to manually expand dynamic
patterns into separate <node>_branch_<N>
nodes.build_pipeline() and populate_pipeline().
Attempting to build/populate a pipeline containing unexpanded patterns
raises a compile-time StructuralError._branch_N are strictly forbidden in
manual definitions to prevent namespace clashes. Duplicate node
detection and branch-aware read_node errors assist in
identifying naming conflicts.pipeline_to_ga: Adds
pipeline_to_ga(p, ...) to generate a complete GitHub
Actions CI workflow YAML. The generated workflow automates Nix-based
pipeline runs on push/PR events.rstats-on-nix cache) and automatically
manages the caching of built Nix artifacts using a repository’s
t-runs branch (archived as .nar files).file parameter to write the workflow YAML directly to
.github/workflows/<name>.yml (defaulting to
auto-detected project names).pipeline_report: Introduces
pipeline_report(p, ...) to generate structured Markdown
(target = "ssh") or HTML (target = "web")
execution reports.which_log
regex parameter to report on historical run records.float_seq: Adds the
float_seq(start, end, n) built-in function to generate a
List of n evenly-spaced floats.{ ... }) no longer silently swallow errors occurring
inside assignment/reassignment statements. The error is bound to the
variable and propagated correctly, enabling robust recovery patterns via
subsequent match blocks.This release:
pipeline_of), first-class artifact export/import
capabilities, meta-pipeline graph visualization, interactive Mermaid
browser rendering, and cache-aware dry runs with programmatic garbage
collection.{ ... }) now abort immediately on encountering a
VError from a bare expression, preventing silent error
accumulation. However, VError values from
Assignment and Reassignment statements no
longer abort the block — the error is bound to the variable and
subsequent statements (typically a match) can handle it.
This enables patterns like
{ x = 42 / 0; match(x) { Error { msg } => 0, default => x } }
where a risky computation is captured and recovered.VError now conditionally fall back to
binary serialize/deserialize rather than
triggering configured custom serializers (like Arrow), preventing
crashes.get_pipeline_member function no
longer eagerly evaluates T expressions when a pipeline is accessed.
Evaluation is deferred to rerun_pipeline at build time via
build_pipeline(). This eliminates redundant re-evaluation
cycles, improves performance for large pipelines, and avoids side-effect
leakage from unbuilt nodes.p.p_nodes instead of
p.p_exprs for identifying resolved nodes from other
pipelines, improving accuracy of dependency inference.build_pipeline(p), downstream nodes now automatically
inherit warnings from ancestor nodes.
warning_msg(downstream) shows warnings with source
provenance
("Ancestor node '<name>' reported following warning: <message>").
Multiple warnings (own + upstream, or from multiple ancestors) are
joined with ". Furthermore, ".inspect_node now shows warnings:
inspect_node(node) returns a warnings field
with a structured list of dicts (source +
message), showing both own and inherited upstream
warnings..warnings from read_node()
return: The .warnings field on
read_node() results has been removed. Use
warning_msg(node) for a formatted warning message or
inspect_node(node).warnings for structured warning
metadata.read_past_node(p.node_name, which_log):
New NSE-captured function to read a pipeline node from a specific
historical build log without the pipeline being in scope. The first
argument is captured before evaluation, so
read_past_node(base_p.raw, which_log = "qcfs") works even
when base_p is not defined. which_log is
mandatory — no default.read_node: The
which_log parameter has been removed from
read_node. Use
read_past_node(p.node_name, which_log = "...") for
historical reads, and read_node(p.node_name) for in-scope
pipeline reads.pipeline_name on
build_pipeline:
build_pipeline(p, pipeline_name = "my_name") now records
the pipeline name in the build log JSON ("pipeline" field).
list_logs() shows a new pipeline column for
disambiguation.pipeline_summary and
pipeline_dot: These convenience aliases have been
removed. Use pipeline_to_frame(p) instead of
pipeline_summary(p), and pipeline_to_dot(p)
instead of pipeline_dot(p). pipeline_to_dot
also handles MetaPipeline.pipeline_to_dot(p, title = na()):
Generates a Graphviz DOT representation of the given pipeline or
meta-pipeline. New optional title parameter auto-detects
the project name from tproject.toml (fallback: none).
Renders as label= in the digraph header.pipeline_to_mermaid(p, title = na(), flatten = false):
Generates a Mermaid flowchart diagram string from the pipeline topology.
New optional title parameter auto-detects from
tproject.toml; new flatten parameter (default
false) renders meta-pipelines as grouped subgraph blocks —
set to true for flat output.---\ntlang-title: ...\n---), visible in the HTML
<h1> via show_plot() but silently
ignored by Mermaid.js to avoid in-diagram title duplication.#ffced0 to #859900 (green) for a cleaner
visual appearance in generated graphs.show_plot:
show_plot(p) now accepts a pipeline or meta-pipeline
directly — it calls pipeline_to_mermaid internally, renders
the DAG as an interactive HTML page, and opens it in the browser.
show_plot(mermaid_string) also renders arbitrary Mermaid
diagram strings.read_node “Did you mean” hint: When
read_node receives a bare symbol
(e.g. read_node(ha) instead of
read_node(p.ha)), the error now suggests the correct form:
Did you mean read_node(p.ha)?read_node error messages: All
non-ComputedNode argument errors now follow a consistent
format guiding the user to build the pipeline first, use dot access, or
use read_past_node for historical builds.export_artifacts: Support
exporting cached Nix artifacts for individual nodes, sub-pipelines,
meta-pipelines, or lists/dictionaries of nodes/pipelines.import_artifacts: Support
both 1-argument import_artifacts(archive_path) (direct Nix
store import) and 2-argument
import_artifacts(target_val, archive_path) (import and
verify paths) calling signatures.inspect_artifacts(archive_path):
Import archive into a temporary, isolated Nix store and return a
DataFrame containing the included nodes, store paths, hashes, sizes in
bytes, and reference basenames without affecting the local store.import_artifacts,
export_artifacts, and inspect_artifacts.populate_pipeline(p, dry_run = true) and
build_pipeline(p, dry_run = true) now perform a dry run via
Nix (--dry-run), parsing the plan to report which nodes
will hit the local cache ("cached"), rebuild
("build"), or fetch from remote substituters
("fetch"), returning the results as a DataFrame.pipeline_gc(p, dry_run = true): Deletes the store paths
of the given pipeline p if safe. By default
(dry_run = true), it returns a DataFrame previewing the
nodes, store paths, and deletion eligibility status. Set
dry_run = false to execute the deletion.t_gc(): Triggers a global Nix garbage collection
(nix-store --gc) directly from the REPL to safely clean up
old, detached derivations.pipeline_of block: A new combinator
that composes multiple pipelines into a higher-order DAG. It allows you
to define relationships between sub-pipelines in a declarative way,
enabling complex, multi-stage workflows.etl.clean in the stats pipeline) to infer the
execution order between sub-pipelines. No manual depends
configuration is required for the flattening engine.meta_flatten
combinator automatically flattens meta-pipelines at execution time. When
a meta-pipeline is populated, queried, or inspected, T-Lang
automatically flattens it internally. This flattening is done on-demand,
so you don’t need to manually flatten meta-pipelines.etl.raw, etl.clean,
stats.summary) to prevent namespace collisions, and all
internal variable references are rewritten accordingly.p_etl.raw) are
automatically rewritten to their namespaced equivalents (e.g.,
etl.raw) during the flattening process.pipeline blocks (e.g.,
\(multiplier) pipeline { ... }) are now fully supported.
Outer variables referenced inside the pipeline nodes are automatically
substituted with their concrete values during compilation, producing
fully independent and Nix-reproducible pipelines.# Define multiple pipelines
p_etl = pipeline { ... }
p_stats = pipeline { ... }
# Compose into a meta-pipeline
meta = pipeline_of {
etl = p_etl
stats = p_stats
}
# Built-in commands automatically handle meta-pipelines
populate_pipeline(meta, build = true)
read_node(meta.stats.summary)
inspect_pipeline(meta)meta_graph = pipeline_of {
raw = pipeline {
src = read_csv("raw.csv")
}
cleaned_a = pipeline {
a = clean(raw.src)
}
cleaned_b = pipeline {
b = clean(raw.src)
}
summary = pipeline {
val = summarize(cleaned_a.a, cleaned_b.b)
}
}
# T-Lang automatically infers the execution order:
# raw -> {cleaned_a, cleaned_b} -> summary
populate_pipeline(meta_graph, build = true)meta_flatten combinator is not exposed as a
first-class function in the CLI or T-Lang AST. It is an internal
implementation detail of the pipeline engine that is automatically
invoked when working with pipeline_of.fivenum, ensuring parity with R’s
fivenum() output.Float.erfc, improving tail accuracy.t_quantile recursive to handle edge cases in quantile
computations correctly.ss_res formula used in leave_one_out_sigma
calculation in lm.ml.cut to
output spaces-free labels formatted with %g format,
matching R’s cut() label style.3.0 == 3) and improved
pnorm accuracy using Float.erfc.VFactor and VString values.read_csv to parse empty strings as NA with
allow-null-strings, and implemented proper dataframe
comparison semantics.jsonlite::write_json to serialize floats at full precision,
preventing rounding-induced mismatches.NA values as JSON
null for correct round-tripping.node.sub.field) in Nix dry-run evaluation expressions.pipeline_to_mermaid to prevent
collisions from similar node names.inspect_artifacts, and restored scalar
TypeError for invalid inputs.NixError fallback behavior for empty trimmed
last_part in builder_internal.ml.__node_result across all
runtimes), fixed R variable naming compatibility (dep_
prefix), and corrected dependency namespace generation.rename_node updated the assoc-list key in
p_nodes but left VComputedNode.cn_name
unchanged, causing downstream lookups (resolved_cn,
computed_node_resolver, read_node) to search
using the stale name and fail with <unbuilt> path
errors.The entire OCaml codebase underwent a systematic safety review following best practices for ML-family languages:
Option.get, bare List.hd on unvalidated lists,
and similar partial operations.match expression across ~100 files was audited and made
total, removing partial wildcard patterns where feasible.failwith, raise, and assert false
in user-facing code to structured VError returns.Float.compare..mli
interface files with abstract types for Arrow,
GroupedTable, and key FFI modules.AGENTS.md as a permanent reference for future
contributions.This release introduces interactive pipeline node debugging via
debug_node, native Nix orchestration features for granular
rebuild control, job parallelisation, Cachix binary caching, and
dry-runs, and the temporal introspection pair
build_log_history and node_diff for tracking
how pipeline outputs change across builds.
Status: Beta
debug_node(p.node)): Introduces a new built-in
function to drop developers directly from the T REPL into a sandboxed
guest REPL (Python, R, or Julia) to step through and debug code using
actual upstream outputs.py>, r>,
jl>) to cleanly signal that you are in a debugger
subshell session, returning immediately to the T REPL upon exit.p_env_vars) are programmatically inherited by the subshell
process.ValueError for unsupported runtimes
(like Quarto or Bash).debug_node(), running R, Julia, or Python directly inside
the development shell started via nix develop will
automatically intercept and block imperative package mutations
(install.packages(), Pkg.add(),
pip install, poetry, uv,
conda, python -m pip, etc.). Running these
commands displays a helpful instruction directing developers to declare
dependencies in tproject.toml, run t update,
and re-enter nix develop, protecting the workspace from
drift and preserving reproducible Nix derivation footprints.build_log_history(p, n = NA)): Exposes the
historical record of builds matching the current pipeline signature as a
sorted DataFrame, ordered from most recent to oldest. Uses the 1-indexed
build_rank convention (where 1 represents the
most recent build, 2 the second most recent, etc.).node_diff(node_a, node_b, log_a = "latest", log_b = "latest")):
Compares outputs of a specific node across two historical builds
(defaulting to the most recent vs. second most recent). Implements
type-sensitive comparison strategies:
diff -u to extract
precise line additions, removals, and diff summaries. Includes a robust
fallback if system tools are sandboxed or missing."default"/"tobj" Deserialization:
Fixed a major correctness bug in read_standard_node_value
where scalar nodes serialized with "default" or
"tobj" formats were not being deserialized when queried via
standard readers, returning a fallback VComputedNode token
instead. Standard readers now correctly deserialize value payloads (like
VInt, VFloat) using OCaml’s Marshal digestion,
enabling precise cross-node value and delta comparisons.targets, force, dry_run,
max_jobs, and cache parameters in
build_pipeline and pipeline_run.targets): Map
targets to -A <derivations> in the
underlying nix build command, allowing specific parts of
the pipeline to be built selectively.force): Map
force to native --check flags. Pass
true to force-rebuild the entire pipeline, or a string/list
of specific node names to force-rebuild only selected steps.max_jobs):
Mapped the max_jobs parameter directly to
--max-jobs <N>, enabling parallel compilation of
sandbox environments and derivations.cache):
Seamless Cachix binary cache integration by dynamically configuring
extra-substituters and
extra-trusted-public-keys (prioritizing
rstats-on-nix as the preferred default cache).dry_run):
Implemented a native dry-run mode that parses
nix-build --dry-run output into a structured T-Lang
DataFrame (containing columns node,
action, path) to inspect build execution plans
without mutating local store state.update_pipeline_with_build_paths) that reconciles
internal ComputedNode paths with the real store paths
generated by Nix.t doctor
Pipeline Dependency Analysist doctor now parses src/pipeline.t and
statically analyses each node’s command block to detect
runtime packages (library(...), import ...,
using ...) that are referenced but absent from
tproject.toml. Missing packages are reported as warnings
with an actionable suggestion to add them to the relevant
[r-dependencies], [py-dependencies], or
[jl-dependencies] section and run t update.
All pipeline definitions in the file are scanned, not just the first
one.src/pipeline.t found) is only emitted when the
project has at least one runtime dependency declared, avoiding noise for
pure R or Julia package projects.test_pipeline.ml to verify dry-run
DataFrame output, validation guards, and advanced parameter passthroughs
(2271/2271 tests passing).docs/pipeline_tutorial.md and
docs/api-reference.md to formally document the new
parameters, along with comparative command mapping tables.populate_pipeline() to support all the new Nix
orchestration arguments (targets, force,
dry_run, max_jobs, cache) in the
exact same manner as build_pipeline().targets and force
node lists in the OCaml pipeline compiler. T-Lang now instantly detects
misspelled or nonexistent node targets and raises highly readable
StructuralError warnings before spawning the Nix
interpreter.model) would erroneously match long node name store
paths (e.g. model_evaluation).node_diff improvementsnode_diff now splits the values
on newlines and produces a proper unified diff with context lines — the
same colourised format already used for text-file nodes. Calling
detailed_summary on the result shows added/removed lines
highlighted in green and red.NaN / NA handling in
DataFrames: Cells that contain NaN or
NA on both sides are no longer incorrectly reported as
changed.key that does not exist in one of the DataFrames,
node_diff now raises a clear error immediately instead of
silently producing wrong counts.node_diff requires ComputedNode
arguments: node_diff now enforces that both
arguments are pipeline node references
(e.g. node_diff(p.my_node, p.my_node)). Passing a plain
string or pipeline object raises a descriptive
TypeError.explain()
— Unicode display→, emoji, …) are displayed as-is in the REPL and inside
explain() tree output, instead of being shown as raw byte
sequences such as \226\134\146.This release finalizes end-to-end Julia ONNX serialization support, fixes pipeline compiler strategy dictionary parsing issues, strengthens runtime safety by protecting reserved keywords, and completes the migration of pipeline introspection to a strict, node-centric dot-access model.
Status: Beta
read_node() to strictly require ComputedNode
arguments (e.g., read_node(p.node_name)), disallowing
legacy string lookup paths.read_node OCaml resolution to prioritize the in-memory
registry (Ast.in_memory_node_values) over disk-based build
log artifacts, resolving transient FileError omissions when
accessing unbuilt or dynamically computed nodes.build_pipeline()
execution, instructing users how to read, inspect, and summarize their
pipeline using their actual variables and first-class node objects.t_demos workspace and workflows (79+ scripts and workflows)
to adopt the new strict dot-access design. Standalone helper scripts are
now automatically self-contained with explicit
import 'src/pipeline.t' prepends.build_log(p)):
Expose the underlying Nix build results as a VBuildLog
record containing node-by-node details, total duration, and a list of
failed nodes. build_pipeline(p) now returns a
BuildLog value instead of a raw output-path string (use
build_pipeline(p).out_path when you need the previous path
value).build_log_to_frame):
Added build_log_to_frame(log) to tabulate build results
(one row per node) for high-level analysis using colcraft
verbs."Pending" or
"Building" states to "Skipped" when the
nix-build process crashes/fails, avoiding confusing out-of-date
states.VBuildLog stringification and REPL pretty-printing to
dynamically count and print all status types
(e.g. 2 succeeded, 9 failed, 3 skipped) rather than
assuming a simple binary success/failure count.collect_exceptions(p)): Gathers all
VError values and warning diagnostics from computed nodes
of a built pipeline into a structured DataFrame (node,
status, code, message), replacing
the legacy collect_errors and error_summary
functions.collect_exceptions(p): Automatically cleans and
extracts the last non-empty line of multi-line error traces (such as
from Python or Arrow exceptions) and truncates the string to 100
characters max, maintaining a neat, legible table in the REPL.error_code(), error_msg(), and
error_context() builtins polymorphic. They now accept
either standard Error objects or first-class pipeline
ComputedNode values (e.g. p.X or
p.combined_df). They automatically resolve the node’s
underlying VError store artifact for soft-failures, or fall
back to parsing log traceback details for hard Nix-build failures.warning_msg() and a matching
.warning_msg property lookup on computed nodes to easily
inspect non-fatal build warnings from successful derivations.t_make():
Upon early termination/build crash, the Nix builder scans the store
paths of completed upstream nodes to extract soft-failed diagnostic
states, printing them directly in the final build failure summary
(e.g. showing (8 captured errors) and listing their node
names).35 characters to
32 characters followed by ... to keep columns
aligned and clean.error_chain(err1, err2)): Chains multiple errors
to preserve failure provenance and causality across dependent
nodes.build_log, print, mean, etc.) are
now strictly protected against accidental user reassignment or
overwriting.= or the
overwrite operator := will raise a highly visible
NameError
(e.g. Cannot overwrite build_log: it's a reserved keyword!).mean)
in their package scope without conflict.:Cast (pass-through identity) and
:Reshape (handling inferred -1 shapes mapping
to Julia Colon()) operators.Base.invokelatest to resolve world age lexical
method updates in isolated Nix builds.Umlaut dependency into [jl-dependencies] and
corrected column-major tracing dimension layouts.nix_emit_node.ml
(get_format) and builder_populate.ml
(extract_format) to correctly parse
VSerializer values inside pipeline deserialization mapping
dictionaries (e.g.,
deserializer = [ julia_model: ^onnx ]).onnx_julia_stress_t and
observability_hardening_t end-to-end stress test suites to
verify prediction parity, safety safeguards, and observability
logs.jn() alias: Eliminated the
undocumented jln() alias jn() from the
evaluator, tests, and all documentation. Use jln()
exclusively for Julia pipeline nodes.Returns docs:
All node-defining functions (node, rn,
pyn, jln, qn, shn)
now correctly document their return type as a NodeDef
pipeline node configuration object, not the evaluated result of the
enclosed code. The code is executed by build_pipeline(),
not immediately.jln serializer default:
Documentation previously stated the default serializer was
^csv; the actual default is the runtime-native binary
serializer (jl_serialize), consistent with rn
and pyn.The focus of this release is the introduction of first-class Julia support, enabling high-performance polyglot pipelines with seamless Julia integration.
Status: Beta
jln):
jln() for executing Julia code directly
within T pipelines.tproject.toml via the [jl-dependencies]
section.^pmml serializer.JavaCall.jl
integration.^onnx serializer.ONNXRunTime.jl for industry-standard
inference performance and ONNX.jl for model
serialization.MethodError: method is too new).Base.invokelatest.Flux.jl or Zygote.jl) remains accessible
within the same execution cycle, even in restricted environments like
the Nix build sandbox.JSON package in Julia nodes, enabling seamless JSON-based
data exchange for Julia-based pipeline steps.show_plot() now supports Julia plots via
TidierPlots.jl, Plots.jl, and
Makie.jl.Makie.jl
objects, CairoMakie is the mandatory backend for
reproducible headless rendering within the Nix sandbox. Ensuring
CairoMakie is in [jl-dependencies].packages is
required for successful visual inspection of Makie nodes.read_node Helpers: Introduced
lightweight packages for R, Python, and Julia (all named
tlang) to simplify consumption of T-Lang build artifacts
from external runtimes.pipeline_nodes() to all companion packages. It returns the
pipeline DAG as an idiomatic data structure (e.g.,
data.frame in R, dict in Python/Julia),
enabling easy programmatic traversal of node relationships.path: field from the default
ComputedNode REPL printer.path: <unbuilt> / path: status lines for
ComputedNodes; users who need explicit artifact paths in
the T runtime can obtain them via inspect_node(node).path
or inspect_log().return_path in Companion
Packages: Added return_path argument to
read_node() in the R, Python, and Julia companion packages.
When set to true, these helpers return the absolute path to the artifact
in the Nix store/project directory instead of deserializing it, allowing
for custom loading logic or direct file inspection.build_log_*.json in
the _pipeline/ directory, providing a stable way to access
node results during development and reporting (e.g., in Quarto). ###
Strict Serialization & Pipeline Stability^ symbols for node serializers and
deserializers (e.g., serializer = ^arrow).TypeError during evaluation,
eliminating a common source of pipeline configuration drift.write_arrow to surface detailed
VError traces instead of failing silently.dataframe as a core standard package,
ensuring stable resolution during Nix builds and preventing “package not
found” errors in isolated environments.to_factor was being incorrectly
emitted; now correctly uses the standard R factor() for
native R-node interoperability. ### API Standardization &
Ergonomicsto_ Naming Convention:
to_ prefix:
as_date() → to_date()as_datetime() → to_datetime()as_factor() / factor() →
to_factor()sym() → to_symbol()dataframe() → to_dataframe()str_string() → to_string()as_* and shorthand aliases
(fct(), fct_infreq() →
to_factor(..., ordered=true) etc.) to ensure a single,
canonical API path.augment() renamed to add_diagnostics() for
better clarity and consistency with the T-Lang philosophy of descriptive
names.to_string() now provides a unified interface for string
conversion across all T types, including proper level resolution for
Factors and recursive formatting for Lists and Vectors.fct() shorthand in favor of the
standardized to_factor().to_factor() now consistently
uses alphabetical sorting for derived levels, removing the previous
“first-appearance” behavior to align with industry standards and
internal consistency.augment() to add_diagnostics() to
better reflect its purpose of appending model-level diagnostics
(residuals, hat values, etc.) to data frames.broom::augment outputs.t_demos to adopt the new standardized API.The focus of this release was to improve language ergonomics for data guardrails, enhance package manager feedback, and increase test coverage across all packages.
Status: Beta
nest() and unnest() operations to eliminate
OCaml-side materialization bottlenecks.GroupedTable to use gint64 row
indices, enabling direct bulk transfer of group subsets to Arrow.merge_horizontal), enabling efficient
expansion of key columns during unnesting.t update feedback to explicitly report counts
for R, Python, Additional Tools, and LaTeX packages being synchronized
to flake.nix.get():
Enhanced the get() primitive to support default value
fallbacks. It now handles:
get(potential_error_or_na, default): Returns the
default if the first argument is an error or NA.get(target, selector, default): Returns the default if
the retrieval from the target fails.get(s.min_age, 0) >= 0, which gracefully handles missing
columns or empty summaries.generate_onnx.py compatibility with scikit-learn
1.6+ and ensured numeric output parity for classification models.1e-2 to preserve
cross-runtime stability without masking meaningful regressions.cv,
fivenum, trimmed_mean, mad,
iqr, range, var, and
cov against R baselines.skewness and kurtosis (excess kurtosis) using
population-moment calculations.pnorm (standard normal approximation), pt,
pf, and pchisq CDFs.winsorize, huber_loss, normalize,
and Pearson cor against R reference values.weights argument for lm()
to support Weighted Least Squares (WLS) regression.mean,
sd, var, cov, cor,
median, quantile, iqr,
fivenum, trimmed_mean, skewness,
and kurtosis.standardize and scale using
iris$Sepal.Length.coef, conf_int, sigma,
nobs, and df_residual for linear models.builder_utils.ml and test_misc_coverage.ml.
Switched to Unix.realpath for canonical path resolution on
macOS (handling /var vs /private/var) and
ensured GNU utilities are explicitly used via Nix environment
wrappers.X to float_input to match
scikit-learn 1.6+ exports.model_type
and mining_function metadata into lm and
PMML-loaded model objects. This ensures that fit_stats()
returns complete, R-compatible diagnostic tables without NA
placeholders for model categories.anova: Updated the
anova builtin to support model labels (e.g.,
anova(m1 = m1, m2 = m2)). The labels are now preserved in
the resulting DataFrame, matching R’s behavior in model comparison
tables.normal_quantile,
t_quantile) where tail approximations were incorrect,
leading to broken confidence intervals. Implemented high-precision
Acklam’s algorithm for normal quantiles and accurate Cornish-Fisher
expansion for t
quantiles.onnxruntime CPU vendor warnings in the golden test suite
and standardized the “✓” success indicator across all statistical test
scripts.base package
builtins, specifically targeting error handling, NA container logic, and
serialization.is_na
vectorization across Vectors and named Lists.serialize, deserialize,
t_write_json, and t_read_json, including
type-mismatch and file-system failure scenarios.core package builtins,
including args, help, apropos,
and write_text.args() builtin on both builtins and lambdas, ensuring
correct parameter name and type extraction.identical (deep equality), sum (edge cases),
seq (auto-descending ranges), and
head/tail (slicing boundaries).ifelse, case_when, and
identical (t_boolean.ml), get with all lens
types (t_get.ml), and all specialized rendering paths in
pretty_print.ml.fill, replace_na, complete,
relocate, count, slice,
unnest, separate, and uncount.
Verified downup direction logic and regex error
handling.pretty_print.help() and
apropos().lens
package, focusing on custom lenses, pipeline orchestration, and
recursive mapping.get and set
functions.node_meta_lens for serializer and
deserializer fields, and filter_lens for batch
updates to pipeline node values.col_lens and idx_lens across nested Lists of
Vectors and DataFrames.modify()
builtin.Functionlengthexpects...).packages.t-coverage Nix output for simplified code coverage
collection.bisect-ppx-report within the coverage derivation to
streamline local reporting workflows.Status: Beta
small, medium, full,
huge).huge reference generation: Implemented
a recursive build process in docs/build.sh that
concatenates the entire documentation ecosystem into a single,
comprehensive reference for high-context agents.t init to be fully interactive. The CLI now
prompts for the AI Agent Context Level during project
or package initialization.AGENTS.md: Automatically generates
project-specific onboarding guides for LLMs based on the workspace
type.T-LANGUAGE-REFERENCE.md in the project root and
automatically added to .gitignore.README, Getting Started, and
LLM Collaboration guides to emphasize T’s unique support
for agentic development.$param):
$param syntax in lambda and
function() parameter lists.$ automatically capture bare
names (like column names) as Symbols rather than
evaluating them.enquo() in simple forwarding cases.get() and New to_symbol()
Builtin:
to_symbol() core builtin for programmatic
symbol creation.get() dispatcher across core
and lens packages, ensuring a single, stable interface for
variable lookup, collection indexing, and lens-based retrieval.lens package is
loaded.StructuralError Category:
StructuralError as a new terminal diagnostic code
for fundamental pipeline orchestration failures.StructuralError.populate_pipeline (e.g., missing dependencies in
tproject.toml, missing Nix build tools, or stalled artifact
directories) now emit StructuralError.StructuralError as a
terminal event that bypasses “Resilient-by-Default”
(resilient=true) settings.1 / 0).TLANG_AUTO_ADD_PIPELINE_DEPS environment variable.StructuralError
instead of implicitly continuing into a broken Nix state.unbuilt states instead of resolved values in complex
dependency graphs.MissingArtifactError in the lens
system to provide precise feedback on unbuilt dependencies during lazy
evaluation.get() when targeting
plotting nodes, ensuring metadata dictionaries are returned
predictably.ggplot2 objects.matplotlib figures,
plotnine (ggplot-style), seaborn grids,
plotly figures, and altair charts.show_plot() Builtin:
show_plot() to render and open pipeline plot
artifacts locally.ggplot2), Python
(Matplotlib, Seaborn, Plotly, Altair, Plotnine), and Julia
(TidierPlots.jl, Plots.jl,
Makie.jl via CairoMakie) plots within the Nix
sandbox.kaleido) and Altair (via
vl-convert).tlang now
automatically suggests or injects cloudpickle when plotting
libraries are detected in Python nodes to ensure reliable serialization
of complex objects containing lambdas.read_node() for Plots:
read_node() now recognizes nodes of class
ggplot, matplotlib, plotnine,
seaborn, plotly, altair,
tidierplots, plotsjl, or
makie.VLens sum type.get() Integration: The
get() builtin now natively supports VLens for
data focus, providing a single, consistent interface for variable
lookup, indexing, and lens-based retrieval.get(), to_symbol(), and related
primitives.t_doc("parse") and t_doc("generate") workflows
for extracting and publishing reference pages for new core
functions.docs/language_overview.md and
docs/quotation.md with comprehensive examples of the new
$param auto-quoting feature.editors/tree-sitter-t.<{ }>
blocks when used inside rn(), pyn(), or
shn() calls (supporting both named and positional
arguments).qn()
as a first-class convenience wrapper around node() with
runtime = Quarto, matching the existing rn()
and pyn() helpers for R and Python nodes.nix_emit_node.ml to support the new visualization injection
logic and improve script-based node robustness.get_sym_demo_t comprehensive demo project with dedicated CI
validation and automated assertions.Printf.sprintf type error in the Python plot rendering
logic that prevented pipeline builds for Python-based
visualizations.show_plot calls so the rendered path
is reported cleanly before local viewer launch.lens.ml.filter_node(is_na($diagnostics.error)) and
which_nodes(is_na($diagnostics.error)) could evaluate
outside the node metadata scope and incorrectly exclude nodes without
diagnostics errors.resilient=true). This ensures that scripts
and pipelines continue execution upon encountering VError
values, aligning with the “Errors are Values” philosophy.--failfast Flag: Replaced the
--resilient CLI flag with --failfast. Users
can now explicitly opt-in to the usual, common behaviour of
short-circuiting upon the first error.t_make() & t_run()
Integration: Added failfast parameter to the main
pipeline orchestrator and script runner for granular control.VError values were deserialized as
Dictionaries. The system now correctly restores the native
Error type across node boundaries, even when using modern
JSON interchange.Status: Beta
VError
objects, allowing independent branches to complete.suppress_warnings combinator to silence high-noise nodes
while maintaining background auditability.node(s) and error(s) formatting,
with clear iconographic reporting (✖, ✓,
○, ?).read_node() &
read_pipeline(): Promoted to first-class tools.
They now accept in-memory objects and return structured results with
values and diagnostics.explain() function: Enhanced to
surface context, tracebacks, and missingness statistics for pipeline
results and errors.verbose=1
support to pipeline builders, mapping directly to Nix build logs for
improved debugging.node_name into all diagnostic records for clear error
attribution in complex DAGs.NAPredicateError: Dedicated error code
for NA values in boolean contexts, enabling robust logic in
filter() and if expressions.na_ignore
semantics across all core math transforms (abs,
log, sqrt, exp, pow)
and aggregations (min, max, sum,
mean).null keyword and VNull type have been
completely removed from the language grammar and AST.NA values.is_null() with is_na() as the
universal missingness predicate.:)
resolution in the native lm() and predict()
engines.fit_stats() API: Unified
goodness-of-fit statistics (R², AIC, BIC) into a single,
language-agnostic DataFrame output.^onnx serialization and native OCaml
scoring via onnxruntime FFI.== and !=. These
now require explicit broadcasting (.==) for collections to
prevent silent logic errors.identical(a, b): New core builtin for
deep structural equality of complex objects.to_dataframe() Constructor: Added
support for Dictionary-based construction and automatic scalar
recycling.pull() and column helpers to support VString
arguments for extraction of special-character column names.tproject.toml.TLANG_AUTO_ADD_PIPELINE_DEPS for CI automation.RSTATS-NIX-DATE for
reproducible, cache-friendly builds.t_demos into 30+ dedicated per-demo
workflows.repl.ml that was causing compilation errors
during scale-wide refactors.None results.:) resolution in native linear model
scoring.Status: Beta
Release Date: 2026-03-28
modify(): A variadic
builtin for applying multiple lens transformations in a single
pass.compose(): Now variadic,
allowing any number of lenses to be chained into a single declarative
path.node_lens() and env_var_lens() for inspecting
and modifying Pipeline node results and environment variables.idx_lens(i) for positional access in
Lists/Vectors, row_lens(i) for specific
DataFrame row targeting, and
filter_lens(p) for predicate-based focus
on elements and rows.rm() Function: New core language
feature for removing variables from the environment. Supports symbols,
strings, and list-based removal (e.g., rm(x, y),
rm("z"), rm(list = vars)).^
symbol prefix for resolving built-in serializers from a centralized
registry (e.g., ^csv, ^arrow,
^pmml, ^json, ^text).<{ ... }>) for reader/writer snippets, ensuring
syntactic separation between T and injected code.t_make()
builtin to the REPL. It defaults to building src/pipeline.t
and supports optional named arguments (e.g., max_jobs,
max_cores) that pass through to the underlying Nix
build.NameError reporting with “Did you mean …?” suggestions when
an unbound variable is accessed.match expression for declarative list and error
destructuring. Includes support for head/tail patterns,
Error { msg } patterns, and automatic error propagation for
unhandled error values.flush stderr after variable reassignment (:=)
warnings to ensure they appear promptly in the REPL.read_node() to automatically fall back to environment
variables (T_NODE_<name>) when build logs are
unavailable. This enables read_node to work inside Nix
build sandboxes (e.g., within Quarto nodes or nested pipeline steps) and
provides automatic deserialization for Arrow, JSON, and PMML artifacts
based on class hints.read_log() now
returns the Nix build log as a VString instead of printing
directly. This allows the log to be captured as a variable or formatted
with cat().print() and
cat() now correctly handle literal newlines and escape
sequences in strings and shell results, ensuring
?<{ls -l}> and logs display correctly.t init
interactive prompts and project scaffolding templates. The
t init command now defaults to the version of the T binary
being used.t init package
and t init project subcommands to
t init --package and t init --project flags
for better clarity.pyn(). Pipeline nodes are now correctly provided as global
variables in the Python runtime.import os to Python serialization snippets, ensuring
standard library access in the Nix sandbox.t update now
automatically handles untracked flake.nix in Git
repositories when running in a CI environment
(CI=true).docs/installation.md.|>) patterns.t_demos and examples/ with the latest
first-class serializer system and ^-prefix notation.Use ':=' to overwrite or rm() to delete the variable).Version history and roadmap for the T programming language.
Status: Beta Release Date: 21st of March 2026
t upgrade to
automatically update projects to the latest T version and refresh the
rstats-on-nix nixpkgs date.VERSION file and propagated across the project.t update and related flake generation logic.read_node
substitution in Quarto reports to prevent syntax errors in R/Python
chunks.Status: Alpha — Syntax and semantics frozen
Release Date: February 2026
t update)t update now
generates a flake.nix that points to this version by
default.min_version now use 0.51.0.✅ Implemented:
$column_name) for concise data
manipulation✅ Implemented:
$column_name for column
references$age > 30 →
\(row) row.age > 30summarize($total = sum($amount)),
mutate($bonus = $salary * 0.1)select, filter,
mutate, arrange, group_by,
summarize✅ Implemented:
✅ Implemented:
+, -, *,
/, %==, !=, <,
>, <=, >=and, or, not|> (conditional, short-circuits on error)?|> (always forwards, including
errors)~ (for regression models)✅ Core Package:
print, pretty_print, type,
length, head, tailmap, filter, sum,
seqis_error✅ Base Package:
error, error_code,
error_message, error_contextassertNA, na_int, na_float,
na_bool, na_string, is_na✅ Math Package:
sqrt, abs, log,
exp, powmin, max✅ Stats Package:
mean, sd, quantile,
cor (with na_rm parameter)lm (linear regression: y ~ x)✅ DataFrame Package:
read_csv (with separator,
skip_lines, skip_header,
clean_colnames)write_csvnrow, ncol, colnamesclean_colnames (symbol expansion, diacritics,
snake_case, collision resolution)✅ Colcraft Package (Data Verbs):
select, filter, mutate,
arrangegroup_by, summarize,
ungrouprow_number, min_rank,
dense_rank, cume_dist,
percent_rank, ntilelag, leadcumsum, cummin,
cummax, cummean, cumall,
cumany✅ Pipeline Package:
pipeline_nodes, pipeline_deps,
pipeline_node, pipeline_run✅ Explain Package:
explain, explain_jsonintent_fields, intent_get✅ Error Handling:
|> and
?|>✅ NA Handling:
na_int(), etc.)na_rm parameter for aggregations✅ Pipelines:
✅ Cross-Language Model Interchange (PMML):
lm) and Python
(scikit-learn)broom-style tidy summaries (summary(),
fit_stats()) for imported models✅ Intent Blocks:
✅ Arrow Integration & Data Formats:
explain(df) surfaces whether a DataFrame is still on
the native Arrow path (storage_backend,
native_path_active)✅ Reproducibility:
✅ Implemented:
✅ Implemented:
✅ Implemented:
T_PACKAGE_PATH flake configurations export the exact
built derivations rather than raw source mapssrc/ and
help/ directories instead of silently failing contextt-lang compiler successfully stripped out of packages’
default recursive buildInputs dependencieshelp() UX fallback bypasses closures gracefully by
searching locally bound environment lambda variables✅ Implemented (this release):
glimpse() documentation for quick DataFrame summaries.