Introduction

matten is a developer-experience-first multidimensional array (tensor) library for Rust — the family car for learning, teaching, small numerical workflows, data exploration, and early prototypes.

Maturity labels in this book — such as production-ready — describe stability within that scope, not performance or scale. matten optimizes for time to first understanding and a runnable PoC, not benchmark leadership.

This book is organized by reader:

• New users — philosophy and a quick start.
• Playground — try broadcasting, reshape, axis reductions, and matmul in the browser, no install required: Playground.
• Reference — the rules that shape the public API.
• Contributors — project layout, milestones, and process.

This documentation tracks the current 0.46 release family, carrying RFC-110, RFC-111, and RFC-112 — no new API, only behaviour changes and a restriction removed. Zero-sized dimensions are now constructible directly (Tensor::try_new(vec![], &[0, 3]) succeeds), not merely reachable by slicing as before: every constructor, reshape, the shape-composition family, linspace/eye, serde, and the matten-ndarray bridge accept them. Display on an empty tensor now shows its shape instead of an empty string; Debug is unchanged. mean_axis/min_axis/max_axis/var_axis/std_axis now error when the reduced axis has length 0, instead of leaking NaN/inf/ -inf — a zero-length surviving axis is a different case and was and remains Ok with an empty result; sum/sum_axis are unchanged, their additive identity was already correct. matten-ndarray’s ZeroSizedAxis error variant is deprecated and never constructed, kept only so existing code matching on it still compiles. matten-mlprep’s standardize_columns/minmax_scale_columns return a different error — not a different outcome — for a zero-row input: previously a shape-rejection error from tensor construction, now an axis-reduction error, both Err, never a panic in any released version — see the [0.46.0] CHANGELOG entry for the complete list.

Philosophy

matten is a developer-experience-first tensor library for Rust. It is shaped for learning, teaching, early prototypes, small numerical workflows, and data exploration where clear tensor code matters more than exposing every specialized engine concern up front.

The project optimizes for time to first understanding: create a tensor, see its shape, transform it, and keep moving. That does not mean hiding Rust. It means using Rust’s packaging, explicit errors, and predictable ownership while keeping the public tensor surface narrow enough to learn.

What matten is

matten gives Rust users a small, concrete Tensor-centered path for vectors, matrices, axes, broadcasting, reductions, simple statistics, dynamic ingestion, and small table-to-tensor workflows. It is useful when the goal is to explore an idea, explain an operation, teach tensor shape, or build a readable proof of concept before choosing heavier tools.

The intended feel is the family car: practical and predictable, comfortable to start, explicit about boundaries, and honest about when another library is a better next step.

Core principles

One primary type. Ordinary numeric work starts with Tensor. The public API avoids generic dtype parameters and lifetime-bearing tensor views in common examples so a new reader can focus on the operation and its shape.

Concrete before abstract. Core Tensor computation is numeric and f64-based by default. Mixed external data enters through the dynamic ingestion path, where cleanup and numeric conversion are explicit steps instead of hidden coercion.

Small surface, visible meaning. Shape, axis, and data movement should be inspectable. Examples and visual explanations are part of the learning path, not decoration.

Panic locally, return Result at boundaries. Trusted local math conveniences may panic with actionable messages. Anything that reads files, parses JSON/CSV, accepts user-provided shapes, or crosses an external boundary returns Result.

Evidence without ranking. Benchmarks and reports exist to explain tradeoffs and catch regressions, not to claim universal speed leadership.

What matten is not

matten is not a dataframe engine, an ML framework, a GPU backend, a sparse tensor library, an automatic differentiation system, or a broad wrapper around external numeric crates. Companion crates and migration docs can help users connect to other ecosystems, but core matten stays small and tensor-centered.

When to move on

If a workflow grows beyond the small, readable, educational, or prototype-oriented scope, move the hot path to the tool that owns that domain. Use ndarray for broader Rust N-D array work, nalgebra for linear algebra structures, Polars or Pandas for dataframe workflows, and Candle for ML tensor/model workflows. The migration guide explains those paths.

The goal is not to keep every project inside matten; the goal is to make the first model clear enough that the next decision is informed.

Quick start

#![allow(unused)]
fn main() {
use matten::Tensor;

let a = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);
assert_eq!(a.shape(), &[2, 2]);
assert_eq!(a.ndim(), 2);
}

Install the lean core only:

matten = { version = "0.46.0", default-features = false }

Want to try shape reasoning first, with nothing to install? See the Playground.

Playground

Try matten’s shape reasoning below — broadcasting, reshape, axis reductions, matrix multiplication, and converting mixed data to numeric — computed by a real build of core matten and shown exactly as it happens, including rejections.

A zero-sized dimension — a shape like 3,0 — is a valid input here. It looks like it should fail, but matten accepts zero-sized dimensions (RFC-111): try it in any of the four forms below and the result computes normally, including a matrix product where one side has a zero dimension.

Rejections are shown, not hidden

Try an incompatible pair — shapes 2,3 and 4 for broadcasting, or 2,3 and 2,2 for matmul — and the page shows the same error matten itself produces for that mistake, not a generic “invalid input”. A rejected operation teaches as much as an accepted one.

Input notes

A shape or values field accepts either commas or newlines as separators — paste a grid the way it looks, one row per line, and it parses (1, 2, 3 on one line and 1, 2, 3\n4, 5, 6 across two both work). A trailing separator is always fine (1,2,3, is the same as 1,2,3).

A blank cell in the middle of a shape or values field — 1,2,,4 — is never silently dropped. On the four numeric forms above it is reported by position, since a numeric tensor has no way to represent “missing” and dropping it would silently shift every value after it. On the Converting mixed data form it is accepted and shown as None instead — a dynamic tensor can represent a missing cell, so there it is data to convert or reject explicitly, not a parsing mistake.

Notes for contributors

Building the WebAssembly module locally

This page needs a WebAssembly build that a plain local mdbook build does not produce. The .wasm module and its JS bindings are generated by a separate step and are git-ignored, not committed — the same policy this workspace already applies to tools/matten-report/tools/matten-migrate/benchmarks’s own Cargo.lock files: build artifacts of a workspace-excluded, publish = false tool are regenerated on demand, not tracked. Build it first:

cargo build --manifest-path tools/matten-playground/Cargo.toml \
  --target wasm32-unknown-unknown --release
wasm-bindgen --target web \
  --out-dir docs/src/playground \
  --out-name matten_playground \
  tools/matten-playground/target/wasm32-unknown-unknown/release/matten_playground.wasm

Then mdbook build docs (or mdbook serve docs) as usual. In CI this happens automatically before every book build (.github/workflows/docs.yaml) — a checkout from main and a fresh mdbook build locally are the only two cases that need the manual step above.

The scope rule

The output on this page is a representation, never a visualization, of what matten computes (RFC-093 §6, as amended by RFC-095 §3):

• Representation (permitted): showing the tensor’s own structure — rows as rows, columns as columns, numbers as numbers. This is how mathematics writes a matrix and how NumPy prints one. It adds no information that is not already in the shape and the values.
• Visualization (forbidden): encoding a value as visual magnitude or colour — bars, sparklines, heat maps, axes, lines, colour scales, SVG, canvas. Forbidden regardless of the medium, including plain characters: a bar chart drawn with # is visualization and stays out of scope.

The test: does the rendering encode a value as something other than that value? A grid does not. A bar does, however it is drawn. A change that crosses this line needs its own RFC that argues against RFC-093 §6 by name.

Architecture

matten is a small core crate plus four independent companion crates. The shape is a star, not a stack: every companion depends on core, and core depends on none of them.

graph LR
    ndarray[matten-ndarray] --> core[matten]
    mlprep[matten-mlprep] --> core
    data[matten-data] --> core
    stats[matten-stats] --> core

That shape is enforced from both directions, not just documented:

• Core depends on no companion. crates/matten/Cargo.toml lists no matten-* dependency, and scripts/check-core-dependency-boundary.sh (RFC-022 §10) fails CI if one is ever added — checked with --all-features, so an optional dependency behind a non-default feature cannot slip past it.
• No companion depends on another companion (RFC-078 §6). Each crates/matten-*/Cargo.toml lists matten as its only workspace dependency.

This is the fact worth knowing before choosing what to depend on: pick core alone, or core plus exactly the companions you need — never a chain.

The crates

Core’s Status is README.md’s own label; it sits outside the companion promotion sequence (RFC-057/080/084/085) tracked in detail in Compatibility and stability.

Not part of the published surface

Three local tools and the benchmark harness live in the repository but are workspace-excluded (exclude in the root Cargo.toml) and publish = false: tools/matten-report, tools/matten-migrate, tools/matten-playground, and benchmarks/. None of them ship to crates.io, and none can affect the dependency boundary above — they are excluded from the workspace specifically so their tool-only dependencies never enter it.

Feature matrix

Core’s Cargo feature matrix (default, serde, json, csv, dynamic) is listed once, on the contributor reference page, to avoid two copies drifting apart: see Contributing → Architecture.

Source layout

This page is the reader’s overview. For the module-by-module source layout, public re-exports, and design invariants, see Contributing → Architecture.

The data model

For what a Tensor actually holds, how a value moves from raw input to a computation, and what state a tensor’s storage moves through, see Data model and lifecycle.

Start here

This is the recommended learning path for matten.

The goal is to learn tensor-shaped computation in small, readable steps: first numeric tensors, then messy-data cleanup, then visual summaries when shapes or axes become hard to reason about from code alone.

Numeric tensors

If your data is already clean numeric values, follow these examples in order:

After these ten examples you understand the numeric core.

For the visual side of the same path, see Visual understanding examples. Use that page when you want to check a shape before reading values:

Dynamic ingestion: messy data with dynamic

If your input has missing values, mixed types, or dirty CSV/JSON:

The lifecycle rule

Always follow this pattern with dynamic data:

messy input
  → ingest as dynamic tensor    (from_json_dynamic / from_csv_dynamic)
  → inspect                     (schema_summary, numeric_mask, count_none)
  → clean                       (fill_none, forward_fill_none)
  → convert                     (try_numeric / try_numeric_with)
  → numeric tensor computation  (&a + &b, matmul, sum_axis, …)

Never call arithmetic, reductions, or slicing on a dynamic tensor directly — those APIs reject dynamic tensors with a clear message directing you to try_numeric() first.

Read the two main learning paths like this:

clean numeric values
        |
        v
Tensor<f64>
        |
        v
shape ops, broadcasting, matmul, reductions

messy values
        |
        v
dynamic Tensor<Element>
        |
        v
inspect -> clean -> try_numeric
        |
        v
Tensor<f64>
        |
        v
shape ops, broadcasting, matmul, reductions

If an operation feels confusing, first ask which shape is being kept and which axis is being collapsed. For example, mean_axis(0) on a [rows, columns] matrix collapses rows and leaves one value per column.

When to graduate from matten

matten is the family car: easy to start, honest about its limits. When you need performance, static shapes, or advanced linear algebra, see Migration to specialised libraries.

Examples index

All matten examples live in examples/. They are grouped by purpose.

Core examples (numeric Tensor)

These examples demonstrate the default matten API. No extra features required.

Math examples

Applied problems (famous small math)

Recognizable small math / numerical-computing problems, used to show what a Tensor can represent. These live in a fresh 30+ band so the core suite above stays stable. Write-ups: Beginner applied math, Matrix iteration, Numerical methods, and ML-like.

Vector distance and cosine similarity are already covered above — see 54_pairwise_distance.rs, 25_normalize_vector.rs, and 26_cosine_similarity.rs rather than a duplicate in this band.

Practical numeric recipes (50_–56_)

Common data-processing patterns that combine multiple primitives. See Practical numeric recipes for the full write-up.

Readability examples

Small terminal summaries that make shapes, axes, and data readiness easier to scan. See Visual understanding examples for the grouped RFC-063 path across core, dynamic, data, mlprep, and the local report tool.

Dynamic examples (--features dynamic)

These require the dynamic feature for heterogeneous data ingestion. JSON and CSV are equal on-ramps here: from_json_dynamic and from_csv_dynamic differ only in the input format — both land messy data in a dynamic tensor that the same inspect → clean → convert workflow turns into a numeric Tensor.

Companion crate examples

These live in each companion crate’s own examples/ directory, not in core matten. See Companion crate examples for the write-up.

Running examples

# Numeric core (no features needed):
cargo run --example 00_quickstart
cargo run --example 27_axis_reductions

# Dynamic:
cargo run --example dynamic_06_numeric_policy --features dynamic
cargo run --example dynamic_07_on_ramp_summary --features dynamic,csv
cargo run --example dynamic_08_json_ingestion --features dynamic,json
cargo run --example dynamic_09_visual_readiness_summary --features dynamic

Scope rule

Every example demonstrates accepted APIs only. Examples are not a back door for adding new mathematical operations, dataframe behavior, or ML scope.

Visual understanding examples

These examples make shapes, axes, data readiness, and preprocessing effects easier to inspect in plain terminal output. They are examples and local tooling only: no plotting dependency, no public visualization API, and no generated image assets.

Use this page when code runs but the shape, axis, or data meaning is still hard to see. The examples are intended for learning and teaching small tensor workflows, not for dashboarding or large-data visualization.

Runnable examples

Local report tool

tools/matten-report is a workspace-excluded, publish = false local tool for deterministic Markdown/plain-text reports and selected local static HTML artifacts. It is not a published crate and not a public API.

cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo data-readiness
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo shape-flow
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo dynamic-readiness
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo mlprep-standardization
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo educational-path

All fixed demos can also write self-contained local HTML files with explicit --output. For example:

cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo data-readiness --format html --output target/matten-report-data-readiness.html
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo mlprep-standardization --format html --output target/matten-report-mlprep-standardization.html

Input mode is currently accepted only for data-readiness:

cargo run --manifest-path tools/matten-report/Cargo.toml -- \
  --input tools/matten-report/fixtures/small.csv \
  --kind data-readiness \
  --select sales,cost

Scope

These examples answer practical inspection questions:

Which shape did this operation produce?
Which axis did the reduction collapse?
Which dynamic values are numeric, text, or missing?
Which selected table columns can become a numeric Tensor?
What did standardization change, and what shape stayed the same?

Worked questions

These small questions are the fastest way to check the shape or data meaning before reading a longer reference page.

Broadcasting shape alignment

Broadcasting is read from the trailing axis leftward. A dimension of 1 expands to match the other side.

left shape:   [3, 1]
right shape:  [1, 4]
              ------
result shape: [3, 4]

axis 1: left has 1, right has 4, so left repeats across 4 columns
axis 0: left has 3, right has 1, so right repeats across 3 rows

One way to picture the values:

left [3, 1]        right [1, 4]        result [3, 4]

[ 1 ]              [10 20 30 40]       [11 21 31 41]
[ 2 ]         +                         [12 22 32 42]
[ 3 ]                                   [13 23 33 43]

Ask for the output shape first. If every aligned pair is equal, 1, or missing on one side, the operation has a shape to compute.

Reshape, flatten, and transpose

Reshape and flatten keep the same row-major tape. They only change the grouping.

shape [2, 3]

[ 1  2  3 ]
[ 4  5  6 ]

flat tape: 1  2  3  4  5  6

reshape [3, 2]

[ 1  2 ]
[ 3  4 ]
[ 5  6 ]

Transpose changes the coordinate meaning instead:

input [2, 3]        transpose [3, 2]

[ 1  2  3 ]         [ 1  4 ]
[ 4  5  6 ]    ->   [ 2  5 ]
                    [ 3  6 ]

Read it this way: reshape asks “where are the row breaks?”, while transpose asks “which axis does each coordinate belong to?”

Axis reductions

For a [rows, columns] matrix, axis reductions answer “which axis disappears?”

input shape: [3, 2]

rows axis    = axis 0
columns axis = axis 1

mean_axis(0): collapse rows, keep columns
  [3, 2] -> [2]
  result has one mean per column

mean_axis(1): collapse columns, keep rows
  [3, 2] -> [3]
  result has one mean per row

Read a reduction from the output shape first: the missing axis is the one the operation summarized.

Matmul shape flow

For matrix multiplication, the inner dimensions must match. The outer dimensions become the output shape.

left shape       right shape       result shape
[m, n]       x   [n, p]        ->  [m, p]
    ^             ^
    |             |
    shared inner dimension

For concrete shapes:

[2, 3] x [3, 4] -> [2, 4]

left rows are kept:       2
right columns are kept:   4
shared dimension:         3

Each result cell is one left row dotted with one right column.

Dynamic readiness

Dynamic tensors are for inspection and cleanup before numeric computation. A readiness question is about which values can cross the try_numeric() boundary.

dynamic values: [ Float(1.0), None, Text("x"), Int(4) ]

none_mask():    [    0.0,     1.0,     0.0,    0.0 ]
numeric_mask(): [    1.0,     0.0,     0.0,    1.0 ]

Interpret the masks like this:

None        -> missing; fill or otherwise handle it first
Text("x")   -> not numeric under the strict policy
Float/Int   -> can become f64

After cleanup, call try_numeric() before arithmetic, reductions, slicing, reshape, or matmul.

Standardization before and after

Standardization changes scale, not shape. A column-standardization workflow should be read like this:

input tensor
  shape [rows, columns]
  columns may have different centers and scales

standardized tensor
  same shape [rows, columns]
  each selected numeric column is centered and scaled

For runnable output, use the existing matten-mlprep visual example:

cargo run -p matten-mlprep --example mlprep_visual_standardize_summary

That example is the source of truth for the exact reported values.

They deliberately do not add:

Tensor::plot or Tensor::show
automatic expression tracing
SVG, Vega-Lite, or public JSON output
notebook, GUI, or dashboard integration
published report or visualization crates

For local report artifacts, tools/matten-report supports private JSON output with --format json --output <path> for fixed demos and CSV data-readiness input. Input-mode JSON reports are bounded, summary-only artifacts for numeric-conversion success or conversion error; they do not export raw CSV rows. This remains a private schema-v0 local-tool format without a public compatibility promise.

Beginner applied math

A small set of recognizable math problems that show what a matten::Tensor can represent and how short vector/matrix algorithms look in matten. They use only the default numeric Tensor API — no extra features, no external crates, and small hard-coded inputs with stable output.

These examples are teaching examples, not a production algorithm package. They sit in a 30+ filename band so the established 00_–28_ suite stays untouched.

Examples

30_magic_square_checker.rs

Difficulty: Beginner. Checks whether a square matrix is a magic square — every row, column, and both diagonals share one sum. Demonstrates 2-D Tensor::new, shape, and element access with get(&[row, col]). Uses the classic 3×3 Lo Shu square (magic constant 15).

cargo run --example 30_magic_square_checker

Source: 30_magic_square_checker.rs

31_fibonacci_matrix_power.rs

Difficulty: Beginner. Computes Fibonacci numbers from the identity Q^n = [[F(n+1), F(n)], [F(n), F(n-1)]] with Q = [[1, 1], [1, 0]]. Demonstrates repeated Tensor::matmul (recall that * is element-wise, never a matrix product) and reading one element with get. A demonstration of the identity, not a big-integer routine.

cargo run --example 31_fibonacci_matrix_power

Source: 31_fibonacci_matrix_power.rs

32_graph_path_counting.rs

Difficulty: Beginner. Counts walks in a directed graph using the fact that (A^k)[i, j] is the number of walks of length k from node i to node j. Demonstrates representing a graph as an adjacency Tensor and taking matrix powers via matmul. Note the distinction between a walk (may repeat nodes/edges) and a simple path (may not).

cargo run --example 32_graph_path_counting

Source: 32_graph_path_counting.rs

Already covered (cross-references)

Two classic beginner problems already ship as examples, so this band does not add duplicates:

• Vector distance — 54_pairwise_distance.rs (and 25_normalize_vector.rs).
• Cosine similarity — 26_cosine_similarity.rs.

What this is not

These examples do not imply that matten is a graph library, a number-theory package, or an ML framework. They are single-file demonstrations of accepted APIs.

Matrix iteration

Intermediate examples built on repeated matrix/vector multiplication. They show how an iterative process — a probability distribution evolving over time, or a ranking settling to a fixed point — is just Tensor::matmul applied in a loop.

Like the rest of the applied band, these use only the default numeric Tensor API, small hard-coded inputs, and deterministic output. They are teaching examples, not a graph or probability library.

Examples

33_markov_chain_weather.rs

Difficulty: Intermediate. Models a two-state (Sunny / Rainy) weather process with a row-stochastic transition matrix P. Each day applies v_next = v · P via vector × matrix matmul, and the distribution converges to the stationary π = [5/6, 1/6].

cargo run --example 33_markov_chain_weather

Source: 33_markov_chain_weather.rs

34_tiny_pagerank.rs

Difficulty: Intermediate. Ranks the nodes of a tiny directed graph with PageRank. A column-stochastic link matrix M is power-iterated with damping (r_next[i] = (1 - d)/N + d·(M·r)[i]) using matrix × vector matmul; the best-connected node wins, and the link-less node keeps only its teleport share.

cargo run --example 34_tiny_pagerank

Source: 34_tiny_pagerank.rs

What this is not

These are single-file demonstrations of accepted APIs. They do not imply a graph framework, a probability toolkit, or a production PageRank implementation.

Numerical methods

Small numerical-method examples that demonstrate how iterative and sampled-grid algorithms look in matten. They use only the default numeric Tensor API (plus the RFC-038 comfort APIs), small hard-coded inputs, and deterministic output.

These are teaching examples, not a SciPy replacement.

Examples

35_linear_regression_gradient_descent.rs

Difficulty: Advanced-small. Fits y = w·x + b by batch gradient descent on mean-squared error. The data is stacked into a design matrix with a bias column, so predictions are X · θ and the gradient is (2/n)·Xᵀ·(ŷ - y) — one matmul for each, with transpose forming Xᵀ once. Converges to the true line y = 2x + 1.

cargo run --example 35_linear_regression_gradient_descent

Source: 35_linear_regression_gradient_descent.rs

36_heat_equation_1d.rs

Difficulty: Advanced-small. Evolves the 1D heat equation on a rod with fixed-end temperatures using the explicit (forward-Euler) finite-difference update. The stencil is encoded as a tridiagonal matrix A (with identity rows at the boundaries), so each time step is u_next = A · u. The profile converges to the steady-state straight line between the boundary temperatures.

cargo run --example 36_heat_equation_1d

Source: 36_heat_equation_1d.rs

39_finite_difference_derivative.rs

Difficulty: Intermediate. Approximates the derivative of f(x) = x³ sampled on a linspace grid using the central difference (f(x+h) − f(x−h)) / (2h). The grid and the function values are Tensors (the latter via elementwise &x * &x). For a cubic the central-difference error is exactly h², so the example shows the approximation quality directly. It is a numerical approximation, not symbolic differentiation.

cargo run --example 39_finite_difference_derivative

Source: 39_finite_difference_derivative.rs

40_trapezoidal_integration.rs

Difficulty: Intermediate. Approximates ∫₀¹ x² dx with the composite trapezoidal rule and compares against the known exact value 1/3. The grid comes from linspace, the values from elementwise squaring, and the running total from a Tensor::sum reduction. It is a numerical approximation, not an integration library.

cargo run --example 40_trapezoidal_integration

Source: 40_trapezoidal_integration.rs

What this is not

These are single-file demonstrations of accepted APIs. They do not imply that matten is an optimization library, a PDE/finite-element framework, or a SciPy replacement.

ML-like

Two small algorithms often associated with machine learning, written with matten to show that a Tensor is enough for recognizable ML-shaped tasks. They use only the default numeric Tensor API, small hard-coded inputs, and deterministic output.

The boundary is deliberate: these are algorithm demonstrations, not an ML framework. There is no training loop abstraction, no model object, no autograd, and no randomness — k, initial centroids, labels, and iteration counts are all fixed and explicit. Both find the nearest point with Tensor::argmin (RFC-038).

Examples

37_kmeans_small.rs

Difficulty: Advanced-small. Clusters six 2-D points into two groups with Lloyd’s algorithm: assign each point to the nearest centroid, then move each centroid to the mean of its points. Deterministic initial centroids make the run reproducible; it converges to the two obvious clusters.

cargo run --example 37_kmeans_small

Source: 37_kmeans_small.rs

38_nearest_neighbor_classification.rs

Difficulty: Beginner. Classifies a query point by the label of its single nearest training point (1-NN) over a labeled [samples, features] data matrix. No training step, no fitted parameters — just a nearest-point search.

cargo run --example 38_nearest_neighbor_classification

Source: 38_nearest_neighbor_classification.rs

What this is not

These are single-file demonstrations of accepted APIs. They do not imply that matten is an ML framework, a clustering/classification library, or a replacement for a dedicated ML toolkit.

Practical numeric recipes

A set of small, self-contained numeric recipes that combine core matten primitives into common data-processing patterns. Each file is a single runnable example with hard-coded data, assertions, and stable output.

These live in the 50_–56_ band, separate from the core tutorial (01_–13_), the numeric building blocks (20_–28_), and the famous-problem examples (30_–40_).

Examples

50_rowwise_scoring.rs

Row-wise weighted scoring: multiply each row of a feature matrix by a weight vector, then sum across columns to produce one score per row. Shows broadcasting between a [rows, cols] tensor and a [cols] weight vector, followed by sum_axis.

cargo run --example 50_rowwise_scoring

Source: 50_rowwise_scoring.rs

51_standardize_columns.rs

Z-score normalisation of each column (zero mean, unit variance) using only mean_axis, broadcasting, and element-wise arithmetic — no external crate needed.

cargo run --example 51_standardize_columns

Source: 51_standardize_columns.rs

52_minmax_scaling.rs

Min-max (0–1) scaling of each column using min_axis, max_axis, and broadcasting. A common feature-normalisation step before ML algorithms.

cargo run --example 52_minmax_scaling

Source: 52_minmax_scaling.rs

53_gram_matrix.rs

Gram matrix: G = X · Xᵀ, computed with matmul. Used in kernel methods and feature covariance. Shows that a single matmul call produces a symmetric [n, n] similarity matrix from an [n, d] data matrix.

cargo run --example 53_gram_matrix

Source: 53_gram_matrix.rs

54_pairwise_distance.rs

Pairwise Euclidean distances between rows using the identity ‖a−b‖² = ‖a‖² + ‖b‖² − 2aᵀb, computed with broadcasting and matmul. Demonstrates efficient distance computation without an explicit loop over pairs.

cargo run --example 54_pairwise_distance

Source: 54_pairwise_distance.rs

55_moving_average.rs

Simple moving average over a 1-D series using slice windows (slice_str). Shows a sliding-window pattern with overlapping slices and mean reduction.

cargo run --example 55_moving_average

Source: 55_moving_average.rs

56_rolling_windows_basic.rs

Rolling window sum and max over overlapping slices of a 1-D series. Extends the moving-average idea to multiple aggregations in one pass.

cargo run --example 56_rolling_windows_basic

Source: 56_rolling_windows_basic.rs

What this is not

These recipes show how to compose accepted matten APIs into common patterns. They do not imply that matten is a feature-engineering framework, a signal-processing library, or a statistics package. For preprocessing helpers with a proper API, see matten-mlprep.

matten-data — table to Tensor

matten-data is a small, production-ready (RFC-085) companion crate for the boring step between a small table-like input (such as a CSV) and a numeric [matten::Tensor]. It is a conversion helper, not a dataframe library or query engine.

For joins, group-by, lazy queries, or datetime handling, use Polars, DataFusion, or Pandas. matten-data deliberately does none of those. Row-count-bounded batched CSV reading is available behind the optional streaming feature (RFC-082) — see below.

Install

[dependencies]
matten = "0.46.0"
matten-data = "0.46.0"

Both crates share one lock-step family version (RFC-030); maturity is a per-crate Status label, not a separate version number.

Quickstart

use matten::Tensor;
use matten_data::Table;

fn main() -> Result<(), matten_data::MattenDataError> {
let csv = "region,sales,cost\nnorth,100,40\nsouth,150,\neast,120,55";

let tensor: Tensor = Table::from_csv_str(csv)?
    .select_columns(["sales", "cost"])? // choose columns by name, in this order
    .fill_missing(0.0)?                  // the missing south/cost becomes 0.0
    .try_numeric()?                      // strict, explicit conversion to f64
    .to_tensor()?;                       // a normal [rows, columns] Tensor

assert_eq!(tensor.shape(), &[3, 2]);
Ok(())
}

The data path is intentionally explicit:

CSV text
  |
  v
Table
  headers: region, sales, cost
  rows:    3
  |
  | select_columns(["sales", "cost"])
  v
Table
  headers: sales, cost
  rows:    3
  |
  | fill_missing(0.0)
  v
Table
  missing cost cell is now an explicit numeric value
  |
  | try_numeric()
  v
NumericTable
  all selected cells are f64-compatible
  |
  | to_tensor()
  v
Tensor shape [3, 2]
  rows    = CSV data rows
  columns = selected columns, in requested order

The example suite

The numbered tutorial suite teaches one step at a time; csv_to_tensor is a single comprehensive overview.

cargo run -p matten-data --example data_00_quickstart
cargo run -p matten-data --example data_06_visual_readiness_summary

Output Tensor shape

to_tensor produces a tensor of shape [rows, columns], where rows are the data rows (the header is not a row) and columns are the selected columns in the order you requested them. The data is row-major: row 0’s values come first, then row 1’s, and so on. Once converted, the result is an ordinary matten::Tensor — every core operation applies.

For the quickstart input, selecting sales and cost gives:

source rows

row 0: region=north
row 1: region=south
row 2: region=east

selected columns: sales, cost

selected table

row 0: sales=100  cost=40
row 1: sales=150  cost=0      (filled explicitly)
row 2: sales=120  cost=55

Tensor shape [3, 2]

[ 100   40 ]
[ 150    0 ]
[ 120   55 ]

flat row-major data:

[100, 40, 150, 0, 120, 55]

Missing-value policy

Missing cells are never silently turned into 0. A missing value that reaches numeric conversion is a precise MissingValue { column, row } error (the row is the 1-based CSV line number). You decide what a missing value means by calling fill_missing with an explicit value before converting.

The policy is visible in the workflow:

missing cell present
        |
        | try_numeric()
        v
MissingValue error

missing cell present
        |
        | fill_missing(value)
        v
explicit value present
        |
        | try_numeric()
        v
NumericTable

Numeric conversion policy

Conversion is strict and explicit (try_numeric then to_tensor): integers and floats become f64; booleans and non-numeric text are rejected (they are never coerced to 1/0); and a remaining missing cell is rejected. This keeps the boundary between “table-like text” and “numbers” honest and visible.

Limitations

matten-data has no joins, group-by, pivot, query DSL, lazy execution, indexing/loc/iloc, rolling/window operations, datetime engine, or categorical dtype system. It is for small, application-validated or trusted data. When you need those capabilities, reach for a dataframe/query engine (Polars, DataFusion, Pandas) instead.

Streaming (optional, streaming feature)

CsvBatchReader (RFC-082) reads a CSV file in row-count-bounded Table batches, off by default behind the streaming feature — the csv feature is implied, no new dependency. This is a memory strategy, not a dataframe engine: no schema evolution, no lenient/skip-malformed mode, no streaming numeric conversion, and no async. A batch is exactly a Table; concatenating every batch reproduces Table::from_csv_path’s output for ordinary input. Two malformed-input edge cases deliberately diverge from Table::from_csv_path (a file that is blank but not empty — one that trims to nothing but contains at least one whitespace character other than a line terminator, such as a space or tab — and invalid UTF-8) — see CsvBatchReader’s own documentation for the exact behavior. matten-data’s production-ready promotion (RFC-085) covers this feature too: stable in what it does, but its scope may still grow (RFC-082 §5 defers nine further items, including async and resumability).

[dependencies]
matten-data = { version = "0.46.0", features = ["streaming"] }

use matten_data::CsvBatchReader;

fn main() -> Result<(), matten_data::MattenDataError> {
let path = std::path::Path::new("large.csv");
let mut reader = CsvBatchReader::open(path, 10_000)?;
while let Some(batch) = reader.next_batch()? {
    // process one Table batch at a time
}
Ok(())
}

Status and maturity

Production-ready (0.46.x family, RFC-085). The table-to-Tensor API is mostly stable but pre-1.0; pin the release explicitly. The crate’s scope is locked and enforced in CI (RFC-042), and core matten never depends on it (RFC-022).

Companion crate examples

Each companion crate ships its own runnable examples, living in that crate’s examples/ directory (never in core matten). They are small, deterministic, and self-checking, and they all respect the one-way dependency rule: companions depend on matten, but core matten depends on no companion.

These examples were audited and improved in place under RFC-048; the program does not add duplicate or renamed companion examples.

matten-ndarray — interop with ndarray

Both conversions copy data (no zero-copy claim) and preserve shape. Only numeric tensors convert to ndarray. The full conversion rules are documented as a bridge conversion contract; the bridge-crate policy covers how bridge crates are structured (own their target dependency, never re-export Tensor).

cargo run -p matten-ndarray --example from_arrayd
cargo run -p matten-ndarray --example to_arrayd

matten-mlprep — small preprocessing

Convention throughout: rows are samples, columns are features; every transform is deterministic with no hidden randomness and no model training.

cargo run -p matten-mlprep --example mlprep_standardize_columns
cargo run -p matten-mlprep --example mlprep_train_test_split
cargo run -p matten-mlprep --example mlprep_visual_standardize_summary

matten-data — table-to-Tensor (production-ready)

matten-data is production-ready (RFC-085) and intentionally small. It is not a dataframe: no group-by, join, merge, pivot, or query. Missing values and numeric conversion are explicit, never silent. See matten-data: table to Tensor for the full guide.

cargo run -p matten-data --example csv_to_tensor

What this is not

Companion examples demonstrate accepted bridge/preprocessing APIs. They do not imply that matten is a dataframe engine, an ML framework, a linear-algebra backend, or a replacement for ndarray, nalgebra, NumPy, or Pandas.

Data model and lifecycle

One type, two modes

Tensor {
    data:  Vec<f64>,
    shape: Vec<usize>,
    dynamic: Option<Box<DynamicTensor>>,   // #[cfg(feature = "dynamic")] only
}

There is one tensor type, with two modes. Every Tensor is at least a numeric Vec<f64> with a shape. With the dynamic feature enabled, it can additionally hold a DynamicTensor — heterogeneous, Element-typed storage — behind the same Tensor handle.

The dynamic field does not exist without the dynamic feature. On the default feature set, Tensor is exactly the numeric pair (data, shape); there is no hidden mode to reach. Everything below that mentions DynamicTensor, Element, or ViewKind applies only when dynamic is enabled.

The lifecycle

A value’s path from raw input to a computed result crosses exactly one gate:

try_numeric() is the one place a heterogeneous value either becomes a plain numeric Tensor or is rejected. Nothing upstream of it is numeric; nothing downstream of it is anything else.

Table (matten-data) is a separate type in a companion crate, not a Tensor variant and not part of core. It reaches core through NumericTable::to_tensor() -> Result<matten::Tensor, _>.

The types involved

DynamicTensor { storage: Arc<Vec<Element>>, shape: Vec<usize>, len: usize, view: ViewKind }
ViewKind       Contiguous { offset: usize } | Indexed(Vec<usize>)
Element        Float(f64) | Int(i64) | Text(Arc<str>) | Bool(bool) | None

Table (matten-data) { headers: Vec<String>, rows: Vec<Vec<CellValue>> }
CellValue      Text(String) | Float(f64) | Int(i64) | Bool(bool) | Missing

Table and CellValue are a companion-crate representation for tabular input, not part of core’s Tensor/Element model — to_tensor() is the only bridge between the two.

The storage state machine

Dynamic storage is copy-on-write (RFC-012): a tensor either owns its storage uniquely, or shares it with other tensors via Arc, and moves between the two.

stateDiagram-v2
    [*] --> ContiguousUnique
    ContiguousUnique --> IndexedShared: slice()
    IndexedShared --> ContiguousUnique: get_element_mut() (materialize)
    ContiguousUnique --> ContiguousUnique: get_element_mut() (no-op, already unique)

Two consequences of this follow directly, and matter independently:

• A slice retains its source’s entire allocation for as long as the slice lives — even after the source tensor is dropped. A one-element slice of a 100,000-element tensor keeps all 100,000 elements in memory. See Slicing (RFC-102 §8.1).
• Mutating a slice releases that allocation, as a side effect. The first write through get_element_mut() materializes a fresh, uniquely-owned copy and detaches from whatever the tensor was sharing — an incidental escape hatch from the retention cost above, arriving from an unrelated operation. See Dynamic feature (RFC-104 §6.1).

Error model

matten uses a single public error type, MattenError, and splits every API into one of two zones. Understanding the split is the key to writing correct code with matten.

Panic zone vs Result zone

Rule of thumb: if the shape or data comes from outside your code (a file, a web request, user input), use the try_* form.

use matten::{MattenError, Tensor};

// Panic zone: shape is a trusted literal
let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);

// Result zone: shape comes from somewhere external
let result = Tensor::try_new(data, &user_shape);
match result {
    Ok(t) => println!("{t:?}"),
    Err(e) => eprintln!("bad input: {e}"),
}

MattenError variants

#![allow(unused)]
fn main() {
#[derive(Debug)]
#[non_exhaustive]
pub enum MattenError {
    Shape     { operation: &'static str, message: String },
    Broadcast { left: Vec<usize>, right: Vec<usize> },
    Allocation { requested_elements: usize, message: String },
    Slice     { input: Option<String>, message: String },
    Parse     { format: DataFormat, message: String },
    Io        { path: std::path::PathBuf, source: std::io::Error },
    Unsupported { operation: &'static str, message: String },
    InvalidArgument { operation: &'static str, argument: &'static str, message: String },
}
}

MattenError is #[non_exhaustive], so match it with a wildcard arm to stay forward-compatible.

DataFormat identifies which parser produced a Parse error:

#![allow(unused)]
fn main() {
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[non_exhaustive]
pub enum DataFormat { Json, Csv }
}

Variant guide

Matching errors

MattenError embeds std::io::Error in Io, which is neither Clone nor PartialEq. Never compare with ==; always match by variant.

#![allow(unused)]
fn main() {
let err = Tensor::try_new(vec![1.0], &[2, 2]).unwrap_err();

// correct
assert!(matches!(err, MattenError::Shape { .. }));

// correct
if let MattenError::Shape { operation, message } = &err {
    println!("{operation}: {message}");
}

// will not compile — MattenError does not implement PartialEq
// assert_eq!(err, MattenError::Shape { .. });
}

Panic message format

Panic-zone APIs always begin with "matten":

matten shape error in reshape: cannot reshape tensor with 6 elements
    from shape [2, 3] into shape [4, 2] requiring 8 elements

The format is matten <category> error in <operation>: <detail>. When something panics unexpectedly, this prefix makes it easy to grep.

Using ? in application code

MattenError implements std::error::Error, so it works with ? and Box<dyn Error>:

#![allow(unused)]
fn main() {
fn load_and_process(path: &str) -> Result<Tensor, Box<dyn std::error::Error>> {
    let t = Tensor::load_json(path)?;  // Io or Parse on failure
    let flat = t.try_reshape(&[t.len()])?;  // Shape on mismatch
    Ok(flat)
}
}

Panic vs Result

This page has moved to Error model in the Reference section, which covers the full panic-zone / Result-zone split, all MattenError variants, and usage patterns.

Construction and conversion

All matten construction produces an owned, contiguous, row-major Vec<f64> paired with a validated shape. Fields are private; users interact only through methods.

Core constructors

#![allow(unused)]
fn main() {
// From data + shape (panic zone)
let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);

// From data + shape (Result zone)
let t = Tensor::try_new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2])?;

// 1-D from flat vector
let t = Tensor::from_vec(vec![1.0, 2.0, 3.0]);   // shape [3]
}

new panics on mismatch; try_new returns MattenError::Shape or MattenError::Allocation.

Fill constructors

#![allow(unused)]
fn main() {
let z = Tensor::zeros(&[3, 4]);       // all 0.0, shape [3, 4]
let o = Tensor::ones(&[3, 4]);        // all 1.0
let f = Tensor::full(&[3, 4], -1.0); // all -1.0
let s = Tensor::scalar(42.0);         // shape [], len 1
}

All fill constructors validate the shape before allocating — a bad shape panics with an actionable message.

Range constructor

// Half-open, step > 0: [0.0, 1.0, 2.0, 3.0, 4.0]
let r = Tensor::arange(0.0, 5.0, 1.0);

// Negative step: [3.0, 2.0, 1.0]
let r = Tensor::arange(3.0, 0.0, -1.0);

// Result zone (step or bounds from user input)
let r = Tensor::try_arange(start, end, step)?;

arange rejects zero or non-finite step, non-finite bounds, and a computed element count above the allocation limit (2²⁸).

Evenly spaced values and identity (RFC-038)

// `count` evenly spaced values, inclusive of both endpoints:
let xs = Tensor::linspace(0.0, 1.0, 5);   // [0.0, 0.25, 0.5, 0.75, 1.0]
let one = Tensor::linspace(2.0, 9.0, 1);  // [2.0]

// n × n identity matrix:
let i3 = Tensor::eye(3);                   // 1.0 on the diagonal, 0.0 elsewhere

// Result zone:
let xs = Tensor::try_linspace(start, end, count)?;
let i = Tensor::try_eye(n)?;

linspace includes both endpoints when count >= 2, returns [start] when count == 1, and returns an empty tensor when count == 0 (RFC-111) — no step is ever computed. eye produces shape [n, n] and returns an empty [0, 0] tensor when n == 0. Both are budget-checked like the fill constructors (oversized results yield MattenError::Allocation).

Shape model

Shapes are runtime Vec<usize>. There is no const-generic or type-level shape arithmetic.

Rules enforced on every constructor:

• Zero-sized dimensions are accepted (RFC-111): a shape containing a 0 yields an empty tensor, len() == 0. The empty product (rank 0, a scalar) is 1, not 0 — a scalar is never empty.
• Rank may not exceed 8.
• Shape product is computed with checked arithmetic; overflow returns MattenError::Allocation.

Nested row construction

#![allow(unused)]
fn main() {
// Panic zone (convenience for trusted literals)
let t: Tensor = vec![vec![1.0, 2.0], vec![3.0, 4.0]].into();

// Result zone (ragged rows return Err)
let t = Tensor::try_from_rows(vec![vec![1.0, 2.0], vec![3.0, 4.0]])?;
}

From<Vec<Vec<f64>>> panics on ragged rows with an actionable message. try_from_rows returns MattenError::Shape with the ragged-row detail.

Inspection

t.shape()      // &[usize]  — no allocation
t.ndim()       // usize     — shape().len()
t.len()        // usize     — element count
t.is_scalar()  // bool      — ndim() == 0
t.is_vector()  // bool      — ndim() == 1
t.is_matrix()  // bool      — ndim() == 2
t.as_slice()   // &[f64]    — flat row-major view

Conversion out

let v: Vec<f64>        = t.to_vec();       // clone
let v: Vec<f64>        = t.into_vec();     // move, no copy
let v: Vec<f64>        = Vec::from(&t);    // borrow-clone
let v: Vec<f64>        = t.into();         // consuming From
let rows: Vec<Vec<f64>> = t.try_into()?;   // fails for non-rank-2

Migration to faster libraries

When a PoC moves to a performance-sensitive path, hand the flat data to a specialised crate:

let flat: Vec<f64> = tensor.into_vec(); // zero-copy move
// pass `flat` to ndarray, nalgebra, candle, etc.

Operators and broadcasting

matten implements element-wise arithmetic for borrowed tensors with NumPy-style right-aligned broadcasting. All results are new owned tensors; operands are never mutated.

Element-wise operators

#![allow(unused)]
fn main() {
use matten::Tensor;

let a = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);
let b = Tensor::full(&[2, 2], 10.0);

let c = &a + &b;  // [11.0, 12.0, 13.0, 14.0]
let d = &a - &b;  // [-9.0, -8.0, -7.0, -6.0]
let e = &a * &b;  // [10.0, 20.0, 30.0, 40.0]  ← element-wise, not matmul
let f = &a / &b;  // [0.1,  0.2,  0.3,  0.4]
let g = -&a;       // [-1.0, -2.0, -3.0, -4.0]
}

* is always element-wise. Matrix multiplication is explicit via matmul / dot.

Scalar operators

All eight scalar forms are supported:

#![allow(unused)]
fn main() {
let t = Tensor::new(vec![1.0, 2.0, 3.0], &[3]);

// tensor on left
let r = &t + 10.0;   // [11.0, 12.0, 13.0]
let r = &t * 2.0;    // [2.0, 4.0, 6.0]

// scalar on left
let r = 10.0 + &t;   // [11.0, 12.0, 13.0]
let r = 2.0 * &t;    // [2.0, 4.0, 6.0]
}

Broadcasting rules

Shapes are compatible when aligned from the right and each dimension pair satisfies one of:

• dimensions are equal;
• one dimension is 1 (it broadcasts to match the other);
• one operand has fewer dimensions (the missing leading axes are treated as 1).

Read broadcasting from the trailing axis leftward:

matrix:  [2, 3]
bias:       [3]
          -----
result:  [2, 3]

axis -1: 3 matches 3
axis -2: bias has no axis, so it behaves like 1 and repeats over 2 rows

#![allow(unused)]
fn main() {
// bias addition: add a [3] bias vector to every row of a [2, 3] matrix
let matrix = Tensor::new(vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0], &[2, 3]);
let bias   = Tensor::new(vec![10.0, 20.0, 30.0], &[3]);
let result = &matrix + &bias;
// [[11.0, 22.0, 33.0],
//  [14.0, 25.0, 36.0]]
}

The data meaning is “repeat the smaller shape where it has a missing axis or a dimension of 1”:

matrix [2, 3]       bias [3]         result [2, 3]

[ 1  2  3 ]       [10 20 30]       [11 22 33]
[ 4  5  6 ]   +   [10 20 30]   =   [14 25 36]

Two one-length axes can expand in different directions:

left [3, 1]        right [1, 4]       result [3, 4]

[1]                [10 20 30 40]      [11 21 31 41]
[2]          +                         [12 22 32 42]
[3]                                    [13 23 33 43]

Incompatible shapes

Incompatible shapes panic in operator code with an actionable message:

matten broadcast error in add: shapes [2, 3] and [2] are not compatible

IEEE 754 semantics

matten does not intercept NaN or inf:

• Division by zero produces inf, -inf, or NaN per IEEE 754.
• NaN propagates through all arithmetic.
• No silent sanitisation.

No intermediate copies

The broadcast implementation maps result coordinates directly to source element indices using zero-stride tricks. No expanded broadcast copies of the operands are allocated.

Elementwise comfort math (RFC-038)

Beyond the operators above, Tensor provides a few familiar elementwise transforms. Each preserves shape, follows ordinary f64 NaN/Inf behavior, and panics on dynamic tensors (call try_numeric() first):

#![allow(unused)]
fn main() {
use matten::Tensor;
let t = Tensor::from_vec(vec![-5.0, 0.5, 9.0]);
assert_eq!(t.clip(0.0, 1.0).as_slice(), &[0.0, 0.5, 1.0]);
}

clip panics if min > max; try_clip(min, max) returns MattenError::InvalidArgument instead (or MattenError::Unsupported on a dynamic tensor).

Shape operations

All shape-transformation methods return new independent owned tensors. The numeric core copies data internally; no view lifetime is ever exposed.

Reshape

#![allow(unused)]
fn main() {
let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0], &[2, 3]);

// Panic zone
let r = t.reshape(&[3, 2]);      // shape [3, 2], same flat order

// Result zone
let r = t.try_reshape(&[3, 2])?; // MattenError::Shape on mismatch
}

Only the element count matters — reshape never fails because of memory layout. Flat data order (row-major) is preserved unchanged.

The easiest way to read reshape is: keep the flat row-major tape, then place cuts in different positions.

shape [2, 3]

[ 1  2  3 ]
[ 4  5  6 ]

flat tape: 1  2  3  4  5  6

reshape [3, 2]

[ 1  2 ]
[ 3  4 ]
[ 5  6 ]

// Any compatible shape works
let flat  = t.reshape(&[6]);        // [6]
let col   = t.reshape(&[6, 1]);     // [6, 1]
let cube  = t.reshape(&[1, 2, 3]);  // [1, 2, 3]

Panic message on mismatch:

matten shape error in reshape: cannot reshape tensor with 6 elements
    from shape [2, 3] into shape [4, 2] requiring 8 elements

Flatten

#![allow(unused)]
fn main() {
let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);
let f = t.flatten();   // shape [4]

// A scalar becomes shape [1]
let s = Tensor::scalar(7.0).flatten();  // shape [1]
}

Flatten is the same row-major tape without any row/column grouping:

[ 1  2 ]
[ 3  4 ]  ->  [1 2 3 4]

Transpose

transpose() reverses the axis order. t() is an alias.

#![allow(unused)]
fn main() {
// 2-D: swap rows and columns
let m  = Tensor::new(vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0], &[2, 3]);
let mt = m.transpose();
// shape [3, 2], data [1.0, 4.0, 2.0, 5.0, 3.0, 6.0]

// Higher rank: axes are fully reversed
// [d0, d1, d2] → [d2, d1, d0]
let t3  = Tensor::new((1..=24).map(|x| x as f64).collect(), &[2, 3, 4]);
let t3t = t3.transpose();  // shape [4, 3, 2]
}

For a matrix, transpose swaps the coordinate meaning:

input shape [2, 3]          transpose shape [3, 2]

coord [0,0] = 1             coord [0,0] = 1
coord [0,1] = 2             coord [1,0] = 2
coord [0,2] = 3             coord [2,0] = 3
coord [1,0] = 4             coord [0,1] = 4
coord [1,1] = 5             coord [1,1] = 5
coord [1,2] = 6             coord [2,1] = 6

[ 1  2  3 ]                 [ 1  4 ]
[ 4  5  6 ]       ->        [ 2  5 ]
                             [ 3  6 ]

Transposing twice is the identity:

assert_eq!(t.transpose().transpose(), t);

Transposing a scalar panics — there are no axes to reverse.

Swap axes

#![allow(unused)]
fn main() {
let t = Tensor::new((1..=24).map(|x| x as f64).collect(), &[2, 3, 4]);
let s = t.swap_axes(0, 2);  // shape [4, 3, 2]
}

transpose() reverses every axis; swap_axes(a, b) swaps only the two axes you name:

shape [2, 3, 4]
axes    0  1  2

transpose()       -> shape [4, 3, 2]   axes 2 1 0
swap_axes(0, 2)   -> shape [4, 3, 2]   axes 2 1 0
swap_axes(0, 1)   -> shape [3, 2, 4]   axes 1 0 2
swap_axes(1, 2)   -> shape [2, 4, 3]   axes 0 2 1

Swapping an axis with itself is a no-op. Out-of-range axes panic:

matten shape error in swap_axes: axis 5 is out of range for rank-3 tensor

Squeeze and expand_dims (RFC-038)

use matten::Tensor;

// squeeze: drop every length-1 axis (data order unchanged)
let t = Tensor::new(vec![1.0, 2.0, 3.0], &[1, 3, 1]);
let s = t.squeeze();           // shape [3]

// an all-ones shape squeezes to a scalar
let one = Tensor::new(vec![5.0], &[1, 1]).squeeze();  // shape []

// expand_dims: insert a length-1 axis at `axis` (0..=ndim)
let v = Tensor::from_vec(vec![1.0, 2.0, 3.0]);
let row = v.expand_dims(0);    // [1, 3]
let col = v.expand_dims(1);    // [3, 1]

// Result zone: axis > ndim is an InvalidArgument
let r = v.try_expand_dims(axis)?;

squeeze removes all length-1 axes and never fails (a scalar stays a scalar). expand_dims accepts axis in 0..=ndim; an out-of-range axis panics, while try_expand_dims returns MattenError::InvalidArgument. Both clone data and reject dynamic tensors (call try_numeric() first).

Element access

#![allow(unused)]
fn main() {
use matten::Tensor;
let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);

t.get(&[0, 1]);  // Some(2.0)
t.get(&[5, 0]);  // None — out of bounds
t.get(&[0]);     // None — rank mismatch

// Scalar element
Tensor::scalar(99.0).get(&[]);  // Some(99.0)
assert_eq!(t.get(&[0, 1]), Some(2.0));
assert_eq!(t.get(&[5, 0]), None);
assert_eq!(t.get(&[0]), None);
assert_eq!(Tensor::scalar(99.0).get(&[]), Some(99.0));
}

get returns Option<f64> and never panics.

Mutable element access (RFC-104)

get_mut and get_flat_mut mirror get and get_flat exactly — same argument shape, same Option return, same panic-on-dynamic guard — but return a mutable reference instead of a copy, so read-modify-write is one expression instead of two lookups:

#![allow(unused)]
fn main() {
use matten::Tensor;
let mut t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);

*t.get_mut(&[0, 1]).unwrap() += 1.0;
assert_eq!(t.get(&[0, 1]), Some(3.0));

assert_eq!(t.get_mut(&[5, 0]), None); // out of bounds -- tensor unchanged
}

There is no set/set_flat: t.set(coord, v) is one line over get_mut (*t.get_mut(coord).unwrap() = v), so it was left out rather than added as redundant sugar. IndexMut (t[[i, j]] = v), iter_mut, and as_mut_slice are deliberately not in this cut either — get_mut covers the read-modify-write case with an Option rather than a panic path; the others are cheap to add later on top of get_mut and not cheap to withdraw.

On a dynamic tensor, use get_element_mut instead — get_mut/get_flat_mut panic on dynamic input, the same guard get/get_flat use.

Numeric Tensor ownership note

Every method above clones or physically reorders data into a fresh contiguous buffer. This keeps the API lifetime-free and predictable, at the cost of higher allocation than a view-based library. When this matters for large data, migrate to ndarray or nalgebra using tensor.into_vec().

See also

To join several tensors into one — along an existing axis (concatenate) or a new axis (stack) — see Shape composition.

Shape composition

Shape composition joins, repeats, or grids tensors. matten provides six functions on the numeric Tensor only, across two themes:

• concatenate — join along an existing axis (RFC-039).
• stack — join along a new axis (RFC-039).
• repeat / repeat_axis — repeat each element (RFC-087).
• tile — repeat the whole tensor (RFC-087).
• meshgrid — build coordinate grids for evaluating f(x, y) (RFC-087).

Each has a panicking convenience form and a non-panicking try_* form. All reject dynamic tensors — convert with try_numeric() first — and check the output allocation against MattenLimits before copying any data.

concatenate

Tensor::concatenate(tensors: &[&Tensor], axis: usize) -> Tensor
Tensor::try_concatenate(tensors: &[&Tensor], axis: usize) -> Result<Tensor, MattenError>

All inputs must have the same rank and the same size on every axis except axis. The output axis size is the sum of the inputs’ axis sizes; all other axes are unchanged. axis must be in 0..rank.

[2, 3] ++ [4, 3]  along axis 0  ->  [6, 3]
[2, 3] ++ [2, 5]  along axis 1  ->  [2, 8]

concatenate extends an axis that already exists:

axis 0 concatenate: add more rows

[ a a a ]      [ b b b ]      [ a a a ]
[ a a a ]  ++  [ b b b ]  ->  [ a a a ]   shape [4, 3]
                              [ b b b ]
                              [ b b b ]

axis 1 concatenate: add more columns

[ a a a ]      [ b b ]      [ a a a b b ]
[ a a a ]  ++  [ b b ]  ->  [ a a a b b ]   shape [2, 5]

A single-element list returns a clone of that tensor (after validating the axis and dynamic status).

stack

Tensor::stack(tensors: &[&Tensor], axis: usize) -> Tensor
Tensor::try_stack(tensors: &[&Tensor], axis: usize) -> Result<Tensor, MattenError>

All inputs must have identical shapes. A new axis of size n (the number of inputs) is inserted at position axis, so the output rank is the input rank plus one. axis may be 0..=rank.

three [2, 4] tensors stacked at axis 0  ->  [3, 2, 4]
three [2, 4] tensors stacked at axis 1  ->  [2, 3, 4]
three [2, 4] tensors stacked at axis 2  ->  [2, 4, 3]

stack creates a new axis whose size is the number of inputs:

two vectors shape [3]

a = [1 2 3]
b = [4 5 6]

stack([a, b], axis 0) -> shape [2, 3]

[ 1 2 3 ]
[ 4 5 6 ]

stack([a, b], axis 1) -> shape [3, 2]

[ 1 4 ]
[ 2 5 ]
[ 3 6 ]

The short rule:

concatenate: existing axis gets longer
stack:       new axis appears

A single-element list inserts a length-1 axis (the analogue of expand_dims).

repeat / repeat_axis

Tensor::repeat(&self, n: usize) -> Tensor
Tensor::try_repeat(&self, n: usize) -> Result<Tensor, MattenError>

Tensor::repeat_axis(&self, n: usize, axis: usize) -> Tensor
Tensor::try_repeat_axis(&self, n: usize, axis: usize) -> Result<Tensor, MattenError>

repeat repeats each element n times, flattening the result to rank 1:

[1, 2, 3].repeat(2)  ->  [1, 1, 2, 2, 3, 3]

repeat_axis repeats each element n times along axis, preserving rank:

[[1, 2], [3, 4]].repeat_axis(2, 0)  ->  [[1, 2], [1, 2], [3, 4], [3, 4]]

A rank-0 scalar .repeat(n) produces a rank-1 tensor of length n; repeat_axis on a rank-0 scalar is a Shape error (there is no axis to repeat along). n = 0 returns an empty tensor (RFC-111): repeat gives shape [0], repeat_axis gives a zero-length result on axis.

repeat is explicit allocation, unlike broadcasting, which is implicit and materializes nothing: [1, 2, 3] * 2 and [1, 2, 3].repeat(2) differ for exactly that reason — the first never allocates a doubled-length tensor, the second always does.

tile

Tensor::tile(&self, reps: &[usize]) -> Tensor
Tensor::try_tile(&self, reps: &[usize]) -> Result<Tensor, MattenError>

tile repeats the whole tensor, one repetition factor per axis:

[1, 2, 3].tile(&[2])         ->  [1, 2, 3, 1, 2, 3]
[[1, 2]].tile(&[2, 1])        ->  [[1, 2], [1, 2]]

repeat repeats elements; tile repeats the whole tensor — the single most confused pair in this area:

[1, 2, 3].repeat(2)   -> [1, 1, 2, 2, 3, 3]   (each element, in place)
[1, 2, 3].tile(&[2])  -> [1, 2, 3, 1, 2, 3]   (the whole tensor, twice)

If reps is shorter than the input’s rank, it is padded with leading 1s (NumPy-compatible). If reps is longer than the rank, this is an explicit Shape error naming both lengths — NumPy would silently promote the tensor’s rank instead, which matten treats as the surprising direction, not the safe one: the result would have more dimensions than the input, with no obvious place for a caller to look. This is a deliberate, one-directional divergence from NumPy (see below). reps must be non-empty; a 0 entry returns a zero-length result on that axis (RFC-111), rather than an error.

meshgrid

Tensor::meshgrid(x: &Tensor, y: &Tensor) -> (Tensor, Tensor)
Tensor::try_meshgrid(x: &Tensor, y: &Tensor) -> Result<(Tensor, Tensor), MattenError>

Builds the two coordinate grids for evaluating a function of two variables over a grid. x and y must both be rank-1 (a rank-2 input is a Shape error, never silently flattened). For x of length m and y of length n, both outputs have shape [n, m], using NumPy’s xy indexing:

out_x[i][j] == x[j]     (each row is a full copy of x)
out_y[i][j] == y[i]     (each row is constant, equal to y[i])

x = [1, 2, 3]        (len 3)
y = [10, 20]         (len 2)

meshgrid(x, y) -> both outputs shape [2, 3]

out_x = [[1, 2, 3], [1, 2, 3]]
out_y = [[10, 10, 10], [20, 20, 20]]

xy is used deliberately, matching NumPy’s default, even though the alternative ij convention (out[i][j] == (x[i], y[j])) can feel like the more natural matrix reading. When x and y have equal length, xy and ij differ only by a transpose — an invisible mistake with no shape error to catch it — so this matches the ecosystem instead of diverging on an axis a caller cannot see. A reader who specifically wants ij gets it by transposing both outputs.

The tile/meshgrid divergence principle

tile’s rank-promotion rejection and meshgrid’s NumPy-matching xy indexing look like opposite choices — one matches the ecosystem, one does not — but both follow the same rule:

MATCH the ecosystem when a divergence would be SILENT — wrong numbers, or a
      wrong shape the caller cannot see.                (meshgrid's indexing)

DIVERGE where the ecosystem's own behaviour is itself implicit, the divergence
      surfaces as an explicit error, and the error teaches.   (tile's rank promotion)

This is not a license to diverge generally — it is consistent with matten’s standing preference for explicit over silent behaviour.

Errors

The convenience forms panic with the same message the try_* forms would return.

Allocation safety

The output shape is checked against MattenLimits before any data is copied, so an oversized result fails with Allocation (or Shape when the stacked rank would exceed the dimension limit) rather than attempting a huge allocation. repeat, tile, and meshgrid all multiply sizes and can overflow trivially, so every output size is computed with a checked product before allocating — never a bare *.

Example

See 14_concatenate_stack.rs for concatenate/stack, and 58_repeat_tile_meshgrid.rs for repeat/tile/meshgrid, for runnable walkthroughs.

Slicing

matten provides two slicing APIs. The builder is the canonical form; slice_str is a NumPy-like convenience. Both return owned tensors and never produce view lifetimes.

Builder API (canonical)

#![allow(unused)]
fn main() {
use matten::Tensor;

let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0], &[2, 3]);

// One method call per axis; finish with .build()
let row  = t.slice().index(0).all().build()?;     // shape [3]
let top2 = t.slice().range(0..2).all().build()?;  // shape [2, 3]
let col1 = t.slice().all().index(1).build()?;     // shape [2]
}

Builder methods:

Index semantics follow NumPy: index(n) removes the axis, collapsing one dimension. range keeps it.

// Shape [2, 3]: index one axis
let scalar_result = t.slice().index(0).index(1).build()?;
assert!(scalar_result.is_scalar());  // both axes indexed out → shape []

slice_str (convenience)

let row  = t.slice_str("0, :")?;      // first row
let top2 = t.slice_str("0:2, :")?;   // first two rows
let step = t.slice_str("::2")?;      // every other element in a 1-D tensor
let last = t.slice_str("-1, :")?;    // last row (RFC-088)

Grammar:

Whitespace around tokens is ignored: "0:2, :" and " 0:2 , : " are equivalent.

slice_str always returns Result and never panics on malformed input. It rejects specs longer than 512 bytes.

Negative indices (RFC-088)

index, start, and end accept an optional leading -, matching Python’s convention: -1 is the last element along that axis, -2 the second to last, and so on. A negative value is resolved as dim + i before the usual bounds check.

#![allow(unused)]
fn main() {
let t = Tensor::new(vec![1.0, 2.0, 3.0], &[3]);
assert_eq!(t.slice_str("-1")?.as_slice(), &[3.0]);       // last element
assert_eq!(t.slice_str("0:-1")?.as_slice(), &[1.0, 2.0]); // everything but the last
assert_eq!(t.slice_str("-2:")?.as_slice(), &[2.0, 3.0]);  // last two
}

Out-of-range negatives error; they do not clamp. slice_str("-10") or slice_str("-10:") on an axis of size 3 is an error — unlike Python, which clamps a negative slice bound silently (a[-10:] on a 3-element list returns the whole list). matten already errors on positive out-of-range values ("0:100" on size 3 errors too), so a spec string is not validated by two different rules depending on its sign. The error message names both the written form and what it resolved to, e.g. index -10 (resolves to -7) is out of range for axis 0 with size 3.

The builder does not accept negative indices. SliceBuilder::index and .range() take usize only; a caller with len in hand writes len - 1 directly. Adding signed range support to the builder would make every existing range(1..3) call ambiguous between usize and isize inference — a source-breaking change RFC-088 declines to make for a convenience feature.

Negative step (“reversal”) is not implemented. step stays positive-only; "::-1" remains a parse error, not a reversed slice.

Builder vs slice_str

The builder is the primary API because it is type-checked at the call site. slice_str is useful for exploratory work and tutorials where NumPy-familiar syntax is more readable.

// These produce the same tensor
let a = t.slice().range(0..2).all().build()?;
let b = t.slice_str("0:2, :")?;
assert_eq!(a, b);

When in doubt, use the builder — it gives better error messages and is documented in examples as canonical.

Numeric Tensor ownership

Every slice of a numeric tensor is a new contiguous owned tensor. No borrowed view of the source tensor is returned. This means slicing always allocates and copies the selected f64s, but the API is lifetime-free and safe to pass across function boundaries without lifetime annotation.

Slicing dynamic tensors (RFC-102, #[cfg(feature = "dynamic")])

slice() and slice_str() also work on dynamic tensors, returning a dynamic tensor (is_dynamic() == true). The grammar, rank rules, and error messages are identical to the numeric case — slicing selects positions; it does not interpret Element values, so Text, None, and Bool survive a slice unchanged alongside Int/Float.

Ownership differs from the numeric case above: a dynamic slice shares storage with its source (Arc::clone, RFC-012’s copy-on-write model) rather than copying elements. Slicing a slice composes through the existing view instead of nesting, so an arbitrarily long chain of slices still shares one underlying allocation.

That sharing has a cost: a slice keeps its source’s entire allocation alive for as long as the slice itself lives — even after the source tensor is dropped. A one-element slice of a 100,000-element tensor retains all 100,000 elements in memory, not just the one selected. If you need to release the rest, materialize the slice into its own storage explicitly:

#![allow(unused)]
fn main() {
#[cfg(feature = "dynamic")] {
use matten::{Element, Tensor};

let t = Tensor::from_elements((0..6).map(Element::Int).collect(), &[2, 3]);
let row = t.slice().index(0).all().build().unwrap();
let released = Tensor::from_elements(row.to_elements(), row.shape());
let _ = released;
}
}

#![allow(unused)]
fn main() {
#[cfg(feature = "dynamic")] {
use matten::{Element, Tensor};

let t = Tensor::from_elements((0..6).map(Element::Int).collect(), &[2, 3]);
let row = t.slice().index(0).all().build().unwrap();
assert!(row.is_dynamic());
assert_eq!(row.get_element(&[1]), Some(Element::Int(1)));
}
}

Error handling

build() and slice_str() both return MattenError::Slice on:

• number of specs ≠ tensor rank;
• index out of bounds;
• range start > end or end > dimension;
• slice_str parse error (carries the original spec string).

let err = t.slice().all().build().unwrap_err(); // too few specs for rank-2
assert!(matches!(err, MattenError::Slice { .. }));

Boundary integration

All external-input APIs in matten are Result-zone: they never panic on malformed data and always return Result<Tensor, MattenError>.

JSON

Canonical object form

The preferred form for programmatic use — unambiguous for any rank:

#![allow(unused)]
fn main() {
use matten::Tensor;

let t = Tensor::from_json(
    r#"{"shape":[2,2],"data":[1.0,2.0,3.0,4.0]}"#
).unwrap();
assert_eq!(t.shape(), &[2, 2]);
}

Convenience nested-array form

Rank 1 and rank 2 nested arrays are also accepted:

#![allow(unused)]
fn main() {
let t = Tensor::from_json("[[1.0,2.0],[3.0,4.0]]").unwrap();
assert_eq!(t.shape(), &[2, 2]);

let v = Tensor::from_json("[1.0,2.0,3.0]").unwrap();
assert!(v.is_vector());
}

Ragged arrays and non-numeric values return MattenError::Parse:

#![allow(unused)]
fn main() {
assert!(Tensor::from_json("[[1.0,2.0],[3.0]]").is_err()); // ragged
assert!(Tensor::from_json(r#"[[1.0,"text"]]"#).is_err()); // non-numeric
}

Serde integration

Tensor implements Serialize and Deserialize using the canonical object form (requires the serde or json feature, both on by default):

use matten::Tensor;

let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);
let json = serde_json::to_string(&t).unwrap();
let t2: Tensor = serde_json::from_str(&json).unwrap();
assert_eq!(t, t2);

File loading

#![allow(unused)]
fn main() {
let t = Tensor::load_json("examples/data/tensor_2x2.json")?;
}

File errors map to MattenError::Io; parse errors to MattenError::Parse.

CSV

Numeric CSV ingestion accepts rectangular numeric-only CSV. Shape is inferred as [rows, cols].

#![allow(unused)]
fn main() {
let t = Tensor::from_csv("1.0,2.0,3.0\n4.0,5.0,6.0\n")?;
assert_eq!(t.shape(), &[2, 3]);
assert_eq!(t.as_slice(), &[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]);
}

Errors include row and column context:

matten csv parse error: at row 1, column 1: expected f64, got "active"

#![allow(unused)]
fn main() {
let t = Tensor::load_csv("examples/data/numeric_2x3.csv")?;
}

Cargo features

Lean install (no I/O dependencies):

matten = { version = "0.46.0", default-features = false }

Error mapping

Reductions and matrix multiplication

matten provides whole-tensor reductions, axis reductions, and explicit matrix/vector multiplication. * remains element-wise — matrix multiplication always requires matmul or dot.

Whole-tensor reductions

#![allow(unused)]
fn main() {
use matten::Tensor;

let v = Tensor::from_vec(vec![1.0, 2.0, 3.0, 4.0]);

v.sum();   // 10.0
v.mean();  // 2.5
v.min();   // 1.0
v.max();   // 4.0
assert_eq!(v.sum(), 10.0);
assert_eq!(v.mean(), 2.5);
assert_eq!(v.min(), 1.0);
assert_eq!(v.max(), 4.0);
}

All four return f64. sum and mean propagate NaN naturally (IEEE 754). min and max return NaN if any element is NaN — this is deliberate and documented (see below).

NaN / Inf policy

#![allow(unused)]
fn main() {
let t = Tensor::from_vec(vec![1.0, f64::NAN, 3.0]);
assert!(t.min().is_nan());
assert!(t.max().is_nan());
}

Inf is handled normally: it participates in comparisons as expected.

Implementation note: min/max detect NaN explicitly and short-circuit. They do not use f64::min/f64::max (which silently ignore NaN).

Index reductions (argmin / argmax, RFC-038)

argmin/argmax return the flat, row-major index of the smallest/largest element, with the first occurrence winning ties:

#![allow(unused)]
fn main() {
use matten::Tensor;
let t = Tensor::new(vec![2.0, 9.0, 3.0, 1.0, 0.0, 4.0], &[2, 3]);
assert_eq!(t.argmin(), 4); // the 0.0
assert_eq!(t.argmax(), 1); // the 9.0
}

Unlike the value reductions above, an index is ill-defined when any element is NaN. These therefore follow the selection branch of the NaN policy: try_argmin/try_argmax return MattenError::InvalidArgument, and the convenience argmin/argmax panic with the same context. (On a dynamic tensor the try_* forms return MattenError::Unsupported; call try_numeric() first.)

Axis reductions

#![allow(unused)]
fn main() {
// [[1,2,3],[4,5,6]]
let m = Tensor::new(vec![1.0,2.0,3.0,4.0,5.0,6.0], &[2,3]);

m.sum_axis(0);   // column sums  -> shape [3]  -> [5,7,9]
m.sum_axis(1);   // row sums     -> shape [2]  -> [6,15]
m.mean_axis(0);  // column means -> shape [3]  -> [2.5,3.5,4.5]
m.mean_axis(1);  // row means    -> shape [2]  -> [2.0,5.0]
assert_eq!(m.sum_axis(0).shape(), &[3]);
assert_eq!(m.sum_axis(0).to_vec(), vec![5.0, 7.0, 9.0]);
assert_eq!(m.sum_axis(1).shape(), &[2]);
assert_eq!(m.sum_axis(1).to_vec(), vec![6.0, 15.0]);
assert_eq!(m.mean_axis(0).shape(), &[3]);
assert_eq!(m.mean_axis(0).to_vec(), vec![2.5, 3.5, 4.5]);
assert_eq!(m.mean_axis(1).shape(), &[2]);
assert_eq!(m.mean_axis(1).to_vec(), vec![2.0, 5.0]);
}

The reduced axis is removed from the output shape. Reducing a vector along its only axis gives a scalar-shaped tensor.

Both panic with an actionable message if axis >= ndim.

Empty reduced axis (RFC-110): mean_axis/try_mean_axis error (MattenError::InvalidArgument, or a panic carrying that message) when the reduced axis has length 0 — the mean of nothing is undefined. sum_axis is unaffected and returns the additive identity 0.0 per output slot, the same boundary RFC-105 drew for whole-tensor sum. A zero-length axis that survives the reduction (the axis you did not reduce) is a different case entirely and still returns Ok with an empty result — no constructor accepts a zero-sized shape, but slicing reaches one (t.slice().range(0..0).all().build()).

Read an axis reduction as “collapse that axis and keep the others”:

input shape [2, 3]
axes         0  1

axis 0 = rows      axis 1 remains, output shape [3]
axis 1 = columns   axis 0 remains, output shape [2]

For a [2, 3] matrix:

            columns / axis 1
             0   1   2
rows 0     [ 1   2   3 ]
axis 0     [ 4   5   6 ]

mean_axis(0): collapse rows, keep columns
             [ (1+4)/2  (2+5)/2  (3+6)/2 ]
          -> [   2.5      3.5      4.5   ]   shape [3]

mean_axis(1): collapse columns, keep rows
             [ (1+2+3)/3  (4+5+6)/3 ]
          -> [     2.0        5.0   ]         shape [2]

Vector dot product

#![allow(unused)]
fn main() {
let a = Tensor::from_vec(vec![1.0, 2.0, 3.0]);
let b = Tensor::from_vec(vec![4.0, 5.0, 6.0]);

let d = a.dot(&b);
assert!(d.is_scalar());
assert_eq!(d.as_slice(), &[32.0]); // 1*4 + 2*5 + 3*6
}

dot on two vectors [n] and [n] returns a scalar tensor (shape []).

try_dot returns Result<Tensor, MattenError> instead of panicking: MattenError::Shape on incompatible shapes or an unsupported rank combination, MattenError::Unsupported on a dynamic tensor (call try_numeric() on each operand first). dot delegates to try_dot and panics with the same message on error.

Matrix multiplication

matmul is an alias for dot, including its try_matmul/try_dot non-panicking form. Use whichever reads more clearly.

Shape flow for the common matrix-matrix case:

left shape       right shape       result shape
[m, n]       x   [n, p]        ->  [m, p]
    ^             ^
    |             |
    shared inner dimension must match

Each output cell is one row from the left dotted with one column from the right:

left [2, 3]          right [3, 2]           result [2, 2]

[ a b c ]            [ x y ]                [ ax+bz+cu   ay+bw+cv ]
[ d e f ]       x    [ z w ]          ->    [ dx+ez+fu   dy+ew+fv ]
                     [ u v ]

#![allow(unused)]
fn main() {
let a = Tensor::new(vec![1.0,2.0,3.0,4.0], &[2,2]);
let b = Tensor::new(vec![5.0,6.0,7.0,8.0], &[2,2]);

let c = a.matmul(&b);
// [[19,22],[43,50]]
assert_eq!(c.as_slice(), &[19.0, 22.0, 43.0, 50.0]);
}

Incompatible shapes panic with an actionable message including both shapes, or with try_matmul, return Err(MattenError::Shape { .. }) with the same message. Batched matmul (rank > 2) is out of scope for the numeric core.

Axis reductions (min and max)

min_axis and max_axis reduce along an axis, removing it from the output shape, and propagate NaN the same way min and max do.

#![allow(unused)]
fn main() {
use matten::Tensor;

// [[3,1,4],[1,5,9]]
let m = Tensor::new(vec![3.0,1.0,4.0,1.0,5.0,9.0], &[2,3]);

m.min_axis(0);  // column minimums -> shape [3] -> [1.0, 1.0, 4.0]
m.max_axis(0);  // column maximums -> shape [3] -> [3.0, 5.0, 9.0]
m.min_axis(1);  // row minimums   -> shape [2] -> [1.0, 1.0]
m.max_axis(1);  // row maximums   -> shape [2] -> [4.0, 9.0]
assert_eq!(m.min_axis(0).to_vec(), vec![1.0, 1.0, 4.0]);
assert_eq!(m.max_axis(0).to_vec(), vec![3.0, 5.0, 9.0]);
assert_eq!(m.min_axis(1).to_vec(), vec![1.0, 1.0]);
assert_eq!(m.max_axis(1).to_vec(), vec![4.0, 9.0]);
}

NaN propagation: if any element along the reduced axis is NaN, the output for that position is NaN.

Empty reduced axis (RFC-110): min_axis/try_min_axis and max_axis/try_max_axis error (MattenError::InvalidArgument, or a panic carrying that message) when the reduced axis has length 0, rather than returning f64::INFINITY/f64::NEG_INFINITY — those are fold identities, not answers. A zero-length axis that survives the reduction still returns Ok with an empty result.

* is always element-wise

#![allow(unused)]
fn main() {
let a = Tensor::new(vec![1.0,2.0,3.0,4.0], &[2,2]);
let b = Tensor::new(vec![5.0,6.0,7.0,8.0], &[2,2]);

let elem = &a * &b;        // [5, 12, 21, 32]  ← element-wise
let mat  = a.matmul(&b);   // [19, 22, 43, 50] ← matrix product
}

matten never overloads * for matrix multiplication. If you need the matrix product, always call matmul or dot explicitly.

Performance note

matmul uses plain nested loops — correct and readable, but not cache-optimised. For large matrices, migrate the flat data to ndarray or nalgebra:

let flat: Vec<f64> = tensor.into_vec();
// hand off to your preferred crate

Display / formatting (RFC-100)

Tensor implements Display ({}) for a human-facing rendering, distinct from the single-line Debug ({:?}) used for diagnostics:

#![allow(unused)]
fn main() {
use matten::Tensor;

let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0], &[2, 3]);
assert_eq!(t.to_string(), "1.0 2.0 3.0\n4.0 5.0 6.0");
assert_eq!(
    format!("{t:?}"),
    "Tensor(shape=[2, 3], data=[1.0, 2.0, 3.0, 4.0, 5.0, 6.0])"
);
}

Rank 0 is the bare scalar; rank 1 is one right-aligned row; rank 2 is a right-aligned grid, per-column widths, no brackets and no commas — matching what this project already renders elsewhere rather than ndarray’s [[1, 2], [3, 4]] syntax:

#![allow(unused)]
fn main() {
use matten::Tensor;

assert_eq!(Tensor::scalar(3.5).to_string(), "3.5");
assert_eq!(Tensor::new(vec![1.0, 2.0, 3.0], &[3]).to_string(), "1.0 2.0 3.0");
}

Every cell uses {:?} (Debug) formatting, not bare Display — matten’s only element type is f64, and bare Display drops the .0 on whole numbers, which would make a grid of floats read as one of integers:

#![allow(unused)]
fn main() {
use matten::Tensor;
// Deliberately diverges from ndarray, which prints "1" here, not "1.0".
assert_eq!(Tensor::new(vec![1.0, 2.0], &[2]).to_string(), "1.0 2.0");
}

Rank > 2 has no honest 2-D arrangement, so it falls back to the flat form used before this RFC existed:

#![allow(unused)]
fn main() {
use matten::Tensor;
let t = Tensor::new((1..=8).map(|x| x as f64).collect(), &[2, 2, 2]);
assert_eq!(
    t.to_string(),
    "shape=[2, 2, 2] values=[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0]"
);
}

A rank-1 row truncates past 12 values, and a rank-2 grid truncates past 12 columns, so Display on a huge tensor cannot flood a terminal. {:#} (the alternate flag) disables truncation:

#![allow(unused)]
fn main() {
use matten::Tensor;
let row: Vec<f64> = (1..=13).map(|x| x as f64).collect();
let t = Tensor::new(row, &[13]);
assert!(t.to_string().ends_with("... 1 more values"));
assert!(!format!("{t:#}").contains("more values"));
}

On a dynamic tensor (#[cfg(feature = "dynamic")]), each cell renders via Element’s own Display in the same grid, except Float, which uses {:?} on the inner f64 instead so a whole-number float stays visibly distinct from an Int of the same value — a dynamic tensor exists precisely to carry mixed types in one grid, and Element’s own Display alone would render Float(2.0) and Int(2) identically as 2:

#![allow(unused)]
fn main() {
use matten::{Element, Tensor};

let t = Tensor::from_elements(
    vec![Element::Float(2.0), Element::Int(2), Element::Float(1.5), Element::Int(7)],
    &[2, 2],
);
assert_eq!(t.to_string(), "2.0 2\n1.5 7");
}

Debug is unchanged by this RFC — it stays the single-line, truncated-at-8 diagnostic form RFC-020 defined, and is still the better choice for logs. Display is for a human looking at the data.

See also

For the three linalg-adjacent helpers norm, trace, and outer — and the list of advanced linear algebra that is intentionally out of core scope — see Linear algebra (core-lite).

For population variance and standard deviation — var, std, var_axis, std_axis — see Statistics (core-lite).

Linear algebra (core-lite)

Core matten provides small linalg-adjacent helpers, not a linear algebra backend. matten prioritizes PoC ergonomics, not numerical linear algebra performance or stability leadership.

matten offers exactly three linalg-adjacent helpers (RFC-041), alongside the dot/matmul already in Reductions and matrix multiplication:

• norm — L2 / Frobenius norm over all elements.
• trace — diagonal sum of a rank-2 tensor.
• outer — rank-1 × rank-1 outer product.

norm

Tensor::norm(&self) -> f64
Tensor::try_norm(&self) -> Result<f64, MattenError>

The L2 / Frobenius norm over all elements: sqrt(sum(x_i^2)). It works at any rank — for a matrix this is the Frobenius norm. NaN propagates (any NaN element yields NaN). No overflow-avoidance scaling is applied, so extreme magnitudes may overflow to infinity.

try_norm returns MattenError::Unsupported on a dynamic tensor; norm panics in that case. Convert with try_numeric() first when working from dynamic data.

norm([3, 4])          = 5            // sqrt(9 + 16)
norm([[1, 2], [2, 4]]) = 5           // Frobenius: sqrt(1 + 4 + 4 + 16)

trace

Tensor::trace(&self) -> f64
Tensor::try_trace(&self) -> Result<f64, MattenError>

The sum of the diagonal of a rank-2 tensor. Rectangular matrices are allowed: the trace sums self[i, i] for i in 0..min(rows, cols).

trace([[1, 2], [3, 4]])             = 5   // 1 + 4
trace([[1, 2, 3], [4, 5, 6]])       = 6   // min(2,3)=2 -> self[0,0] + self[1,1]

try_trace returns MattenError::Shape if the tensor is not rank-2, or MattenError::Unsupported on a dynamic tensor; trace panics in those cases.

outer

Tensor::outer(&self, other: &Tensor) -> Tensor
Tensor::try_outer(&self, other: &Tensor) -> Result<Tensor, MattenError>

The outer product of two rank-1 tensors: out[i, j] = self[i] * other[j], with shape [self.len(), other.len()]. The output is checked against MattenLimits before allocation.

[1, 2, 3] ⊗ [4, 5]  ->  [[4, 5], [8, 10], [12, 15]]   // shape [3, 2]

try_outer returns MattenError::Shape if either input is not rank-1, MattenError::Unsupported on a dynamic tensor, or MattenError::Allocation if the result exceeds the limit; outer panics in those cases.

Out of scope for core

The following are intentionally not in core matten (RFC-041 §5):

inverse        determinant     solve          least_squares
eigenvalues    eigenvectors    SVD            QR
LU             Cholesky        sparse         BLAS / LAPACK

For serious numerical linear algebra, use a specialized crate such as nalgebra or ndarray-linalg. A future matten-nalgebra / matten-ndarray-linalg bridge would require its own RFC.

Example

See 15_norm_trace_outer.rs for a runnable walkthrough.

Statistics (core-lite)

Core matten provides exactly four statistics reductions (RFC-040), alongside the mean/mean_axis already in Reductions and matrix multiplication:

• var / std — population variance / std over all elements.
• var_axis / std_axis — the same along one axis.

Anything with significant statistical policy — quantile, percentile, histogram, covariance, correlation, z-score, sample variance — is out of core scope (a possible future matten-stats companion). matten is a family-car PoC library, not a statistics package.

Population variance, not sample variance

All four use population variance (ddof = 0):

mean = sum(x) / n
var  = sum((x - mean)^2) / n
std  = sqrt(var)

There is no sample-variance (ddof = 1) variant, no var_with_ddof, and no nanvar/nanstd in core. A single-element tensor has variance 0.0. A two-pass algorithm is used (mean first, then squared deviations) to avoid the avoidable cancellation of the naive one-pass E[x^2] - E[x]^2.

NaN propagates: any NaN element yields NaN (per-slice for the axis variants), consistent with the other f64 reductions. Use try_numeric() to convert a dynamic tensor first; the statistics methods reject dynamic tensors.

var / std

Tensor::var(&self) -> f64
Tensor::std(&self) -> f64
Tensor::try_var(&self) -> Result<f64, MattenError>
Tensor::try_std(&self) -> Result<f64, MattenError>

[1, 2, 3, 4]  ->  mean 2.5,  var 1.25,  std sqrt(1.25) ≈ 1.118

The try_* forms return MattenError::Unsupported on a dynamic tensor. They also guard the empty-tensor case with MattenError::InvalidArgument (RFC-105) — reachable via slicing to a zero-sized shape and, since RFC-111, directly from any constructor as well.

var_axis / std_axis

Tensor::var_axis(&self, axis: usize) -> Tensor
Tensor::std_axis(&self, axis: usize) -> Tensor
Tensor::try_var_axis(&self, axis: usize) -> Result<Tensor, MattenError>
Tensor::try_std_axis(&self, axis: usize) -> Result<Tensor, MattenError>

The reduced axis is removed from the output shape (no keepdims), matching the existing axis reductions (mean_axis, sum_axis):

[[1, 2, 3], [4, 5, 6]]  var_axis(0)  ->  [2.25, 2.25, 2.25]   // shape [3], per column
[[1, 2, 3], [4, 5, 6]]  var_axis(1)  ->  [2/3, 2/3]            // shape [2], per row

The shape behavior is the same for var_axis and std_axis; std_axis just takes the square root of each variance:

input shape [2, 3]
axes         0  1

var_axis(0): collapse rows, keep columns

[ 1  2  3 ]
[ 4  5  6 ]  ->  [ var([1,4])  var([2,5])  var([3,6]) ]
             ->  [    2.25        2.25        2.25    ]   shape [3]

var_axis(1): collapse columns, keep rows

[ 1  2  3 ]
[ 4  5  6 ]  ->  [ var([1,2,3])  var([4,5,6]) ]
             ->  [     2/3            2/3     ]           shape [2]

The try_* forms return MattenError::Shape if axis >= rank, or MattenError::Unsupported on a dynamic tensor.

Empty reduced axis (RFC-110): var_axis/try_var_axis and std_axis/try_std_axis additionally return MattenError::InvalidArgument (or, for the panicking forms, a panic carrying that message) when the reduced axis has length 0 — variance and standard deviation of nothing are undefined, not NaN. A zero-length axis that survives the reduction (a different axis than the one being reduced) is unaffected and still returns Ok with an empty result.

Out of scope for core

sample variance (ddof = 1)    quantile        percentile
histogram                     covariance      correlation
z-score                       nanvar/nanstd   statistical tests

These stay out of core; RFC-040 §9’s gate for a future companion was cleared once at least three clearly-useful, well-scoped APIs were accepted, and that companion — matten-stats — now exists (RFC-078, expanded by RFC-083; see below). Some (z-score) overlap with matten-mlprep and must not be duplicated there.

The matten-stats companion (RFC-078, RFC-083, RFC-084, RFC-090)

matten-stats is a separate, production-ready candidate companion crate (promoted in RFC-084 once its surface settled) computing statistical summaries that core deliberately excludes: covariance, covariance_population, correlation, quantile, skewness, kurtosis, and histogram. A summary is returned as f64 where it is scalar, and as a small owned struct (Histogram) where it is inherently vector-valued — matten-stats never returns a Tensor (RFC-090 §5 amended the original Tensor -> f64 framing to make room for histogram’s non-scalar result). Its estimator conventions differ per function, matching what each function’s name is expected to mean in the wider ecosystem (NumPy/SciPy/R/pandas):

covariance             sample,     ddof = 1        (NumPy/R/pandas `cov`/`corrcoef` default)
covariance_population  population, ddof = 0        (explicit in the name; no default to choose)
correlation            ddof-invariant                 (the n - 1 factors cancel)
skewness               g1,  uncorrected                (SciPy `skew(bias=True)` default; NOT pandas' `.skew()`)
kurtosis               g2,  uncorrected, EXCESS         (SciPy `kurtosis(fisher=True, bias=True)` default; NOT pandas' `.kurt()`)

kurtosis reports excess kurtosis (Fisher’s definition): a normal distribution scores 0.0, not 3.0. pandas’ .skew()/.kurt() bias-correct and so return a different number than skewness/kurtosis for the same input — this divergence is deliberate, not an oversight. See the crate’s own README for the full API and error model.

histogram — bin count is the caller’s choice (RFC-090)

pub struct Histogram {
    pub counts: Vec<usize>,
    pub edges: Vec<f64>,
}

pub fn histogram(x: &Tensor, bins: usize) -> Result<Histogram, MattenStatsError>;

There is no automatic bin-count rule — not Sturges, not Freedman-Diaconis, not Scott, no "auto". bins is required because bin count is a genuine analytical choice; a histogram whose bin count was picked for the caller teaches the wrong lesson (RFC-090 §4.1).

The range is always [min(x), max(x)]; there is no range parameter. The last bin is closed at the top ([edges[bins - 1], edges[bins]]), unlike every other bin ([edges[i], edges[i + 1]), half-open) — otherwise the maximum value would fall in no bin and silently vanish from the counts. A constant input errors (ZeroVariance) rather than widening the range the way NumPy does ((v - 0.5, v + 0.5)); that 0.5 comes from nowhere in the data.

Example

See 16_variance_std.rs for a runnable walkthrough of core’s var/std, and histogram.rs (cargo run -p matten-stats --example stats_histogram) for matten-stats’s histogram.

Dynamic feature (Element model)

The dynamic feature enables heterogeneous dynamic tensors. Enable it in Cargo.toml:

matten = { version = "0.46.0", features = ["dynamic"] }

matten is not a dataframe library. The dynamic feature is for ingesting and cleaning messy PoC data before converting to numeric tensors or handing off to a specialised crate.

The lifecycle is deliberately staged:

messy JSON / CSV / Elements
        |
        v
dynamic Tensor<Element>
        |
        | inspect: schema_summary, count_none, none_mask, numeric_mask
        v
dynamic Tensor<Element> with known issues
        |
        | clean: fill_none, forward_fill_none
        v
dynamic Tensor<Element> ready for policy
        |
        | convert: try_numeric / try_numeric_with
        v
numeric Tensor<f64>
        |
        v
ordinary matten computation

The important boundary is the conversion step. Arithmetic, slicing, reshape, and reductions belong after try_numeric(), not before it.

Element variants

use matten::Element;

Element::Float(1.5)            // IEEE 754 f64
Element::Int(42)               // i64
Element::text("active")        // UTF-8 text (Arc<str> internally)
Element::Bool(true)            // boolean
Element::None                  // missing / null

size_of::<Element>() == 24 bytes on 64-bit targets (all text representations give the same size; Arc<str> was chosen for cheap clone in CoW slices).

Constructing dynamic tensors

use matten::{Element, Tensor};

let t = Tensor::from_elements(
    vec![
        Element::Float(1.0), Element::text("ok"), Element::Bool(true),
        Element::Int(2),     Element::None,        Element::Bool(false),
    ],
    &[2, 3],
);

// Boundary-safe variant:
let t = Tensor::try_from_elements(data, &[2, 3])?;

Element predicates and coercion

#![allow(unused)]
fn main() {
use matten::Element;
Element::None.is_none();         // true
Element::Float(1.0).is_numeric(); // true
Element::Int(42).is_numeric();   // true
Element::Bool(true).is_numeric(); // false — no silent bool coercion

Element::Float(1.5).try_as_f64();  // Some(1.5)
Element::Int(7).try_as_f64();      // Some(7.0)
Element::text("3").try_as_f64();   // None — no silent text coercion
Element::None.try_as_f64();        // None
assert!(Element::None.is_none());
assert!(Element::Float(1.0).is_numeric());
assert!(Element::Int(42).is_numeric());
assert!(!Element::Bool(true).is_numeric());
assert_eq!(Element::Float(1.5).try_as_f64(), Some(1.5));
assert_eq!(Element::Int(7).try_as_f64(), Some(7.0));
assert_eq!(Element::text("3").try_as_f64(), None);
assert_eq!(Element::None.try_as_f64(), None);
}

Coercion policy (RFC-011 §11)

Use fill_none or explicit conversion helpers to clean data before arithmetic.

Accessing elements

t.get_element(&[0, 1])  // Option<Element> — None if out of bounds
t.is_dynamic()          // true for dynamic tensors
t.to_elements()         // Vec<Element> in row-major order

Mutable element access (RFC-104)

get_element_mut mirrors get_element, returning Option<&mut Element> instead of a copy. The caller reads, changes, or replaces the variant — the library never chooses one, so writing a 42.0 into what was an Int column raises no coercion question; it simply becomes a Float because that is what was written.

#![allow(unused)]
fn main() {
#[cfg(feature = "dynamic")] {
use matten::{Element, Tensor};

let mut t = Tensor::from_elements(vec![Element::Int(1), Element::Int(2)], &[2]);
*t.get_element_mut(&[1]).unwrap() = Element::text("two");
assert_eq!(t.get_element(&[1]), Some(Element::text("two")));
assert_eq!(t.get_element_mut(&[9]), None); // out of bounds
}
}

If this tensor’s storage is shared — for example it is a slice — the first write materializes it: a fresh, uniquely owned copy is made and the tensor detaches from whatever it was sharing with, so the write can never reach a shared parent. This is a no-op when the storage is already contiguous and unique — a second write on an already-detached tensor does not reallocate.

Worth knowing: materializing a slice releases the source’s allocation it was otherwise keeping alive for as long as the slice lived — the retention cost documented in Slicing gets an incidental escape hatch here, as a side effect of an unrelated operation (mutation), not a feature built for that purpose.

Missing-value utilities

use matten::{Element, Tensor};

let t = Tensor::from_elements(
    vec![Element::Float(1.0), Element::None, Element::Float(3.0), Element::None],
    &[4],
);

// Count None values
t.count_none()          // 2

// Boolean-like mask: 1.0 where None, 0.0 elsewhere (numeric f64 tensor)
let mask = t.none_mask();   // [0.0, 1.0, 0.0, 1.0]
// RFC-011 named alias:
let mask = t.is_none_mask(); // identical result

// Constant fill
let filled = t.fill_none(Element::Float(0.0)); // [1.0, 0.0, 3.0, 0.0]

// Forward-fill: carry last non-None value forward (fallback for leading None)
let t2 = Tensor::from_elements(
    vec![Element::None, Element::Float(1.0), Element::None, Element::Float(4.0)],
    &[4],
);
let fwd = t2.forward_fill_none(Element::Float(-1.0));
// [-1.0, 1.0, 1.0, 4.0]  (leading None takes fallback)

// Sum skipping None (panics on non-numeric non-None elements)
t.sum_skip_none()  // 4.0  (1.0 + 3.0, None values skipped)

Masks make readiness visible without changing the data:

dynamic values:     [ Float(1.0), None, Text("x"), Int(4) ]

none_mask():        [    0.0,     1.0,     0.0,    0.0 ]
numeric_mask():     [    1.0,     0.0,     0.0,    1.0 ]

meaning:
  none_mask    = where missing values are
  numeric_mask = which values strict try_numeric() can accept

Parsing mixed data

#![allow(unused)]
fn main() {
// JSON: null→None, booleans→Bool, strings→Text, integers→Int, floats→Float
#[cfg(feature = "json")]
let t = Tensor::from_json_dynamic(r#"[[1, "active", true], [2, null, false]]"#)?;

// CSV: empty field→None, "true"/"false"→Bool, integers→Int, floats→Float, rest→Text
#[cfg(feature = "csv")]
let t = Tensor::from_csv_dynamic("1,active,true\n2,,false\n")?;
}

Slicing (RFC-102)

slice() and slice_str() work on dynamic tensors, returning a dynamic tensor. Slicing selects positions — it does not interpret Element values, so heterogeneity is irrelevant and Text/None/Bool survive a slice unchanged alongside Int/Float. Storage is shared, not copied (Arc::clone), the same copy-on-write model fill_none and the other element-producing methods above already use internally:

#![allow(unused)]
fn main() {
#[cfg(feature = "dynamic")] {
use matten::{Element, Tensor};

let t = Tensor::from_elements((0..6).map(Element::Int).collect(), &[2, 3]);
let row = t.slice().index(0).all().build().unwrap();
assert!(row.is_dynamic());
assert_eq!(row.get_element(&[1]), Some(Element::Int(1)));
}
}

See Slicing for the full contract.

Current limitations (guard model)

In the current release, many numeric operations reject dynamic tensors with a clear matten unsupported error message. You must convert to a numeric tensor first using try_numeric().

Guarded (will panic or return Err):

• reshape, flatten, transpose, swap_axes
• all arithmetic operators and reductions
• dot / matmul
• as_slice, to_vec, into_vec, get, get_flat
• Serialize / serde

The underlying Arc-based CoW storage (DynamicTensor) is implemented internally; slicing (above) is its first public use. reshape is not yet wired to it.

// Correct pattern: ingest → clean → convert → arithmetic
let raw = Tensor::from_csv_dynamic("1.0,2.0\n3.0,4.0\n")?;
let filled  = raw.fill_none(Element::Float(0.0));
let numeric: Tensor = filled.try_numeric()?; // convert to numeric
let result = &numeric * 2.0;                 // numeric arithmetic

Workflow pattern

#![allow(unused)]
fn main() {
use matten::{Element, Tensor};

fn process_messy_csv(input: &str) -> Result<Tensor, Box<dyn std::error::Error>> {
    // 1. Ingest as dynamic
    let raw = Tensor::from_csv_dynamic(input)?;

    // 2. Fill missing values
    let clean = raw.fill_none(Element::Float(0.0));

    // 3. Convert to numeric tensor for arithmetic
    let numeric = clean.try_numeric()?;

    // 4. Use numeric arithmetic, reductions, matmul...
    Ok(numeric)
}
}

For a dirty row, the same workflow looks like this:

input row:       1.0, "", "active", 4

dynamic parse:   Float(1.0)  None  Text("active")  Int(4)

inspect:         none_mask    -> [0, 1, 0, 0]
                 numeric_mask -> [1, 0, 0, 1]

clean:           fill_none(0.0)
                 Float(1.0)  Float(0.0)  Text("active")  Int(4)

convert:         strict try_numeric() still rejects Text("active")
                 allow_text_parse() only helps text that actually parses as f64

Limitations

• No dataframe joins, group-by, pivot, or query operations.
• No date/time dtype.
• No categorical dtype.
• No silent text-to-number or bool-to-number coercion.
• Batched matmul on dynamic tensors requires try_numeric first.
• For large datasets, consider specialised crates (polars, ndarray).

Migration to specialised libraries

For the full narrative guide — when to stay vs. migrate, a target-selection matrix, and per-target playbooks — see the Production migration guide. This reference page is the quick, copy-paste companion: data-export snippets and minimal conversions.

matten is a starting point, not an endpoint. When a PoC graduates to production or numerical performance becomes critical, migrate the data to a specialised crate. This page shows how.

When to migrate

Exporting data from matten

Every matten tensor exposes its flat row-major data. The simplest data-export path is:

let flat: Vec<f64> = tensor.into_vec();  // consuming, no copy
// or
let flat: Vec<f64> = tensor.to_vec();    // borrowing clone

The shape is available as:

let shape: &[usize] = tensor.shape();

To ndarray

The bridge-first path uses the matten-ndarray crate (copies, numeric-only, rejects dynamic tensors, preserves logical row-major order — see the bridge contract):

#![allow(unused)]
fn main() {
use matten::Tensor;
use matten_ndarray::to_arrayd;

let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0], &[2, 3]);
let arr = to_arrayd(&t)?;   // ArrayD<f64>, logical row-major
println!("{arr}");
}

Without the bridge crate, convert manually from the flat Vec<f64> plus shape:

use matten::Tensor;
use ndarray::ArrayD;

let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0], &[2, 3]);
let shape: Vec<usize> = t.shape().to_vec();
let flat: Vec<f64>    = t.into_vec();
let arr = ArrayD::from_shape_vec(shape, flat).unwrap();
println!("{arr}");

ndarray supports BLAS-backed matrix multiplication, advanced indexing, views, and strided arrays.

To nalgebra

use matten::Tensor;
use nalgebra::DMatrix;

let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);
let flat: Vec<f64> = t.into_vec();

// DMatrix is column-major; transpose if needed
let mat = DMatrix::from_row_slice(2, 2, &flat);
println!("{mat}");

nalgebra provides static and dynamic matrices, LU/QR/SVD decomposition, and linear algebra operations.

To candle (ML tensors)

#![allow(unused)]
fn main() {
use matten::Tensor;
// candle_core = { version = "0.x", features = ["..."] }

let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);
// matten uses f64; convert to f32 if the Candle workflow wants f32.
let flat_f32: Vec<f32> = t.as_slice().iter().map(|&v| v as f32).collect();
let shape = t.shape().to_vec();
// let candle_t = candle_core::Tensor::from_vec(flat_f32, shape, &device)?;
println!("data ready for candle: {flat_f32:?}, shape: {shape:?}");
}

candle targets GPU-accelerated ML workflows (transformers, training loops).

Dynamic tensors: clean then migrate

If your data went through matten’s dynamic feature, convert to a numeric tensor numeric first:

#![allow(unused)]
fn main() {
use matten::{Element, Tensor};

let raw = Tensor::from_csv_dynamic("1.0,2.0\n3.0,4.0\n")?;
let filled  = raw.fill_none(Element::Float(0.0));
let numeric: Tensor = filled.try_numeric()?; // MattenError if non-numeric
let flat: Vec<f64>  = numeric.into_vec();    // hand off
}

Allocation warning

matten clones on every reshape and slice. For large datasets, migrate before performing many transformations:

// Prefer this pattern for large data:
let result = compute_in_matten(&small_data);
let flat   = result.into_vec();
// then pass `flat` to ndarray/nalgebra for the heavy lifting

Compatibility promise (v0.x)

During v0.x, API changes are allowed but minimised after a release. The core Tensor type, the public error model, and the panic-vs-Result split are stable design decisions and will not change without a documented breaking change. See the public API snapshot and the CHANGELOG for the exact current export surface.

v1.0.0 requires explicit maintainer confirmation and a full public API review. See the project CHANGELOG for migration notes on any breaking changes.

Compatibility and stability policy

Public API contract

matten exports the following public names from the crate root:

#![allow(unused)]
fn main() {
use matten::Tensor;                  // always
use matten::{MattenError, DataFormat}; // always
use matten::MattenLimits;            // always (RFC-018)
use matten::SliceBuilder;            // always; returned by Tensor::slice()
use matten::Element;                 // #[cfg(feature = "dynamic")]
use matten::NumericPolicy;           // #[cfg(feature = "dynamic")] (RFC-017)
}

SliceBuilder is returned by Tensor::slice() and is held by value; users do not need to import it by name in the common case.

IntoSliceRange and SliceConvert are hidden implementation plumbing for SliceBuilder::range. They are exported #[doc(hidden)] and use a private sealed::Sealed supertrait so downstream crates cannot meaningfully implement them. Users never need to name them in imports.

SliceSpecRepr is #[doc(hidden)]; it is a visibility-chain artefact and not part of the stable API.

Panic zone vs Result zone

This split is a permanent design decision and will not change:

See Error model for the full list of each zone’s APIs.

Feature flags

Disabling default features is supported: default-features = false gives the lean core. Enabling dynamic does not rename or remove any numeric Tensor API.

v0.x compatibility

matten is on the v0.x line. The policy:

• Breaking changes are allowed but must be documented in CHANGELOG.
• Public API churn decreases after each minor release.
• Feature-gated additions (new #[cfg] methods) are not breaking.
• #[non_exhaustive] on MattenError and DataFormat means match arms must include a wildcard — new variants may be added without a semver break.

Family version and core dependency requirement

The published crates are released as one lock-step family (RFC-030): matching crate versions are the supported, documented set. Downstream examples therefore show explicit matched pins such as matten = "0.46.0" plus a companion at the same release.

Inside the workspace, companion crates inherit the core dependency from [workspace.dependencies] as a broad pre-1.0 requirement (matten = "0" plus a local path). This is a maintenance policy (RFC-064): it reduces version-line churn in member manifests while keeping release identity, maturity labels, and user-facing install guidance tied to the lock-step family.

matten-stats — a deliberate ddof divergence (RFC-078, RFC-083, RFC-084)

matten-stats is a companion crate at production-ready candidate maturity (RFC-084, promoted from Experimental once its six-function surface settled — no usage-history claim is made; that is what the candidate label is for). Its covariance and correlation functions use the sample estimator (ddof = 1, i.e. divide by n - 1), diverging deliberately from core matten’s population var/std (ddof = 0). This matches the near-universal default in inferential statistics (NumPy, R, pandas) and is the only behavioral divergence between a companion and core in the family. correlation is unaffected by the choice — the n - 1 factors cancel — so only covariance’s numeric output actually differs from what core’s population convention would produce. covariance_population (RFC-083) gives the population estimator explicitly, and skewness/kurtosis (RFC-083) each match their own ecosystem default rather than always bias-correcting. See RFC-078 §4.1 and RFC-083 §4.1 for the full rationale.

v1.0 requirements

v1.0.0 requires explicit maintainer confirmation. Before that can happen:

• public API review must be complete;
• cargo public-api snapshot must be taken and approved;
• the panic/Result split must be finalised;
• the serde canonical format must be declared stable;
• limitations and non-goals must be clearly documented;
• if any lock-step family crate remains production-ready candidate, the v1.0 release RFC must include the RFC-067 family maturity table and explicitly decide that crate’s inclusion without silently promoting its maturity label.

MSRV

rust-version = "1.85" (Rust 2024 edition). The MSRV may be relaxed in a future release; it will not be raised without a documented breaking change.

Formatting contract

Tensor implements Debug for compact, single-line inspection output, and — as of RFC-100 — Display for a human-facing rendering: rank 0/1/2 as a right-aligned grid, rank > 2 as a flat shape=... values=... list, truncated past 12 values/columns unless {:#} is used. See Display / formatting for the full contract. Both are now a documented compatibility surface — a change to either’s exact output is a breaking change to be treated with the same care as any other public API.

Deferred items

The following items were considered and explicitly deferred:

Phase status

The v0.20 family completed the materialization phase: the core numeric comfort APIs (RFC-038 — elementwise, selection, creation, and shape helpers) and the 30_–40_ famous-problem examples program (RFC-043–048). The matten-data CSV→tensor ingestion API first shipped in this family (RFC-034, RFC-035).

The v0.21 family delivered selective boundary implementation: shape composition (concatenate / stack), small statistics (var / std), linalg-lite helpers (norm / trace / outer), and the matten-data scope guard. These are additive under lock-step family versioning (RFC-030).

The v0.22 family promotes matten-data to Beta: the RFC-036 example suite (data_00–data_05) plus an explicit malformed-CSV test complete the documented Beta gate (RFC-023 §9). Maturity is a per-crate Status label, not a separate version, under lock-step family versioning (RFC-030).

The v0.23 family adds the production migration guide (RFC-050–052): when to stay vs. migrate, per-target playbooks (ndarray, nalgebra, Polars/Pandas, Candle, NumPy), and the bridge conversion-contract template with the matten-ndarray reference contract. This is documentation only — no public API, runtime, or dependency change, and core matten gains no dependency.

The v0.24 family completes the reduction surface (RFC-055 / RFC-056): every scalar value reduction (try_sum / try_mean / try_min / try_max / try_norm) and every axis reduction (try_sum_axis / try_mean_axis / try_min_axis / try_max_axis) now has a non-panicking Result form, joining try_var / try_std and their axis variants. The panic forms are unchanged in behaviour and remain convenience wrappers. These are additive under lock-step family versioning (RFC-030); no existing signature, numeric result, output shape, NaN policy, or dependency changes, and core matten gains no dependency.

The v0.25 family opens the companion-maturity line by promoting matten-ndarray from production-ready candidate to production-ready (RFC-057). This is a maturity Status label only — no API, runtime, error-variant, dependency, copy-semantics, or ndarray-version change — and it does not imply v1.0, which still requires explicit maintainer confirmation. Under lock-step family versioning (RFC-030) the crate stays on the shared family version. matten-mlprep and matten-data remain at Beta pending their own maturity decisions.

The v0.26 family continues the companion-maturity line by promoting matten-mlprep from Beta to production-ready candidate (RFC-058). Label/docs only — no API, runtime, error-variant, or dependency change. The candidate rung reflects an honest limitation: train_test_split is ordered-only (no shuffle), acceptable if that documented limit is acceptable; full production-ready is deferred (RFC-058 §5.1). This does not imply v1.0. matten-ndarray remains production-ready; matten-data remains Beta pending its own maturity decision.

The v0.27 family completes the companion-maturity line by promoting matten-data from Beta to production-ready candidate (RFC-059), with two promotion-blocking hygiene fixes first (a maturity-neutral package description; required-features = ["csv"] on the data_0X examples). Label/docs/packaging only — no API, runtime, error-variant, or dependency change, and no scope expansion: the RFC-042 lock holds (still a CSV→tensor on-ramp, not a dataframe engine). Full production-ready is deferred to a separate future review. This does not imply v1.0. The ladder then read matten-ndarray production-ready, matten-mlprep and matten-data production-ready candidates.

matten-mlprep was subsequently promoted from production-ready candidate to production-ready (RFC-080), now that RFC-058 §5.1’s Option B exit criterion is satisfied: RFC-077 added train_test_split_seeded, closing the ordered-only-split caveat that held the promotion at candidate. Label/docs only — no API, runtime, error-variant, dependency, or version change, and no other crate’s maturity changes. matten-data was production-ready candidate at this point and was later promoted to production-ready as well (RFC-085), once RFC-059 §6’s deferred full-production review ran; matten-stats was Experimental at this point and was later promoted to production-ready candidate as well, once its surface settled (RFC-083, RFC-084).

The v0.28 family moves the matten-ndarray bridge to ndarray 0.17 (RFC-062): the supported requirement changes from the 0.16 minor to 0.17 (CI targets 0.17.2). Because to_arrayd/from_arrayd expose ndarray::ArrayD<f64>, the supported ndarray minor is part of the bridge’s public type identity — consumers build against ndarray 0.17. (ndarray 0.17.0 is yanked; use a non-yanked 0.17 patch.) RFC-062 first evaluated supporting 0.16 and 0.17 together via a bounded range; the maintainer chose the single-version requirement to keep Cargo.toml simple and readable — the architect ruling listed this as an acceptable alternative. No bridge API, behavior, copy-semantics, error, or zero-copy change, and core matten still carries no ndarray dependency. A public-dependency compatibility event handled as a lock-step family minor (RFC-030); it does not imply v1.0.

The v0.29 release family adds RFC-063 visual-understanding work: Markdown / ASCII diagrams for broadcasting, shape operations, matrix multiplication, axis reductions, statistics reductions, dynamic readiness, and the matten-data table-to-Tensor flow; canonical visual-summary examples; and the local-only tools/matten-report Markdown/plain-text report tool. This release does not change public API, published crate dependency graphs, core default features, runtime behavior, MSRV, maturity labels, or scope boundaries. Richer visualization/reporting work remains deferred and requires separate handoffs or RFC approval.

The v0.30 release family adds RFC-065 educational visualization and tensor-learning path work on top of the RFC-063 base: positioning guardrails, worked learner questions for shape/data meaning, and a local-only tools/matten-report --demo educational-path report. This release remains docs/examples/local-tool only: no public visualization/report crate, plotting dependency, expression tracer, autograd, public API change, published dependency change, runtime behavior change, MSRV change, or maturity-label change.

The v0.31 release family is an RFC-066/RFC-067 policy-cleanup release: the reviewed v1.0 readiness audit is recorded, BF-1 is remediated, RFC-067 resolves the MD-1 companion-maturity policy question, and the v1.0 gates now require the RFC-067 family maturity table for any future v1.0 release RFC with candidate-labeled companions. This release does not authorize v1.0 release preparation and does not change public API, published crate dependency graphs, runtime behavior, MSRV, feature flags, maturity labels, or companion scope.

The v0.36 release family adds the RFC-069 input-mode local HTML slice: tools/matten-report can write a bounded static self-contained HTML summary for --input <csv> --kind data-readiness --select <cols> when --format html --output <path> is explicit. Markdown/plain text remains the default report output, HTML still requires explicit --output, and generated artifacts remain local files rather than checked-in assets. The input-mode HTML report is summary-only: it escapes user-controlled strings and bounds column lists, long fields, conversion errors, and tensor previews. This release does not change public API, published crate dependency graphs, core runtime behavior, MSRV, feature flags, maturity labels, or companion scope. Public matten-report / matten-viz crates, core visualization APIs, expression tracing, autograd, SVG, Vega-Lite, JSON report output, notebook, GUI, browser-app scope, and general raw CSV HTML rendering remain deferred.

The v0.37 release family adds the RFC-071 fixed-demo private JSON slice: tools/matten-report can write deterministic private-local JSON artifacts for the five fixed demos when --format json --output <path> is explicit. Markdown remains the default report output, JSON remains local-file-only, and schema_version: 0 means the format is not a public compatibility contract. This release does not change public API, published crate dependency graphs, core runtime behavior, MSRV, feature flags, maturity labels, or companion scope. Public matten-report / matten-viz crates, public report schemas, core visualization APIs, expression tracing, autograd, input-mode JSON, SVG, Vega-Lite, notebook, GUI, browser-app scope, and general raw CSV JSON rendering remain deferred.

The v0.38 release family adds the RFC-073 private input-mode JSON slice: tools/matten-report can write a bounded private-local JSON artifact for the existing CSV data-readiness input path when --format json --output <path> is explicit, covering both successful strict numeric conversion and bounded conversion-error outcomes. input_mode: "csv" extends the schema_version: 0 envelope; the five fixed-demo JSON artifacts remain byte-identical. Every user-controlled string and list (paths, headers, column names, conversion-error text, tensor previews) is bounded with structured truncation metadata; raw CSV rows/cells and unselected cell values are never emitted; non-finite tensor values are rejected before any write. This release does not change public API, published crate dependency graphs, core runtime behavior, MSRV, feature flags, maturity labels, or companion scope. Public matten-report / matten-viz crates, public report schemas, input-mode JSON for other report kinds, raw CSV export, core visualization APIs, expression tracing, autograd, SVG, Vega-Lite, notebook, GUI, and browser-app scope remain deferred.

Public API snapshot

This page lists every public item in matten at the current v0.46 release family. It serves as the baseline for tracking breaking changes toward v1.0.0 and as the review gate required by RFC-015. Core matten’s public API most recently changed in RFC-104, which added get_mut/get_flat_mut (numeric) and get_element_mut (dynamic) — all three mirroring their existing getters — and RFC-108, which added is_empty(); all four are additive, none changing an existing signature or message. Before those, RFC-099 added try_dot/ try_matmul, and RFC-100 added Display for Tensor — both additive as well. Earlier still, RFC-087 added repeat, repeat_axis, tile, and meshgrid (see the shape composition section below). RFC-088 followed with negative indices in slice_str, but changed no public item — a signature-level grammar extension behind an existing method, not a new row here. RFC-102 is the same shape: slice() and slice_str() now accept dynamic tensors and return one instead of MattenError::Unsupported — a behavior change behind two existing methods, with no signature change and no new row. RFC-105 and RFC-108’s mm_mul fix are likewise behavior changes with no new row: mean/min/max/ argmin/argmax now return Err/panic-with-message on an empty tensor instead of panicking with a raw index error or returning NaN/inf/-inf, and dot/matmul/try_dot/try_matmul no longer panic on a zero-column product. RFC-110 and RFC-111 are the same shape again: mean_axis/min_axis/ max_axis/var_axis/std_axis now return Err/panic-with-message when the reduced axis has length 0, and every constructor, reshape, the shape-composition family, linspace/eye, and serde now accept a zero-sized shape instead of rejecting it — behavior changes behind existing signatures, no new row. matten-ndarray’s ZeroSizedAxis error variant (companion crate, not core) is deprecated by the same RFC and never constructed, kept rather than removed since the enum is #[non_exhaustive] but a removed variant still breaks a caller matching on it. The RFC-082 streaming feature, RFC-083’s functions before it, and RFC-090’s histogram were companion-crate (matten-data/matten-stats) additions, and the RFC-080/084/085 maturity promotions were label changes; none of the three touched core matten’s root exports.

Root exports

#![allow(unused)]
fn main() {
// Primary user-facing types
pub use matten::Tensor;
pub use matten::MattenError;
pub use matten::DataFormat;
pub use matten::MattenLimits;  // RFC-018: resource safety limits
pub use matten::SliceBuilder;

// Feature-gated
#[cfg(feature = "dynamic")]
pub use matten::Element;
#[cfg(feature = "dynamic")]
pub use matten::NumericPolicy; // RFC-017: numeric conversion policy

// Compiler-visibility plumbing — #[doc(hidden)], NOT user-facing extension points.
// IntoSliceRange and SliceConvert use a private sealed::Sealed supertrait;
// downstream crates cannot meaningfully implement either trait.
// Users never need to name them in imports.
#[doc(hidden)] pub use matten::IntoSliceRange;
#[doc(hidden)] pub use matten::SliceConvert;
#[doc(hidden)] pub use matten::SliceSpecRepr;
}

Dynamic tensor behaviour

Methods marked numeric-only panic with a matten unsupported error message when called on a dynamic tensor. Call try_numeric() to convert first.

Tensor — formatting (RFC-100)

See Display / formatting for the full contract and examples.

Tensor — construction

Tensor — shape inspection

Tensor — data access (numeric Tensor)

Tensor — shape operations (numeric Tensor)

Tensor — shape composition (numeric Tensor, RFC-039)

Associated functions (called as Tensor::concatenate(...)), not methods. Both take a borrowed slice &[&Tensor] and reject dynamic inputs.

repeat, tile, and meshgrid were added in RFC-087, closing RFC-039 §8’s three deferred shape-composition APIs.

Tensor — slicing (numeric Tensor)

SliceBuilder methods

Tensor — arithmetic (numeric Tensor)

Operator traits implemented for &Tensor: Add, Sub, Mul, Div, Neg — element-wise with broadcasting.

Scalar operators: &Tensor + f64, &Tensor - f64, &Tensor * f64, &Tensor / f64 (and reverse: f64 + &Tensor, f64 - &Tensor, f64 * &Tensor, f64 / &Tensor).

All panic on dynamic tensors.

Tensor — elementwise comfort math (numeric Tensor, RFC-038)

All panic on dynamic tensors (except try_clip, which returns Unsupported).

Tensor — linalg core-lite (numeric Tensor, RFC-041)

Small linalg-adjacent helpers — not a linear algebra backend. inverse, determinant, solve, eigen-decomposition, SVD, QR, LU, Cholesky, sparse, and BLAS/LAPACK are out of scope for core (use nalgebra or ndarray-linalg).

Tensor — statistics (numeric Tensor, RFC-040)

Population variance only (ddof = 0): var = sum((x_i - mean)^2) / n, std = sqrt(var), two-pass, NaN-propagating. Sample variance, quantile, percentile, histogram, covariance, correlation, and z-score are out of core scope.

Tensor — boundary / serde

Tensor — dynamic (#[cfg(feature = "dynamic")])

MattenLimits (RFC-018)

#![allow(unused)]
fn main() {
pub struct MattenLimits {
    pub max_dimensions: usize, // default: 8
    pub max_elements: usize,   // default: 1 048 576 (~1 M / ~8 MiB)
    pub max_parse_bytes: usize, // default: 128 MiB
}
}

Methods: MattenLimits::default(), MattenLimits::strict().

NumericPolicy (RFC-017, #[cfg(feature = "dynamic")])

Controls how Element values coerce to f64 in try_numeric_with.

Builder methods: .strict(), .permissive(), .allow_bool(), .allow_text_parse(), .none_as(value), .none_as_nan().

Conversion traits

MattenError variants

#[non_exhaustive]
pub enum MattenError {
    Shape      { operation: &'static str, message: String },
    Broadcast  { left: Vec<usize>, right: Vec<usize> },
    Allocation { requested_elements: usize, message: String },
    Slice      { input: Option<String>, message: String },
    Parse      { format: DataFormat, message: String },
    Io         { path: PathBuf, source: std::io::Error },
    Unsupported { operation: &'static str, message: String },
    InvalidArgument { operation: &'static str, argument: &'static str, message: String },
}

DataFormat variants

#![allow(unused)]
fn main() {
pub enum DataFormat { Json, Csv }
}

Element variants (#[cfg(feature = "dynamic")])

pub enum Element {
    Float(f64),
    Int(i64),
    Text(Arc<str>),
    Bool(bool),
    None,
}

Methods: try_as_f64() -> Option<f64>, is_numeric() -> bool, is_none() -> bool, as_text() -> Option<&str>, as_bool() -> Option<bool>, and the text(s) constructor.

Production migration guide

matten is the family car: small, approachable, Tensor-centered, and good for proof-of-concept work, learning, and small serious workflows. It stays deliberately dependency-light and does not try to become a dataframe engine, an ML framework, or a high-performance linear-algebra backend.

This guide is about the other half of that promise: helping you know when and how to leave matten when a workflow outgrows it. Moving a hot path to a production-oriented ecosystem is not a failure — outgrowing matten is a successful PoC outcome. It means the idea earned the move.

What this guide is — and is not

This guide helps you migrate intentionally. It is:

• a way to decide when to stay with matten and when to migrate;
• a target-selection matrix from your workload to the right ecosystem;
• a set of playbooks for specific targets (ndarray, nalgebra, Polars/Pandas, Candle, and NumPy);
• guidance on the bridge crates that own dependency-specific conversion.

It is explicitly not:

• a claim that matten is faster, or a promise that you can swap matten out unchanged;
• a claim that any target is universally “better” — it depends entirely on the workload;
• a tool that rewrites your code for you. matten helps you understand and plan a migration. The local matten-migrate helper can draft an advisory report, but it is not a source rewriter.

tools/matten-migrate is now available as a local, unpublished, advisory helper for generating a first migration-readiness report. It is a heuristic text/dependency scan, not a source rewriter and not a correctness oracle: it may miss or over-report usage, has not been validated against real downstream projects yet, and should be treated as a starting point for manual review.

The layered idea

core matten   →  owns Tensor; stays small; no heavy target-library dependencies
bridge crates →  own dependency-specific conversion (e.g. matten-ndarray)
docs (here)   →  when to stay, when to migrate, and how

Core matten gains no new heavy dependency from any of this. The conversion to a specific ecosystem lives in a dedicated bridge crate (such as matten-ndarray) or in your own code — never inside core matten.

Where to go next

• When to migrate — signals that you have outgrown matten, and the equally important signals that you should stay.
• Choosing a target — a matrix from workload shape to ecosystem.
• Common pitfalls — mistakes to avoid when moving data out.
• Readiness checklist and report template — turn the signals into an explicit, advisory decision you can record and review.
• Target playbooks — step-by-step, per-target migration guides.

For quick, copy-paste data-export snippets, the reference page Migration to specialised libraries is the companion to this narrative guide.

When to migrate

The honest default is: stay with matten until a concrete signal tells you to move. matten is built for PoC, learning, and small serious workflows, and most of those never need to leave. Migration is a deliberate response to pressure, not a rite of passage.

Signals that you have outgrown matten

Treat any of these as a real reason to plan a migration of the affected part of your workflow:

• Data-size pressure. Your arrays are large enough that matten’s copy-on-every- reshape/slice behavior shows up in profiles, or you are pushing past comfortable in-memory sizes.
• Runtime pressure. A dense numeric kernel (matrix multiply, matrix–vector products, operator application) is a measured hot path. In the accepted RFC-049 Rust peer comparison, dense matmul and matrix–vector tasks showed a noticeably larger gap to ndarray/nalgebra than lighter vector tasks did — so those are the kernels most worth moving when they get hot.
• Linear-algebra pressure. You need decompositions (LU, QR, SVD), solvers, or eigenvalues. matten intentionally does not provide these.
• Dataframe pressure. You need group-by, joins, pivots, or query-style operations. matten-data is an ingestion on-ramp (CSV/table → Tensor) and will not grow into a dataframe engine.
• ML / device pressure. You need autodiff, training loops, or GPU execution.
• Dynamic-ingestion pressure. You are leaning heavily on the dynamic feature for large or repeated messy-data cleanup, beyond a one-time on-ramp.

Signals that you should stay

Equally important — these are reasons not to migrate:

• The numeric work is small and not on a hot path.
• You are wiring data into a web API (serde in, serde out) with light math in between.
• You are learning, prototyping, or teaching, and approachability matters more than raw speed.
• Your messy data needs a one-time clean-then-compute pass, which matten’s dynamic on-ramp handles.

If none of the pressure signals above apply, staying with matten is the right call, and adding a heavyweight dependency would cost you simplicity for no real gain.

Migrate the hot path, not the whole program

Migration is rarely all-or-nothing. The common, healthy pattern is to keep matten for construction, ingestion, and glue, and move only the measured hot kernel into a specialised crate:

matten            →  build / ingest / shape the data, light math
specialised crate →  the heavy kernel (matmul, decomposition, training, group-by)

The target-selection matrix helps you map each pressure signal to a destination, and the playbooks show the per-target mechanics.

Choosing a target

There is no universally “best” target — the right destination depends on the shape of the pressure you are feeling. Use the matrix below to map a signal to an ecosystem, then open the matching playbook.

Target-selection matrix

A quick decision path

1. Is the bottleneck a dense numeric kernel (matmul, matrix–vector, operator application, axis reductions) that you have measured as hot? → ndarray (general N-D) or nalgebra (if it is fundamentally small/mid dense linear algebra needing decompositions).
2. Do you need linear-algebra results matten does not provide (LU/QR/SVD, solvers, eigenvalues)? → nalgebra.
3. Is the real need tabular (group-by/join/pivot/query)? → Polars (Rust) or Pandas (Python). Not matten-data.
4. Is it ML (autodiff/training/GPU)? → Candle or another ML framework.
5. Are you already in Python? → NumPy/Pandas, with matten as the upstream Rust producer if useful.
6. None of the above, or the work is small? → stay with matten.

Playbooks

Full per-target playbooks are available for every destination above: ndarray, nalgebra, Polars/Pandas, Candle, and NumPy. The two Rust array/linalg targets carry task-scoped positioning notes from the accepted RFC-049 peer comparison; the dataframe, ML, and Python targets are different paradigms with no such benchmark (see each playbook).

Common pitfalls

A few mistakes come up repeatedly when moving data out of matten. None are hard to avoid once you know to look for them.

Memory order: matten is row-major

matten stores tensor data in row-major (C order) logical layout. Some targets differ: nalgebra’s DMatrix is column-major. If you hand a flat Vec<f64> to a column-major constructor as if it were column-major, you will silently transpose your data. Always use a constructor that interprets the source order explicitly (for example nalgebra::DMatrix::from_row_slice, which reads row-major source), or transpose deliberately. The per-target playbooks show the correct constructor for each.

Conversions copy — plan for it

Both directions of a bridge conversion copy the underlying data. That is the right default for safety, but it means converting inside a tight loop is wasteful. Convert once, at the boundary between “build/ingest in matten” and “compute in the specialised crate”, not on every iteration.

do:    build in matten → convert once → run the hot loop in the target
avoid: convert ↔ on every iteration of the hot loop

f64 vs other dtypes

matten tensors are f64. Targets that want f32 (common in ML, e.g. Candle) need an explicit conversion, which is another copy and a precision change. Decide this at the boundary and do it once.

Dynamic tensors must be made numeric first

If your data came through the dynamic feature (the messy-data on-ramp), it may hold non-numeric or missing elements. Bridges reject dynamic tensors rather than guess. Resolve to a numeric tensor first (fill or drop missing values, then try_numeric()), and only then convert. See the dynamic reference.

Don’t expect matten-data to grow dataframe features

matten-data is an ingestion on-ramp (CSV/table → Tensor). If you find yourself wanting group-by, joins, pivots, or query expressions, that is a signal to move the tabular work to Polars or Pandas — not a gap to be filled in matten-data. It will not grow those features.

Migrate the kernel, keep the glue

The goal is rarely to rewrite everything. Keep matten for construction, ingestion, and glue; move only the measured hot kernel. A migration that replaces your whole program is usually a sign of over-migrating.

Migration readiness checklist

This checklist turns the vague question “should I leave matten?” into a set of concrete pressure signals. Work through it for the part of your workflow under question, mark each signal, and follow the mapping to a playbook. It is an advisory self-assessment — there is no tool that scans your code, and a high score does not by itself mean you must migrate.

How to read it: each signal is a yes/no probe. The more signals you mark “yes” — especially the first six — the stronger the case to move that hot part of the workflow. If almost everything is “no”, staying with matten is the right answer.

Pressure signals → target

Reading the result

• Signals 1–3 dominate → a dense-array hot path: ndarray.
• Signal 4 is present → you need capability matten lacks: nalgebra.
• Signal 5 dominates → the work is tabular, not array math: Polars/Pandas. Remember matten-data will not grow these.
• Signal 6 is present → you have crossed into ML: Candle or another framework.
• Signals 9–10 point to Python → NumPy/Pandas, with matten as an upstream producer.
• Signal 8 is “yes”, or almost everything is “no” → stay with matten. Adding a dependency would cost simplicity for no measured gain.

Migration is usually partial: move the signalled hot kernel, keep matten for construction, ingestion, and glue. When you are ready to write the result down, use the readiness report template; a filled-in example is in examples/.

Migration readiness report

When you have worked through the readiness checklist and want to record a decision — for a code review, a design doc, or just your own notes — fill in the report template below. It is a manual template: you write it, drawing on what you know about your own workload. There is no generator and no source-scanner.

This report is advisory. It does not prove production readiness, does not guarantee a target library is better, and does not perform automatic conversion.

Keep that framing in mind: the report’s job is to make a migration decision explicit and reviewable, not to certify anything.

For a local first draft, tools/matten-migrate can inspect a project and emit a Markdown report using this vocabulary. Its detection is heuristic: it may miss real matten usage, may over-report source-like text, and has not been validated against real downstream projects. Use its output as a starting point for manual review, not as an automated decision.

How to use the template

Copy the skeleton below into your own doc and fill each section. Sections you have nothing to say about can be marked “none” — that is itself useful information (e.g. “Manual redesign areas: none” means the move is a near-direct port).

Template

# matten Migration Readiness Report

## Summary
One or two sentences: what is being assessed, and the headline recommendation
(stay, or migrate which part to which target).

## Current matten usage
What the code does in matten today — the shapes, the operations (matmul, reductions,
slicing, dynamic ingestion), and which examples it resembles.

## Production pressure signals
Which checklist signals are present, and the evidence (a profile, a data size, a
required capability). Be concrete; "runtime pressure: the per-step matmul dominates
at N samples" beats "it feels slow".

## Recommended target(s)
The target(s) the signals point to, and why. It is fine to recommend "stay with
matten" if the signals are weak.

## Direct conversion candidates
The operations that map cleanly onto the target (e.g. matmul → ndarray `.dot()`),
including which bridge function carries the data across.

## Manual redesign areas
The parts that do not port mechanically and need rethinking (e.g. an iterative loop,
or switching an algorithm to a decomposition-based form). "none" is a valid answer.

## Bridge crates / tools
Which bridge crate applies (e.g. matten-ndarray) or that the conversion is manual
(e.g. nalgebra). Note copy/precision boundaries.

## Risks
What could go wrong: precision changes (f64 → f32), memory-order traps (row- vs
column-major), converting inside a hot loop, or scope creep into over-migration.

## Next steps
The concrete plan: profile to confirm the hot path, convert once at the boundary,
move the kernel, keep matten for setup/glue, and a checkpoint to reassess.

A filled-in example

See Linear regression (GD) readiness for the template applied to the 35_linear_regression_gradient_descent example.

Worked example: linear regression (gradient descent) readiness

This applies the readiness report template to the 35_linear_regression_gradient_descent example. It is illustrative — the example itself runs on toy data; the report imagines the same code scaled to real data and asks what would change.

This report is advisory. It does not prove production readiness, does not guarantee a target library is better, and does not perform automatic conversion.

matten Migration Readiness Report

Summary

Batch gradient descent for a linear model ŷ = X · θ. At the example’s toy size, stay with matten. If the same code runs on a real design matrix (thousands of samples, many features), move the per-step matrix products to ndarray via the matten-ndarray bridge, keeping matten for setup. A closed-form solve in nalgebra is an optional redesign.

Current matten usage

• X is a [samples, 2] Tensor (leading bias column, so θ = [b, w]).
• Each step runs two Tensor::matmul calls: predictions X · θ ([n,2] × [2] → [n]) and the gradient Xᵀ · residual ([2,n] × [n] → [2]).
• Xᵀ is formed once with Tensor::transpose and reused.
• The residual and the θ update are plain Rust (zip/map).
• The loop runs many iterations (2000 in the example).

Production pressure signals

• Runtime pressure (signal 2): present at scale. The two matmuls per step, over many iterations, are the hot path once X is large. This is the kernel worth moving.
• Data-size pressure (signal 1): present at scale. A large design matrix stresses matten’s copy-on-reshape/slice behavior.
• Linear-algebra pressure (signal 4): partial. The problem can be solved without iteration, via the normal equations — which needs a solver/decomposition matten lacks.
• Dependency policy (signal 8): low cost. matten-ndarray is already an available bridge, so the ndarray path adds little.
• Ecosystem/team (signals 9–10): Rust.
• Axis-reduction, dataframe, ML/device, and dynamic-ingestion signals are not present here.

Recommended target(s)

• ndarray (primary). Keep the gradient-descent structure as-is and run the two matrix products as ndarray .dot() (BLAS-backed for large X). This is a near-direct port.
• nalgebra (optional redesign). If you would rather not iterate, reformulate as a closed-form normal-equation solve using a decomposition. That is a change of algorithm, chosen for capability, not a mechanical port.
• Toy size: stay with matten. The signals only bite at real data sizes.

Direct conversion candidates

• X, Xᵀ, and θ → ArrayD<f64> with matten_ndarray::to_arrayd, converted once before the loop.
• The two matmul calls → ndarray .dot().
• Final θ back to a Tensor with from_arrayd if downstream code expects one.

Manual redesign areas

• The plain-Rust residual and θ update become ndarray elementwise operations — small, but not a literal copy-paste.
• The optional closed-form solve is a genuine redesign (assemble XᵀX and Xᵀy, solve), not a translation of the existing loop.

Bridge crates / tools

• matten-ndarray (to_arrayd / from_arrayd): copies both directions, f64 on both sides, so no precision change. See the bridge contract.
• The nalgebra option has no bridge crate; conversion is manual via DMatrix::from_row_slice (mind the row- vs column-major boundary).

Risks

• Converting inside the loop. Convert X/Xᵀ/θ once, before iterating — not per step.
• Column-major trap (nalgebra option only). Build DMatrix with from_row_slice so the row-major data is not silently transposed.
• Over-migration. Keep matten for constructing X and y; only the kernel needs to move.

Next steps

1. Profile at a realistic data size to confirm the matmuls are the bottleneck.
2. Convert X, Xᵀ, θ once via matten-ndarray; run the loop with ndarray .dot().
3. Keep matten for data construction and glue.
4. Reassess later: if you want a non-iterative solve, move to nalgebra; if the model grows into trained ML with autodiff/GPU, that is a Candle question.

Bridge conversion contracts

A bridge crate converts a matten::Tensor to and from a specific external type (for example matten-ndarray ↔ ndarray::ArrayD<f64>). Because a conversion can silently lose or reshape data if its rules are vague, every bridge crate documents a conversion contract: a fixed set of dimensions that say exactly what the conversion does. This page gives the template and the filled-in contract for the reference bridge, matten-ndarray.

The contract template

Every bridge contract documents these dimensions:

Two rules are constant across all bridges (see bridge-crate policy): conversions return Result and never panic on rejected input, and a bridge crate does not re-export core Tensor.

Reference contract: matten-ndarray

matten-ndarray converts between matten::Tensor and ndarray::ArrayD<f64>.

Examples

use matten::Tensor;
use matten_ndarray::{to_arrayd, from_arrayd};

// Tensor -> ArrayD<f64> (copies; row-major)
let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);
let arr = to_arrayd(&t)?;
assert_eq!(arr[[1, 0]], 3.0);

// ArrayD<f64> -> Tensor preserves *logical* order, even for a transposed array
let back = from_arrayd(arr.t().to_owned())?; // logical shape [2, 2], transposed
Ok::<(), matten_ndarray::MattenNdarrayError>(())

A dynamic tensor is rejected rather than guessed:

// to_arrayd(&dynamic_tensor)  ->  Err(MattenNdarrayError::DynamicTensor)
// Resolve to a numeric tensor first (e.g. try_numeric()), then convert.

Error-category note

The generic error categories sketched in RFC-051 (UnsupportedTensorKind, UnsupportedRank, …) are illustrative for future bridges, not a required enum schema. matten-ndarray’s existing variants (DynamicTensor / NdarrayShape / Matten, plus the deprecated, never-constructed ZeroSizedAxis) document its contract clearly and are compliant as-is; a bridge need not rename or expand its error enum to match the sketch.

Bridge-crate policy

Bridge crates are how matten connects to specific external ecosystems without dragging their dependencies into core. This page states the rules every bridge crate follows, and the checklist a future bridge crate must satisfy.

Why bridges are separate crates

Core matten owns the Tensor type and stays small and dependency-light. It must not gain a dependency on ndarray, nalgebra, Polars, Candle, or any other target library. Each target-specific conversion therefore lives in its own crate (for example matten-ndarray), which is the only place that target’s dependency appears.

core matten      →  owns Tensor; no target-library dependency
matten-ndarray   →  owns the ndarray dependency; converts Tensor ↔ ArrayD<f64>
(future bridges) →  own their target's dependency; same pattern

This boundary is CI-enforced: scripts/check-published-dependency-isolation.sh proves that the published core and companion crates do not pull in target/benchmark dependencies, with matten-ndarray → ndarray as the one allowed, documented exception.

Rules every bridge crate follows

• Own the target dependency. The bridge crate is the only published crate that depends on its target library.
• Do not add a dependency to core matten. A bridge never causes core to gain a target-library dependency.
• Do not re-export core Tensor. A bridge takes and returns matten::Tensor, but users import Tensor from matten, not from the bridge. (For example, matten-ndarray exports only to_arrayd, from_arrayd, and MattenNdarrayError.)
• Return Result, never panic on rejected input. Document the rejection cases.
• Publish a conversion contract. Fill in every dimension of the contract template.
• Name conversions to_<target> / from_<target>. This follows the to_arrayd / from_arrayd precedent (e.g. to_dmatrix / from_dmatrix, to_dvector / from_dvector). Deviate only if the target ecosystem has a stronger idiom, and justify it in that bridge’s RFC.

Current bridges

• matten-ndarray — the reference bridge (Tensor ↔ ndarray::ArrayD<f64>). Its contract is documented in bridge contracts.

There is no matten-nalgebra bridge today; the nalgebra playbook documents the manual conversion path, and a dedicated bridge is only a possible future direction, not a commitment.

Future bridge-crate checklist

Before a new bridge crate is created (which requires separate approval — see below):

• The target library has a clear, recurring conversion need that does not fit an existing bridge.
• The crate owns the target dependency; core matten gains nothing.
• Conversions are to_<target> / from_<target> and return Result.
• The crate does not re-export Tensor.
• A full conversion contract is filled in (copy/shape/memory order/dynamic/NaN/missing/dtype/error/performance).
• scripts/check-published-dependency-isolation.sh is extended so the new crate’s allowed/forbidden dependencies are enforced.
• The dynamic-tensor policy is explicit (reject, or document the numeric-first step).

No new bridge crate without approval

This policy page does not authorize creating new bridge crates. A new bridge (such as a hypothetical matten-nalgebra, matten-polars, or matten-candle) requires its own RFC and explicit approval. Documenting the pattern here does not pre-approve any specific crate.

Target playbooks

Each playbook is a step-by-step guide for moving a matten workflow to one specific ecosystem. They share a common structure: when to choose (and not choose) the target, how matten concepts map onto it, worked example migrations drawn from the examples, the conversion path, pitfalls, task-scoped positioning notes, and a minimal checklist.

Available now

• ndarray — general Rust N-D arrays; the first stop for dense numeric workloads at scale, with a contract-backed bridge crate (matten-ndarray).
• nalgebra — dense linear algebra: vectors, matrices, decompositions, and solvers.
• Polars / Pandas — dataframe analytics (group-by, joins, pivots, query). matten-data is an on-ramp and will not grow these.
• Candle — ML tensors, training, and device execution — without implying matten is an ML framework.
• NumPy — the Python scientific path, as a manual/conceptual hand-off.

Decision tree

measured dense numeric hot path?
├─ general N-D arrays / BLAS matmul / axis reductions   → ndarray
└─ small/mid dense linear algebra, decompositions       → nalgebra

need LU / QR / SVD / solvers / eigenvalues?             → nalgebra
need group-by / join / pivot / query?                   → Polars (Rust) / Pandas (Python)
need autodiff / training / GPU?                          → Candle / ML framework
already in Python / NumPy ecosystem?                     → NumPy (matten as upstream producer)
small work, ingestion, glue, learning?                   → stay with matten

If you are unsure whether you have outgrown matten at all, start with When to migrate.

Migrating to ndarray

ndarray is the general Rust N-D array crate: strided arrays, views, broadcasting, and (with a BLAS backend) fast matrix multiplication. It is the natural first production target when a dense numeric workload outgrows matten. A dedicated bridge crate, matten-ndarray, provides a contract-backed conversion in both directions.

Choose this target when

• You have general N-D numeric arrays and need production-grade array operations.
• Dense matmul or axis reductions are a measured hot path at scale.
• You want strided views, advanced indexing, or a BLAS backend.

Do not choose this target when

• The work is small or not on a hot path — staying with matten is simpler.
• You fundamentally need linear-algebra results (LU/QR/SVD, solvers, eigenvalues) → prefer nalgebra.
• The real need is tabular (group-by/join/pivot) → Polars/Pandas, not an array crate.

Concept mapping

Example migrations

These map directly from the shipped examples:

• 22_matrix_multiplication → ndarray when the matrices are large or the multiply is hot.
• 27_axis_reductions → ndarray; axis reductions are exactly where matten’s internal baseline flagged the widest internal cost, so this is a strong candidate to move.
• 35_linear_regression_gradient_descent → ndarray (or nalgebra) once the GD loop runs on real-sized design matrices.
• 50_rowwise_scoring → ndarray if rows get large, otherwise stay with matten.

Conversion path

The clean path is the matten-ndarray bridge, which copies, is numeric-only, rejects dynamic tensors, and preserves logical row-major order:

use matten::Tensor;
use matten_ndarray::{to_arrayd, from_arrayd};

let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0], &[2, 3]);

// matten Tensor -> ndarray ArrayD<f64> (copies)
let arr = to_arrayd(&t)?;

// ... heavy ndarray work here (dot, BLAS matmul, axis reductions, views) ...

// ndarray ArrayD<f64> -> matten Tensor (copies)
let back = from_arrayd(arr)?;
Ok::<(), matten_ndarray::MattenNdarrayError>(())

If you are not using the bridge crate, the manual path via flat data also works (see the reference page): t.into_vec() / t.shape() into ArrayD::from_shape_vec(shape, flat).

Common pitfalls

• Convert once. Both directions copy — do it at the boundary, not inside the hot loop.
• Dynamic tensors are rejected. Make the tensor numeric (try_numeric()) before converting; the bridge returns a DynamicTensor error rather than guessing.
• Reshape APIs moved. Prefer ndarray’s current reshape APIs over the deprecated into_shape; check the ndarray version matten-ndarray pins before copying snippets.

Performance / positioning notes

In the accepted RFC-049 Rust peer comparison (task-scoped, small fixed sizes, single machine — not a ranking), dense matmul and matrix–vector tasks showed the widest gap to ndarray (roughly an order of magnitude at those sizes), while a lighter vector task was competitive. The practical reading: if dense matmul, matrix–vector, or axis-reduction kernels are your measured hot paths, moving those to ndarray is where the benefit is concentrated. This is positioning, not a claim that either library is “better” in general.

Minimal checklist

• The hot path is a dense array kernel you have actually measured.
• You convert once at the boundary, not per iteration.
• The tensor is numeric (no dynamic elements) before conversion.
• You kept matten for construction/ingestion/glue where it was already fine.

Migrating to nalgebra

nalgebra is the dense linear-algebra crate: statically- and dynamically-sized vectors and matrices, plus decompositions (LU, QR, SVD), solvers, and eigenvalues. It is the right target when your workload is fundamentally small/mid dense linear algebra, especially when you need results matten intentionally does not provide.

There is no matten-nalgebra bridge crate today — conversion is manual (a few lines) and a dedicated bridge is only a documented future direction, not a commitment.

Choose this target when

• You need decompositions or solvers: LU, QR, SVD, eigenvalues, linear systems.
• Your data is naturally small/mid dense vectors and matrices.
• You want a typed linear-algebra API rather than general N-D arrays.

Do not choose this target when

• You need general N-D arrays or BLAS-backed bulk array ops → prefer ndarray.
• The work is small or not hot → stay with matten.
• The real need is tabular or ML → Polars/Pandas or Candle.

Concept mapping

Example migrations

• 20_dot_product / 21_matrix_vector_product → nalgebra DVector/DMatrix operations.
• 22_matrix_multiplication → nalgebra &a * &b (or ndarray for general N-D).
• 31_fibonacci_matrix_power → nalgebra matrix powers.
• 35_linear_regression_gradient_descent → nalgebra when you want a typed matrix/vector formulation (or want to switch to a closed-form solve via a decomposition).

Conversion path

Manual, via flat row-major data. DMatrix is column-major, so build it from a row-major slice with from_row_slice, which reads the source in row-major order:

use matten::Tensor;
use nalgebra::{DMatrix, DVector};

// vector
let v = Tensor::from_vec(vec![1.0, 2.0, 3.0]);
let dv = DVector::from_vec(v.into_vec());

// matrix (row-major source -> from_row_slice keeps the logical layout)
let m = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);
let shape = m.shape().to_vec();
let dm = DMatrix::from_row_slice(shape[0], shape[1], &m.into_vec());

// ... decompositions / solvers / matrix algebra here ...

To return to matten, read the matrix back in row-major order (transpose as needed) and rebuild with Tensor::new(data, &shape).

Common pitfalls

• Column-major trap. Do not feed a row-major flat Vec to a column-major constructor as if it were column-major — use from_row_slice or transpose deliberately, or you will silently transpose your data.
• Convert once at the boundary; conversions copy.
• Make dynamic tensors numeric first (try_numeric()).

Performance / positioning notes

In the accepted RFC-049 Rust peer comparison (task-scoped, small fixed sizes, single machine — not a ranking), nalgebra had lower overhead than matten on dense matmul and matrix–vector kernels, while a lighter vector task was competitive. The value of nalgebra, though, is usually capability rather than raw speed: decompositions and solvers that matten does not implement at all. If you need those, the migration is about what you can compute, not just how fast.

Minimal checklist

• You need dense linear algebra or a decomposition/solver matten does not provide.
• You build DMatrix with from_row_slice (or transpose deliberately).
• You convert once at the boundary; the tensor is numeric first.
• You kept matten for the parts where it was already a good fit.

Migrating to Polars / Pandas (dataframes)

Polars (Rust, with Python bindings) and Pandas (Python) are dataframe libraries: labeled columns, heterogeneous types, group-by, joins, pivots, and query expressions. They solve a different problem than matten, which is a numeric Tensor. Reach for them when your real need is tabular analytics, not array math.

The boundary you are crossing

matten-data is an ingestion on-ramp: it reads CSV/table data and hands you a numeric Tensor. It is intentionally minimal and will not grow group-by, joins, pivots, or query expressions. If you find yourself wanting any of those, that is the signal to move the tabular layer to a dataframe library — it is not a missing matten-data feature.

matten-data →  CSV/table → numeric Tensor   (on-ramp only)
Polars/Pandas →  group-by / join / pivot / query / labeled columns

Choose this target when

• You need group-by, joins, pivots, windowing, or query-style selection.
• Your data is tabular and heterogeneous (mixed column types, labels, nulls as a first-class concept).
• You want to explore/clean tabular data interactively before any numeric step.

Do not choose this target when

• Your data is already a clean numeric array and you only need array math → ndarray or stay with matten.
• You need decompositions/solvers → nalgebra.
• The tabular step is a one-time CSV-to-numeric on-ramp → matten-data already covers it.

Concept mapping

Example migrations

• data_00_quickstart → if the next step is group-by/join/pivot rather than array math, read the CSV straight into Polars/Pandas instead of matten-data.
• A CSV → clean → single numeric pass with no tabular analytics → stay with matten-data; it is the right size for that.

Conversion path

The usual pattern is not to convert a matten tensor into a dataframe, but to enter the dataframe library at the data source:

have a CSV and need tabular analytics?  → read it directly into Polars/Pandas
already have a numeric matten Tensor?    → export its columns if you must, but prefer
                                           doing tabular work upstream of matten

If you genuinely need to move a numeric Tensor into a dataframe, export its data (tensor.to_vec() / tensor.shape()) and build columns in the dataframe library; the exact constructor depends on the library and version, so follow its current docs.

Common pitfalls

• Don’t wait for matten-data to grow tabular features. It will not. Recognize the boundary early.
• Round-tripping is usually wrong. If tabular work is the point, do it in the dataframe library from the start rather than converting back and forth with matten.

Performance / positioning notes

There is no matten-vs-dataframe benchmark: a numeric tensor and a dataframe are different paradigms, and a cross-library/cross-language ecosystem comparison would be RFC-049 Phase 3, which is not authorized. Choose by capability and ecosystem fit (do you need tabular operations?), not by a measured speed comparison.

Minimal checklist

• Your real need is tabular (group-by/join/pivot/query), not array math.
• You enter the dataframe library at the data source rather than round-tripping.
• You are not waiting for matten-data to grow dataframe features.

Migrating to Candle (ML tensors)

Candle is a Rust ML tensor framework: autograd, neural-network layers, model loading, and CPU/GPU execution. Move here when your workflow becomes machine learning — training loops, autodiff, or device acceleration.

matten is not an ML framework and does not aim to become one. It has no autograd, no layers, no optimizers, and no device backend. When you need those, that capability lives in Candle (or another ML framework), not in a future matten feature.

Choose this target when

• You need automatic differentiation / backprop.
• You are building or running a model (layers, training loop, inference).
• You need GPU/device execution.

Do not choose this target when

• You are doing plain numeric array math with no learning → ndarray or stay with matten.
• You need classical linear-algebra results (decompositions/solvers) → nalgebra.
• The “ML” is actually a small hand-written numeric step (e.g. a single gradient-descent update) that matten already expresses clearly — it may not be worth a framework yet.

Concept mapping

Example migrations

• 35_linear_regression_gradient_descent → Candle once you want autodiff and an optimizer instead of a hand-written gradient step.
• 37_kmeans_small / 38_nearest_neighbor_classification → Candle (or a dedicated ML crate) if these grow into trained models on real data; for small teaching versions, matten is fine.

Conversion path

matten is f64; Candle workflows are commonly f32, so the boundary involves a precision conversion as well as a copy. The shape carries over directly. Illustratively (Candle is not a matten dependency):

#![allow(unused)]
fn main() {
use matten::Tensor;
// candle_core = { version = "0.x", features = ["..."] }

let t = Tensor::new(vec![1.0, 2.0, 3.0, 4.0], &[2, 2]);
let shape = t.shape().to_vec();
let flat_f32: Vec<f32> = t.as_slice().iter().map(|&v| v as f32).collect();
// let device = candle_core::Device::Cpu;
// let candle_t = candle_core::Tensor::from_vec(flat_f32, shape, &device)?;
}

Common pitfalls

• f64 → f32 is a precision change, not just a copy. Do it once at the boundary and be aware of the loss.
• Don’t expect matten to provide autograd or layers. If you reach for those, you have already crossed into ML-framework territory.
• A single update step is not a model. If your “training” is one hand-written step, consider whether you actually need a framework yet.

Performance / positioning notes

There is no matten-vs-Candle benchmark. They occupy different layers (a plain numeric tensor vs. an autodiff/device ML framework), and a cross-framework comparison would be RFC-049 Phase 3, which is not authorized. Choose Candle for ML capability and device support, not on the basis of a measured speed comparison.

Minimal checklist

• You actually need autodiff, layers/optimizers, or device execution.
• You handle the f64 → f32 precision change once, at the boundary.
• You are not treating matten as an ML framework it never claimed to be.

Migrating to NumPy (Python scientific stack)

NumPy is the foundation of the Python scientific ecosystem (SciPy, scikit-learn, Pandas, and most ML tooling sit on top of it). Move here when the workflow’s center of gravity is Python, or when you need a library that only exists in that ecosystem.

This is a cross-language boundary, so it is manual/conceptual: there is no automatic Rust↔Python conversion, and matten does not provide one. The realistic pattern is to use matten as an upstream Rust producer and hand the data to Python through a serialization format.

Choose this target when

• Your team or downstream pipeline is in Python.
• You need a Python-only library (SciPy, scikit-learn, a specific ML/stats package).
• The numeric work belongs next to Python data tooling rather than in a Rust binary.

Do not choose this target when

• You want to stay in Rust → ndarray (general arrays) or nalgebra (linear algebra).
• The work is small and already lives happily in matten.

Concept mapping

Example migrations

• Any numeric example (e.g. 35_linear_regression_gradient_descent, 36_heat_equation_1d) → reimplement in NumPy when the surrounding pipeline is Python; the row-major layout and shape transfer directly.

Conversion path

Hand data across the language boundary via a serialization format. The flat data is row-major, which matches NumPy’s default, so only the shape needs to travel with it:

matten (Rust):   tensor.to_vec()  +  tensor.shape()
   ↓  write to a shared format (CSV, JSON, or .npy / Arrow for larger data)
NumPy (Python):  np.loadtxt(...)   /  np.load("data.npy").reshape(shape)

For small data, CSV/JSON is simplest; for larger or repeated transfers, a binary format (.npy, or Arrow) avoids text parsing overhead. There is no in-process bridge — the two runtimes do not share memory here.

Common pitfalls

• No automatic bridge. Plan an explicit serialization step; do not expect in-process conversion between Rust and Python.
• Carry the shape. The flat buffer is row-major (NumPy’s default), but you must reattach the shape on the Python side.
• f64 everywhere in matten. If the Python side wants float32, cast there.

Performance / positioning notes

There is no matten-vs-NumPy benchmark, and one would be a cross-language RFC-049 Phase 3 comparison, which is not authorized. NumPy is C/BLAS-backed and fast on dense numeric work, but the reason to migrate here is usually ecosystem and language fit, not a measured speed comparison against matten.

Minimal checklist

• The workflow’s home is Python (team, pipeline, or a Python-only library).
• You have a concrete serialization hand-off (CSV/JSON for small, .npy/Arrow for larger) and you carry the shape across.
• You are not expecting an in-process Rust↔Python conversion.

Benchmarks

matten keeps a small, reproducible benchmarking and positioning program (RFC-049). Its goal is to describe matten’s position honestly and with evidence, not to win a performance contest.

The program answers questions like:

• What is matten good at?
• Where is it intentionally simpler?
• Where is it slower but acceptable?
• Where would performance become a blocker?
• How much code does a user write to solve small problems?

It deliberately does not claim that matten replaces ndarray, nalgebra, NumPy, SciPy, Pandas, or Candle. matten is a small, approachable, Tensor-centered Rust numeric crate for PoC, learning, and small workflows; the benchmarks exist to make that position legible.

Current status

The benchmark program is staged.

• Phase 1 — internal Rust baseline: implemented and accepted. A benchmark harness (benchmarks/, kept outside the workspace and unpublished); a core micro set and five scenario workloads drawn from the examples; a peak-RSS memory note on Linux; and an accepted internal baseline report.
• Phase 2 — Rust peer comparison (ndarray/nalgebra): complete and accepted. The official peer comparison was filled from a maintainer run on the baseline’s machine class and accepted by architect ruling on 2026-06-25. Peer tasks are opt-in behind the peers feature (off by default).
• Phase 3 — Python reference comparison (NumPy/Pandas): implemented and accepted. Optional scripts record ELOC, dependency footprint, versions, and code-shape notes. Runtime context is omitted by default and must never be used as a ranking. The report was refreshed with pinned Python dependencies and accepted by review on 2026-07-11.

Still deferred (designed in RFC-049, not yet implemented/authorized):

• Phase 4 — regression tracking policy and hard thresholds/gates.

Read next

Two paths, depending on what you need:

• Just want the results? → Results — a curated, readable summary of the latest numbers (Phase 1 internal baseline and Phase 2 peer comparison), with the “positioning, not ranking” framing. This is the reader’s page.
• Need to regenerate or extend the benchmarks? → Methodology for what is measured and the rules that keep the program honest, then the harness README.md in benchmarks/ for the maintainer path: the environment-capture snippet and the exact cargo bench … commands under How to regenerate (with environment capture).

The full reports (complete tables, environment, regeneration commands) live in benchmarks/reports/.

Benchmark methodology

This page records how matten’s benchmarks are measured and the rules that keep the program honest. It reflects RFC-049 Phase 1 (internal Rust baseline), Phase 2 (Rust peer comparison), and Phase 3 (Python reference comparison).

Purpose

Clarify matten’s position with reproducible evidence: execution time, memory behavior, example-code size (ELOC), and dependency footprint. The output is a positioning and regression-visibility tool, not a ranking or a marketing claim.

Non-goals

The benchmark program must not:

• claim matten is faster than NumPy, or a replacement for ndarray/nalgebra;
• include SciPy, Candle, GPU suites, or broad Pandas dataframe benchmarks;
• add hard CI speed-fail thresholds (initially);
• change any public API merely to make a benchmark faster;
• pressure the project into scope creep.

Metrics

• Execution time — measured with criterion for Rust microbenchmarks: inputs are pinned and built outside the timed body, no printing happens inside the measured section, and black_box is used to prevent the optimizer from deleting the work.
• Memory — peak resident set size (see below). Informative, not a gate.
• Example ELOC and dependency footprint — reported alongside timings when available, and as the main Phase 3 Python-reference evidence, to show approachability and dependency trade-offs.

Workloads (Phase 1)

A core micro set: construction, reshape/flatten, elementwise add/mul, broadcasting, sum/mean, sum_axis/mean_axis, matmul, and a small slice. An optional dynamic try_numeric micro-workload is available behind the harness’s dynamic feature.

A scenario set of five small, well-known computations taken from the examples: cosine similarity, a Markov-chain step, a tiny PageRank step, a linear-regression gradient-descent step, and a 1-D heat-equation step.

Heavier examples (k-means, nearest-neighbor, finite differences, trapezoidal integration) remain outside the benchmark set.

Memory measurement policy

Phase 1 uses Linux peak RSS, which is coarse but adequate and requires no allocator instrumentation:

/usr/bin/time -v cargo bench --manifest-path benchmarks/Cargo.toml --bench scenarios -- --noplot
# record "Maximum resident set size"

Measuring smaller per-scenario commands gives a more useful figure than one giant run. No custom global allocator and no allocation-level instrumentation are added in Phase 1. macOS (/usr/bin/time -l) and Windows are deferred; memory must never block Phase 1 if allocation-level measurement is not ready.

Environment recording

Every report records: OS, kernel, CPU, RAM, rustc version, target, build profile, the exact command, and the peak-RSS tool. Benchmarks are workload- and environment-specific; numbers from different machines are not directly comparable.

A runnable capture snippet for these fields, plus the full regenerate steps, lives in the harness README under How to regenerate (with environment capture).

CI policy

CI compile-checks the harness (cargo bench --manifest-path benchmarks/Cargo.toml --no-run) but does not run full benchmarks. CI may fail if the harness does not compile, a report generator breaks, or a result schema is invalid — but never because a run is slower or uses more memory than a previous run. There are no hard performance gates.

Required disclaimer (in every report)

These results are workload-specific and environment-specific. They are for positioning and regression visibility, not universal ranking.

Phase 2 — Rust peer comparison (implemented)

Phase 2 was authorized once the maintainer-run internal baseline was accepted, and the peer-comparison harness is implemented. Peer comparison is:

• task-scoped, not library-scoped — a task is included only if the compared implementations solve the same small mathematical problem with comparable data representation and no hidden extra library capability. It is a Rust peer comparison for positioning, never a competitor ranking or a “faster than X” claim;
• opt-in — behind the peers feature (ndarray/nalgebra as optional deps), off by default, so the default harness build and ordinary CI stay peer-free. The peers bench is compile-checked only in a separate, manually/scheduled workflow, never with speed gates;
• isolated — published crates are positively proven free of peer dependencies by scripts/check-published-dependency-isolation.sh (the matten-ndarray → ndarray bridge is the one allowed exception).

Run it with cargo bench --manifest-path benchmarks/Cargo.toml --features peers --bench peers -- --noplot; results go in benchmarks/reports/peer-comparison-v0.1.md.

The Phase 2 harness, report template, and official peer report are complete: the official Rust peer comparison was filled from a maintainer run on the same machine class as the accepted internal baseline and accepted by architect ruling on 2026-06-25 (benchmarks/reports/peer-comparison-v0.1.md, Report ID matten-rfc049-rust-peer-comparison-v0.1).

Phase 3 — Python reference comparison (implemented and accepted)

Phase 3 is implemented as a code-shape-first reference slice. It is:

• optional — Python, NumPy, and Pandas are not required for ordinary Rust CI;
• not a runtime ranking — the first report omits runtime context entirely;
• narrow — NumPy covers the five scenario tasks; Pandas is limited to CSV/table cleanup into a numeric matrix;
• dependency-explicit — benchmarks/python/requirements.txt pins exact versions, setup may contact PyPI, and reference runs must not access the network.

The Phase 3 runner records:

• ELOC without imports and with imports, comparing Python scripts with minimal matten task-equivalent snippets rather than didactic examples;
• direct dependency pins and installed versions;
• transitive dependency counts where packages are installed;
• short code-shape notes;
• missing optional dependency behavior when NumPy/Pandas are not installed.

Run it with:

python3 benchmarks/python/run_references.py --environment
python3 benchmarks/python/run_references.py --all

If runtime context is added later, it must be same-machine, same-report context for both matten and the Python references, and it must not be sorted or worded as a winner/loser ranking.

The report was refreshed with pinned Python dependencies installed and accepted by review on 2026-07-11. Hard performance gates remain not authorized.

Benchmark results

This page is the reader’s view: a curated summary of matten’s benchmark results so they are readable from inside the book. It is a small representative selection, not the full matrix — the complete numbers, environment details, and regeneration steps live in the reports under benchmarks/reports/. If you want to run the benchmarks, see the methodology and the harness README.md.

These numbers are workload-specific and environment-specific. They were produced on one virtualized machine with microbenchmark methodology. They are a positioning and regression-visibility reference — not a ranking, and not a “faster than X” claim. matten optimizes for time to a runnable PoC, not benchmark leadership.

The Phase 1/2 timing numbers below are the v0.2 maintainer refresh at workspace 0.28.3, produced under the unchanged RFC-049 methodology. The architect-accepted Rust reference baseline is v0.1 (see the reports); the relative positioning matches v0.1. Absolute timings drift run-to-run with VM load — all libraries move together — so the shape of the results is the signal, not the exact microseconds. Phase 3 adds code-shape evidence, not timing numbers.

Phase 1 — internal baseline

matten measured against itself, to establish a reference point and make future regressions visible (RFC-049 Phase 1).

• Baseline ID: matten-rfc049-internal-baseline-v0.2 — maintainer refresh at v0.28.3 (reference: …-v0.1, accepted 2026-06-24).
• Environment: Ubuntu 26.04, 8 vCPU AMD (virtualized), rustc 1.93.1, profile bench (opt-level 3), Criterion defaults; git 5953c9f, workspace 0.28.3. Not comparable across machines.

Representative medians (full table in the report):

Peak RSS was not captured in this refresh (the VM lacked GNU /usr/bin/time); it is informative-only and never a gate. The accepted v0.1 baseline recorded ~44 MiB for the full scenario run under the same methodology, dominated by Criterion’s own footprint rather than the small tensors.

The clearest signal is that axis reductions are currently matten’s most expensive core path — the combined sum_axis/mean_axis workload (~1.30 ms) is roughly 400× the whole-tensor sum/mean (~3.23 µs) and ~17× a 64×64 matmul. This is recorded as positioning / regression-visibility information, not a defect: it is the natural first place to look if axis-reduction cost ever matters for your workload.

Phase 2 — Rust peer comparison

The same small problems placed next to two established Rust numeric crates, ndarray and nalgebra, each in its native type (RFC-049 Phase 2). This shows where matten’s approachable Tensor API sits — including where it is slower but acceptable — not a ranking of libraries.

• Report ID: matten-rfc049-rust-peer-comparison-v0.2 — maintainer refresh at v0.28.3 (reference: …-v0.1, accepted 2026-06-25).
• Environment: same machine class as the baseline; git 5953c9f, workspace 0.28.3, ndarray 0.17.2, nalgebra 0.33.3. Peer tasks are opt-in behind the peers feature (off by default). This run was taken at ndarray 0.17.2, so the harness now matches the matten-ndarray bridge’s supported ndarray version. Not comparable across machines.

Representative Criterion medians (full six-task table in the report):

On these small dense kernels the production-oriented peers generally carry less overhead than matten’s Tensor API — expected, and consistent with matten’s DX-first role. The size of the gap is the useful part, and it is not uniform: a vector×matrix step (markov) is competitive here — ahead of both peers at this size — while dense matmul and matrix×vector steps (heat, pagerank) show the widest gaps (~7.5–9×). A consistent internal pattern is that matten’s matrix×vector path is its widest gap while its vector×matrix path is competitive — echoing the axis-reduction signal from Phase 1.

Phase 3 — Python reference comparison

The Python reference comparison is code-shape-first: it records ELOC, dependency footprint, and short code-shape notes for small NumPy/Pandas references next to matching minimal matten solution snippets. Runtime context is intentionally omitted in the first report.

• Report ID: matten-rfc049-python-reference-comparison-v0.1.
• Scope: NumPy for the five scenario tasks; Pandas only for CSV → selected columns → missing fill → numeric matrix.
• Environment note: NumPy 2.3.5 and Pandas 2.3.3 were installed from the pinned requirements for the refreshed report. Dependency counts exclude optional extras.

Representative ELOC rows (without imports / with imports):

ELOC is an approachability signal, not a quality or performance score. NumPy/Pandas are mature ecosystems with broad native surfaces; matten keeps the same small workflows in a Rust-first Tensor path without claiming to replace those ecosystems.

Read next

• Methodology — what is measured, what is not, and the rules that keep the program honest.
• Full reports with complete tables, environment, and regeneration commands: benchmarks/reports/internal-baseline-v0.2.md, benchmarks/reports/peer-comparison-v0.2.md, and benchmarks/reports/python-reference-comparison-v0.1.md (with accepted v0.1 references alongside the Rust reports).

Phase 4 (regression gates) is designed in RFC-049 but deferred and not yet measured.

Reports

matten includes a local development tool, matten-report, that renders small fixed demonstrations of shape reasoning, dynamic-tensor readiness, and preprocessing over a handful of built-in scenarios. The five pages under this section are its Markdown output, generated once and committed so they are readable here without a checkout.

What these are

• Fixed demos, not a live tool. Each page is the output of one matten-report --demo <kind> invocation against data baked into the tool. Nothing here runs in your browser, and nothing here reads data you provide — see the Playground for the page that does compute live, on shapes you enter.
• Not automatic expression tracing. Every page says so in its own ## Input section. These demonstrate specific, hand-chosen operations — they do not observe or replay arbitrary code.

What matten-report is — and is not

matten-report is a local development tool: workspace-excluded, publish = false, and never published to crates.io. It is not a matten public API, and using it does not require depending on anything beyond the crates you already use.

The tool can also render HTML and JSON, and can run against a CSV file you supply (--input <path> --kind data-readiness). Neither the HTML/JSON output nor that live-input mode is published here, or anywhere public — RFC-070 declined a public reporting or visualization surface, and generating these five Markdown pages does not reopen that decision (RFC-097 §3). What you are reading is rendered output, not an interface anything can build against.

Running it yourself

# Any of the five fixed demos, Markdown to stdout:
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo shape-flow

# Against your own CSV:
cargo run --manifest-path tools/matten-report/Cargo.toml -- \
  --input your-data.csv --kind data-readiness --select column_a,column_b

The five demos

• shape-flow — broadcasting, reshape, axis reductions, and matmul, the same operations as the Playground, shown as fixed output.
• educational-path — a longer walk through shape reasoning, dynamic readiness, and standardization in one report.
• mlprep-standardization — before/after column standardization.
• data-readiness — CSV column selection, missing-value counts, and strict numeric conversion.
• dynamic-readiness — a mixed-type dynamic tensor, its readiness masks, and two conversion policies.

Staying accurate

These pages are generated, not hand-maintained — regenerating them from the commands above must reproduce them byte for byte. scripts/check-report-demos.sh enforces that in CI; if the tool’s output ever changes, these pages are regenerated and recommitted in the same change, never edited by hand.

matten shape-flow report

Input

demo: shape-flow note: fixed demo report, not automatic expression tracing

Broadcast add

input a: shape [2, 3] input b: shape [3] operation: a + b shape flow: [2, 3] + [3] -> [2, 3] result values:

11.0 22.0 33.0
14.0 25.0 36.0

Reshape

input: shape [2, 3] operation: reshape([3, 2]) shape flow: [2, 3] -> [3, 2] result values:

1.0 2.0
3.0 4.0
5.0 6.0

Axis reductions

input: shape [2, 3] mean_axis(0): [2, 3] -> [3] mean_axis(0) values: [2.5, 3.5, 4.5] mean_axis(1): [2, 3] -> [2] mean_axis(1) values: [2.0, 5.0]

Matrix multiplication

left: shape [2, 3] right: shape [3, 2] operation: left.matmul(right) shape flow: [2, 3] @ [3, 2] -> [2, 2] result values:

22.0 28.0
49.0 64.0

matten educational-path report

Input

demo: educational-path note: fixed educational demo report, not automatic expression tracing

How to read shapes first

1. ask what shape each input has
2. ask which axes align, disappear, or remain
3. read the output shape before reading values
4. convert dynamic data before numeric computation

Broadcasting

shape flow: [3, 1] + [1, 4] -> [3, 4] axis 1: left repeats across 4 columns axis 0: right repeats across 3 rows result values:

11.0 21.0 31.0 41.0
12.0 22.0 32.0 42.0
13.0 23.0 33.0 43.0

Reshape and transpose

reshape: [2, 3] -> [3, 2] reshape values:

1.0 2.0
3.0 4.0
5.0 6.0

transpose: [2, 3] -> [3, 2] transpose values:

1.0 4.0
2.0 5.0
3.0 6.0

meaning: reshape changes grouping; transpose changes coordinate meaning

Axis reductions

mean_axis(0): [2, 3] -> [3] mean_axis(0) keeps columns: [2.5, 3.5, 4.5] mean_axis(1): [2, 3] -> [2] mean_axis(1) keeps rows: [2.0, 5.0]

Matrix multiplication

shape flow: [2, 3] @ [3, 4] -> [2, 4] shared inner dimension: 3 result values:

38.0 44.0  50.0  56.0
83.0 98.0 113.0 128.0

Dynamic readiness

dynamic shape: [2, 3] none mask:

0.0 0.0 1.0
0.0 0.0 0.0

numeric mask: strict policy readiness

1.0 0.0 0.0
1.0 0.0 1.0

Text values are not numeric-ready under the strict mask next step: clean values, then call try_numeric()

Standardization

operation: standardize_columns(input) shape flow: [3, 2] -> [3, 2] before column mean: [10.000, 100.000] before column population std: [1.633, 16.330] after column mean: [0.000, 0.000] after column population std: [1.000, 1.000]

What this report is not

• not a public API
• not source scanning
• not a renderer
• not model-quality analysis

matten mlprep-standardization report

Input

demo: mlprep-standardization note: fixed demo report, not automatic model-quality analysis

Operation

operation: standardize_columns(input) meaning: each column is centered to mean 0 and population standard deviation 1

Before

shape: [3, 2] row-major values:

 8.000  80.000
10.000 100.000
12.000 120.000

column mean: [10.000, 100.000] column population std: [1.633, 16.330]

After

shape: [3, 2] row-major values:

-1.225 -1.225
 0.000  0.000
 1.225  1.225

column mean: [0.000, 0.000] column population std: [1.000, 1.000]

Shape meaning

shape flow: [3, 2] -> [3, 2] rows: samples unchanged columns: features unchanged

matten data-readiness report

Input

demo: data-readiness

Source columns

• region
• sales
• cost
• note

Selected columns

• sales
• cost

Columns left out

• region
• note

Missing values

Numeric conversion

strict conversion: success

Tensor preview

shape: [3, 2] row-major values:

100.0 40.0
150.0 45.0
120.0 55.0

matten dynamic-readiness report

Input

demo: dynamic-readiness note: fixed demo report, not automatic data profiling

Dynamic values

shape: [2, 3] row-major values:

• [0, 0] Float(1.0)
• [0, 1] Text(“2.5”)
• [0, 2] None
• [1, 0] Int(4)
• [1, 1] Text(“6.0”)
• [1, 2] Float(8.0) schema summary:
• Float: 2
• Int: 1
• Text: 2
• None: 1

Readiness masks

none mask:

0.0 0.0 1.0
0.0 0.0 0.0

numeric mask: strict policy readiness

1.0 0.0 0.0
1.0 0.0 1.0

strict numeric-ready: false

Strict conversion

result: error: strict conversion rejects Text and None values

Explicit policy conversion

policy: none_as(0.0) + allow_text_parse() converted shape: [2, 3] converted row-major values:

1.0 2.5 0.0
4.0 6.0 8.0

Contributing

The contributing documentation has moved:

• Development process — QA commands, reviewer checklist, definition of done, file-size guidelines.
• Architecture — module layout, design invariants, feature matrix, milestone sequence.

Development process

This page distils the workflow that applies to every PR in matten. It is drawn from the common sections that appear across all implementation handoffs (RFC-002 through RFC-008).

Required QA commands

Run these before requesting review unless the PR is explicitly documentation-only:

cargo fmt --all --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-targets
cargo test --doc

When the PR touches feature-gated behaviour, also run:

cargo test --all-targets --no-default-features
cargo test --all-targets --features serde
cargo test --all-targets --features json
cargo test --all-targets --features csv
cargo test --all-targets --features dynamic
cargo test --all-features

When the PR touches parser, JSON, CSV, indexing, or shape arithmetic, add at least one targeted invariant or regression test. Property tests and fuzz targets are future hardening candidates rather than current release gates; if a fuzz crate is introduced for a slice, its targets should compile even when they do not run on every PR.

Reviewer checklist

Every PR is reviewed against this list regardless of which RFC it implements:

• The implementation keeps the matten::Tensor public surface simple.
• No public lifetime, storage, or dimension generic leaks into user-facing examples.
• Panic-zone APIs have actionable matten … error messages.
• Result-zone APIs do not panic for malformed external input.
• Shape product and allocation-sensitive paths use checked helpers.
• Documentation examples compile and match implemented behaviour.
• Any deferred work is listed explicitly rather than hidden in TODO comments.

Definition of done

A milestone or RFC is complete when:

• all planned PRs are merged;
• acceptance criteria in the RFC are satisfied;
• all QA commands pass;
• README and rustdoc examples for the affected API surface compile and are accurate.

File-size guideline

• Consider splitting a .rs file if it exceeds 300 effective lines of code (ELOC) (non-blank, non-comment-only lines).
• Splitting is strongly recommended above 500 ELOC.
• Test code within src/ lives in tests.rs (a sibling file) or in a tests/ subdirectory; use the 2018+ module style (foo.rs + foo/ coexistence, no mod.rs).

Design-before-code sequence

Requirements → External design → RFC → Implementation → Tests

Do not widen the public API beyond what an accepted RFC specifies without a follow-up RFC or maintainer approval.

Release checklist

This page documents the steps required before publishing any matten release. It is the canonical gate referenced by RFC-015.

This page covers whether a release is fit to go out. It does not decide when one happens — that is RFC-094’s release cadence policy: a correctness fix to published code is a patch and ships as soon as it is reviewed; anything adding public API is a minor and batches until two or more themes have landed, 28 days have passed, or the owner asks; and a change that does not reach crates.io is not a release at all. The last is testable rather than editorial — if git diff --name-only <last-tag>..HEAD -- crates/ is empty, there is nothing to release.

Before every release

Release tags use bare SemVer with no v prefix, for example 0.46.0.

1. Source verification

cargo fmt --all --check
cargo fmt --manifest-path tools/matten-report/Cargo.toml --check
cargo fmt --manifest-path tools/matten-migrate/Cargo.toml --check
bash scripts/check-core-dependency-boundary.sh   # RFC-022 core boundary gate
bash scripts/check-published-dependency-isolation.sh  # RFC-049 §B1 per-crate peer-dep isolation
bash scripts/check-matten-data-scope.sh          # RFC-042 matten-data anti-scope guard
bash scripts/check-benchmark-dependency-sync.sh  # benchmark harness ndarray pin == workspace requirement
bash scripts/check-streaming-scope.sh            # RFC-037 streaming / large-CSV anti-scope guard
bash scripts/check-release-docs.sh               # doc-truth + examples naming-band guards
bash scripts/check-doc-code.sh                   # every non-ignored ```rust block in docs/src compiles
bash scripts/check-report-demos.sh               # docs/src/reports/*.md match matten-report's current output
bash scripts/check-tool-tests.sh                 # RFC-117: workspace-excluded tools' own shell test suites
cargo clippy --all-targets --all-features -- -D warnings
cargo clippy --all-targets --no-default-features -- -D warnings
cargo clippy --all-targets --no-default-features --features dynamic -- -D warnings
RUSTFLAGS="-D warnings" cargo check --all-targets --all-features
cargo test --all-targets
cargo test --doc --all-features

2. Feature matrix

cargo test --no-default-features
cargo test --no-default-features --features serde
cargo test --no-default-features --features json
cargo test --no-default-features --features csv
cargo test --no-default-features --features dynamic
cargo test --no-default-features --features dynamic,json
cargo test --no-default-features --features dynamic,csv
cargo test --no-default-features --features dynamic,json,csv
cargo test --all-features

3. Examples

cargo check --examples
cargo check --examples --all-features
cargo run --example 00_quickstart
cargo run --example 06_broadcasting
cargo run --example 08_slicing_builder
cargo run --example 12_boundary_error_handling
cargo run --example 57_visual_shape_axis_summary
cargo run --example dynamic_00_quickstart --features dynamic,json,csv
cargo run --example dynamic_05_dirty_csv_cleanup --features dynamic,json,csv
cargo check -p matten --example dynamic_09_visual_readiness_summary
cargo run -p matten --example dynamic_09_visual_readiness_summary --features dynamic
cargo run -p matten-data --example data_06_visual_readiness_summary
cargo run -p matten-mlprep --example mlprep_visual_standardize_summary
cargo check --manifest-path tools/matten-report/Cargo.toml
cargo test --manifest-path tools/matten-report/Cargo.toml
bash tools/matten-report/tests/process-boundary.sh
bash tools/matten-report/tests/module-boundaries.sh
bash tools/matten-report/tests/module-boundaries.sh --self-test
cargo clippy --manifest-path tools/matten-report/Cargo.toml -- -D warnings
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo data-readiness
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo data-readiness --output target/matten-report-demo.md
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo data-readiness --format html --output target/matten-report-data-readiness.html
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo data-readiness --format json --output target/matten-report-data-readiness.json
cargo run --manifest-path tools/matten-report/Cargo.toml -- --input tools/matten-report/fixtures/small.csv --kind data-readiness --select sales,cost
cargo run --manifest-path tools/matten-report/Cargo.toml -- --input tools/matten-report/fixtures/small.csv --kind data-readiness --select sales,cost --format html --output target/matten-report-input.html
cargo run --manifest-path tools/matten-report/Cargo.toml -- --input tools/matten-report/fixtures/non_numeric.csv --kind data-readiness --select sales,cost --format html --output target/matten-report-input-error.html
cargo run --manifest-path tools/matten-report/Cargo.toml -- --input tools/matten-report/fixtures/small.csv --kind data-readiness --select sales,cost --format json --output target/matten-report-input.json
cargo run --manifest-path tools/matten-report/Cargo.toml -- --input tools/matten-report/fixtures/non_numeric.csv --kind data-readiness --select sales,cost --format json --output target/matten-report-input-error.json
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo shape-flow
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo shape-flow --output target/matten-report-shape-flow.md
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo shape-flow --format html --output target/matten-report-shape-flow.html
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo shape-flow --format json --output target/matten-report-shape-flow.json
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo dynamic-readiness
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo dynamic-readiness --output target/matten-report-dynamic-readiness.md
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo dynamic-readiness --format html --output target/matten-report-dynamic-readiness.html
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo dynamic-readiness --format json --output target/matten-report-dynamic-readiness.json
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo mlprep-standardization
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo mlprep-standardization --output target/matten-report-mlprep-standardization.md
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo mlprep-standardization --format html --output target/matten-report-mlprep-standardization.html
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo mlprep-standardization --format json --output target/matten-report-mlprep-standardization.json
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo educational-path
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo educational-path --output target/matten-report-educational-path.md
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo educational-path --format html --output target/matten-report-educational-path.html
cargo run --manifest-path tools/matten-report/Cargo.toml -- --demo educational-path --format json --output target/matten-report-educational-path.json
cargo check --manifest-path tools/matten-migrate/Cargo.toml
cargo test --manifest-path tools/matten-migrate/Cargo.toml
cargo clippy --manifest-path tools/matten-migrate/Cargo.toml -- -D warnings
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- list-targets
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- inspect tools/matten-migrate/fixtures/simple-core-project
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- report tools/matten-migrate/fixtures/simple-core-project
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- report tools/matten-migrate/fixtures/simple-core-project --output target/matten-migration-report.md
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- suggest --target ndarray tools/matten-migrate/fixtures/receiver-method-project
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- suggest --target polars-pandas tools/matten-migrate/fixtures/common-rust-collisions-project
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- suggest --target stay-with-matten tools/matten-migrate/fixtures/simple-core-project
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- explain-api Tensor::matmul
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- explain-api matmul
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- explain-api matten_ndarray::to_arrayd
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- explain-api matten_data::Table
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- check-bridges tools/matten-migrate/fixtures/ndarray-bridge-project
cargo run --manifest-path tools/matten-migrate/Cargo.toml -- check-bridges tools/matten-migrate/fixtures/simple-core-project

4. MSRV

cargo +1.85.0 build
cargo +1.85.0 test --all-features --quiet

5. Public API audit

Compare the current public surface against docs/src/reference/public-api-snapshot.md.

Allowed root exports:

• Tensor
• MattenError
• DataFormat
• MattenLimits
• SliceBuilder
• Element (under #[cfg(feature = "dynamic")])
• NumericPolicy (under #[cfg(feature = "dynamic")])

Allowed #[doc(hidden)] exports (compiler visibility only, not user-facing):

• IntoSliceRange
• SliceConvert
• SliceSpecRepr

Run a spot-check:

grep -n "^pub use" src/lib.rs

Verify no module accidentally became pub mod.

cargo public-api snapshot (manual, pre-v1.0 minimum-viable step; RFC-066 NF-2, re-confirmed open by the RFC-074 re-audit). Not wired as a CI gate or project dependency. Before a minor/major release that touches public API surface, run it by hand for matten and any companion whose surface changed, and reconcile the output against docs/src/reference/public-api-snapshot.md and each crate’s ## Public API README block:

cargo install cargo-public-api   # one-time, not a project dependency
cargo public-api --manifest-path crates/matten/Cargo.toml
cargo public-api --manifest-path crates/matten-ndarray/Cargo.toml
cargo public-api --manifest-path crates/matten-mlprep/Cargo.toml
cargo public-api --manifest-path crates/matten-data/Cargo.toml
cargo public-api --manifest-path crates/matten-stats/Cargo.toml

Wiring this into CI (toolchain pinning, nightly requirements) is a separate, explicit decision — do not add it silently as part of a routine release.

6. Documentation truth pass

# No stale version strings in user-facing files
grep -R "Status:.*0\.[0-9]\{2\}\." README.md docs/src/ src/lib.rs || true

# No stale "matten 0.x" in runtime messages
grep -rn "matten 0\." src/ | grep -v "CHANGELOG\|#\[" || true

# No version-specific claims in lib.rs crate docs
grep "This is.*0\." src/lib.rs || true

7. CHANGELOG

• Every API change has a changelog entry.
• Changelog entries describe actual changes, not planned ones.
• No changelog entry claims a fix that is not in the code.
• If the entire scope of this release is local-tool-only (workspace-excluded, publish = false crates such as tools/matten-report or tools/matten-migrate), the CHANGELOG entry must include a one-line justification for cutting a lock-step family checkpoint despite no published-crate change (RFC-075 §3.1).

8. Version bump

Update Cargo.toml version. During v0.x, patch releases (0.13.x) should not introduce new public API unless a minor release (0.14.0) is intended.

Additional gates for minor releases (0.14.0, 0.15.0, …)

• New public API has a corresponding accepted RFC.
• Public API snapshot is regenerated and reviewed.
• mdBook examples for new APIs compile and run.
• Migration guide updated if any method signature changed.

Public-dependency-minor changes

When a published crate re-exposes a third-party type in its public API (for example matten-ndarray exposing ndarray::ArrayD<f64> through to_arrayd/from_arrayd), changing the supported minor of that dependency is a public-API compatibility event — not a routine cargo update — and is handled as a lock-step family minor (RFC-030). Before releasing such a change:

• The change has an accepted RFC recording the supported version(s) and the decision (a single bump vs. a bounded range). Precedent: RFC-062 (ndarray → 0.17), which weighed a 0.16+0.17 range before the maintainer chose a single-version requirement to keep Cargo.toml simple.
• If a range is supported, CI verifies the crate’s tests, doctests, and examples against each supported minor — e.g. cargo update -p <dep> --precise <ver> in a fresh checkout (so the per-job lockfile edit is not committed). A single-version requirement needs only the normal job against the resolved patch (document which patch CI targets).
• No version-conditional bridge/crate code. If the unchanged crate cannot compile against every supported minor, narrow the range instead of adding #[cfg(...)] branches or per-version feature flags.
• Docs state that the resolved dependency minor is part of the crate’s public type identity, name any yanked patch that is excluded and not a tested target, and note that docs.rs renders a single resolved minor even though CI verifies the full range.
• MSRV is re-verified with the new dependency version in the graph. A dependency’s own rust-version is not sufficient — its transitive dependencies can raise the floor independently.
• Core matten dependency isolation is re-confirmed (the published-dependency-isolation guard still passes; the change must not leak a peer dependency into the core graph).
• If the dependency is also used by the workspace-excluded benchmark harness (e.g. a peer pin in benchmarks/Cargo.toml), its pin is synced by hand and check-benchmark-dependency-sync.sh passes — the harness cannot inherit { workspace = true }, so this guard catches a forgotten sync.

Workspace core-dependency requirement

Companion crates inherit core matten through [workspace.dependencies] as matten = { version = "0", path = "crates/matten", default-features = false } (RFC-064). Do not narrow this requirement during routine family releases. User docs and examples still show explicit matched release pins so downstreams see the supported family set.

Push, confirm CI green, then publish

This section applies to every release, patch included — not only minor ones. Pushing, confirming CI, tagging, and publishing happen regardless of release size, so this lives outside ## Additional gates for minor releases on purpose: a patch release must not be able to read that heading and skip straight to tagging.

Push, confirm CI green, then tag

The release sequence is push → confirm CI green → tag → publish, in that order, for a reason at each step:

1. Push main. CI runs on push; it cannot report on a commit that has not been pushed yet.

2. Confirm CI is green on the commit just pushed — not “CI was green recently,” not the previous run, the commit that is about to be tagged. A red run on that commit stops the release. Do not tag or publish until it is green. Check with:

   gh run list --limit 5
   

   or the repository’s Actions tab, and match the run against the commit SHA just pushed.

3. Tag, only after step 1 has already landed on the remote. A tag pointing at a commit absent from the remote is the orphaned-tag defect this project repaired once, for 0.38.0/0.39.0.

4. Publish — see below.

0.46.0 was tagged and published across four consecutive red CI runs because this step did not exist: every local gate passed, and the workflow result on the commit just pushed was never checked (RFC-117, RFC-118). This step does not automate that check — it only makes it impossible to miss by accident.

Publishing: one workspace command, not five per-crate ones

cargo publish --workspace --dry-run
cargo publish --workspace

Publish the whole workspace in one invocation. Cargo resolves the order itself and — the reason this matters — verifies every crate before uploading any, so a failure in the last companion aborts before core is irreversibly live. crates.io has no unpublish, only yank, which makes a half-published family a permanent artifact of the registry rather than a mistake you can undo.

This supersedes the older instruction to publish matten first and then each companion in dependency order. That sequence predates cargo publish --workspace and carried exactly the partial-publication hazard above; it was still being followed as late as the 0.42.0 release, where the owner stopped it before the first upload. Keep the following in mind, but do not turn them back into a manual sequence:

• matten is still published before the companions — cargo does this for you.
• A companion dry-run run on its own may fail before core is visible on crates.io. That is a sequencing artifact, not a dependency-policy failure, and --workspace avoids it entirely.
• If the broad version = "0" requirement is intentionally changed, update RFC-030/RFC-064, this checklist, companion README compatibility notes, and package dry-run expectations in the same review slice.

Verify afterwards against the sparse index, not the JSON API:

curl -s https://index.crates.io/ma/tt/matten-stats | grep '"vers":"<version>"'

https://crates.io/api/v1/crates/<name> now returns HTTP 403 under the crates.io data-access policy, so a verification step built on it reports nothing and looks like a failed publish.

v1.0.0 gate

v1.0.0 requires explicit confirmation from the maintainer (nabbisen). It is not triggered automatically by any feature or test passing.

Before v1.0.0, the project should have:

• stable core public API;
• clear dynamic on-ramp story;
• strong, scoped examples;
• reliable diagnostics;
• documented companion-crate boundary (RFC-022);
• clean feature matrix across all profiles;
• an RFC-067 family maturity table in the v1.0 release RFC if any lock-step family crate remains production-ready candidate.

Architecture

This is the contributor reference: source layout, re-exports, invariants, milestone history. For the reader-facing crate overview and the data-model/lifecycle picture, see Architecture and Data model and lifecycle.

Source layout

src/
  lib.rs          crate root: public re-exports, #![forbid(unsafe_code)]
  error.rs        MattenError + DataFormat (RFC-005)
  shape.rs        validate_shape, strides, coord↔flat helpers (RFC-003)
  tensor.rs       Tensor struct, constructors, accessors, arange
  tensor/
    ops.rs        shape ops, slicing, boundary APIs (split per 300-ELOC rule)
  limits.rs       MattenLimits — single source of truth for allocation budgets
  convert.rs      From/TryFrom trait impls (RFC-004)
  reshape.rs      permute_axes, reshape helpers (RFC-007)
  slice.rs        SliceSpec, SliceBuilder, slice_str parser (RFC-008)
  ops.rs          ops/ module root
  ops/
    broadcast.rs  broadcast_shape, BroadcastCtx, apply_binary (RFC-006)
    broadcast/
      tests.rs    BroadcastCtx unit tests
    tensor_ops.rs Add/Sub/Mul/Div for &Tensor pairs
    scalar_ops.rs &Tensor op f64, f64 op &Tensor
    unary_ops.rs  Neg
  tests.rs        test module root
  tests/
    tensor.rs     construction, shape validation, fill ctors, arange, limits
    convert.rs    From/TryFrom
    error.rs      MattenError / DataFormat model
    shape.rs      row-major index helpers
    ops.rs        broadcasting, scalar ops
    reshape.rs    reshape/flatten/transpose/swap_axes/get
    slice.rs      SliceBuilder, slice_str
    math.rs       reductions, axis reductions, matmul, NaN policy
    dynamic.rs    dynamic test dispatcher
    dynamic/
      element.rs  Element model tests
      tensor.rs   dynamic construction, JSON, CSV
      lifecycle.rs storage, utility, is_none_mask, lifecycle
      guards.rs   accessor guards, diagnostics
      policy.rs   NumericPolicy, inspection helpers

Module style: foo.rs + foo/ coexistence (Rust 2018+). No mod.rs files.