Engineering a GPU-native DataFrame engine.

Context

Analytical workloads combine expressions, filtering, and mathematical operations over columns of data. Tachyon is a NeoCraft DataFrame engine exploring GPU-native execution for these workloads, with an Apache Arrow-inspired columnar model.

Contribution

The project brings together expression parsing and type inference, column representations, vectorized operations, and GPU evaluation. Its public README documents runtime-compiled GPU kernels and support for half-precision numeric types, including f16 and bf16.

Decisions that shape the engine

Make columns the unit of work

The columnar model organizes data by field. It gives the execution engine a consistent representation for vectorized math, filtering, and expression evaluation.

Compile kernels at runtime

JIT-compiled GPU kernels allow execution to be specialized at runtime. The README presents this as a core part of Tachyon’s GPU execution model.

Keep expression semantics explicit

The documented examples parse an expression, infer its type against a schema, and evaluate an expression over a column. Schema context and error modes make the interpretation of an expression explicit at the API boundary.

Validation

The public development workflow separates CPU and GPU testing. Default setup and CPU tests do not require CUDA. GPU tests require a compatible NVIDIA compute driver and CUDA compiler and runtime support; explicit CPU-only and GPU-required test modes are documented.

The README includes assertions for expression type inference and GPU evaluation. These examples demonstrate the expected API behavior; they do not provide workload timings or benchmark results.

Evidence you can inspect

The public repository contains the project description, supported features, setup and test commands, and API examples. Consult the current source for its implementation and development status.

Working on a demanding compute path?

We can help assess the workload, understand where acceleration fits, and define what a useful result would look like.