Research

Classification loss experiments

Comparisons of softmax, hyperspherical losses, and complex-valued output models.

SimAct

Studying feedback from local activation and context-learning modules.

Waves

Recursive composition and readout of bounded two-dimensional states.

Code chooser

Learning to select a program from several generated candidates.

NPC Brain

Experiments in character reactions and small learned action selectors.

Lean Intuition

Learning search heuristics for Lean theorem proving.

TPCN

Experiments in continuous-time predictive coding.

TimWM

Latent world-model experiments with learned action effects.

Token branching

Comparing continuations generated from isolated tokens.

MKAR

Attention Residuals reproduction and experiments with retrieval across model paths.

Learned Error Objectives

Experiments using an error-optimization network as a learned objective for language-model training.

Attentional EON

Retrieving stored state changes during predictive-coding inference.

Persistent Attention Memory

Experiments with persistent key/value memory and boundary-triggered updates in transformer attention.

Predictive Coding for Density Updates

Updating electron-density estimates in a simplified energy model.

Foggy PCN

Predictive-coding experiments with noise, attention, and reconstruction.

Embodied Predictive Coding

Experiments linking predictive-coding state updates to actions in simple control tasks.

Temporal Transformer

Processing token chunks across several temporal scales.

Predictive-Coding Attention

Predictive-coding attention experiments for text classification and image reconstruction.

PCPPO

An experiment using reinforcement-learning agents to choose structural changes in a predictive-coding network.

Adaptive Predictive Coding

Experiments with neuron and synapse updates in a network that can add or prune connections.

Activation Neural Network

Character-level text-generation experiments with an MLP network and an attention-based decoder.

TTRL

Teacher-student training and reinforcement learning for text agents.

TimFinPPL

Reinforcement-learning experiments with portfolio allocation and trading simulations.

AutoTarget Transformer

Experiments with learned target representations for training transformer models.

Multidimensional Transformer

Character-level sequence tagging with character, word, and within-word positions.

Landscape Transformer

Adding attention over learned prototypes to token representations.

Signal Accumulative Model

Carrying decaying accumulated signals between successive model inputs.

WiseR

An unfinished reinforcement-learning environment for navigating text embeddings.

WordScrunch

Updating word embeddings using attention scores from word descriptions.

Input PPO

Experiments using reinforcement learning to adjust the inputs supplied to a classifier.

Associative memory and predictive coding

Experiments combining associative memory, predictive coding, and attention.

CarRacing world model

A small world-model prototype with a learned visual representation and controller.

Directional predictive coding

Experiments with directions, rotations, and local relationships in neural models.

Distributed residual graph networks

Comparing local and dense communication between persistent node states.

EBM reconstruction

Using an energy-based model as a learned reconstruction loss.

EEG experiments

Applying AEON models to EEG and P300 tasks.

Explicit objectives

Experiments with representing and checking the objectives supplied to a model.

Fractional Transformer

Testing long-memory mechanisms inside a decoder transformer.

GiRT

A proposed persistent 3D world model with explicit uncertainty.

GlyphJEPA

Representing token embeddings as image patches for latent prediction.

GPT World Space

Communication-agent experiments using GPT-2 representations.

Habitat navigation

A scaffold for learning navigation with language-model demonstrations.

Hindsight

Experiments with planning context and feedback between reasoning and execution models.

Latent terrain

Predicting logits between fixed language-model trajectories.

Latent-code dialogue

A reinforcement-learning chatbot experiment using latent codes.

Local prediction

Experiments with node-local errors, messages, and inference updates.

Memory Diffusion Blocks

Refining next-token representations drawn from fast-weight memory.

Memory routing

Experiments with write and retrieval modules across language-model context chunks.

NCM continuation experiments

Studying additional training and refinement in the released NCM generator.

Neural audio compression

Representing audio with a coordinate-based neural network.

Plasticity and task relevance

Notes and experiments on retaining the ability to learn new tasks.

Predictive-coding exercises

Small experiments with predictive coding and cellular dynamics.

RLLM

Reinforcement-learning experiments with world models, planning, and communication tasks.

Self-correcting computational phases

An investigation of error correction through state dynamics.

Signal-gated graph

Language-model experiments with local prediction and recurrent graph updates.

Software design rules

A reading companion to A Philosophy of Software Design.

Trading simulations

Experiments with backtesting, paper trading, and multiple trading agents.