This experiment adds fractional-calculus-inspired memory to a decoder transformer. Two implementations provide different ways to retain a long history: a causal convolution and a sum of exponentials.
The initial task is a synthetic copy problem. A model sees a sequence, crosses a delimiter, and must reproduce the earlier tokens. A conventional transformer provides a comparison within the same setup.
The question is whether the additional memory helps as the gap or sequence grows, and what it costs to compute. Success on this task would still leave open whether the mechanism helps with natural-language tasks.