ANTS / CANTS / EvoTS

Nature-Inspired Neural Architecture Search for Time Series Forecasting

Overview

This research program develops evolutionary and nature-inspired methods for automatically discovering neural network architectures — eliminating the need for hand-crafted designs. Starting from ant colony optimization (ACO) applied to recurrent networks in 2017, the work has grown into a full framework spanning continuous search spaces, graph-based representations, and most recently, transformer architecture search.

A consistent theme across all stages: the evolved architectures are evaluated on real-world time series from engineering and science — turbine vibration, power systems, and multi-variate forecasting benchmarks — not just toy problems.

ANTS

The Ant-based Neural Topology Search (ANTS) algorithm applies ant colony optimization to directly search the topology of recurrent neural networks. Rather than fixing a layer structure and tuning hyperparameters, ANTS lets ant agents construct the network graph from scratch.

ANTS search animation
A live ANTS search run. Each ant selects its own input node and output node, then constructs a path through the network graph — choosing which recurrent cell to place at each node and which connections to wire across layers and time steps. The glowing dots are pheromone traces: deposited along paths taken by ants whose candidate architectures performed well, they strengthen signals in promising regions of the search space and guide subsequent foragers toward them. Traces decay over time, letting the colony forget poor regions and concentrate on what works.
  • Cell types: agents select from Δ-RNN, GRU, LSTM, MGU, and UGRNN cells at each node.
  • Recurrent connections: can span multiple layers and multiple time steps, not just adjacent ones.
  • Sparsity via pheromones: pheromone update rules are designed to mirror L1 and L2 regularization, biasing the search toward sparser, more efficient topologies.
  • Specialized ant roles: explorer ants build the feed-forward skeleton; social ants then select nodes to wire recurrent memory connections.
  • Communal weight sharing: the colony shares the best-found weights for initialization, reducing backpropagation epochs needed per candidate and speeding up the entire search.

ANTS-evolved networks significantly outperformed single- and two-layer baselines of the same cell types, as well as NEAT, on challenging engineering time series datasets.

CANTS

The Continuous Ant-based Neural Topology Search (CANTS) extends ANTS from a discrete graph search to a continuous 3D (and later 4D) space. Ant agents traverse a spatial volume from inputs to outputs, placing nodes and connections anywhere in that space rather than choosing from a fixed set of candidates. This removes the need to pre-specify the search graph and allows far greater architectural variety.

A later variant eliminates backpropagation entirely during the search, relying on weight perturbation and colony-level fitness signals — making it viable in settings where gradient computation is impractical or unavailable.

CANTS was also extended into CG-CANTS-N, a graph-based framework that generalizes the approach to problems beyond time series forecasting, including combinatorial optimization domains.

EvoTS Latest · 2026

Evolutionary Transformer Search for Time Series Forecasting (EvoTS) brings the same neuro-evolutionary philosophy to transformer architectures. Rather than adopting a fixed transformer design, EvoTS uses evolutionary search to discover task-specific transformer topologies from modular building blocks.

Key design choices:

  • Modular genome: architectures are encoded as combinations of attention, feed-forward, and projection modules, giving the search a rich but structured space to explore.
  • Repair mechanism: a validation step during evolution detects and corrects structurally invalid configurations, keeping the population feasible throughout.
  • Benchmark evaluation: tested on all four ETT datasets (ETTh1, ETTh2, ETTm1, ETTm2) at forecast horizons of 96, 192, 336, and 720 steps in the multivariate-to-multivariate setting.

Evolved architectures achieve competitive and in several cases improved mean squared error relative to strong transformer baselines, demonstrating that evolutionary search can find meaningful structure even in the large, modular space of transformer designs.

arXiv:2607.00154  ·  GECCO 2026

Publications

  1. EvoTS: Evolutionary Transformer Search for Time Series Forecasting — GECCO 2026
  2. CG-CANTS-N: A Versatile Graph-Based Framework for Scalable and Adaptive Problem Solving Across Domains — 2025
  3. Continuous Ant-Based Neural Topology Search — Software Impacts, 2024
  4. Backpropagation-Free 4D Continuous Ant-Based Neural Topology Search — Applied Soft Computing, 2023
  5. Ant-Based Neural Topology Search (ANTS) for Optimizing Recurrent Networks — EvoApplications 2020
  6. An Empirical Exploration of Deep Recurrent Connections and Memory Cells Using Neuro-Evolution — 2020
  7. Investigating Recurrent Neural Network Memory Structures using Neuro-Evolution — GECCO 2019
  8. Optimizing Long Short-Term Memory Recurrent Neural Networks Using Ant Colony Optimization to Predict Turbine Engine Vibration — Applied Soft Computing, 2018