No Common Yardstick for Inverse-Designed Colour

Question: How are inverse design methods for structural colour in nanophotonic systems—such as adjoint optimisation, generative models, and surrogate-assisted optimisation—being benchmarked and compared against one another?

Answer

They are barely being compared at all, and where they are, the comparison is not commensurable. Across the retrieved literature there is no shared structural-colour test problem, no shared baseline set, and no shared metric. Each method paper demonstrates itself on its own device—an achromatic RGB beam deflector [4], a CMOS image-sensor colour router [6], a thermophotovoltaic metasurface [13], a chiral metasurface [8]—against baselines drawn from its own methodological lineage, and reports whichever quantity flatters it: total figure of merit, forward-solver call count, wall-clock speedup, inference throughput, image-fidelity proxies, or "accuracy" undefined. The closest thing to a common currency is the number of full-wave simulations, but because papers differ on whether dataset generation, precomputation, and VAE/network training count toward that budget, the headline "order-of-magnitude" claims cannot be ranked against one another. The single most reproducible pattern is directional rather than quantitative: hybrid pipelines that couple adjoint gradients to a learned latent or generative representation beat both pure gradient-based topology optimisation and pure simulation-free generative inference on their own chosen tasks [4] [6] [22] [3]. For structural colour specifically, the field is running on borrowed benchmarks.

Every comparison is run on its own device, so no two papers share a test problem

The comparison unit in this literature is a bespoke device, not a community task. HiLAB is evaluated on an achromatic RGB beam deflector reaching ~25% balanced diffraction efficiency [4]; AdjointDiffusion is evaluated on a bent waveguide and a colour router [6]; PearSAN on a thermophotovoltaic metasurface reaching 97% maximum design efficiency [13]; a neural-network-vs-genetic-algorithm study on chiral GaP/air and PMMA/air metasurfaces [8]. Because no two of these share a geometry, material palette, spectral target, or resolution, the reported gains are improvements relative to different denominators, and a method that wins on a deflector is never tested on a colour target. The one paper that names the problem explicitly calls out the absence of "an open-source, standardized infrastructure and evaluation benchmark" as what limits cross-disciplinary comparison [9].

Forward-solver call count is the de facto common currency, but the accounting is not consistent

Simulation budget is the nearest thing to a shared axis. AdjointDiffusion reports roughly 2×10² simulations against 10⁵–10⁶ for pure deep-learning approaches [6]; HiLAB reports over an order of magnitude fewer full simulations than conventional topology optimisation "even after accounting for training overhead" [4]; PearSAN claims at least an order of magnitude speedup over previous methods [13]; SP2RINT claims 1825× over simulation-in-the-loop adjoint design at digital-comparable accuracy [22]; PNGF claims three orders of magnitude in design time with no accuracy loss versus full-wave [2]; a physics-tailored surrogate predicts fields four orders of magnitude faster than the solver it was trained on [19]. These are six different quantities—solver calls, design wall-clock, per-iteration cost, field-prediction cost, training-adjusted totals—measured against six different reference implementations, so the numbers are internally meaningful and mutually incomparable.

Baselines are chosen from within each paper's own lineage, not from the strongest available method

Where a head-to-head exists, the comparator is typically the method the authors are arguing against rather than the current best. AdjointDiffusion benchmarks against MMA and SLSQP [6]; HiLAB against "conventional TO pipelines prone to local optima" [4]; the chiral-metasurface study against a genetic algorithm [8]. A multi-agent reinforcement-learning paper does position itself against "previous state-of-the-art gradient-based optimization" and explicitly offers its environment as a reusable benchmark [17], and a representation-learning review frames the output-side/input-side split as a genuine trade-off in data requirements, generalisation, and capacity for novel discovery rather than a ranking [3]—but no retrieved paper runs three or more paradigms under one roof on one device.

Generative and surrogate methods are scored on proxies, not on the colour target

Where a generative model is used, the reported quality metric is usually a proxy for geometry reconstruction rather than a measure of the optical or perceptual target. A diffusion-based spectrum-to-geometry framework reports PSNR, SSIM, Dice, IoU, boundary F-score, and spectral-consistency RMSE/R² evaluated through a frozen forward surrogate [18]; a fabrication-variation generative model is scored by IoU, KL divergence, and Wasserstein distance [10]; MOCLIP reports 97% "accuracy" for generative latent-space optimisation and 0.2 million zero-shot samples per second [5]; an LLM benchmark reports accuracy as a function of parameter count [20]. Outside photonics, a comparison of generative optimisers against Bayesian optimisation does the opposite and uses a Pareto-front distance metric, with qEHVI reaching GD = 0.0 versus 15.03 for a weaker BO baseline [7]—showing an evaluation convention the photonics papers have not adopted.

Hybrids win, but only on their own chosen ground

The most consistent qualitative finding is that coupling a physics gradient into a learned representation outperforms either ingredient alone: HiLAB's adjoint-seeded VAE plus Bayesian search explores near-global optima with far fewer simulations than topology optimisation [4]; AdjointDiffusion injects adjoint gradients into each denoising step and beats nonlinear optimisers on both efficiency and manufacturability [6]; SP2RINT alternates transfer-matrix training with adjoint correction to reach digital-comparable accuracy far faster than simulation-in-the-loop [22]; the review literature independently concludes that hybrid physics-plus-data frameworks help escape poor local optima [3]. Because each of these wins against baselines chosen within its own paper, the convergence is suggestive rather than established.

Conflicts and gaps

The papers disagree on whether data generation is part of the cost. HiLAB counts it explicitly [4]; the foundation-model and surrogate papers foreground throughput and speedup without an equivalent amortised accounting [5] [19]. They disagree on what "no loss" means: PNGF claims accuracy parity with full-wave simulation [2] while an inverse-designed chiral metasurface reports 115% simulated helical dichroism against 62% measured [14]—a gap that makes any simulation-only ranking suspect as a predictor of realised performance. Non-uniqueness undermines single-point scoring directly: papers on one-to-many mapping and on range-aware acquisition argue that the right output is a diverse set of valid designs, not a single optimum [11] [12] [21]. Most consequentially, perceptual colour difference under defined illuminants and angular acceptance—the metric that would actually define a structural-colour benchmark—does not appear anywhere in the retrieved corpus as a comparison axis, and the one colour-relevant device that is used as a test problem (the CMOS image-sensor colour router) appears in a single paper [6]. Finally, at least one prominent commentary argues that black-box design tooling should be paired with physical understanding rather than ranked by output metrics alone [1].

What would change the answer

A structural-colour benchmark suite would settle most of this: a fixed target set expressed as reflectance or transmission spectra tied to colorimetric specifications (sRGB and wide-gamut primaries under stated illuminants, with an angular cone), a fixed design domain and material palette, and a mandated reporting table covering total forward-solver calls including dataset generation, wall-clock on named hardware, final figure of merit, perceptual colour error, and fabrication-constraint satisfaction. Existing infrastructure could host it: MAPS-InvDes already exposes adjoint optimisation steps with fabrication-variation models [9], and the AdjointDiffusion colour router is open-source and could serve as the seed problem [6]. The protocol should borrow the Pareto-distance and hypervolume conventions already used in cross-domain generative-versus-Bayesian comparisons [7] together with diversity and validity reporting [12] [21], and report budget-scaling curves over multiple random seeds rather than best-of-N. Benchmark suites of the kind proposed for differentiable-physics inverse problems [16] and for ML-enhanced inverse design on shared photonic tasks [15] show the format is achievable; what is missing is a single team running adjoint, generative, and surrogate-assisted methods on the same colour target with held-out experimental validation as arbiter.

References

[1] P. Lalanne, O. Miller. "The hidden dimension in nanophotonics design: understanding". arxiv. 2026-09-29T00:00:00.000Z. URL: https://doi.org/10.1088/2399-1984/ae8631

[2] Thi Phuong Thao Nguyen, James Wang, Constantine Sideris. "Fast Nanophotonic Inverse Design using Precomputed Numerical Green Functions". arxiv. 2026-10-05T00:00:00.000Z. URL: https://arxiv.org/abs/2610.02821

[3] Reza Marzban, Ali Adibi, Raphael Pestourie. "Inverse Design in Nanophotonics via Representation Learning". arxiv. 2025-10-15T00:00:00.000Z. URL: https://arxiv.org/abs/2507.00546

[4] Reza Marzban, Hamed Abiri, Raphael Pestourie, Ali Adibi. "HiLAB: A Hybrid Inverse-Design Framework". arxiv. 2025-09-17T00:00:00.000Z. URL: https://arxiv.org/abs/2505.17491

[5] S. Rodionov, A. Burguete-Lopez, M. Makarenko, Q. Wang, F. Getman, A. Fratalocchi. "MOCLIP: A Foundation Model for Large-Scale Nanophotonic Inverse Design". arxiv. 2026-08-25T00:00:00.000Z. URL: https://doi.org/10.1038/s41467-026-76714-x

[6] Dongjin Seo, Soobin Um, Sangbin Lee, Jong Chul Ye, Haejun Chung. "Physics-guided and fabrication-aware inverse design of photonic devices using diffusion models". arxiv. 2025-04-25T00:00:00.000Z. URL: https://arxiv.org/abs/2504.17077

[7] Muhammad Bilal Awan, Abdul Razzaq, Abdul Shahid. "Benchmarking Generative AI Against Bayesian Optimization for Constrained Multi-Objective Inverse Design". arxiv. 2025-11-04T00:00:00.000Z. URL: https://arxiv.org/abs/2511.0007

[8] Davide Filippozzi, Alexandre Mayer, Nicolas Roy, Wei Fang, Arash Rahimi-Iman. "Advancing Machine Learning Optimization of Chiral Photonic Metasurface: Comparative Study of Neural Network and Genetic Algorithm Approaches". arxiv. 2025-12-16T00:00:00.000Z. URL: https://arxiv.org/abs/2512.13656

[9] Pingchuan Ma, Zhengqi Gao, Meng Zhang, Haoyu Yang, Mark Ren, Rena Huang, Duane S. Boning, Jiaqi Gu. "MAPS: Multi-Fidelity AI-Augmented Photonic Simulation and Inverse Design Infrastructure". arxiv. 2025-03-04T00:00:00.000Z. URL: https://arxiv.org/abs/2503.01046

[10] Rambod Azimi, Yuri Grinberg, Dan-Xia Xu, Odile Liboiron-Ladouceur. "Gen-Fab: A Variation-Aware Generative Model for Predicting Fabrication Variations in Nanophotonic Devices". arxiv. 2026-03-13T00:00:00.000Z. URL: https://doi.org/10.1007/s00158-026-04272-3

[11] Bolin Chen, Dex Doksoo Lee, Wei "Wayne'' Chen, Wei Chen. "RAG: A Random-Forest-Based Generative Design Framework for Uncertainty-Aware Design of Metamaterials with Complex Functional Response Requirements". arxiv. 2026-01-21T00:00:00.000Z. URL: https://arxiv.org/abs/2601.13233

[12] Muhammad Arif Hakimi Zamrai. "Generative Inverse Design: From Single Point Optimization to a Diverse Design Portfolio via Conditional Variational Autoencoders". arxiv. 2025-10-08T00:00:00.000Z. URL: https://arxiv.org/abs/2510.0516

[13] Michael Bezick, Blake A. Wilson, Vaishnavi Iyer, Yuheng Chen, Vladimir M. Shalaev, Sabre Kais, Alexander V. Kildishev, Alexandra Boltasseva, Brad Lackey. "PearSAN: A Machine Learning Method for Inverse Design using Pearson Correlated Surrogate Annealing". arxiv. 2025-12-30T00:00:00.000Z. URL: https://arxiv.org/abs/2412.19284

[14] Chia-Chun Pan, Yu-Hsi Chen, Munseong Bae, Haejun Chung, Dan Smith, Daniel Fan, Sejeong Kim, Ranjith R Unnithan. "Inverse-Designed Chiral Metasurfaces for Enhanced Helical Dichroism". arxiv. 2026-09-24T00:00:00.000Z. URL: https://doi.org/2609.28

[15] Luka Grbcic, Juliane Müller, Wibe Albert de Jong. "AutoTandemML: Active Learning Enhanced Tandem Neural Networks for Inverse Design Problems". arxiv. 2025-02-24T00:00:00.000Z. URL: https://arxiv.org/abs/2502.15643

[16] Xiang Chen, Huanhuan Xia. "Benchmarking Optimizers to Solve Inverse Problems with Differentiable Physics Simulators". arxiv. 2026-09-15T00:00:00.000Z. URL: https://arxiv.org/abs/2609.13819

[17] Yannik Mahlau, Maximilian Schier, Christoph Reinders, Frederik Schubert, Marco Bügling, Bodo Rosenhahn. "Multi-Agent Reinforcement Learning for Inverse Design in Photonic Integrated Circuits". arxiv. 2025-06-24T00:00:00.000Z. URL: https://arxiv.org/abs/2506.18627

[18] Waleed Waseer, Muhammad Shahid Jabbar, Muhammad Sohail Ibrahim, Shujaat Khan. "Two-Stage Deformable-Convolutional Inverse Design of Nanophotonic Absorbers from Optical Spectra". arxiv. 2026-08-13T00:00:00.000Z. URL: https://arxiv.org/abs/2608.1186

[19] Viktor A. Lilja, Khosro Zangeneh Kamali, Mikael Käll, Philippe Tassin. "Deep learning enables large-scale inverse design of free-form metasurfaces". arxiv. 2026-10-07T00:00:00.000Z. URL: https://arxiv.org/abs/2610.05021

[20] Huanshu Zhang, Lei Kang, Sawyer D. Campbell, Douglas H. Werner. "Chat to Chip: Large Language Model Based Design of Arbitrarily Shaped Metasurfaces". arxiv. 2026-01-28T00:00:00.000Z. URL: https://doi.org/10.1515/nanoph-2025-0343

[21] Shengli Jiang, Jason Wu, Charles M. Schroeder, Michael A. Webb. "Range-Aware Bayesian Optimization for Discovering Diverse Designs within Target Property Windows". arxiv. 2026-06-11T00:00:00.000Z. URL: https://arxiv.org/abs/2606.11574

[22] Pingchuan Ma, Ziang Yin, Qi Jing, Zhengqi Gao, Nicholas Gangi, Boyang Zhang, Tsung-Wei Huang, Zhaoran Huang, Duane S. Boning, Yu Yao, Jiaqi Gu. "SP2RINT: Spatially-Decoupled Physics-Inspired Progressive Inverse Optimization for Scalable, PDE-Constrained Meta-Optical Neural Network Training". arxiv. 2025-05-30T00:00:00.000Z. URL: https://arxiv.org/abs/2505.18377

How this survey was made

This survey answers one question: “How are inverse design methods for structural colour in nanophotonic systems—such as adjoint optimisation, generative models, and surrogate-assisted optimisation—being benchmarked and compared against one another?”

We keep our own collection of recent preprints and refresh it every day from bioRxiv, medRxiv, arXiv, PsyArXiv, SocArXiv and ChemRxiv. For each survey, we search that collection for recent work and keep the strongest matches: papers that use the same terms as the question, and papers on the same topic in different words.

The text was written from the titles and abstracts of those papers; the full texts were not read. Statements that draw on a particular paper are marked with a number in brackets. The reference list just above is the 22 papers whose work this text draws on, chosen from the 111 papers those searches returned; the rest of the shortlist is not printed. It is not a complete review of everything published on the question.

Treat this as a starting point and a reading list, not a peer-reviewed review. The searches behind this survey ran on October 21, 2026 and covered work from the previous two years.