Abstract
The design of autonomous quantum systems constitutes an entirely new paradigm for next-generation computing by having machine intelligence do the engineering work to build and optimise successful quantum architectures. We address the problem of self-optimizing quantum systems that perform adaptive control, circuit synthesis and decision making in both accurate and uncertain controllable environments via a two-pronged approach: Reinforcement Learning (RL) together with Evolutionary Strategies (ES). In this work, we find that reinforcement learning provides a general framework for sequential decision-making, where agents learn optimal policies from reward signals by interacting with quantum environments. Evolutionary strategies, which are based on ideas from biological evolution, provide an alternative to RL by performing global optimization of quantum circuit parameters and architectures using the standard canonical processes of mutation, selection and recombination
Recent developments in quantum technologies show, again, that machine learning tools (ML), especially RL can be very useful to optimize unavailable closed blow-ups related with quantity functions. Moreover, evolutionary optimization based hybrids with RL have demonstrated better performance on the complex multi-objective quantum problems due to balanced exploitation and exploration. In this paper, we provide an overview of a versatile framework for autonomous quantum system design that showcases example algorithmic architectures, hybrid machine learning models and real world applications in areas such as quantum control, quantum circuit synthesis and energy optimization. In addition, we discuss well-known obstacles such as the scalability problem, the noise sensitivity issue, barren plateaus of quantum optimization and computational complexity.
PDF
Reference
[1] Sutton, R.S. and Barton, A.G. (2018) Reinforcement Learning: An Introduction. 2nd end. MIT Press.
[2] Richard S. Sutton (1988) Learning to predict by the methods of temporal differences. Machine Learning, 3(1), pp.9–44.
[3] Andrew G. Barton (1995) Reinforcement learning in neural networks.
[4] David E. Goldberg (1989) Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley.
[5] John H. Holland (1992) Adaptation in Natural and Artificial Systems. MIT Press.
[6] Kenneth O. Stanley and Miikkulainen, R. (2002) Evolving neural networks through augmenting topologies. Evolutionary Computation.
[7] Thomas Back (1996) Evolutionary Algorithms in Theory and Practice. Oxford University Press.
[8] Nikolaus Hansen (2006) The CMA evolution strategy.
[9] Seth Lloyd (2013) Quantum machine learning. Arrive preprint arrive: 1307.0411.
[10] Jacob Belmonte et al. (2017) Quantum machine learning. Nature, 549, pp.195–202.
[11] Maria Schulz and Petruccione, F. (2018) Supervised Learning with Quantum Computers. Springer.
[12] Michele Carazo et al. (2021) Variational quantum algorithms. Nature Reviews Physics.
[13] Jarrod R. McLean et al. (2018) Barren plateaus in quantum neural network training landscapes. Nature Communications.
[14] Edward Farsi et al. (2014) A quantum approximate optimization algorithm. arrive.
[15] Patrick Rebentrost et al. (2014) Quantum support vector machine for big data classification. Physical Review Letters.
[16] Rotech Havelock et al. (2019) Supervised learning with quantum-enhanced feature spaces. Nature.
[17] Kosice Mitral et al. (2018) Quantum circuit learning. Physical Review A.
[18] Marcello Benedetti et al. (2019) Parameterized quantum circuits as machine learning models. Quantum Science and Technology.
[19] Reskill, J. (2018) Quantum computing in the NISQ era and beyond. Quantum.
[20] Nielsen, M.A. and Chuang, I.L. (2010) Quantum Computation and Quantum Information. Cambridge University Press.
[21] Love Grover (1996) A fast quantum mechanical algorithm for database search.
[22] Peter Short (1994) Algorithms for quantum computation.
[23] Aram Harrow et al. (2009) Quantum algorithm for linear systems.
[24] Daniel Gateman (2010) Quantum error correction.
[25] Sugar Endo et al. (2018) Practical quantum error mitigation.
[26] Kristen Time et al. (2017) Error mitigation for short-depth quantum circuits.
[27] Scott Aaronson (2013) Quantum Computing since Democritus.
[28] John Atreus (2018) The Theory of Quantum Information.
[29] Murphy, K.P. (2012) Machine Learning: A Probabilistic Perspective. MIT Press.
[30] Bishop, C.M. (2006) Pattern Recognition and Machine Learning. Springer.
[31] Good fellow, I., Bagnio, Y. and Carville, A. (2016) Deep Learning. MIT Press.
[32] Yarn Gal (2016) Uncertainty in Deep Learning.
[33] Carl Edward Rasmussen and Williams, C.K.I. (2006) Gaussian Processes for Machine Learning.
[34] Eye Huller Meier and Aegean, W. (2021) Aleatoric and epistemic uncertainty.
[35] IBM Quantum (2025) Quantum computing research.
[36] Google Quantum AI (2024) Quantum AI advancements.
[37] Microsoft Quantum (2025) Quantum development tools.
[38] Regatta Computing (2024) Hybrid quantum systems.
[39] Landau (2024) Photonic quantum computing.
[40] Jürgen Schmidhuber (2015) Deep learning overview.
[41] Volodymyr Minho et al. (2015) Human-level control through deep RL. Nature.
[42] David Silver et al. (2016) Mastering the game of Go with deep neural networks and tree search.
[43] Sergey Levine (2018) Deep RL overview.
[44] Chelsea Finn et al. (2017) Model-agnostic meta-learning.
[45] Died Erik P. Kingman and Welling, M. (2014) Auto-encoding Variational Bayes.