Preface to the Second Edition#
In the last few years, the field of tensor network method has undergone a very fast development. Since the publication of the first edition of this book, there have been several exciting developments in different directions. At the same time, I moved from Ulm University back to Italy, where I am now heading the Quantum Information and Matter group at the Department of Physics and Astronomy of Padova University. The research group has grown substantially which allowed me to explore different research directions, most of them devoted to the development and exploitation of new features of tensor network methods. Meanwhile, the Italian part of the EU Recovery Plan—NextGenerationEU has impacted my research as well, as it funded the Italian National Center for High-Performance Computing, Big Data, and Quantum Computing (Foundation ICSC), which has boosted the Italian activities on quantum computation, has increased the interaction and cohesion between universities, national research centers and industries in the field of quantum computing, and, as far as concern us, has greatly tightened our interactions with the CINECA, the Italian HPC center. At the same time, in 2022, Padova University funded via an open call a Quantum Computing and Simulation Center (QCSC) which I have the honor to direct. The QCSC was co-funded by several Quantum Centers and universities in Italy, together with the Italian National Institute for Nuclear Physics (INFN), and develops coordinated activities on quantum information among those institutions.
This thriving environment put us in the conditions to explore some of the exciting developments of tensor network methods, especially concerning future quantum computations and simulations. Indeed, tensor network methods are becoming more and more relevant as tools to benchmark, verify, and guide the development of future quantum computers and simulators. On the one hand, tensor network quantum computing emulations are becoming the ultimate quantum advantage certification tools. The ability of tensor network emulators to simulate quantum computations with hundreds of moderately entangled qubits makes them the perfect tool to discern where the quantum computing advantage threshold lays. In the last years, starting from the first claim of Google’s quantum supremacy experiment in 2018— right after the publication of the first edition of this book—we have seen an increasing number of experiments claiming to have reached quantum advantage that have been seriously challenged by tensor network emulations. This friendly competition is currently ongoing and has surely contributed to improving our profound understanding of the limitations and the potential of current quantum chips and quantum computations.
Alongside, tensor network methods have kept attacking the most serious challenge they have been facing for a couple of decades, that is, their effectiveness in describing many-body quantum systems with dimensions higher than one. Indeed, since the introduction of the Density Matrix Renormalization Group, tensor network methods have been recognized as the method of choice for describing one-dimensional systems in regimes where Monte Carlo methods are hindered by the sign problem, i.e., described by a negative or imaginary action. However, for higher dimensional systems, the situation is different. On the one side, there are many rigorous and important theoretical results related to tensor network ansatzes for high-dimensional systems, like as PEPS and MERA. On the other side, their high algorithmic complexity prevented their application in many practical uses, and their adoption is not yet as ubiquitous as methods for one-dimensional systems, i.e., based on Matrix Product States.
For many years, the difficulties of tensor network methods in studying highdimensional systems have prevented their application in a large number of fundamental problems, such as the study of lattice gauge theories at finite chemical potential or out of equilibrium. One of the exciting developments of the last years has been a combination of new developments that are starting to unlock the possibility of studying them. Indeed, a combination of algorithmic developments, new tensor network ansatzes (e.g., the augmented Tree Tensor Networks), and computational improvements (the use of modern programming languages, of GPUs, and HPC environments, etc.) are such that simulations of 2D and even 3D dimensional systems are becoming available and start to produce interesting results in different fields, from condensed matter theory to high-energy physics. Once again, the results that will be obtained in the years to come will guide future research in different fields and will serve as future highly nontrivial benchmarks for future quantum computers and simulators that will eventually realize Feynman’s vision.
On a parallel direction, tensor network methods are finding increasing direct applications in computer science problems. Indeed, since the seminal work of Stoudenmire and Schwab in 2016, it is becoming increasingly evident that the interplay of tensor network methods and machine learning can lead to very interesting developments, possibly opening up new routes toward the solution of some longstanding limitations of neural networks, from their explainability to their energy and computational extreme needs. Moreover, tensor network machine learning is a sub-class of quantum machine learning, that is, it is a perfect bridge between quantum and classical machine learning, and as such, it is the perfect playground to study their interplay and properties such as scalability, resilience to errors, dependence on the entanglement of the trial function, different architectures, and so on. Similarly, thanks to the new developments and the new ability to efficiently simulate high-dimensional systems, tensor network methods can be used to attack hard combinatorial classical optimizations, one of the main attractive real-world applications of quantum computers for industries. Once again, they can be used to develop new classical optimizers that might become competitive and relevant for industrial applications, and naturally bridge the quantum and classical optimizers, providing a perfect playground to study hard instances, their scalability to thousands of variables, the role of entanglement, and various strategies and architectures.
The aforementioned developments urged us to update the previous version of the book, which now includes some of the latest developments in the field that we consider promising for the future. We have added new chapters to introduce the new concepts and one part of the book with more advanced technical concepts. The new Part III of the book contains all the technical details to prepare the interested reader to understand and code any tensor network algorithm in a loopless structure, that is, in a Tree Tensor Network (TTN) ansatz. This ansatz is shown to be powerful enough to describe highly relevant physics in dimensions higher than one, with very high efficiency. Moreover, whenever necessary, it is possible to extend this ansatz via the augmented TTN (aTTN) that encodes the area law in higher dimensions keeping the algorithmic complexity of the overall computation to the minimum, the bond dimension to the fourth power. This improvement paves the way for many exciting applications, some of which are described in the new chapters in Part IV of the book: Hamiltonian Lattice Gauge theories, both Abelian and Non-Abelian, described in detail in the new Chap. 10. Finally, in the new Chap. 12, the reader is introduced to Tensor Network Machine learning a completely new field of application of Tensor Network Methods that has been recently unveiled.
Most of the book’s new content comes from the Ph.D. theses of two brilliant Ph.D. students who, under my supervision, have developed new tensor network methods and applied them to highly challenging problems: Timo Felser and Giovanni Cataldi have just entered this fascinating world and have already demonstrated their value.
Most of the ideas and applications presented in this book have been explored, coded, and applied by members of my group in the last 20 years. The result of this accumulated knowledge is now available as the open-source software suite QuantumTEA1 for anybody who wants to run or further develop it. It is also preinstalled on Leonardo, the CINECA supercomputer for ease of use. This could not be possible without the hard daily work and support of Daniel Jaschke who is leading this effort.
The research group I have the pleasure of coordinating is made up of amazing people, from which I get the stimuli to keep going every day, learning something new, and getting excited by new developments. They are too many to be thanked here, but the group could not smoothly work without the constant efforts of Marco Di Liberto, Carmelo Mordini, Ilaria Siloi, and Pietro Silvi.
Finally, I would like to thank the Physics and Astronomy Department of Padova University, which supported me from the first day I arrived: the past and present directors Francesca Soramel and Flavio Seno, all my colleagues, and the administrative and technical staff for the very stimulating, cooperative, and open environment they contributed to creating. It is impressive how solutions to practical problems can be always found if working together toward a common goal.
Padova, Italy
December 2024
Simone Montangero
After Simone’s overview of the progress in the field of tensor networks following the first edition, let me give some background to my journey with tensor network methods, which ultimately concluded several contributions in this new edition.
The implementation and development of tensor networks for solving cuttingedge problems in research have become an interdisciplinary field combining elements from mathematics, such as linear algebra and graph theory, with highperformance computing and Quantum Information. I started my journey of tensor network developments back in 2015 in Simone’s group—at the time at the University of Ulm. After attending his lecture on quantum computational physics myself as a physics student, I was hooked by the passion for simulating quantum systems on a classical computer—especially since my interest in numerical simulations existed a long time before: Starting in the motorsports team of the Einstein Automotive e.V. which builds a competitive race car each year, I had the pleasure to dive deep into numerical simulations ranging from solid-state simulations of car components and their structural optimization up to computational fluid dynamics (CFD)-simulations for optimizing the aerodynamic parts—all areas where tensor computation form the mathematical foundation.
My first encounter with tensor networks was the implementation of non-Abelian symmetries and a time evolution (concretely a TEBD) algorithm for Matrix-Product States—the arguably easiest, non-trivial geometry of a tensor network. The exploitation of symmetries—as illustrated in Chap. 6—has become a crucial, state-of-the-art add-on for many tensor network algorithms nowadays deployed to investigate quantum many-body systems. It reduces the numerical resources needed and thereby helps to push forward the limitations of simulating quantum systems on classical computers via Tensor Networks.
During my PhD, I had the great pleasure of getting a set of challenging problems to solve under the supervision of Simone. The target was to develop a tensor network algorithm for higher systems’ dimensions, i.e., 2D and 3D, that is efficiently contractable and encodes the area law by construction. Several tensor network geometries have been established over the last decades, the most prominent of which being the Matrix Product states (MPS), the Tree Tensor Networks (TTN), the Multiscale Entanglement Renormalization Ansatz (MERA), and the Projected Entangled Pair States (PEPS). However, when going to higher system dimensions, the algorithms either do not encode the area law—and thus eventually fail to accurately capture the underlying entanglement with increasing system size failing to faithfully describe the quantum states’ properties—or are numerically inefficient—up to the point where the computation of the quantum state or certain properties becomes practically infeasible. This conflict limited the scalability of tensor network algorithms in simulating higher dimensional quantum systems for a long time. After several years of development, we introduced the augmented Tree Tensor Network (aTTN) as a novel approach to simulating highdimensional systems which is introduced in the new Chap. 9. Along the way, we applied TTN methods to problems from various fields ranging from low-energy to high-energy to medical physics. These applications include the simulation of Lattice Gauge Theories in higher dimensions and the applications of tensor networks to optimizing radiotherapy plans in cancer treatments as an alternative optimization approach to simulated or quantum annealing. Based on these research endeavors, the new Chap. 8 provides an in-depth overview of the implementation TTN algorithms for any dimensionality of the underlying systems and includes an HPC perspective to boost their performance.
During that time, the tensor network community grew quite large with different novel application areas arising. For instance, in quantum gravity, MERA has been linked to the geometry of space via the Anti-de Sitter (AdS)/Conformal Field Theory (CFT) correspondence which in return has been brought in context to research of black holes by, to some extent, resolving the black hole information paradox. Furthermore, tensor networks have become increasingly of interest to the applied math community. In this community, the idea of Matrix Product States was rediscovered as Tensor Trains and Tree Tensor Networks have been proposed as Hirarchical Tucker Trees, thus highlighting the potential of tensor networks as a general tool for information processing. This potential of tensor networks was further proven by its application to Machine Learning tasks as a novel competitor to state-of-the-art neural networks. The first approaches of machine learning with tensor networks already yielded comparable results when performing supervised learning on standardized datasets. During my PhD time, I had the pleasure of getting into contact with Miles Stoudenmire, one of the first to publish together with D. Schwab the concept of tensor network machine learning using MPS. In this context, I transferred the concept to general loopless-TN geometries, which later on provided the basis for one of the first applications of tensor network machine learning to real data from the LHCb hardron collider at CERN. In the new Chap. 12, we introduce the concept of tensor network machine learning, giving technical insights into the great potential of the method for information processing.
After my PhD, I headed a research transfer project to spin off the method of tensor network machine learning into the company Tensor AI Solutions GmbH, which applies it to industry applications across various sectors. It turns out that tensor networks are a great tool for addressing current challenges in the lack of transparency of AI systems, providing an interpretable AI alternative to neural networks. We applied them as an explainable AI to projects ranging from credit scoring, where the learned patterns and the decisions made have to be accessible, over autonomous driving, where tensor networks are more robust against adversarial attacks or data poisoning compared to neural networks, up to real-time process optimization, where the ability to compress the network post-learning is a great benefit to adapt the performance at low cost to the requirements of the hardware. Furthermore, it has been shown that tensor networks as a numerical tool can address combinatorial optimization problems with the potential to improve areas like medical treatment, logistics optimization or energy management. Nevertheless, since in the last years, quantum technologies have emerged at the brink of transferring from research to industry, a large amount of investments have been made all over the world to foster this technology. In this light, tensor networks have emerged as the main working horse for simulating quantum circuits, i.e., as quantum simulators, provide challenging benchmarks for the development of quantum computers and are about to empower the field of quantum machine learning. All of these emerging application areas and the perspective impact of tensor networks in research and industry highlight the significance of this method in the future.
During my tensor network journey, I had the great pleasure of working with outstanding people who had an important impact on me and with whom I share valuable memories. Therefore, I would like to thank all of them for putting and keeping me on this journey with the common passion we shared. These include among others F. Tschirsich, M. Gerster, P.Silvi, F.Schrodi, G. Bishop, L. Kohn with whom I spent most of my working time at the University of Ulm; A. Daley as early inspiration with discussions on TEBD algorithms giving me the confidence to proceed with research in this direction; S. Notarnicola, M. Collura, G. Magnifico, with whom I had the pleasure to work at the University of Padova and enjoyed countless of inspiring dinner discussions; G. Morigi who among others made my PhD research as co-tutelle between University of Saarland and the University of Padova possible; M. Stoundenmire who inspired me to go in the direction of tensor network machine learning and supported me with a lot of inspiring discussions; F. Schinnerling, N. Rach, M. Trenti with whom I worked extensively together in our spin-off project and J. Ankerhold who supported the project at the University of Ulm. Last but not least, I have to thank my co-author S. Montangero who put his trust in me when I started my tensor network journey and who accompanied me throughout my tensor network path as a mentor, a stimulating research partner, and a trusted friend. Without his impact, my journey would certainly not have been the same.
Ulm, Germany December 2024
Timo Felser