Research led by Alina Yousaf

Writers: Ibrahim Sadat, Hamna Azeem, Shahzadi Fatima, Arhama Javed

‍1.Introduction:

Entropy production is a fundamental concept in thermodynamics, particularly in the study of non-equilibrium systems. It quantifies the irreversibility of Processes and serves as a measure of the energy dissipation within a system as it evolves over time. In equilibrium thermodynamics, entropy is maximized in isolated systems, and processes are reversible. However, most natural and engineered systems operate far from equilibrium, where energy and matter are continuously exchanged with their surroundings. In such systems, entropy Production becomes a Central quantity, providing insights into the efficiency, stability, and evolution of dynamic processes.

Non-equilibrium Thermodynamics extends the principles of classical thermodynamics to systems that are not in thermodynamic equilibrium. It addresses phenomena such as heat conduction, diffusion, chemical reactions, and fluid flow, which are ubiquitous in nature and technology. Entropy Production plays a pivotal role in understanding these phenomena, as it characterizes the rate at which energy is dissipated and the degree to which a system deviates from equilibrium. This article explores the concept of entropy Production, its historical development, and its significance in non-equilibrium Thermodynamics.

The study of nonequilibrium thermodynamics and physical systems often relies on fundamental theoretical frameworks to describe the behavior of complex systems. Among these cornerstone concepts are Onsager's reciprocity relations and variational principles, which provide profound insights into the behavior of natural systems. Onsager's relations, obtained from statistical mechanics, describe the symmetry in transport phenomena, and variational principles are the foundation for understanding optimization and stability of physical systems across mechanics, thermodynamics, and quantum physics. This paper intends to give an in-depth analysis of these two concepts, with mathematical formulations, physical interpretations, and real-world applications.

2.Historical Remarks:

The concept of entropy was first introduced by Rudolf Clausius in the mid-19th century as part of the second law of thermodynamics. Clausius defined entropy as a state function that increases in isolated systems, providing a mathematical formulation for the irreversibility of natural processes. However, the early development of thermodynamics primarily focused on equilibrium states, where entropy is maximized, and processes are reversible.

The study of non-equilibrium thermodynamics began to gain traction in the early 20th century, driven by the need to understand irreversible processes such as heat conduction and diffusion. Lars Onsager made a groundbreaking contribution in 1931 with his formulation of the Onsager reciprocal relations, which describe the linear relationships between thermodynamics forces and fluxes in near-equilibrium systems. These relations laid the foundation for the modern theory of non-equilibrium thermodynamics and highlighted the importance of entropy production in characterizing irreversible processes.

In the 1940s and 1950s, Ilya prigogine further advanced the field by developing the concept of minimum entropy production for steady- state system near equilibrium. Prigogine’s work demonstrated that such systems tend to evolve toward states of minimum entropy production, providing a variational principle for non-equilibrium thermodynamics. His contributions earned him the Nobel Prize in chemistry in 1977 and solidified the role of entropy production as a central quantity in the study of irreversible processes.

The latter half of the 20th century saw the extension of non-equilibrium thermodynamics to far from equilibrium systems, where nonlinear effects and complex dynamics dominate. Researchers such as Nicolis and Prigogine explored the emergence of dissipative structures, self-organizing patterns that arise in open systems driven far from equilibrium. These studies revealed that entropy production is not merely a measure of dissipation but also a driver of complexity and order in non-equilibrium systems.

Today, entropy production remains a key concept in non-equilibrium thermodynamics, with applications ranging from biological systems and climate science science to engineering and materials science. Its role in understanding the efficiency of energy conversion, the stability of dynamic systems, and the emergence of complex structures continues to inspire research and innovation in the field.

3.The Role of Entropy in Non-Equilibrium Thermodynamics:

 The Second Law of Thermodynamics and Irreversibility:

“The second law of thermodynamics states that the entropy of an isolated system never decreases.”

0 ≤ dS

where S is the entropy of the system. In equilibrium, entropy is maximum for a given set of constraints, and there is no further entropy production. In non-equilibrium conditions, entropy production occurs because of irreversible processes such as heat conduction, diffusion, and chemical reactions.

 Entropy Production in Irreversible Thermodynamics:

Non-equilibrium thermodynamics/ irreversible thermodynamics, introduces local entropy balance equations to describe the spatial and temporal variation of entropy. The local entropy production per unit volume is given by:

σ = (1/T) * (∇⋅J),

where "T" is the absolute temperature, "∇" is the gradient operator, and "J" represents the flux of any irreversible process occurring within the system.

Key examples of entropy production mechanisms include:

●       Heat conduction:  where is the heat flux and is temperature.
       

●       Mass diffusion:  where is the mass flux and is the chemical potential.
       

●       Viscous dissipation: where is the viscous stress tensor and is velocity.

Onsager’s Reciprocal Relations:

In the linear regime close to equilibrium, Onsager’s theory of irreversible processes establishes that thermodynamic fluxes and forces are linearly related:

L_ij = L_ji,

where L_ij represents the coefficient coupling flux "j" to force "i" and L_ji represents the coefficient coupling flux "i" to force "j" ; essentially, the "cross-coupling" effects between different processes are equal in both directions.

These relations arise from microscopic time-reversal symmetry and play a important role in transport processes such as electrical conduction, diffusion, and thermoelectric effects.

 The Principle of Minimum Entropy Production:

 For systems in steady-state non-equilibrium conditions, Prigogine’s principle states that entropy production is minimized under certain constraints. This principle applies to linear, near-equilibrium systems and has significant implications in transport processes and self-organizing systems.

Entropy Production and Kinetic Theory:

 The Boltzmann Equation and H-Theorem

The kinetic theory of gases provides a microscopic description of entropy production using the Boltzmann equations. This establishes that entropy always increases, leading to the macroscopic irreversibility of thermodynamic processes in spite of the time-reversible nature of microscopic equations of motion.

 Chapman-Enskog Expansion and Transport Coefficients:

The Chapman-Enskog method allows the derivation of transport coefficients such as viscosity, thermal conductivity, and diffusion coefficients from kinetic theory. Expanding the distribution function in powers of the Knudsen number leads to expressions for entropy production in terms of these transport processes.

Entropy production in kinetic theory manifests through:

●   Collisional entropy generation: Irreversible scattering leads to entropy increase.

● Macroscopic fluxes: Diffusion, viscosity, and heat conduction generate entropy in accordance with thermodynamic laws.
        ‍

Entropy production works on self-organizing systems far from equilibrium. Prigogine’s work on dissipative structures explains how ordered patterns arise in reaction-diffusion systems.

Stochastic Thermodynamics and Fluctuation Theorems

Recent advances in stochastic thermodynamics have refined our understanding of entropy production in small systems. The fluctuation theorem generalizes the second law, showing that entropy production can temporarily fluctuate in small systems but obeys an overall tendency toward increase.

Research into entropy production, its role in non-equilibrium thermodynamics, and its connection to irreversible thermodynamics and kinetic theory has led to many surprising questions. These questions challenge our understanding of irreversibility, statistical mechanics, and the fundamental nature of time and order in the universe.

If the microscopic laws of physics are time-reversible, why does entropy production create an irreversible arrow of time?

Can entropy production be reconciled with deterministic, time-symmetric microscopic dynamics without invoking special starting conditions?

Could there be areas of the universe where entropy decreases, and what would be the physical implications of such "entropy anomalies"?

4. Onsager's Reciprocity Relations:

Historical Background:

Lars Onsager, a Norwegian-American physicist, introduced his reciprocity relations in the 1930s and was awarded the Nobel Prize in Chemistry in 1968. His work formed the basis of modern nonequilibrium thermodynamics, providing a framework to describe how systems respond to external forces when not in equilibrium.

Theoretical Framework:

 Onsager's reciprocity relations are based on the linear response theory in which small disturbances from equilibrium induce proportional responses in thermodynamic systems.

 The general form of Onsager's relations is expressed as:

Where:

‍

‍Onsager's Reciprocity Principle:

Lij=Lji

This symmetry relation implies that the cross-coefficients are equal, provided the system obeys microscopic reversibility (time-reversal symmetry).

Physical Interpretation:

The reciprocity relations show that in coupled processes, the effect of one thermodynamic force on a different flux is the same as the effect of the second force on the first flux. This is not intuitively obvious but is a profound consequence of microscopic time-reversibility.

Example: Thermoelectric Effects

In the Seebeck effect, a temperature gradient induces an electric current.

In the Peltier effect, an electric current induces heat flow.

Onsager's relations predict that the coefficients describing these effects are symmetric .

Applications:

●       Thermodynamics and Transport Phenomena:

1. Heat conduction, diffusion, and viscosity in fluids.
          
2. Coupled transport in electrolytes and semiconductors.

●       Biological Systems:

          1. Ion transport across membranes.
          2. Active transport in cells where energy gradients couple with matter flows.

●       Quantum Systems:

          1.Quantum Hall effect and mesoscopic systems, where reciprocity holds even under quantum mechanical constraints.

●       Modern Technologies:

1. Design of thermoelectric devices for energy conversion.
          
2. Optimization of chemical reactors and separation processes.
   
   

Mathematical Definition

The fluctuation-dissipation theorem relates the response of a system to external perturbations to its natural fluctuations at equilibrium, and through which Onsager's relations are derived. Key steps are:

1. Consideration of the entropy production rate in non-equilibrium systems.
          
2. Linear irreversible thermodynamics as applied to fluxes and forces.
          
3. Principle of microscopic reversibility in transport coefficients, providing symmetry.

5. Variational Principles:

Introduction to Variational Principles:

Variational principles are foundational in physics and mathematics, which provide a robust approach to determine the governing equations of physical systems. In simple terms, the

evolution of the system is supposed to follow an extremal path that minimizes (or extremizes) some quantity, generally known as action or functional.

The general statement of a variational principle can be written in the following way:

‍‍

Principle of Least Action:‍

The best-known variational principle is indeed the Principle of Least Action introduced by Pierre-Louis Maupertuis and perfected by Leonhard Euler and Joseph-Louis Lagrange. "The path followed by a mechanical system from an initial to a final state is that for which the action is minimized."

Mathematically, for classical mechanics:

‍‍

Physical Interpretation:

Variational principles show that nature acts at its best—in minimizing energy, action, or entropy. The approach here provides a unified platform for various physical laws: apparently different systems all follow the same optimization rules.

●       In mechanics, bodies move on least action trajectories.
       

●       In optics, light moves along least time paths (Fermat's principle).
       

●       In thermodynamics, systems proceed towards maximizing entropy.

Applications:

●       Classical Mechanics:

1. Determination of Newton's laws through Lagrangian and Hamiltonian formulations.
          
2. Analysis of more complicated systems-such as pendulums, planetary motion, and rigid bodies.
   
   

●       Quantum Mechanics:

1. The quantum mechanical path integral of Richard Feynman generalizes  the action principle to the quantum regime, where all paths contribute, but the classical path is most probable.     
   

●       General Relativity:

1. Einstein's field equations can be inferred from the Einstein-Hilbert    action as describing the covariant evolution of spacetime curvature under the influence of matter and energy. 
   

●       Engineering and Optimization:

1. Structural analysis, fluid dynamics and theory of control all use variational methods to find optimal configurations.
   

●       Economics and Biology:

1. Optimization of resource allocation, game theory, strategies evolutionary biology; 2.5 Mathematical Derivation Outline
2. ‍
   

Calculus of variations that provides the mathematical basis:

6.Stochastic Thermodynamics and Entropy Production:

Stochastic thermodynamics is a theoretical framework that extends classical thermodynamics to describe systems at the microscopic scale, where fluctuations play a crucial role. This includes systems such as molecular motors, colloidal particles, biomolecules, and small electronic circuits. Unlike macroscopic thermodynamics, which deals with averaged quantities, stochastic thermodynamics focuses on the thermodynamic properties of individual trajectories of a system's evolution.

A key aspect of stochastic thermodynamics is entropy production, which characterises the irreversibility of processes at the microscopic scale. Even in small systems, the second law of thermodynamics holds on average, but individual fluctuations allow for temporary violations, which are described by fluctuation theorems.

Fundamental Concepts of Stochastic Thermodynamics:

 Stochastic Trajectories and System Dynamics:

Microscopic systems are subject to thermal fluctuations, and their evolution is often described by stochastic processes such as:

●       Langevin equation (for systems with continuous states)
       

●       Master equation (for discrete-state systems)


The probability distribution of microstates changes over time, and the system follows a stochastic trajectory in phase space. Unlike macroscopic thermodynamics, where heat, work, and entropy are well-defined for bulk properties, stochastic thermodynamics defines these quantities at the level of individual trajectories.

First Law of Stochastic Thermodynamics:

 The energy balance at the trajectory level is given by the stochastic version of the first law:

dE=dW+dQ

where:

●       dE is the change in internal energy.  
       

●       dW is the work done on the system. 
       

●        dQ  is the heat exchanged with the environment. 

For small systems, these quantities exhibit fluctuations, but their averages conform to traditional thermodynamics.

Second Law and Entropy Production:

 Entropy production quantifies the irreversibility of a process and consists of two contributions:

ΔS_tot​=ΔS_sys​+ΔS_env​

where:

●       ΔS_sys   is the system ​entropy change, defined from the probability distribution over states:

                                           S_sys​=−kB​x∑​p(x)lnp(x)

●       ΔS_env​   is the entropy change of the surrounding heat bath, given by:
                                                

                                                         ΔS_env​=Q/T

where Q is the heat dissipated into the environment at temperature T.

According to the second law, the total entropy production satisfies:

⟨ΔS_tot​⟩≥0

on average, ensuring that the system evolves irreversibly.

6.Entropy Production and Fluctuation Theorems:

 Fluctuation Theorems:

In small systems, entropy production fluctuates due to thermal noise, sometimes appearing to decrease momentarily. However, statistical laws govern these fluctuations.

Crooks Fluctuation Theorem:

The probability of observing a trajectory with entropy production S_tot versus the probability of observing its time-reversed counterpart with entropy production −S_tot is given by:

P(−S_tot​)/P(S_tot​)​=e^Stot​/kB

This relation quantifies how likely spontaneous violations of the second law are in small systems.

Jarzynski Equality:

A crucial result linking equilibrium and non equilibrium thermodynamics is:

⟨e^−W/kB​T⟩=e^{−ΔF/kB​T}

where W is the work done on the system and ΔF is the free energy difference between equilibrium states.

This equality allows free energy differences to be extracted from non equilibrium experiments, such as single-molecule pulling experiments.

_{Entropy Production in Non equilibrium Steady States:}

For systems driven out of equilibrium by external forces, entropy production is a key measure of dissipation. In steady states, the mean entropy production rate is given by:

S_tot​=_x,y∑​p(x)W_x→y​ln p(y)W_y→x​ / p(x)W_x→y​​

where W_x→y represents transition rates between states.

‍

7.Approaches to Entropy Production in Non-equilibrium Systems:

 Linear Irreversible Thermodynamics (Near-Equilibrium Systems):

When a system is close to equilibrium, entropy production can be analysed using Onsager’s reciprocity relations and the linear response framework. This approach works well for small perturbations but fails when the system is far from equilibrium.

Far-From-Equilibrium Systems:

For strongly driven systems, traditional thermodynamic descriptions break down. Some attempts to generalise entropy production in this regime include:

●       Extended Onsager principles
        

●       Macroscopic fluctuation theory
        

●       Large deviation theory

However, a universal framework for entropy production in far-from-equilibrium systems remains an open challenge.

Stochastic Thermodynamics and Information Theory:

The link between entropy production and information processing has become a growing area of research, particularly in the context of:

●       Maxwell’s demon and information engines
        

●       Landauer’s principle (minimum entropy   cost of erasing information)
       

●       Thermodynamics of computation

Information-theoretic entropy measures, such as relative entropy and mutual information, are used to refine our understanding of entropy production in feedback-controlled systems.

8.Open Challenges and Future Directions:

Despite its successes, stochastic thermodynamics still faces unresolved challenges:

1. Entropy Production in Far-From-Equilibrium Systems
   A universal framework beyond local equilibrium assumptions is needed.
2. Role of Quantum Effects
   Quantum fluctuations modify entropy production at small scales.
3. Interplay Between Information and Thermodynamics
   The relationship between entropy, information processing, and feedback control is still being explored.

Stochastic thermodynamics provides a powerful framework for understanding entropy production at the microscopic scale. While near-equilibrium systems are well understood, far-from-equilibrium processes remain a challenge. The connection between entropy production, fluctuation theorems, and information theory opens exciting new directions for future research.

Conclusion:

The primary aim of this paper is to explore the physical significance of entropy production in non-equilibrium processes. Originally introduced by De Donder over eighty years ago, entropy production was adopted as an alternative to Clausius’ concept of non-compensated heat. Setting aside this historical shift in interpretation, a key question arises when dealing with irreversible phenomena: how should they be approached?

For systems that fall within the scope of Linear Irreversible Thermodynamics—where the local equilibrium assumption holds—there is no ambiguity in handling entropy production. However, the challenge emerges when considering systems that operate far from equilibrium. In such cases, not only does the concept of entropy production become unclear, but even the definition of entropy itself lacks precision. This paper examines various attempts to address

these issues, both from a macroscopic perspective and through kinetic theory. The conclusion drawn is that neither approach yields a definitive and universally accepted answer.

Several efforts in the literature have aimed to extend Onsager’s variational principle to tackle these challenges. However, these approaches are generally limited to systems that remain close to equilibrium. An earlier attempt by Biot is also noteworthy, but it has not been applied to real-world, practical situations.

Additionally, this paper critically evaluates the field of Stochastic Thermodynamics, which primarily deals with small systems and short timescales. This framework extends traditional thermodynamic concepts to situations where their validity is questionable. While the case studies presented in the literature are significant in their own right, the way they are linked to conventional thermodynamic quantities remains problematic. For example, a well-established theory rooted in thermodynamics and statistical mechanics was developed over fifty years ago to describe small systems, yet it has been largely overlooked in contemporary research.

Ultimately, a robust theoretical framework for far-from-equilibrium processes remains elusive. The paper argues that such a framework cannot be built upon conventional thermodynamic principles alone.

10.References:

●        Clausius, R. (1865).(On Different Convenient Forms of the Fundamental Equations of the Mechanical Theory of Heat).125: 353–400.

●       Callen, H. B. (1985)."Thermodynamics and an Introduction to Thermostatistics."

●       Prigogine, I. (1947).(Thermodynamic Study of Irreversible Phenomena).

●       Onsager, L. (1931)."Reciprocal Relations in Irreversible Processes. I."

●       Popov, P. & GEORG WOLSCHIN. (2020). BOLTZMANN EQUATION AND H-THEOREM.

●       VALANIS, K. C., HOBOKEN. (1972). Irreversible thermodynamics with internal inertia. Archives of Mechanics, 24(5–6), 849–862. https://rcin.org.pl/Content/203893/WA727_118331_P.262-Valanis-Irreversible.pdf

●       Murakami, T. (1961). Entropy production and variation principle in the kinetic Theory of Rigid-Sphere gas mixture. Journal of the Physical Society of Japan, 16(4), 633–654. https://doi.org/10.1143/jpsj.16.633

●       Semenov, A. M. (2008). IRREVERSIBLE THERMODYNAMICS. In Begellhouse eBooks. https://doi.org/10.1615/atoz.i.irreversible_thermodynamics

●       Libretexts. (2024, May 20). 1.14.37: Irreversible Thermodynamics: Onsager Phenomenological Equations. Chemistry LibreTexts. https://chem.libretexts.org/Bookshelves/Physical_and_Theoretical_Chemistry_Textbook_Maps/Topics_in_Thermodynamics_of_Solutions_and_Liquid_Mixtures/01%3A_Modules/1.24%3A_Misc/1.14.37%3A_Irreversible_Thermodynamics%3A_Onsager_Phenomenological_Equations

●       Onsager, L. (1931). "Reciprocal Relations in Irreversible   Processes I." Physical Review, 37(4), 405-426.
  

●       De Groot, S. R., & Mazur, P. (1962). Non-Equilibrium        Thermodynamics. North-Holland Publishing.
  

●       Feynman, R. P., & Hibbs, A. R. (1965). Quantum Mechanics and Path Integrals. McGraw-Hill.
  

●       Goldstein, H., Poole, C., & Safko, J. (2002). Classical       Mechanics (3rd ed.). Addison-Wesley.
       

●       Landau, L. D., & Lifshitz, E. M. (1980). Statistical Physics,    Part 1 (3rd ed.). Pergamon Press.
       

●       Lanczos, C. (1970). The Variational Principles of Mechanics.        Dover Publications.
       

●       Einstein, A. (1916). "The Foundation of the General Theory of      Relativity." Annalen der Physik, 354(7), 769-822

●        39th IFF Spring School, Institut of  Solid State Research, Research Centre Jülich (2008). “Stochastic    Thermodynamics” University of Stuggart.  
       

●       Michael J. Kewming, Mark T. Mitchison, and Gabriel T. Landi, "Diverging         current fluctuations in critical Kerr resonators", Physical    Review A 106 3, 033707 (2022).
        

●       Gonzalo Manzano, Jordan M. Horwitz and Juan M. R. Parrando. “Quantum fluctuation theorems on arbitrary    environments: Adiabatic and non adiabatic entropy production.” America Physical Society (2018)

●       D. Hahn, M. Dupont, Markus Schmidt, David J. Luitz and Marin Bukov. “Quantum many-body jarzynski equality and dissipative noise on a digital quantum computer.” American Physical Society (2023)
  

●       Sosuke Ito “ Stochastic thermodynamics interpretation of information geometry.” American Physical Society (2018)

‍