Entropy, cross-entropy, relative entropy: Deformation theory (a)

Attempts at generalizing Shannon entropy and Kullback-Leibler divergence (relative entropy) led to a plenthora of deformation models in theoretical physics, including q -model, κ -model, etc. Naudts and Zhang (Inf. Geom. , 1 (2018) 79) established that these models can be unified under two notions: deformed ϕ -exponential family (Naudts, J., J. Inequal. Pure Appl. Math. , 5 (2004) 102) and conjugate -embedding (Zhang J., Neural Comput. , 16 (2004) 159) of probability functions. Conjugate -embedding has a gauge freedom which, upon its fixing, subsumes the U -model of Eguchi (Sugaku Expositions , 19 (2006) 197) proposed in a statistical machine learning context. The generalization by -entropy, -cross-entropy, -divergence, when applied to the ϕ -exponential family, yields either a Hessian structure or a conformal Hessian structure under different gauge selections —this “splitting” is the hallmark when deforming the exponential family with its dually flat (Hessian) geometry. This letter provides a unified information geometric perspective of deformation of the exponential model, with calculations for Tsallis q -model.