Generalized spectral characterization of mixed graphs.

A mixed graph $G$ is a graph obtained from a simple undirected graph by orientating a subset of edges. $G$ is self-converse if it is isomorphic to the graph obtained from $G$ by reversing each directed edge. For two mixed graphs $G$ and $H$ with Hermitian adjacency matrices $A(G)$ and $A(H)$, we say $G$ is $\mathbb{R}$\emph{-cospectral} to $H$ if, for any $y\in \mathbb{R}$, $yJ-A(G)$ and $yJ-A(H)$ have the same spectrum, where $J$ is the all-one matrix. A self-converse mixed graph $G$ is said to be determined by its generalized spectrum, if any self-converse mixed graph that is $R$-cospectral with $G$ is isomorphic to $G$. Let $G$ be a self-converse mixed graph of order $n$ such that $2^{-\lfloor n/2\rfloor}\det W$ (which is always a real or pure imaginary Gaussian integer) is square-free in $\mathbb{Z}[i]$, where $W=[e,Ae,\ldots,A^{n-1}e]$, $A=A(G)$ and $e$ is the all-one vector. We prove that, for any self-converse mixed graph $H$ that is $\mathbb{R}$-cospectral to $G$, there exists a Gaussian rational unitary matrix $U$ such that $Ue=e$, $U^*A(G)U=A(H)$ and $(1+i)U$ is a Gaussian integral matrix. In particular, if $G$ is an ordinary graph (viewed as a mixed graph) satisfying the above condition, then any self-converse mixed graph $H$ that is $\mathbb{R}$-cospectral to $G$ is $G$ itself (in the sense of isomorphism). This strengthens a recent result of the first author.