Spin structure at zero magnetic field and field-induced spin reorientation transitions in a layered organic canted antiferromagnet bordering a superconducting phase

We attempted to assign the spin structure of a layered organic antiferromagnet, $\ensuremath{\kappa}$-(d8-BEDT-${\mathrm{TTF})}_{2}\mathrm{Cu}[\mathrm{N}{(\mathrm{CN})}_{2}]\mathrm{Br}$, which is a key material located closest to the Mott boundary at ambient pressure among this family of compounds, by investigating its macroscopic magnetization thoroughly, motivated by a recent successful assignment of the spin structure of an isostructural material, $\ensuremath{\kappa}$-(BEDT-${\mathrm{TTF})}_{2}\mathrm{Cu}[\mathrm{N}{(\mathrm{CN})}_{2}]\mathrm{Cl}$ [BEDT-TTF and d8-BEDT-TTF are bis(ethylenedithio)tetrathiafulvalene and its deuterated molecule, respectively]. We measured the isothermal magnetization after careful choice of the measurement temperatures and cooling speed at around 80 K, so that the magnetism of the antiferromagnetic phase can be effectively extracted. Consequently, we observed hysteresis loops signifying ferromagnetism and steplike behavior when the magnetic field applied parallel to the crystallographic $b$ and $a$ axes was swept, respectively. The possible spin structure consistent with these results was discussed in terms of probable interactions between the spins, such as exchange interactions and the Dzyaloshinskii-Moriya interaction. Eventually, we asserted that $\ensuremath{\kappa}$-(d8-BEDT-${\mathrm{TTF})}_{2}\mathrm{Cu}[\mathrm{N}{(\mathrm{CN})}_{2}]\mathrm{Br}$ has a spin structure with the easy axis being the $c$ axis and the net canted moment parallel to the $b$ axis, which is surprisingly different from that of $\ensuremath{\kappa}$-(BEDT-${\mathrm{TTF})}_{2}\mathrm{Cu}[\mathrm{N}{(\mathrm{CN})}_{2}]\mathrm{Cl}$. We suggested that this difference originates from the difference of the sign of the interlayer interaction between the two materials. We also elucidated the overall picture of the magnetization processes of this material under the magnetic fields parallel to the three principle axes, which are also in contrast to those of $\ensuremath{\kappa}$-(BEDT-${\mathrm{TTF})}_{2}\mathrm{Cu}[\mathrm{N}{(\mathrm{CN})}_{2}]\mathrm{Cl}$. In particular, the spin-reverse transition at which half of the spins rotate by ${180}^{\ensuremath{\circ}}$ was not induced by the $b$-axis magnetic field, as in the case of $\ensuremath{\kappa}$-(BEDT-${\mathrm{TTF})}_{2}\mathrm{Cu}[\mathrm{N}{(\mathrm{CN})}_{2}]\mathrm{Cl}$, but by the $a$-axis magnetic field. Finally, numerical simulations and magnetic symmetry analysis enabled us to confirm the validity of the spin structures proposed for the two antiferromagnets under zero and high magnetic fields.