Neural processing genes are linked to divergence in visual mate preference in sympatric Heliconius butterflies

Abstract Divergence in mating behaviors plays a major role during speciation, but we know little about the genetic mechanisms underlying the evolution of such traits. Reproductive isolation between the Neotropical butterflies Heliconius melpomene and H. cydno depends on shifts in mimetic warning patterns, which also act as mating cues. Preferences for conspecific warning patterns have a strong genetic component, but unlike the warning pattern cues, the exact genes responsible remain unknown. Here, we reduce the number of candidate genes associated with a large effect QTL for changes in visual mate preference from 200 to just 6. First, we confirm that genomic regions previously associated with male courtship modulate preference behaviors, rather than other traits that may influence courtship time. We then investigate gene expression in the brains of H. melpomene, H. cydno, and their hybrids, across development. We identify two genes, regucalcin2 and an ionotropic glutamate receptor, that are differentially expressed both in species and hybrid comparisons, in a manner consistent with known patterns of dominance. We also find substitutions in the protein-coding regions of 4 genes predicted to affect protein function. These six candidates are located in regions resistant to interspecific gene flow, as expected for genes that contribute to reproductive isolation. Our candidates have been implicated in key components of neural signaling, suggesting shifts in visual mating preference do not rely on primary photoreceptor genes, but instead involve changes in visual integration or processing. Such genetic changes would allow mate preference evolution without altering perception of the wider environment. Significance statement Many species remain separate not because they fail to produce viable hybrids but because they “choose” not to mate. Although these changes in behavior can be hardwired in the genome, we know little about the specific genes involved. Heliconius butterflies display a striking diversity of warning color patterns, which they use as cues to recognize conspecifics. We analyze a genomic region associated with changes in these visual behaviors in two Heliconius species. We couple quantitative trait locus (QTL) studies with gene expression and population genomic analyses to identify candidate genes associated with these visual preferences. This is an important step towards understanding how behavioral differences, crucial to speciation, are generated both during development and across evolutionary time.