Applying unsupervised transformer models to sperm whale click codas and Egyptian fruit bat vocalizations decodes combinatorial phonetic alphabets, rhythm-and-tempo modulation, and contextual syntax in non-human bioacoustics.