It was never audible, which is why it goes missing. Break the word into its parts and it reappears: elae is the olive, agnus is the chaste tree, and the a at the front of agnus belongs to the second word. Slide the two together and that a lands immediately after the ae with nothing to separate them. Elaegnus keeps the digraph and loses the joint.