No. There is no y anywhere in it, and no y sound either. The temptation comes from the neighbouring vowel: English has a small set of words where a bare u after a consonant carries a hidden y, as in the second half of accumulate. Kumquat is not one of them, and the fruit name has never had a y in any of its English spellings.