Hi, I used to have old email being pushed twice, which is really annoying..
I used to use Delete Prior from my handset, but when I deleting my email manually, such problem never occurs to me again (it's been more than a month). Selecting all your email manually and deleting them is quite a task if you got hundreds of them, but it's better than having the old one pushed back into your device again
It happens on my Gmail account, never on Yahoo.. so perhaps the problem lies with Gmail.