I disagree, lay it to rest and people will keep thinking they can get away with it. He should without a doubt in my mind receive a 3 match ban for that. Violent conduct would empower a 3 match ban, IMO that is worse than violent conduct.
The referee saw it and he got booked. If the FA did that, it'd be an absolute joke after the Rooney elbow.