An improvement is proposed for the removal of the precipitates of a phosphine oxide compound as a by-product from a mixture thereof with an olefin compound obtained by the Wittig reaction for the preparation of the olefin compound from a phosphorane compound and a carbonyl compound. Instead of the conventional procedure of filtration to remove the precipitates, the mixture in a hydrocarbon solvent, after replacement of a non-hydrocarbon solvent with a hydrocarbon solvent, if necessary, is admixed with a lower carboxylic acid such as acetic acid so that the precipitates of the phosphine oxide are dissolved forming a viscous fluid which is immiscible with the mixture of the olefin compound and the hydrocarbon solvent. The fluid containing the dissolved phosphine oxide compound can be easily removed by phase separation so that a great increase is obtained in the yield of the desired olefin compound.