The Bauschinger effect (BE) of AZ31 magnesium alloy was tested by the experiment of cyclical tension and compression, and the mechanisms of BE were investigated. The test results show that the BE is apparent during tension after compression pre-strain, but in compression after tension pre-strain the reverse Bauschinger effect (RBE) is displayed. Furthermore, the BE is much stronger than the RBE. The microstructure and grain orientation evolvement during cyclical tension and compression show that the cause of the BE is the combination of the re-orientation in pre-compression and the effect of de-twinning during subsequent tension, which make the tension yield limit decrease. Though the grain orientation is not changed, the decrease of c/a ratio after pre-tension makes the twinning more difficult in re-compression, resulting in the RBE.