That could be the side that you sleep so baby tends to want to face the mother. You can try changing his cot position or sleeping position atleast during the day time.