Perhaps you are backing away mate ?
I used to have a similar problem until I developed a back and across movement.
I took gaurd at off stump and if the ball was out side off I would move backwards and push across before deciding to play or leave it helped alot especially with the out swing bowlers as I became a better judge of a leave and left most of the out swingers alone.