I agree with Michael, you need to check with your doctor. He should have given you instructions regarding this. In my experience, I have always been told to do what's the most comfortable. If I take it out of the sling and it hurts then put it back in. Keep it in if I'm in a crowd of people or need to protect it for whatever reason. I was never given any kind of time limit. It was all subject to my comfort level.
If my surgery involved a repair, like a rotator cuff repair, then it was at least six weeks in the sling, period.