I don't want to have my Visit Count shown to others.
You can adjust the privacy setting for Visit Count.
 
Click [Settings] in the top right corner of  the visitor graph or go to [Settings > MyStory Privacy Settings > Visit Count]. You can now choose from [Public/Friends/Only Me].