I frankly have little insightful to say. I'm going to have to lean on your to think up something cool about having a Dexterity of 3. For many of the same reasons that Strength could be a 3, so could Dexterity: Overweight, athletically unpracticed, feeble, clumsy, and blind or nearly-blind.
But it could also be due to Parkinson's Disease or Syndrome, or ALS, or polio. Some muscular-skeletal-nervous system disorder that is nonlethal, and for some reason you man was able to live through childhood despite his disability. While most people in that situation would be lame beggars, your man has decided to seek his fortune in some way. Because of his penalty wit missile weapons and a worse AC, he will be at a disadvantage. But because the penalties are small he will not be at a huge disadvantage like he would be in a new school game.
Can you think of other reasons why your man might have a Dex of 3?