If your bird acts fearful toward you, take him out with you (in his cage, of course). In unfamiliar territory, a bird's attitude toward familiar humans can change. Sometimes just taking your bird for a ride generates noticeable improvement - even if your bird never leaves his carrier. Your bird will enjoy new, enhanced experiences and - when he comes home safely - his sense of security will improve.
The development of a confident relationship with even one human can be a breakthrough for a fearful bird because he might then develop more confidence in all areas of behavior.
For more information, read Fearful Parrot.