People are terrified of other people or difficult projects because they tell themselves that they could fail or be rejected. Failure can lead to sorrow, regret, frustration and annoyance - all healthy, negative feelings without which people couldn't exist. But then they add, "I absolutely must succeed and must be loved by significant persons, and if I don't, it's terrible and I'm no good." Those are irrational beliefs. As long as people keep them, they'll be terrified of life and will put themselves down when they get rejected.
Life is indeed difficult, partly because of the real difficulties we must overcome in order to survive, and partly because of our own innate desire to always do better, to overcome new challenges, to self-actualize. Happiness is experienced largely in striving towards a goal, not in having attained things, because our nature is always to want to go on to the next endeavor.
You never truly need what you want. That is the main and thoroughgoing key to serenity. You have only to exist as you do and to live your life as best you can.