I felt bad when I said No to him, because he tried to back off silently, so I decided to give it to him.
I felt bad:悪いと思った
when I said No to him:彼にNoと行ったとき
because he tried to back off silently:静かに引き下がったから
so I decided to give it to him.:だからあげることにした
という意味になります。