They ended up putting the blame on me for bringing the mood down.
In the end, they blamed me for bringing the mood down.
「shift the responsibility」でもいいんですが、ネガティブな感じがするので、「responsibility」より「blame」の方がいいです。
盛り下げる = bring the mood down
盛り上げる = bring the mood up, warm up, bring excitement, get everyone excited