We were supposed to be enjoying our time together, but we ended up fighting instead.
この場合、次のような言い方ができますよ。
ーWe were supposed to be enjoying our time together, but we ended up fighting instead. Sorry about that.
「一緒に時間を楽しむはずだったのに、喧嘩になっちゃってごめん」
ここの「楽しい時間」は to enjoy one's time together のように言うと自然で良いかなと思います。
ご参考まで!