When you were in Japan, I had so much fun drinking with you!
When you were in Japan, I enjoyed our time together!
1) When you were in Japan, I had so much fun drinking with you!
（あなたが日本にいた時、一緒に飲めてとても楽しかった！）
drinking ＝ 飲む。to have a drink というと大体は「お酒を一杯飲む」という意味です。
2) When you were in Japan, I enjoyed our time together!
（あなたが日本にいた時、一緒に過ごせて楽しかった！）
こちらはお酒を飲むという内容よりかは無難に一緒に過ごした時間が楽しかった、という意味です。言いたいことは変わらないと思うのでご参考にどうぞ。
I really enjoyed sharing a drink with you when you were visiting Japan.
I had so much fun drinking with you while you were in Japan.
1) I really enjoyed sharing a drink with you when you were visiting Japan.
「あなたが日本に来ていた時、一緒にお酒を飲めて本当に楽しかった。」
share a drink で「酒を分かち合う」というニュアンスです。
2) I had so much fun drinking with you while you were in Japan.
「あなたが日本にいた間に、一緒にお酒を飲んで本当に楽しかった。」
have fun 動詞のing で「〜するのを楽しむ」という言い方です。
ご参考になれば幸いです！
日本 
