私がその店に着いたときまだ4人しかいなかったって英語でなんて言うの？

この場合、次のような言い方ができますよ。

ーWhen I arrived at the restaurant, there were only four people there.
「レストランに着いた時、まだ４人しかいなかった」
there were only four people there を使って言うと自然です。

ーWhen I got to the restaurant, only four other people had arrived.
「レストランに着いた時、まだ４人しか来ていなかった」
only four other people had arrived のように過去完了を使っても表現できます。

ご参考まで！

"私がその店に着いたときまだ4人しかいなかった"
- When I got there, there were only four people there.
- When I arrived at the restaurant there were only four people there.

「そこには4人しかまだいなかった」
始まる時間に着いた場合：
- There were only four people there
- Only four people had arrived.
- Only four others had arrived.
- There were only four other people there.
もう始まったときから少し時間がたった、それかパーティの最後の方に着いた場合：
- There were still four people left.
- There were only four people remaining.
- Only four others were still there.
- Only four people remained.