There are hardly any books that I want to read in that library.
There are hardly any books which I want to read in that library.
There are almost no books that I want to read in that library.
はい、この場合は関係代名詞である"that"、または"which"を使うことができます。
"There are hardly any books"＝「ほとんど本が無い」を別の言い方ですると、三番目の文章のように"There are almost no books"というフレーズも使えます。
Another similar example:
"The kind of vegetables that I am looking for cannot be found at this store." （この店では、私が探している種類の野菜は見つかりません）
There are almost zero books at the library that I want to read.
There are very few books at that library that I'm interested in.
この場合、次のような言い方ができますよ。
ーThere are almost zero books at the library that I want to read.
「その図書館には私が読みたい本がほとんど1冊もない」＝「読みたい本がない」
ーThere are very few books at that library that I'm interested in.
「その図書館には私が興味を持つ本がほとんどない」
very few で「ほとんどない」と言えます。
ご参考まで！
アメリカ合衆国 
カナダ 
