Do you mean books you have already brought into your local device cannot be accessed when you are offline?
Then, no. Not normal.
If you have tried to modify the destination for your books from the default, this can happen. The mapping to the new location is only done while online. So if you haven't done that either, then: Not normal.