It won't suffer in that short time, so no need to worry. Refrigeration isn't really appropriate for SLA anyway, and freezing would destroy it. Cold storage is mainly relevant to lithium batteries which age whether used or not, a process which is slowed at low temperatures.
In fact your battery won't be actually empty, just at the level where charge can no longer be issued, a lead acid characteristic.
.