Typically salt domes are used for storage. At high enough pressure to be usable the salt melts. Rocks and steel would eventually, too. If you extract the heat you are subject to the Carnot limits to efficiency. Batteries avoid this.
Ok, now I feel dirty. Using 'Carnot' in a general discussion feels wrong.