Florida is basically a giant dead reef. This means that the shallow bedrock is limestone, which is dominated by the mineral calcite (CaCO3, or calcium carbonate). Acid dissolve calcite, and rain is naturally weakly acid, but rapidly reacts with soil and rock to become neutral or slightly basic (which is what most surface water is). That reaction dissolves base salts like calcite. Eventually, infiltration of surface waters into the near subsurface dissolves the rock into caves and tunnels.
This type of geography is called "karst". Many caves and cavers are developed by karst processes. Mammoth Cave in Kentucky or Carlsbad Caverns in New Mexico are pretty famous examples of this. Florida is riddled with underground caverns and tunnels that formed by dissolution of rock after sea level fell.
Not all caves are in limestone and formed by karst-type dissolution of the rock, but that is probably the most common origin for cave systems. Very common in Pennsylvania and southward along the outer flanks of the Appalachians, for example. Places with shallow limestone bedrock tend to develop karst-related caverns and sinkholes. Florida has a pretty serious problem with it, actually.