Our zoning ordinance has an Article on "Nonconformities".
The following is stated:
Section 13.01 Nonconforming Uses of Land
Where, at the date of adoption of this Ordinance, lawful use of land exists that is made no longer permissible under the terms of this Ordinance as enacted or amended, such use may be continued as long as it remains otherwise lawful, subject to the following provisions:
1) No such nonconforming use shall be enlarged or increased, nor extended to occupy a greater area of land than was occupied at the effective date of adoption or amendment of this Ordinance.
2) No such nonconforming use shall be moved in whole or in part to any other portion of the lot or parcel occupied by such use at the effective date of adoption or amendment of this Ordinance.
3) If any such nonconforming use of land is abandoned for a period of more than ninety days, any subsequent use of such land shall conform to the regulations specified by the Ordinance for the district in which such land is located.
Since your gravel pit was abandoned for five or ten years, it could not be continued per paragraph 3.
This is pretty standard stuff.