The word “ordinance” and it’s origination:

“…..by historical notation, the Presbyterian Puritans, for example, used the term “ordinance” as much as they used the term “sacrament,” especially in their personal writings. They were used interchangeably. In the WSC the word “ordinance” is used 4 times and the word “sacrament” 10 times. In the WLC, “ordinance” is used 26 times, and “sacrament” 67 times. Michael Harrison wrote “Infant Baptism God’s Ordinance” and John Cotton wrote “Singing of Psalms a Gospel Ordinance”, where Samuel Bolton wrote, “The Guard of the Tree of Life, a Discourse on the Sacraments,” and Ursinus in his commentary on the HC describes both “What are the signs, and what the things signified in the Sacraments” and “the ordinance” of baptism. John Owen uses the term “ordinance” a whopping 742 times in his writings. He uses the term “sacrament” 84 times. Burroughs in “Gospel Worship” uses the term “ordinance” 104 times. He uses the term “sacrament” 109 times. Theologically, they closely mean the same thing, and are both part of explaining elements in the idea of the unbreakable covenant of Christ. The difference lies in the fact that praying, preaching and singing are also ordinances, but baptism and the supper are sacraments. See the difference here:

Sacrament. “The word, having been transferred from military affairs to sacred uses, was employed by ecclesiastical writers to signify any mystery or sacred and not obvious doctrine. Scripture more properly calls them “signs of the covenant” (Gen. 9:12,13; 17:11), “signs and seals” of the righteousness of faith (Rom. 4:11), and simply “signs” (Ex. 12:13), “patterns” (hypodeigmata, Heb. 8:5; 9:23) and “figures” (antitypa, 1 Pet. 3:21).” (Turretin, Institutes of Elenctic Theology (Vol. 3) 338.

Ordinance. I think Thomas Ridgley has a good definition for the nature of an ordinance, “an outward and ordinary means of grace….by which Christ communicates to us the benefits of his mediation.” The Works of Thomas Ridgley (Vol. 4), 41.”

Thanks to C. Matthew McMahon