A custom was a long-standing way of life or belief of a species. Beginning as a new practice or usage, a custom could, with the passage of many generations, become a tradition. As well, a custom could become a force of law. Custom was a part of culture. (TOS: "The Cage"; TNG: "Half a Life")
The records of the USS Enterprise revealed to the Talosians that the customs and history of Humans showed a unique hatred of captivity. Even when the captivity was pleasant and benevolent, Humans preferred death. (TOS: "The Cage")
A Human custom was "to try and avoid misunderstanding whenever possible." (TOS: "The Corbomite Maneuver")
On Amerind, it was against custom to interrupt a tribal elder at council, even for a medicine chief. (TOS: "The Paradise Syndrome")
In 2364, Ligonian custom made it clear that a visiting leader became an honored guest. (TNG: "Code of Honor")
When Seven of Nine questioned why Captain Kathryn Janeway had required Lieutenant commander Tuvok to accompany her an away mission, she told her that it was not her custom to send an away team of one. Sending at least two crewmembers was, in fact, recommended Starfleet protocol. (VOY: "Hunters")