In addition to the penalty affixed by express terms, to every neglect or violation of official duty on the part of public officers, State, county, city, or township, where it is not so expressly provided, they may, in the discretion of the Court, be removed from office.