The civil service shall further have the power to make investigations concerning all matters touching enforcement and effect of the provisions of this chapter and of the rules of the Civil Service Commission prescribed thereunder; and, in the course of any such investigation or at any trial on appeal, the presiding officer of the Commission shall have the power to subpoena and require the attendance and testimony of witnesses, the power to administer oaths and affirmations and to take testimony relative to any such investigation or hearing, and the power to require the production by subpoena of any books, papers, public records or other documentary evidence pertinent to such investigation, inquiries or hearing.