This page describes a proprietary configuration compliance tool that I built while employed as a Linux system administrator and Linux architect.

Key details
Brief description:An extensible shell-script based tool to detect and correct endpoint configuration discrepancies.
Consumer:Linux system administration team
Impact to consumer:
  • Significantly reduced team workload
  • Improved security posture
  • Reduced risk of human error
Technical features:
  • Simple, shell-based framework that is easily extensible by the team
  • Structured output for re-use in pipelines and for integration with the endpoint management system
  • Unified interface covering multiple Linux variants
  • Minimal overhead with no complicated frameworks or dependencies
Technologies used:Bash

The configuration compliance tool grew out of a server configuration baseline assessment script that I wrote to assist with applying frameworks such as CIS to corporate systems. It is a command-line tool, written in Bash.

Some examples of the checks this tool performs:

The tool has two primary modes: check and correct.

In check mode, the tool simply runs each check function in turn, and reports on the results, with the appropriate exit status.

In correct mode, the tool will run the defined correction functions for each item that failed its check, and report on any failures.

The standardised output means that as well as being run manually, this tool can also be integrated with the endpoint management system. In practice, this is how it is usually used.

The tool is built with cross-platform compatibility as a key feature, providing standardised interfaces for check and correction functions to call in order to report a fault, check whether a package is installed, install or remove a package, check the status of a service, enable or disable a service, or start or stop a service. It has support for CentOS 5 through 7, AlmaLinux 8 and 9, OpenSUSE Leap 15.3 and 15.4, Oracle Enterprise Linux 6, 7, and 9, Red Hat Enterprise Linux 7 and 9, and Debian 7 through 12.

Additional packages are installed depending on the type of endpoint, providing more check and correction functions appropriate to that endpoint type. Also, where appropriate, an operator can configure the tool to ignore specific compliance items on that particular endpoint; the reason for the exception is requested and then automatically recorded.

In practice, it is used to ensure that all endpoints of the same type are configured in the same way regarding security policies, baseline packages, and internal standards such as login banners. It also integrates with the endpoint management system's AV configuration set mechanism to ensure that the selected AV software is correctly installed and configured.

This tool could be thought of as similar to a subset of systems such as Ansible, Chef, SCAP Workbench, etc. It was found that across the whole corporate estate of different Linux endpoint types, attempting to use standard recipes from such tools would require customisation efforts equivalent to or greater than the effort required to maintain this internal tool - with no added benefit, since the customisation would then make them non-standard enough that new team members would need at least as much training on those customisations as they would on using and extending this tool.

A key benefit of this tool's simplicity is that it allows team members to focus on practical system administration skills directly related to the command-line, rather than writing abstract recipes in Ruby, YAML, or XML, and being required to become part of - and maintain - a large ecosystem where the tool itself becomes more important than the systems it governs. With a team of fewer than 10 people managing a heterogeneous estate of over 6,000 endpoints, all team members must be able to spend their time managing those endpoints and not managing the management systems.