Mike Geppert 6ddbfb93fe Update guidelines.md with Approaches Considered and Lessons Learned section

2025-07-20 23:35:05 -05:00

21 KiB

Raw Blame History

SSL Management Project Guidelines

This document provides guidelines for developing and maintaining the SSL Management project, which is a Python-based tool for managing SSL certificates for UniFi devices such as UDM-SE. The tool helps automate the process of obtaining, validating, and deploying SSL certificates to UniFi devices.

Build/Configuration Instructions

Prerequisites

Python 3.6+: The project is written in Python and requires Python 3.6 or higher.
OpenSSL: Required for certificate operations. Must be installed and available in your PATH.
Virtual Environment: The project uses a virtual environment for dependency isolation.
Linux Environment: The script is designed to run on Linux systems such as Ubuntu 24.04. While development can occur on any platform, deployment is expected on a Linux server.

Setup

Clone the repository:

git clone <repository-url>
cd ssl-managment

Create and activate a virtual environment:

python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

No external Python packages are required as the project uses standard library modules.

Configuration

The SSL Manager uses a configuration file (config.json) to store default settings. The configuration file should be placed in the project root directory. Here's an example of the configuration file:

{
  "cert_dir": "~/.ssl-certs",
  "default_port": 443,
  "connection_timeout": 3.0,
  "default_validity_days": 365,
  "key_size": 2048,
  "debug": false,
  "unifi": {
    "host": "unifi.example.com",
    "username": "admin",
    "password": "password",
    "site": "default",
    "ssh_port": 22,
    "ssh_username": "root",
    "ssh_password": "",
    "ssh_key_path": "~/.ssh/id_rsa"
  },
  "letsencrypt": {
    "email": "admin@example.com",
    "validation_method": "standalone",
    "use_staging": false,
    "agree_tos": true
  }
}

Configuration options:

cert_dir: Directory where certificates and keys will be stored
default_port: Default port to use when checking certificate expiration
connection_timeout: Timeout in seconds for SSL connections
default_validity_days: Default validity period in days for generated certificates
key_size: Key size in bits for generated certificates
debug: Enable debug logging with line numbers and file names (default: false)
unifi: UniFi device connection parameters
- host: Hostname or IP address of the UniFi device
- username: Username for authenticating with the UniFi device
- password: Password for authenticating with the UniFi device
- site: Site name for the UniFi device (default: 'default')
- ssh_port: SSH port for the UniFi device (default: 22)
- ssh_username: Username for SSH authentication with the UniFi device
- ssh_password: Password for SSH authentication (leave empty to use SSH key)
- ssh_key_path: Path to the SSH private key file for authentication
letsencrypt: Let's Encrypt certificate settings
- email: Email address for Let's Encrypt registration and important notifications
- validation_method: Method to use for domain validation (standalone, webroot, dns)
- use_staging: Whether to use Let's Encrypt's staging environment for testing (true/false)
- agree_tos: Whether to automatically agree to the Terms of Service (true/false)

Usage

The SSL Manager provides three main commands. All commands support the following global options:

--config: Path to the config file (default: config.json)
--cert-dir: Directory to store certificates (overrides config)
--debug: Enable debug logging with line numbers and file names

Check Certificate Expiration:
```
python src/ssl_manager.py check example.com [--port PORT]
```
The --port option overrides the default_port from the config file.
Generate Certificate:
```
python src/ssl_manager.py generate [COMMON_NAME] [--type TYPE] [--days DAYS] [--email EMAIL] [--validation-method METHOD] [--staging|--production]
```
The COMMON_NAME parameter is optional. If not provided, the UniFi host from the config file will be used. This ensures that the certificate is valid for the UniFi device.

Options:
- --type: Type of certificate to generate (self-signed or letsencrypt, default: letsencrypt)
- --days: Days valid (overrides config, only for self-signed certificates)
- --email: Email address for Let's Encrypt registration (overrides config)
- --validation-method: Method to use for domain validation (standalone, webroot, dns)
- --staging: Use Let's Encrypt's staging environment (for testing)
- --production: Use Let's Encrypt's production environment

Validate Certificate Chain:

python src/ssl_manager.py validate path/to/certificate.crt [--ca-path path/to/ca.crt]

Let's Encrypt Validation Methods

When generating certificates with Let's Encrypt, you need to prove that you control the domain. The SSL Manager supports three validation methods:

Standalone (--validation-method standalone):
- Starts a temporary web server to respond to Let's Encrypt's validation requests
- By default, uses port 80, which requires root privileges
- Can use a non-privileged port (e.g., 8080) with the --http-port option or http_port in config.json
- Requires the specified port to be available and accessible from the internet
- Best for servers where you don't have a web server running
- Requires the hostname to be in public DNS with an A/AAAA record pointing to your server
Webroot (--validation-method webroot):
- Uses an existing web server to serve validation files
- Requires write access to the web server's document root (default: /var/www/html)
- Best for servers with an existing web server
- Requires the hostname to be in public DNS with an A/AAAA record pointing to your server
DNS (--validation-method dns):
- Uses DNS TXT records for validation
- Requires manual intervention to add DNS records
- Best for validating wildcard certificates or when port 80 is not accessible
- Requires the hostname to be in public DNS where you can add TXT records

By default, the SSL Manager uses Let's Encrypt's production environment, which issues trusted certificates. For testing purposes, use the --staging flag to use Let's Encrypt's staging environment, which has higher rate limits but issues untrusted certificates. Once you've confirmed everything works with the staging environment, you can remove the --staging flag to use the production environment.

Public DNS Requirements

Yes, the hostname must be in a public DNS for Let's Encrypt certificates. Let's Encrypt needs to verify that you control the domain before issuing a certificate. The SSL Manager automatically checks if the hostname is in public DNS before attempting to generate a Let's Encrypt certificate and stops with an error if it's not.

This verification process requires:

For standalone and webroot validation methods:
- The domain must have a public DNS record (A or AAAA) pointing to your server
- Your server must be publicly accessible on port 80
- Let's Encrypt servers must be able to reach your server over the internet
For DNS validation method:
- The domain must have public DNS records where you can add TXT records
- You don't need a publicly accessible server, but you need control over the domain's DNS records

Alternative Approaches for Private Networks

If you're using the SSL Manager in a private network where the hostname isn't in public DNS, consider these alternatives:

Self-signed certificates: Use --type self-signed for internal use only (browsers will show warnings)
Private CA: Set up your own Certificate Authority for your internal network
Split DNS: Configure your DNS to resolve the domain internally while also having it in public DNS
Domain with DNS API: Use a domain you control with DNS API support for automated DNS validation

Certificate Verification

The SSL Manager automatically verifies the current certificate for the UniFi host after initialization. When you run any command, the SSL Manager will:

Check if a certificate file exists for the UniFi host in the certificate directory
If it exists, validate it using OpenSSL
Display the verification status

Example output:

Certificate for unifi.example.com:
  Status: Valid
  Path: /home/user/.ssl-certs/unifi.example.com.crt
  Message: Certificate for unifi.example.com is valid

Possible status values:

Valid: The certificate exists and is valid
Invalid: The certificate exists but is invalid (e.g., expired, self-signed, or not trusted)
Missing: No certificate file was found for the UniFi host
Not configured: No UniFi host is configured in the config file

Host and Certificate Validity

For a certificate to be valid for a UniFi device, the Common Name (CN) in the certificate must match the hostname of the device. This is why the SSL Manager uses the UniFi host from the config file as the default common_name when generating certificates.

When you access your UniFi device through a web browser, the browser checks that the hostname in the URL matches the Common Name in the certificate. If they don't match, the browser will display a security warning.

For example:

If your UniFi device is accessed at https://udm-se.example.com
The Common Name in the certificate should be udm-se.example.com

By configuring the host field in the config file and using it as the default common_name, the SSL Manager ensures that the generated certificate will be valid for your UniFi device.

Testing Information

Running Tests

Tests are written using the Python unittest framework. To run all tests:

cd ssl-managment  # Ensure you're in the project root
python -m unittest discover tests

To run a specific test file:

python -m tests.test_ssl_manager

Adding New Tests

Create a new test file in the tests directory, following the naming convention test_*.py.

Import the necessary modules and the SSLManager class:

import os
import sys
import unittest
from unittest.mock import patch

# Add the src directory to the Python path
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../src')))

from ssl_manager import SSLManager

Create a test class that inherits from unittest.TestCase:

class TestYourFeature(unittest.TestCase):
    def setUp(self):
        # Set up test fixtures
        self.ssl_manager = SSLManager()

    def test_your_feature(self):
        # Test your feature
        result = self.ssl_manager.your_method()
        self.assertEqual(result, expected_value)

Use mocking to avoid actual network or system calls:

@patch('ssl_manager.subprocess.run')
def test_with_mock(self, mock_run):
    mock_run.return_value = MagicMock(returncode=0)
    # Test with mock

Test Example

Here's a simple test that verifies the SSLManager initializes correctly:

def test_init_creates_cert_dir(self):
    """Test that the constructor creates the certificate directory."""
    test_path = os.path.expanduser("~/test-ssl-certs")
    
    # Remove the directory if it exists
    if os.path.exists(test_path):
        os.rmdir(test_path)
    
    # Create a new SSLManager with the test path
    ssl_manager = SSLManager(cert_dir=test_path)
    
    # Verify the directory was created
    self.assertTrue(os.path.exists(test_path))
    self.assertTrue(os.path.isdir(test_path))
    
    # Clean up
    os.rmdir(test_path)

Development Guidelines

Code Style

PEP 8: Follow the PEP 8 style guide for Python code.
Docstrings: Use docstrings for all modules, classes, and functions. Follow the Google style for docstrings.
Type Hints: Use type hints for function parameters and return values.

Project Structure

src/: Contains the source code
- ssl_manager.py: Main module with the SSLManager class
tests/: Contains test files
- test_ssl_manager.py: Tests for the SSLManager class

Best Practices

Error Handling: Use try-except blocks to handle exceptions gracefully.
Logging: Use the Python logging module for logging instead of print statements.
Testing: Write tests for all new features and bug fixes.
Documentation: Update documentation when making changes to the code.

Approaches Considered and Lessons Learned

This section documents approaches that were tried during development but were modified or replaced because they didn't fully meet requirements. Understanding these decisions can help future developers avoid repeating work on approaches that have already been determined to be unsuitable.

Certificate Generation Approaches

Self-Signed Certificates vs. Let's Encrypt

Initially, the project used self-signed certificates as the default certificate type. While self-signed certificates are easy to generate and don't require external validation, they have several limitations:

Browsers display security warnings for self-signed certificates
Each client device must manually trust the certificate
No automatic renewal process
Not suitable for production environments accessed by multiple users

The project switched to Let's Encrypt certificates as the default because they:

Are trusted by browsers without warnings
Support automatic renewal
Provide better security assurances
Are free and widely supported

Self-signed certificates are still supported with the --type self-signed option for testing or internal networks where Let's Encrypt validation isn't possible.

Let's Encrypt Staging vs. Production Environment

During initial development, the project used Let's Encrypt's staging environment by default (use_staging: true in config.json). The staging environment has higher rate limits but issues certificates that aren't trusted by browsers.

This approach was changed to use the production environment by default (use_staging: false) because:

Production certificates are trusted by browsers without warnings
Most users want trusted certificates for production use
The staging environment is primarily useful for testing and development

The --staging flag is still available for testing purposes to avoid hitting rate limits during development.

HTTP Validation Approaches

Default Port 80 vs. Configurable Port

Initially, the Let's Encrypt validation using the standalone method required port 80, which needs root privileges. This was problematic because:

Running the script as root introduces security concerns
Many users don't have root access on their servers
Port 80 might already be in use by a web server

The solution was to add support for configurable HTTP ports through:

The http_port configuration option in the Let's Encrypt section of config.json
The --http-port command-line option

This allows users to specify a non-privileged port (e.g., 8080) for HTTP validation, eliminating the need for root privileges.

DNS Validation Approaches

Manual DNS Checking vs. Automated Validation

The project initially relied on Let's Encrypt's built-in validation to check if a hostname is in public DNS. However, this led to confusing error messages when validation failed.

To improve the user experience, the project added a pre-validation step that:

Resolves the hostname to an IP address
Checks if the IP is public (not in private IP ranges)
Provides a clear error message if the hostname isn't in public DNS

This approach helps users identify DNS issues before attempting certificate generation, saving time and avoiding Let's Encrypt rate limits.

UniFi Device Connection Approaches

UnifiControl Library

The project attempted to use the unifiControl Python library to interact with UniFi devices. However, this approach encountered several challenges:

Authentication issues (401 Unauthorized errors)
Potential API version compatibility issues
Limited flexibility for different UniFi device configurations

Instead of relying solely on the unifiControl library, the project adopted a more flexible approach:

Storing UniFi connection parameters in the config file
Supporting both API-based authentication and SSH-based authentication
Allowing users to customize connection parameters based on their specific UniFi device

This approach provides more flexibility and reliability across different UniFi device types and configurations.

Lessons Learned

Prioritize user experience: Choose defaults that work for most users without configuration (e.g., production Let's Encrypt environment).
Provide flexibility: Support multiple approaches (self-signed vs. Let's Encrypt, different validation methods) to accommodate various use cases.
Minimize dependencies: Avoid relying on external libraries when possible to reduce compatibility issues.
Clear error messages: Provide specific, actionable error messages to help users troubleshoot issues.
Pre-validate inputs: Check requirements (like DNS entries) early to provide better feedback before attempting operations that might fail.

Deployment and Automation

Cross-Machine Deployment

The SSL Management tool can be developed on one machine and deployed to another server for production use. When deploying across machines:

Ensure the target server meets all the prerequisites (Python 3.6+, OpenSSL, etc.)
Transfer the entire project directory to the target server
Set up the virtual environment on the target server
Configure the config.json file with the appropriate settings for the target environment
Test the deployment by running a simple command like python src/ssl_manager.py check <hostname>

Automated Certificate Updates

To automate certificate updates using cron:

Create a shell script wrapper for the SSL Manager:

#!/bin/bash
# File: update_certificates.sh

# Change to the project directory
cd /path/to/ssl-managment

# Activate the virtual environment
source .venv/bin/activate

# Run the SSL Manager to update certificates with Let's Encrypt
python src/ssl_manager.py generate your-unifi-device.example.com --type letsencrypt --email admin@example.com --validation-method standalone --production

# Additional commands to deploy the certificate to the UniFi device can be added here

Make the script executable:
```
chmod +x update_certificates.sh
```

Add a cron job to run the script periodically (e.g., every 60 days):

# Edit the crontab
crontab -e

# Add a line like this to run at 2:30 AM on the 1st of every other month
30 2 1 */2 * /path/to/update_certificates.sh >> /var/log/certificate-updates.log 2>&1

Verify the cron job is set up correctly:
```
crontab -l
```

Troubleshooting

Common Issues

OpenSSL Command Not Found

Ensure OpenSSL is installed and in your PATH.
Permission Denied When Creating Certificates

Check that you have write permissions to the certificate directory.
Import Errors in Tests

Make sure you're running tests from the project root directory.
Hostname Not in Public DNS Error

When generating a Let's Encrypt certificate, you may see an error like:
```
Error generating Let's Encrypt certificate: Hostname example.com is not in public DNS. Let's Encrypt requires the hostname to be in public DNS.
```
This means the hostname you're trying to use doesn't resolve to a public IP address. To fix this:
- Verify that the hostname has a public DNS record (A or AAAA) pointing to your server
- Check that the DNS record has propagated (this can take up to 48 hours)
- If you're using a private hostname, consider using a self-signed certificate instead with --type self-signed
- For testing purposes, you can use a hostname that is already in public DNS

Debugging

Use the --debug flag to enable detailed logging with line numbers and file names:
```
python src/ssl_manager.py --debug check example.com
```

Set the debug option to true in the config.json file to always enable debug logging:

{
  "cert_dir": "~/.ssl-certs",
  "default_port": 443,
  "debug": true,
  "connection_timeout": 3.0
}

Debug logs include:
- Line numbers and file names for each log message
- Detailed information about each operation
- Command execution details
- Error messages with stack traces
Check the OpenSSL version with openssl version
Verify certificate paths are correct and accessible

21 KiB Raw Blame History