About

regularbm provides a robust, automated solution for backing up an Open WebUI instance running in Docker to an Amazon S3 bucket. It includes a Python script for performing the backup and an AWS CloudFormation template for deploying the necessary cloud infrastructure securely.

The script performs the following actions:

1. Exports all chats for every user into individual JSON files.
2. Exports the application configuration into a JSON file.
3. Backs up the entire SQLite database file (webui.db).
4. Packages all exported files into a single, compressed archive (.tar.lz4).
5. Uploads the compressed archive to the designated Amazon S3 bucket.

Features

• Comprehensive Backup: Captures user data, chats, and system configuration.
• Efficient Archiving: Uses LZ4 compression for fast packaging.
• Secure Cloud Storage: Uploads backups to a private S3 bucket.
• Automated & Scheduled: Designed to be run automatically on a schedule using cron.
• Secure by Default: Leverages IAM Roles for AWS EC2 instances, eliminating the need for long-lived access keys on the server.
• Flexible: Includes documented procedures for both AWS-native servers and external (on-premises or multi-cloud) servers.
• Configurable: Easily set parameters like container name and S3 path via command-line arguments.

Core Architecture

This solution supports two primary security patterns for granting server backup permissions:

1. Pattern A: AWS EC2 Instances (Recommended): An IAM Role is attached to the EC2 instance via an Instance Profile. The server automatically receives short-lived, temporary credentials, which is the most secure method.
2. Pattern B: External/On-Premises Servers: A dedicated IAM User with a narrowly scoped policy and long-lived static access keys is created. This pattern requires careful credential management.

Setup and Deployment Guide

Step 1: Prerequisites & Dependencies

System Dependencies (Debian/Ubuntu)

You need python3, docker, and development libraries installed on the host machine.

sudo apt update
sudo apt install -y python3-venv docker.io liblz4-dev

Note on Docker Permissions: The user running this script must have permission to execute docker commands. The easiest way is to add the user to the docker group:

sudo usermod -aG docker $USER
# IMPORTANT: You must log out and log back in for this change to take effect.

AWS Account

You will need an AWS Account with permissions to create S3, IAM, and CloudFormation resources.

Step 2: Deploying the AWS Infrastructure (CloudFormation)

The cloud infrastructure (S3 buckets, IAM Role) is defined in cloudformation_template.yaml for consistent deployment.

1. Navigate to CloudFormation in the AWS Console.
2. Click Create stack -> With new resources (standard).
3. Select Template is ready -> Direct input and paste the contents of cloudformation_template.yaml.
4. Specify a stack name and provide globally unique names for the S3 buckets.
5. Acknowledge that CloudFormation will create IAM resources and click Create stack.
6. Wait for CREATE_COMPLETE status. Note the InstanceProfileName from the Outputs tab.

Step 3: Configuring Server Permissions

Choose the appropriate path based on where your server is hosted.

For AWS EC2 Instances (Pattern A)

1. Tag the EC2 Instance: Apply a tag to the instance with Key hostname and a unique value (e.g., openwebui-prod-01).
2. Attach the IAM Role: Select the instance, go to Actions -> Security -> Modify IAM role, and attach the Instance Profile created by CloudFormation.

For External/On-Premises Servers (Pattern B)

1. Create a Dedicated IAM Policy: In the IAM console, create a new policy that allows the s3:PutObject action on the desired S3 prefix (e.g., arn:aws:s3:::your-backup-bucket/your-prefix/*).
2. Create an IAM User: Create a new user with Programmatic access, attaching the policy created above. Save the generated Access Key and Secret Key.
3. Configure Credentials on the Server:
   • Create the credentials file: mkdir -p ~/.aws && nano ~/.aws/credentials.
   • Add the keys in the [default] profile format.
   • Set strict permissions: chmod 600 ~/.aws/credentials.

Step 4: Preparing the Python Script Environment

Due to modern Linux protections (externally-managed-environment), you have two good options for installing the required Python packages (boto3, lz4):

1. The OS-Native Way (Simple): Install the dependencies using apt.

   sudo apt install python3-boto3 python3-lz4
   

2. The Virtual Environment Way (Robust): Create an isolated environment for the script.

   cd /path/to/your/script
   python3 -m venv venv
   source venv/bin/activate
   pip install boto3 lz4
   

Step 5: Usage and Configuration

Script Arguments

The regularbm.py script is configured via command-line arguments:

• --s3-bucket (Required): The name of your S3 bucket.
• --s3-region (Required): The AWS region of your S3 bucket (e.g., us-east-1).
• --container-name: The name of your Open WebUI Docker container. Defaults to open-webui.
• --s3-prefix: A prefix (folder path) within your S3 bucket. Defaults to openwebui_backups/.
• --tmp-dir: A temporary directory for staging files. Defaults to /tmp.

Manual Execution

Make the script executable (chmod +x regularbm.py) and run it manually for testing:

# Basic usage
./regularbm.py --s3-bucket "your-backup-bucket" --s3-region "us-west-2"

# With custom parameters
./regularbm.py \
  --s3-bucket "your-backup-bucket" \
  --s3-region "us-west-2" \
  --container-name "my-custom-webui" \
  --s3-prefix "production_backups/"

Step 6: Automation with Cron

Schedule the script for automated execution using cron.

1. Open your crontab: crontab -e.

2. Add a job entry. Ensure you use absolute paths. If you used a virtual environment, you must use the Python interpreter from that environment.

   Example Cron Job (runs daily at 09:00 UTC):

   0 9 * * * /path/to/script/venv/bin/python /path/to/script/regularbm.py --s3-bucket "your-bucket" --s3-region "us-east-1" --s3-prefix "your-prefix/" >> /var/log/regularbm.log 2>&1
   

   Explanation of the cron line:

   • 0 9 * * *: The schedule (minute 0, hour 9, every day).
   • /path/to/script/.../python: Absolute path to the Python interpreter. Use /usr/bin/python3 for the OS-native install or the path to your venv interpreter.
   • /path/to/script/.../regularbm.py: Absolute path to the backup script.
   • --s3-bucket ...: Your required script arguments.
   • >> /var/log/regularbm.log 2>&1: Highly recommended. Redirects all output and errors to a log file for debugging.

Troubleshooting Common Errors

• externally-managed-environment: Your OS is protecting its Python installation. Follow the instructions in Step 4 to use apt or a virtual environment.
• Unable to locate credentials: The script cannot authenticate with AWS. If on EC2, ensure the IAM Instance Profile is attached (Step 3A). If external, ensure the ~/.aws/credentials file is correctly created and secured (Step 3B).
• AccessDenied on s3:PutObject: The IAM policy is incorrect. The user/role has authenticated successfully but is not authorized to write to the specified S3 path. Edit the IAM policy and ensure the Resource ARN exactly matches the bucket and prefix your script is trying to use.