ephemere/CLAUDE.md

Ephemere Project Guidelines

This document contains project-specific instructions for working with the Ephemere codebase.

Project Overview

Ephemere is a collection of ephemeral scripts for various tasks, written in TypeScript, Python, and Bash. It contains utilities for:

• S3 operations (upload, bulk upload, delete, content type correction)
• Discord bot utilities
• Discourse forum management
• Gitea/GitHub operations
• Security analysis tools
• Music-related scripts
• Cohort programme management (Python + Bash)
• YubiKey SSH key and permission utilities (Bash)

Project Structure

ephemere/
├── typescript/          # TypeScript scripts
│   └── src/
│       ├── s3/         # S3 operations (upload, delete, bulk operations)
│       ├── discord/    # Discord bot and utilities
│       ├── discourse/  # Discourse forum management
│       ├── gitea/      # Gitea API interactions
│       ├── github/     # GitHub API interactions
│       ├── security/   # Security analysis tools
│       ├── music/      # Music-related utilities
│       └── utils/      # Shared utilities
├── python/
│   └── cohort/        # Cohort programme management scripts
├── bash/
│   ├── cohort/        # GitHub team management for cohorts
│   └── yubikey/       # YubiKey SSH key and permission utilities
├── data/              # Input/output data files (gitignored)
└── prod.env           # 1Password vault references (safe to commit)

Development Standards

TypeScript Scripts

• All TypeScript scripts must follow Naomi's Node.js project standards
• Use @nhcarrigan/typescript-config and @nhcarrigan/eslint-config
• Run scripts using the interactive runner: make run (select TypeScript → category → script)
• Interactive scripts should use @inquirer/prompts for user input

Python Scripts

• Use uv for package management
• Linting and formatting with ruff
• Scripts live in subdirectories of python/ (e.g. python/cohort/)
• Run scripts using the interactive runner: make run (select Python → category → script)

Bash Scripts

• Scripts live in subdirectories of bash/ (e.g. bash/cohort/, bash/yubikey/)
• Run scripts using the interactive runner: make run (select Bash → category → script)
• Or run directly: bash bash/<category>/<script>.sh
• Bash scripts do not use 1Password injection (they use gh CLI auth or system tools)

S3 Scripts Specifics

All S3 scripts in typescript/src/s3/ follow these patterns:

• Use environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, S3_ENDPOINT
• Interactive prompts using @inquirer/prompts
• Progress bars using cli-progress for bulk operations
• Proper error handling with success/error counts
• ESLint disable comments for AWS SDK naming conventions
• No CLI arguments - everything is prompted interactively

Running Scripts

Always use the interactive runner to run scripts (it handles 1Password integration for TypeScript and Python):

make run  # Interactive menu to select language, category, and script

Or run directly:

# TypeScript (from project root)
cd typescript && op run --env-file=../prod.env -- pnpm tsx src/<category>/<script>.ts

# Python (from project root)
cd python && op run --env-file=../prod.env -- uv run python <category>/<script>.py

# Bash (no 1Password needed)
bash bash/<category>/<script>.sh

Script Patterns

TypeScript Script Template

/**
 * @copyright NHCarrigan
 * @license Naomi's Public License
 * @author Naomi Carrigan
 */
import { input, confirm, select } from "@inquirer/prompts";

// Environment variable checks
const requiredVar = process.env.REQUIRED_VAR;
if (requiredVar === undefined) {
  throw new Error("REQUIRED_VAR is not set");
}

// Interactive prompts
const userInput = await input({
  message: "Enter your input:",
  validate: (value) => {
    if (!value.trim()) {
      return "Input cannot be empty";
    }
    return true;
  },
});

// Main logic here
console.log("Processing...");

// Always provide feedback
console.log("✅ Operation completed successfully!");

Python Script Template

#!/usr/bin/env python3
"""
Script description here.

@copyright NHCarrigan
@license Naomi's Public License
@author Naomi Carrigan
"""
import os
import sys
from typing import Optional

def main() -> None:
    """Main function."""
    # Environment variable checks
    required_var = os.getenv("REQUIRED_VAR")
    if not required_var:
        print("Error: REQUIRED_VAR is not set")
        sys.exit(1)

    # Interactive input (if needed)
    user_input = input("Enter your input: ").strip()
    if not user_input:
        print("Error: Input cannot be empty")
        sys.exit(1)

    # Main logic here
    print("Processing...")

    # Always provide feedback
    print("✅ Operation completed successfully!")

if __name__ == "__main__":
    main()

Environment Variables

All secrets are managed through 1Password and referenced in prod.env:

• AWS credentials for S3 operations
• Discord bot tokens
• GitHub/Gitea API tokens
• Discourse API credentials

The prod.env file contains 1Password vault references (like op://Private/Hetzner/S3 Endpoint) and is safe to commit.

Important Notes

1. No hardcoded values - All configuration should come from environment variables
2. Interactive scripts - Scripts should prompt for all required input, not use CLI arguments
3. Progress feedback - Long-running operations should show progress bars
4. Error handling - Always track and report success/failure counts
5. Consistent patterns - Follow the existing script patterns in each category

Best Practices

Error Handling

• Always validate environment variables at script start
• Provide clear error messages that help users fix the issue
• Use try-catch blocks for external API calls
• Track success/failure counts for bulk operations
• Exit with appropriate status codes (0 for success, 1 for errors)

User Experience

• Use emojis in console output for visual feedback (✅ ❌ ⚠️ 🚀 📦 etc.)
• Show progress bars for operations with multiple items
• Confirm destructive operations (require typing confirmation + yes/no)
• Provide summaries at the end of bulk operations
• Keep output concise but informative

Code Quality

• Add JSDoc comments for TypeScript functions
• Use type annotations in Python
• Follow the established linting rules (no overrides without good reason)
• Keep functions focused and single-purpose
• Extract reusable logic to utility functions

Security

• Never log sensitive information (tokens, passwords, keys)
• Validate all user inputs
• Use parameterized queries for any database operations
• Follow the principle of least privilege for API tokens
• Sanitize file paths and names

Common Tasks

Adding a new S3 script

1. Create the script in typescript/src/s3/
2. Follow the pattern of existing S3 scripts (see deleteContents.ts)
3. Use environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, S3_ENDPOINT
4. Use @inquirer/prompts for all user input (no CLI arguments)
5. Include proper error handling and progress bars for bulk operations
6. Add ESLint disable comments for AWS SDK naming conventions

Adding a new Discord script

1. Create the script in typescript/src/discord/
2. Use environment variables: DISCORD_TOKEN, DISCORD_CLIENT_ID, etc.
3. Follow Discord.js best practices
4. Use @inquirer/prompts for any configuration input
5. Handle rate limits and API errors gracefully

Adding a new GitHub/Gitea script

1. Create the script in typescript/src/github/ or typescript/src/gitea/
2. Use environment variables: GITHUB_TOKEN or GITEA_TOKEN
3. Use the Octokit library for GitHub operations
4. Include proper pagination for list operations
5. Handle API rate limits appropriately

Adding a new Discourse script

1. Create the script in typescript/src/discourse/
2. Use environment variables: DISCOURSE_URL, DISCOURSE_API_KEY, DISCOURSE_API_USERNAME
3. Follow the Discourse API documentation
4. Use @inquirer/prompts for interactive inputs
5. Handle API errors and rate limits

Adding a new Security script

1. Create the script in typescript/src/security/
2. Use environment variables for any API tokens (e.g., DOJO_TOKEN for DefectDojo)
3. Follow security best practices - never log sensitive data
4. Include proper validation for all inputs
5. Consider adding audit logs for security operations

Adding a new Python script

1. Create the script in the appropriate subdirectory of python/ (e.g. python/cohort/)
2. Use type hints for all functions and variables
3. Follow PEP 8 style guide (enforced by ruff)
4. Add the script to requirements.txt if it needs new dependencies
5. Use argparse for CLI arguments if needed (though prefer interactive)
6. Include docstrings for all functions and classes

Adding a new utility function

1. TypeScript utilities go in typescript/src/utils/
2. Python utilities can be imported from a shared module
3. Utilities should be pure functions when possible
4. Include comprehensive JSDoc/docstring documentation
5. Add unit tests if the utility is complex

Adding a new script category

1. Create a new directory under typescript/src/, python/, or bash/ as appropriate
2. Follow the naming convention (lowercase, descriptive)
3. Create at least one example script showing the pattern
4. Create a README.md in the new directory documenting each script
5. Update this CLAUDE.md with specific guidelines for the category
6. Add any new environment variables to prod.env with 1Password references

Testing

Before committing:

make lint        # Run all linters
make build       # Type check TypeScript
make test        # Run tests (if any)

Troubleshooting

• If a script fails with "not set" errors, ensure you're running through the Makefile or with op run
• For S3 scripts, verify your 1Password vault has the correct S3 endpoint, access key, and secret key
• TypeScript import errors: ensure you use .js extensions in imports (even for .ts files)