ephemere/CLAUDE.md

# Ephemere Project Guidelines

This document contains project-specific instructions for working with the Ephemere codebase.

## Project Overview

Ephemere is a collection of ephemeral scripts for various tasks, written in TypeScript, Python, and Bash. It contains utilities for:
- S3 operations (upload, bulk upload, delete, content type correction)
- Discord bot utilities
- Discourse forum management
- Gitea/GitHub operations
- Security analysis tools
- Music-related scripts
- Cohort programme management (Python + Bash)
- YubiKey SSH key and permission utilities (Bash)

## Project Structure

```
ephemere/
├── typescript/          # TypeScript scripts
│   └── src/
│       ├── s3/         # S3 operations (upload, delete, bulk operations)
│       ├── discord/    # Discord bot and utilities
│       ├── discourse/  # Discourse forum management
│       ├── gitea/      # Gitea API interactions
│       ├── github/     # GitHub API interactions
│       ├── security/   # Security analysis tools
│       ├── music/      # Music-related utilities
│       └── utils/      # Shared utilities
├── python/
│   └── cohort/        # Cohort programme management scripts
├── bash/
│   ├── cohort/        # GitHub team management for cohorts
│   └── yubikey/       # YubiKey SSH key and permission utilities
├── data/              # Input/output data files (gitignored)
└── prod.env           # 1Password vault references (safe to commit)
```

## Development Standards

### TypeScript Scripts
- All TypeScript scripts must follow Naomi's Node.js project standards
- Use `@nhcarrigan/typescript-config` and `@nhcarrigan/eslint-config`
- Run scripts using the interactive runner: `make run` (select TypeScript → category → script)
- Interactive scripts should use `@inquirer/prompts` for user input

### Python Scripts
- Use `uv` for package management
- Linting and formatting with `ruff`
- Scripts live in subdirectories of `python/` (e.g. `python/cohort/`)
- Run scripts using the interactive runner: `make run` (select Python → category → script)

### Bash Scripts
- Scripts live in subdirectories of `bash/` (e.g. `bash/cohort/`, `bash/yubikey/`)
- Run scripts using the interactive runner: `make run` (select Bash → category → script)
- Or run directly: `bash bash/<category>/<script>.sh`
- Bash scripts do not use 1Password injection (they use `gh` CLI auth or system tools)

### S3 Scripts Specifics
All S3 scripts in `typescript/src/s3/` follow these patterns:
- Use environment variables: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `S3_ENDPOINT`
- Interactive prompts using `@inquirer/prompts`
- Progress bars using `cli-progress` for bulk operations
- Proper error handling with success/error counts
- ESLint disable comments for AWS SDK naming conventions
- No CLI arguments - everything is prompted interactively

## Running Scripts

Always use the interactive runner to run scripts (it handles 1Password integration for TypeScript and Python):

```bash
make run  # Interactive menu to select language, category, and script
```

Or run directly:
```bash
# TypeScript (from project root)
cd typescript && op run --env-file=../prod.env -- pnpm tsx src/<category>/<script>.ts

# Python (from project root)
cd python && op run --env-file=../prod.env -- uv run python <category>/<script>.py

# Bash (no 1Password needed)
bash bash/<category>/<script>.sh
```

## Script Patterns

### TypeScript Script Template
```typescript
/**
 * @copyright NHCarrigan
 * @license Naomi's Public License
 * @author Naomi Carrigan
 */
import { input, confirm, select } from "@inquirer/prompts";

// Environment variable checks
const requiredVar = process.env.REQUIRED_VAR;
if (requiredVar === undefined) {
  throw new Error("REQUIRED_VAR is not set");
}

// Interactive prompts
const userInput = await input({
  message: "Enter your input:",
  validate: (value) => {
    if (!value.trim()) {
      return "Input cannot be empty";
    }
    return true;
  },
});

// Main logic here
console.log("Processing...");

// Always provide feedback
console.log("✅ Operation completed successfully!");
```

### Python Script Template
```python
#!/usr/bin/env python3
"""
Script description here.

@copyright NHCarrigan
@license Naomi's Public License
@author Naomi Carrigan
"""
import os
import sys
from typing import Optional

def main() -> None:
    """Main function."""
    # Environment variable checks
    required_var = os.getenv("REQUIRED_VAR")
    if not required_var:
        print("Error: REQUIRED_VAR is not set")
        sys.exit(1)

    # Interactive input (if needed)
    user_input = input("Enter your input: ").strip()
    if not user_input:
        print("Error: Input cannot be empty")
        sys.exit(1)

    # Main logic here
    print("Processing...")

    # Always provide feedback
    print("✅ Operation completed successfully!")

if __name__ == "__main__":
    main()
```

## Environment Variables

All secrets are managed through 1Password and referenced in `prod.env`:
- AWS credentials for S3 operations
- Discord bot tokens
- GitHub/Gitea API tokens
- Discourse API credentials

The `prod.env` file contains 1Password vault references (like `op://Private/Hetzner/S3 Endpoint`) and is safe to commit.

## Important Notes

1. **No hardcoded values** - All configuration should come from environment variables
2. **Interactive scripts** - Scripts should prompt for all required input, not use CLI arguments
3. **Progress feedback** - Long-running operations should show progress bars
4. **Error handling** - Always track and report success/failure counts
5. **Consistent patterns** - Follow the existing script patterns in each category

## Best Practices

### Error Handling
- Always validate environment variables at script start
- Provide clear error messages that help users fix the issue
- Use try-catch blocks for external API calls
- Track success/failure counts for bulk operations
- Exit with appropriate status codes (0 for success, 1 for errors)

### User Experience
- Use emojis in console output for visual feedback (✅ ❌ ⚠️ 🚀 📦 etc.)
- Show progress bars for operations with multiple items
- Confirm destructive operations (require typing confirmation + yes/no)
- Provide summaries at the end of bulk operations
- Keep output concise but informative

### Code Quality
- Add JSDoc comments for TypeScript functions
- Use type annotations in Python
- Follow the established linting rules (no overrides without good reason)
- Keep functions focused and single-purpose
- Extract reusable logic to utility functions

### Security
- Never log sensitive information (tokens, passwords, keys)
- Validate all user inputs
- Use parameterized queries for any database operations
- Follow the principle of least privilege for API tokens
- Sanitize file paths and names

## Common Tasks

### Adding a new S3 script
1. Create the script in `typescript/src/s3/`
2. Follow the pattern of existing S3 scripts (see deleteContents.ts)
3. Use environment variables: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `S3_ENDPOINT`
4. Use @inquirer/prompts for all user input (no CLI arguments)
5. Include proper error handling and progress bars for bulk operations
6. Add ESLint disable comments for AWS SDK naming conventions

### Adding a new Discord script
1. Create the script in `typescript/src/discord/`
2. Use environment variables: `DISCORD_TOKEN`, `DISCORD_CLIENT_ID`, etc.
3. Follow Discord.js best practices
4. Use @inquirer/prompts for any configuration input
5. Handle rate limits and API errors gracefully

### Adding a new GitHub/Gitea script
1. Create the script in `typescript/src/github/` or `typescript/src/gitea/`
2. Use environment variables: `GITHUB_TOKEN` or `GITEA_TOKEN`
3. Use the Octokit library for GitHub operations
4. Include proper pagination for list operations
5. Handle API rate limits appropriately

### Adding a new Discourse script
1. Create the script in `typescript/src/discourse/`
2. Use environment variables: `DISCOURSE_URL`, `DISCOURSE_API_KEY`, `DISCOURSE_API_USERNAME`
3. Follow the Discourse API documentation
4. Use @inquirer/prompts for interactive inputs
5. Handle API errors and rate limits

### Adding a new Security script
1. Create the script in `typescript/src/security/`
2. Use environment variables for any API tokens (e.g., `DOJO_TOKEN` for DefectDojo)
3. Follow security best practices - never log sensitive data
4. Include proper validation for all inputs
5. Consider adding audit logs for security operations

### Adding a new Python script
1. Create the script in the appropriate subdirectory of `python/` (e.g. `python/cohort/`)
2. Use type hints for all functions and variables
3. Follow PEP 8 style guide (enforced by ruff)
4. Add the script to `requirements.txt` if it needs new dependencies
5. Use `argparse` for CLI arguments if needed (though prefer interactive)
6. Include docstrings for all functions and classes

### Adding a new utility function
1. TypeScript utilities go in `typescript/src/utils/`
2. Python utilities can be imported from a shared module
3. Utilities should be pure functions when possible
4. Include comprehensive JSDoc/docstring documentation
5. Add unit tests if the utility is complex

### Adding a new script category
1. Create a new directory under `typescript/src/`, `python/`, or `bash/` as appropriate
2. Follow the naming convention (lowercase, descriptive)
3. Create at least one example script showing the pattern
4. Create a `README.md` in the new directory documenting each script
5. Update this CLAUDE.md with specific guidelines for the category
6. Add any new environment variables to prod.env with 1Password references

## Testing

Before committing:
```bash
make lint        # Run all linters
make build       # Type check TypeScript
make test        # Run tests (if any)
```

## Troubleshooting

- If a script fails with "not set" errors, ensure you're running through the Makefile or with `op run`
- For S3 scripts, verify your 1Password vault has the correct S3 endpoint, access key, and secret key
- TypeScript import errors: ensure you use `.js` extensions in imports (even for `.ts` files)