ephemere/CLAUDE.md

# Ephemere Project Guidelines

This document contains project-specific instructions for working with the Ephemere codebase.

## Project Overview

Ephemere is a collection of ephemeral scripts for various tasks, written in TypeScript and Python. It contains utilities for:
- S3 operations (upload, bulk upload, delete, content type correction)
- Discord bot utilities
- Discourse forum management
- Gitea/GitHub operations
- Security analysis tools
- Music-related scripts
- Various utility functions

## Project Structure

```
ephemere/
├── typescript/          # TypeScript scripts
│   ├── src/
│   │   ├── s3/         # S3 operations (upload, delete, bulk operations)
│   │   ├── discord/    # Discord bot and utilities
│   │   ├── discourse/  # Discourse forum management
│   │   ├── gitea/      # Gitea API interactions
│   │   ├── github/     # GitHub API interactions
│   │   ├── security/   # Security analysis tools
│   │   ├── music/      # Music-related utilities
│   │   └── utils/      # Shared utilities
│   └── data/          # Data files for S3 uploads
├── python/            # Python scripts
└── prod.env          # 1Password vault references (safe to commit)
```

## Development Standards

### TypeScript Scripts
- All TypeScript scripts must follow Naomi's Node.js project standards
- Use `@nhcarrigan/typescript-config` and `@nhcarrigan/eslint-config`
- Run scripts using the Makefile: `make run-ts src/path/to/script.ts`
- Interactive scripts should use `@inquirer/prompts` for user input

### Python Scripts
- Use `uv` for package management
- Linting and formatting with `ruff`
- Run scripts using the Makefile: `make run-py script_name.py`

### S3 Scripts Specifics
All S3 scripts in `typescript/src/s3/` follow these patterns:
- Use environment variables: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `S3_ENDPOINT`
- Interactive prompts using `@inquirer/prompts`
- Progress bars using `cli-progress` for bulk operations
- Proper error handling with success/error counts
- ESLint disable comments for AWS SDK naming conventions
- No CLI arguments - everything is prompted interactively

## Running Scripts

Always use the Makefile commands to run scripts (they handle 1Password integration):

```bash
# TypeScript scripts
make run-ts src/s3/deleteContents.ts
make run-ts src/discord/bot.ts

# Python scripts
make run-py analyse_availability.py
```

Or use the interactive runner:
```bash
make run  # Interactive menu to select language, category, and script
```

## Script Patterns

### TypeScript Script Template
```typescript
/**
 * @copyright NHCarrigan
 * @license Naomi's Public License
 * @author Naomi Carrigan
 */
import { input, confirm, select } from "@inquirer/prompts";

// Environment variable checks
const requiredVar = process.env.REQUIRED_VAR;
if (requiredVar === undefined) {
  throw new Error("REQUIRED_VAR is not set");
}

// Interactive prompts
const userInput = await input({
  message: "Enter your input:",
  validate: (value) => {
    if (!value.trim()) {
      return "Input cannot be empty";
    }
    return true;
  },
});

// Main logic here
console.log("Processing...");

// Always provide feedback
console.log("✅ Operation completed successfully!");
```

### Python Script Template
```python
#!/usr/bin/env python3
"""
Script description here.

@copyright NHCarrigan
@license Naomi's Public License
@author Naomi Carrigan
"""
import os
import sys
from typing import Optional

def main() -> None:
    """Main function."""
    # Environment variable checks
    required_var = os.getenv("REQUIRED_VAR")
    if not required_var:
        print("Error: REQUIRED_VAR is not set")
        sys.exit(1)

    # Interactive input (if needed)
    user_input = input("Enter your input: ").strip()
    if not user_input:
        print("Error: Input cannot be empty")
        sys.exit(1)

    # Main logic here
    print("Processing...")

    # Always provide feedback
    print("✅ Operation completed successfully!")

if __name__ == "__main__":
    main()
```

## Environment Variables

All secrets are managed through 1Password and referenced in `prod.env`:
- AWS credentials for S3 operations
- Discord bot tokens
- GitHub/Gitea API tokens
- Discourse API credentials

The `prod.env` file contains 1Password vault references (like `op://Private/Hetzner/S3 Endpoint`) and is safe to commit.

## Important Notes

1. **No hardcoded values** - All configuration should come from environment variables
2. **Interactive scripts** - Scripts should prompt for all required input, not use CLI arguments
3. **Progress feedback** - Long-running operations should show progress bars
4. **Error handling** - Always track and report success/failure counts
5. **Consistent patterns** - Follow the existing script patterns in each category

## Best Practices

### Error Handling
- Always validate environment variables at script start
- Provide clear error messages that help users fix the issue
- Use try-catch blocks for external API calls
- Track success/failure counts for bulk operations
- Exit with appropriate status codes (0 for success, 1 for errors)

### User Experience
- Use emojis in console output for visual feedback (✅ ❌ ⚠️ 🚀 📦 etc.)
- Show progress bars for operations with multiple items
- Confirm destructive operations (require typing confirmation + yes/no)
- Provide summaries at the end of bulk operations
- Keep output concise but informative

### Code Quality
- Add JSDoc comments for TypeScript functions
- Use type annotations in Python
- Follow the established linting rules (no overrides without good reason)
- Keep functions focused and single-purpose
- Extract reusable logic to utility functions

### Security
- Never log sensitive information (tokens, passwords, keys)
- Validate all user inputs
- Use parameterized queries for any database operations
- Follow the principle of least privilege for API tokens
- Sanitize file paths and names

## Common Tasks

### Adding a new S3 script
1. Create the script in `typescript/src/s3/`
2. Follow the pattern of existing S3 scripts (see deleteContents.ts)
3. Use environment variables: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `S3_ENDPOINT`
4. Use @inquirer/prompts for all user input (no CLI arguments)
5. Include proper error handling and progress bars for bulk operations
6. Add ESLint disable comments for AWS SDK naming conventions

### Adding a new Discord script
1. Create the script in `typescript/src/discord/`
2. Use environment variables: `DISCORD_TOKEN`, `DISCORD_CLIENT_ID`, etc.
3. Follow Discord.js best practices
4. Use @inquirer/prompts for any configuration input
5. Handle rate limits and API errors gracefully

### Adding a new GitHub/Gitea script
1. Create the script in `typescript/src/github/` or `typescript/src/gitea/`
2. Use environment variables: `GITHUB_TOKEN` or `GITEA_TOKEN`
3. Use the Octokit library for GitHub operations
4. Include proper pagination for list operations
5. Handle API rate limits appropriately

### Adding a new Discourse script
1. Create the script in `typescript/src/discourse/`
2. Use environment variables: `DISCOURSE_URL`, `DISCOURSE_API_KEY`, `DISCOURSE_API_USERNAME`
3. Follow the Discourse API documentation
4. Use @inquirer/prompts for interactive inputs
5. Handle API errors and rate limits

### Adding a new Security script
1. Create the script in `typescript/src/security/`
2. Use environment variables for any API tokens (e.g., `DOJO_TOKEN` for DefectDojo)
3. Follow security best practices - never log sensitive data
4. Include proper validation for all inputs
5. Consider adding audit logs for security operations

### Adding a new Python script
1. Create the script in `python/`
2. Use type hints for all functions and variables
3. Follow PEP 8 style guide (enforced by ruff)
4. Add the script to `requirements.txt` if it needs new dependencies
5. Use `argparse` for CLI arguments if needed (though prefer interactive)
6. Include docstrings for all functions and classes

### Adding a new utility function
1. TypeScript utilities go in `typescript/src/utils/`
2. Python utilities can be imported from a shared module
3. Utilities should be pure functions when possible
4. Include comprehensive JSDoc/docstring documentation
5. Add unit tests if the utility is complex

### Adding a new script category
1. Create a new directory under `typescript/src/` or in `python/`
2. Follow the naming convention (lowercase, descriptive)
3. Create at least one example script showing the pattern
4. Update this CLAUDE.md with specific guidelines for the category
5. Add any new environment variables to prod.env with 1Password references

## Testing

Before committing:
```bash
make lint        # Run all linters
make build       # Type check TypeScript
make test        # Run tests (if any)
```

## Troubleshooting

- If a script fails with "not set" errors, ensure you're running through the Makefile or with `op run`
- For S3 scripts, verify your 1Password vault has the correct S3 endpoint, access key, and secret key
- TypeScript import errors: ensure you use `.js` extensions in imports (even for `.ts` files)