---
title: Scripts
---

Scripts Collection (hereinafter the "Application") is a collection of TypeScript-based automation utilities for managing translations, auditing repositories, automating Discord tasks, and processing media files.

## 1. User Documentation

This section is for those interacting with a live instance of the Application.

### Overview

The Scripts Collection provides standalone automation tools that can be executed individually as needed. Each script serves a specific purpose and operates independently.

### Available Scripts

#### 1.1 Crowdin Translation Management

**Purpose:** Manage translations and translation memory in Crowdin projects.

##### Write Data (`src/crowdin/writeData.ts`)

Fetches and persists Crowdin project data locally for use by other scripts.

**Usage:**
```bash
op run --env-file=.env -- tsx src/crowdin/writeData.ts
```

**What it does:**
- Retrieves all files from your Crowdin project
- Retrieves all translatable strings
- Saves data to `data/crowdin-files.json` and `data/crowdin-strings.json`

**Required environment variables:**
- `CROWDIN_PROJECT_ID` - Your Crowdin project identifier
- `CROWDIN_API_URL` - Crowdin API base URL
- `CROWDIN_TOKEN` - Your Crowdin API token

##### Clear Hidden Translations (`src/crowdin/clearHiddenTranslations.ts`)

Removes translations for strings that have been marked as hidden in Crowdin.

**Usage:**
```bash
op run --env-file=.env -- tsx src/crowdin/clearHiddenTranslations.ts
```

**What it does:**
- Reads cached string data from `data/crowdin-strings.json`
- Identifies strings marked as hidden
- Deletes existing translations for hidden strings across all languages
- Tracks processed strings in `data/crowdin-strings-hidden.txt` to avoid reprocessing

**Prerequisites:**
- Run `writeData.ts` first to cache the latest string data
- Same environment variables as above

##### Reapply Translations (`src/crowdin/reapplyTranslations.ts`)

Applies translation memory to project files using Crowdin's pre-translation feature.

**Usage:**
```bash
op run --env-file=.env -- tsx src/crowdin/reapplyTranslations.ts
```

**What it does:**
- Initiates pre-translation jobs for all files using translation memory
- Uses "perfect match only" strategy with auto-approval
- Skips already approved translations
- Polls every 5 seconds for completion status
- Displays progress for each file

**Prerequisites:**
- Run `writeData.ts` first to cache the latest file data
- Same environment variables as above

#### 1.2 GitHub Repository Auditing

##### Audit NPM Packages (`src/github/auditNpmPackages.ts`)

Scans GitHub organisation repositories for known vulnerable npm packages.

**Usage:**
```bash
op run --env-file=.env -- tsx src/github/auditNpmPackages.ts
```

**What it does:**
- Fetches all repositories from the FreeCodeCamp GitHub organisation
- Searches for 18 known vulnerable package versions in `package.json` files
- Generates a report of vulnerable dependencies
- Saves results to `data/npm-vulnerabilities.txt`

**Monitored vulnerable packages include:**
- ansi-regex (< 5.0.1)
- chalk (< 4.1.2)
- debug (< 4.3.1)
- color/color-string libraries (various versions)
- json-schema (< 0.4.0)
- And 13 others

**Required environment variables:**
- `GITHUB_TOKEN` - GitHub personal access token with repo read permissions

**Output format:**
```
Repository: <org>/<repo>
File: package.json
Vulnerable package: <package-name>@<version>
---
```

#### 1.3 Discord Automation

##### Create Conference Threads (`src/discord/cycThreads.ts`)

Automatically creates forum threads for conference presentations.

**Usage:**
```bash
op run --env-file=.env -- tsx src/discord/cycThreads.ts
```

**What it does:**
- Creates threads for 100+ conference talk entries
- Each thread is titled with speaker name and talk title
- Adds a welcome message to each thread
- Implements rate limiting with automatic retry on Discord API limits

**Required environment variables:**
- `DISCORD_TOKEN` - Discord bot token with thread creation permissions

**Target channel:** 1417221194536714393

**Note:** The conference data is currently hardcoded in the script and includes talks on web development, cloud technologies, AI/ML, architecture patterns, and more.

#### 1.4 Media File Processing

##### Add ID3 Tags (`src/music/id3v2.ts`)

Batch processes MP3 files to add metadata tags and album artwork.

**Usage:**
```bash
tsx src/music/id3v2.ts
```

**What it does:**
- Scans `/home/naomi/down` for MP3 files
- Extracts title from filename (removing special characters)
- Sets artist to configured string.
- Adds album artwork from configured path.
- Displays progress bar during processing

**Required system dependencies:**
- `eyeD3` - Python-based MP3 tag editor
- `id3v2` - Command-line ID3 tag tool

**Note:** File paths are currently hardcoded and should be modified for your use case.

### General Usage Tips

1. **Environment Variables:** Most scripts require environment variables. Use 1Password CLI (`op run`) for secure credential management.

2. **Execution Pattern:**
   ```bash
   # With environment variables
   op run --env-file=.env --no-masking -- tsx src/path/to/script.ts

   # Without environment variables
   tsx src/path/to/script.ts
   ```

3. **Data Directory:** Scripts store cached data and results in the `data/` directory. Ensure this directory exists and is writable.

4. **Error Handling:** Scripts implement automatic retry logic for rate-limited API calls (429 errors) with a 5-second backoff.

5. **Progress Tracking:** Most scripts provide console output to track progress. Some use progress bars for visual feedback.

## 2. Technical Documentation

This section is for those interested in running their own instance of the Application.

### 2.1. System Requirements

**Runtime:**
- Node.js (compatible with v24.3.0 type definitions)
- pnpm 10.15.0 (package manager)
- TypeScript 5.9.2

**External Tools (for media processing):**
- eyeD3 (Python package)
- id3v2 (system package)

**Optional:**
- 1Password CLI (`op`) for secure environment variable management

### 2.2. Installation

1. **Clone the repository:**
   ```bash
   git clone <repository-url>
   cd scripts
   ```

2. **Install dependencies:**
   ```bash
   pnpm install
   ```

3. **Create data directory:**
   ```bash
   mkdir -p data
   ```

4. **Set up environment variables:**

   Create a `.env` file with required credentials:
   ```sh
   # Crowdin (for translation scripts)
   CROWDIN_PROJECT_ID=your_project_id
   CROWDIN_API_URL=https://api.crowdin.com/api/v2
   CROWDIN_TOKEN=your_api_token

   # GitHub (for audit script)
   GITHUB_TOKEN=your_github_token

   # Discord (for thread creation script)
   DISCORD_TOKEN=your_discord_bot_token
   ```

5. **Install external tools (for media processing):**
   ```bash
   # On Ubuntu/Debian
   sudo apt-get install id3v2
   pip install eyeD3

   # On macOS
   brew install id3v2
   pip install eyeD3
   ```

### 2.3. Configuration

**TypeScript Configuration (`tsconfig.json`):**
- Extends `@nhcarrigan/typescript-config`
- Source directory: `./src`
- Output directory: `./prod`
- ES Modules enabled

**ESLint Configuration (`eslint.config.js`):**
- Extends `@nhcarrigan/eslint-config`
- Console statements allowed
- Await in loops permitted (needed for sequential API calls)
- Relaxed unsafe assignment rules

**Modifying Target Locations:**

To adapt scripts for your use:

1. **Crowdin scripts:** Update environment variables in `.env`
2. **GitHub audit:** Modify organisation name in `src/github/auditNpmPackages.ts:48`
3. **Discord threads:** Update channel ID and talk data in `src/discord/cycThreads.ts`
4. **Media processing:** Update file paths in `src/music/id3v2.ts:7-8`

### 2.4. Project Structure

```
scripts/
├── src/                          # Source code
│   ├── crowdin/                  # Crowdin automation
│   │   ├── interfaces/           # TypeScript type definitions
│   │   ├── utils/                # Helper functions
│   │   ├── writeData.ts
│   │   ├── clearHiddenTranslations.ts
│   │   └── reapplyTranslations.ts
│   ├── discord/                  # Discord automation
│   │   └── cycThreads.ts
│   ├── github/                   # GitHub utilities
│   │   └── auditNpmPackages.ts
│   ├── music/                    # Media processing
│   │   └── id3v2.ts
│   ├── utils/                    # Shared utilities
│   │   ├── sleep.ts
│   │   ├── backoffAndRetry.ts
│   │   └── serialiseJsonOrError.ts
│   └── index.ts                  # Usage instructions
├── data/                         # Cached data and output
├── prod/                         # Compiled JavaScript
├── package.json                  # Dependencies and scripts
├── tsconfig.json                 # TypeScript configuration
└── eslint.config.js              # Linting rules
```

### 2.5. Development

**Build the project:**
```bash
pnpm build
```

**Run linting:**
```bash
pnpm lint
```

**Execute scripts:**
```bash
# With tsx (no compilation needed)
tsx src/path/to/script.ts

# Or compile first, then run
pnpm build
node prod/path/to/script.js
```

### 2.6. API Integration Details

**Crowdin API:**
- Base URL: `https://api.crowdin.com/api/v2`
- Authentication: Bearer token in Authorization header
- Pagination: 500 items per page (limit parameter)
- Rate limiting: Handled with automatic retry (5-second backoff)

**GitHub API:**
- Client: `@octokit/rest`
- Authentication: Personal access token
- Pagination: 100 repositories per page
- Permissions needed: `repo` (read access)

**Discord API:**
- REST endpoint: `https://discord.com/api/v10`
- Authentication: Bot token
- Rate limiting: Automatic retry with backoff on 429 responses
- Required intents: Message content, guild messages

### 2.7. Data Storage

**Generated files:**
- `data/crowdin-files.json` (~12MB) - Cached Crowdin file metadata
- `data/crowdin-strings.json` (~275MB) - Cached translatable strings
- `data/crowdin-strings-hidden.txt` (~1.3MB) - Processed hidden string IDs
- `data/npm-vulnerabilities.txt` (~2KB) - Audit results

**Data freshness:**
- Crowdin data should be refreshed before running dependent scripts
- GitHub audit data is generated per run
- Discord thread creation is one-time execution

### 2.8. Security Considerations

1. **Secrets Management:**
   - Use 1Password CLI or similar tools for environment variable injection
   - Never commit `.env` files to version control
   - Rotate API tokens regularly

2. **API Token Permissions:**
   - Crowdin: Project member with translation management access
   - GitHub: Read-only repository access (minimal scope)
   - Discord: Bot token with thread creation and message posting permissions

3. **Rate Limiting:**
   - All scripts implement retry logic for rate limit responses
   - 5-second backoff before retrying
   - Recursive retry until success

4. **Input Validation:**
   - API responses are typed with TypeScript interfaces
   - Safe JSON parsing with error handling
   - File existence checks before processing

### 2.9. Troubleshooting

**Common Issues:**

1. **"Cannot find module" errors:**
   - Ensure `pnpm install` completed successfully
   - Check Node.js version compatibility

2. **API authentication failures:**
   - Verify environment variables are set correctly
   - Check token permissions and expiration
   - Ensure `op run` is used for secure variable injection

3. **Rate limit errors persisting:**
   - Check API quota limits on your account
   - Increase backoff delay in `src/utils/backoffAndRetry.ts`

4. **Large JSON files causing memory issues:**
   - Increase Node.js memory limit: `NODE_OPTIONS=--max-old-space-size=4096`
   - Process data in smaller batches

5. **Media processing failures:**
   - Verify `eyeD3` and `id3v2` are installed and in PATH
   - Check file permissions for input/output directories
   - Ensure cover image exists at specified path

## 3. Legal Documentation

This section is for expansions to our legal policies specific to the Application.

### 3.1. License

The Application is distributed under **Naomi's Public Licence**. See `LICENSE.md` for full terms.

### 3.2. Privacy Policy

Usage of these scripts may involve processing data from third-party services. Refer to `PRIVACY.md` for our privacy practices regarding:
- API credentials and tokens
- Cached translation data
- Repository metadata
- Discord message content

### 3.3. Terms of Service

By using these scripts, you agree to:
- Comply with third-party API terms of service (Crowdin, GitHub, Discord)
- Use automation responsibly and within rate limits
- Not use the scripts for malicious purposes
- Maintain security of API credentials

See `TERMS.md` for complete terms.

### 3.4. Security Policy

To report security vulnerabilities in the Application itself, please refer to `SECURITY.md`.

**Third-Party Security:**
- The GitHub audit script identifies known vulnerable npm packages
- Keep dependencies updated regularly
- Review `pnpm audit` output periodically

### 3.5. Data Retention

**Local Data Storage:**
- Cached data in `data/` directory is stored indefinitely until manually deleted
- No data is transmitted to external services except through official APIs
- API tokens are loaded at runtime and not persisted to disk

**Third-Party Data:**
- Crowdin: Translation data is fetched and cached locally
- GitHub: Repository metadata is accessed read-only
- Discord: Messages are posted to threads as configured

## 4. Contributing Documentation

This section is for documentation related to contributing to the Application's codebase.

### 4.1. Contributing Guidelines

Please refer to `CONTRIBUTING.md` for detailed contribution guidelines. Key points:

**Code Style:**
- Follow ESLint configuration (`@nhcarrigan/eslint-config`)
- Use TypeScript with strict type checking
- Include JSDoc comments for exported functions
- Add copyright headers to new files

**Commit Messages:**
- Use conventional commit format
- Examples: `feat:`, `fix:`, `docs:`, `refactor:`

**Testing:**
- While formal tests are not currently required, manually test all changes
- Verify scripts work with environment variables
- Test error handling and edge cases

### 4.2. Adding New Scripts

To add a new automation script:

1. **Create script file:**
   ```bash
   # Choose appropriate directory or create new one
   touch src/<category>/<script-name>.ts
   ```

2. **Add copyright header:**
   ```typescript
   /*
    * Copyright (C) 2025 Naomi Carrigan
    * Licensed under Naomi's Public Licence
    */
   ```

3. **Import shared utilities:**
   ```typescript
   import { sleep } from "../utils/sleep.js";
   import { backoffAndRetry } from "../utils/backoffAndRetry.js";
   ```

4. **Implement type safety:**
   - Create interfaces in `<category>/interfaces/` if needed
   - Use TypeScript types for all API responses
   - Handle errors with try-catch blocks

5. **Add environment variable validation:**
   ```typescript
   const token = process.env.SERVICE_TOKEN;
   if (!token) {
     console.error("Missing SERVICE_TOKEN environment variable");
     process.exit(1);
   }
   ```

6. **Document usage:**
   - Add JSDoc comments to main functions
   - Update this documentation file
   - Include example usage in comments

7. **Test thoroughly:**
   - Test with valid and invalid inputs
   - Verify error handling
   - Check rate limit handling

### 4.3. Code Organization

**Shared Utilities (`src/utils/`):**
Place reusable functions here:
- Async utilities (sleep, retry logic)
- Data transformation functions
- Error handling helpers
- Common type definitions

**Category-Specific Utilities:**
Place service-specific helpers in `<category>/utils/`:
- API client wrappers
- Data fetching functions
- Service-specific transformations

**Interfaces (`<category>/interfaces/`):**
Define TypeScript interfaces for:
- API request/response types
- Configuration objects
- Data models

### 4.4. Dependency Management

**Adding Dependencies:**
```bash
pnpm add <package-name>
pnpm add -D <dev-package-name>
```

**Updating Dependencies:**
```bash
pnpm update
pnpm audit
```

**Preferred Libraries:**
- API clients: Official SDKs when available (`@octokit/rest`, etc.)
- CLI interactions: `@inquirer/prompts`
- Progress visualisation: `cli-progress`
- HTTP requests: Native `fetch` with `backoffAndRetry` wrapper

### 4.5. Documentation Standards

**Code Comments:**
- Use JSDoc for all exported functions
- Include `@param`, `@returns`, and `@throws` tags
- Add inline comments for complex logic

**Example:**
```typescript
/**
 * Fetches all files from a Crowdin project with pagination.
 * @param projectId - The Crowdin project identifier
 * @returns Promise resolving to array of file objects
 * @throws Error if API request fails after retries
 */
export async function getFiles(projectId: string): Promise<File[]> {
  // Implementation
}
```

**README Updates:**
- Document new scripts in README.md
- Include usage examples
- List required environment variables
- Note any system dependencies

**Changelog:**
- Maintain a changelog for significant changes
- Use semantic versioning principles
- Document breaking changes prominently

### 4.6. Architecture Principles

**Script Independence:**
- Each script should be runnable standalone
- Avoid tight coupling between scripts
- Use data files for inter-script communication when needed

**Error Handling:**
- Always validate environment variables at startup
- Use try-catch blocks for external API calls
- Provide descriptive error messages
- Exit with non-zero code on failure

**API Best Practices:**
- Implement rate limit handling with exponential backoff
- Use pagination for large data sets
- Cache data locally when appropriate
- Respect API quotas and limits

**Type Safety:**
- Define interfaces for all external data
- Avoid `any` types when possible
- Use strict TypeScript configuration
- Validate data shapes at runtime when needed

### 4.7. Testing Approach

While formal unit tests are not currently implemented, contributors should:

1. **Manual Testing Checklist:**
   - Run script with valid environment variables
   - Test with missing environment variables
   - Verify output files are created correctly
   - Check error handling with invalid inputs
   - Test rate limit retry logic (if applicable)

2. **Integration Testing:**
   - Test against real APIs in development environments
   - Verify data format compatibility
   - Check for breaking changes in API responses

3. **Future Testing Goals:**
   - Add unit tests for utility functions
   - Mock API responses for integration tests
   - Set up CI/CD pipeline for automated testing
   - Implement test coverage reporting

### 4.8. Code of Conduct

All contributors must adhere to the Code of Conduct outlined in `CODE_OF_CONDUCT.md`. Key principles:
- Be respectful and inclusive
- Welcome newcomers and help them learn
- Focus on constructive feedback
- Respect differing opinions and experiences

### 4.9. Getting Help

**Resources:**
- Review existing scripts for examples
- Check JSDoc comments for function documentation
- Read official API documentation for services
- Consult TypeScript and ESLint documentation for style questions

**Contact:**
- Open an issue for bug reports or feature requests
- Submit pull requests for proposed changes
- Follow contribution guidelines for all submissions

---
