nhcarrigan/ephemere
9
0
Fork 0
generated from nhcarrigan/template
Code Issues Pull Requests Actions Releases Activity

feat: reorganise bash scripts and add comprehensive documentation #6

Merged
naomi merged 5 commits from feat/more-cohort into main 2026-02-23 20:18:42 -08:00
Conversation 0 Commits 5 Files Changed 65
+1985 -203
14 changed files with 1985 additions and 203 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit ae081cb54c - Show all commits
CLAUDE.md
+45 -30
View File
@@ -4,31 +4,36 @@ This document contains project-specific instructions for working with the Epheme
## Project Overview ## Project Overview
Ephemere is a collection of ephemeral scripts for various tasks, written in TypeScript and Python. It contains utilities for: Ephemere is a collection of ephemeral scripts for various tasks, written in TypeScript, Python, and Bash. It contains utilities for:
- S3 operations (upload, bulk upload, delete, content type correction) - S3 operations (upload, bulk upload, delete, content type correction)
- Discord bot utilities - Discord bot utilities
- Discourse forum management - Discourse forum management
- Gitea/GitHub operations - Gitea/GitHub operations
- Security analysis tools - Security analysis tools
- Music-related scripts - Music-related scripts
- Various utility functions - Cohort programme management (Python + Bash)
- YubiKey SSH key and permission utilities (Bash)
## Project Structure ## Project Structure
``` ```
ephemere/ ephemere/
├── typescript/ # TypeScript scripts ├── typescript/ # TypeScript scripts
│ ├── src/ │ └── src/
│ │ ├── s3/ # S3 operations (upload, delete, bulk operations) │ ├── s3/ # S3 operations (upload, delete, bulk operations)
│ │ ├── discord/ # Discord bot and utilities │ ├── discord/ # Discord bot and utilities
│ │ ├── discourse/ # Discourse forum management │ ├── discourse/ # Discourse forum management
│ │ ├── gitea/ # Gitea API interactions │ ├── gitea/ # Gitea API interactions
│ │ ├── github/ # GitHub API interactions │ ├── github/ # GitHub API interactions
│ │ ├── security/ # Security analysis tools │ ├── security/ # Security analysis tools
│ │ ├── music/ # Music-related utilities │ ├── music/ # Music-related utilities
│ │ └── utils/ # Shared utilities │ └── utils/ # Shared utilities
│ └── data/ # Data files for S3 uploads ├── python/
├── python/ # Python scripts │ └── cohort/ # Cohort programme management scripts
├── bash/
│ ├── cohort/ # GitHub team management for cohorts
│ └── yubikey/ # YubiKey SSH key and permission utilities
├── data/ # Input/output data files (gitignored)
└── prod.env # 1Password vault references (safe to commit) └── prod.env # 1Password vault references (safe to commit)
``` ```
@@ -37,13 +42,20 @@ ephemere/
### TypeScript Scripts ### TypeScript Scripts
- All TypeScript scripts must follow Naomi's Node.js project standards - All TypeScript scripts must follow Naomi's Node.js project standards
- Use `@nhcarrigan/typescript-config` and `@nhcarrigan/eslint-config` - Use `@nhcarrigan/typescript-config` and `@nhcarrigan/eslint-config`
- Run scripts using the Makefile: `make run-ts src/path/to/script.ts` - Run scripts using the interactive runner: `make run` (select TypeScript → category → script)
- Interactive scripts should use `@inquirer/prompts` for user input - Interactive scripts should use `@inquirer/prompts` for user input
### Python Scripts ### Python Scripts
- Use `uv` for package management - Use `uv` for package management
- Linting and formatting with `ruff` - Linting and formatting with `ruff`
- Run scripts using the Makefile: `make run-py script_name.py` - Scripts live in subdirectories of `python/` (e.g. `python/cohort/`)
- Run scripts using the interactive runner: `make run` (select Python → category → script)
### Bash Scripts
- Scripts live in subdirectories of `bash/` (e.g. `bash/cohort/`, `bash/yubikey/`)
- Run scripts using the interactive runner: `make run` (select Bash → category → script)
- Or run directly: `bash bash/<category>/<script>.sh`
- Bash scripts do not use 1Password injection (they use `gh` CLI auth or system tools)
### S3 Scripts Specifics ### S3 Scripts Specifics
All S3 scripts in `typescript/src/s3/` follow these patterns: All S3 scripts in `typescript/src/s3/` follow these patterns:
@@ -56,22 +68,24 @@ All S3 scripts in `typescript/src/s3/` follow these patterns:
## Running Scripts ## Running Scripts
Always use the Makefile commands to run scripts (they handle 1Password integration): Always use the interactive runner to run scripts (it handles 1Password integration for TypeScript and Python):
```bash
# TypeScript scripts
make run-ts src/s3/deleteContents.ts
make run-ts src/discord/bot.ts
# Python scripts
make run-py analyse_availability.py
```
Or use the interactive runner:
```bash ```bash
make run # Interactive menu to select language, category, and script make run # Interactive menu to select language, category, and script
``` ```
Or run directly:
```bash
# TypeScript (from project root)
cd typescript && op run --env-file=../prod.env -- pnpm tsx src/<category>/<script>.ts
# Python (from project root)
cd python && op run --env-file=../prod.env -- uv run python <category>/<script>.py
# Bash (no 1Password needed)
bash bash/<category>/<script>.sh
```
## Script Patterns ## Script Patterns
### TypeScript Script Template ### TypeScript Script Template
@@ -232,7 +246,7 @@ The `prod.env` file contains 1Password vault references (like `op://Private/Hetz
5. Consider adding audit logs for security operations 5. Consider adding audit logs for security operations
### Adding a new Python script ### Adding a new Python script
1. Create the script in `python/` 1. Create the script in the appropriate subdirectory of `python/` (e.g. `python/cohort/`)
2. Use type hints for all functions and variables 2. Use type hints for all functions and variables
3. Follow PEP 8 style guide (enforced by ruff) 3. Follow PEP 8 style guide (enforced by ruff)
4. Add the script to `requirements.txt` if it needs new dependencies 4. Add the script to `requirements.txt` if it needs new dependencies
@@ -247,11 +261,12 @@ The `prod.env` file contains 1Password vault references (like `op://Private/Hetz
5. Add unit tests if the utility is complex 5. Add unit tests if the utility is complex
### Adding a new script category ### Adding a new script category
1. Create a new directory under `typescript/src/` or in `python/` 1. Create a new directory under `typescript/src/`, `python/`, or `bash/` as appropriate
2. Follow the naming convention (lowercase, descriptive) 2. Follow the naming convention (lowercase, descriptive)
3. Create at least one example script showing the pattern 3. Create at least one example script showing the pattern
4. Update this CLAUDE.md with specific guidelines for the category 4. Create a `README.md` in the new directory documenting each script
5. Add any new environment variables to prod.env with 1Password references 5. Update this CLAUDE.md with specific guidelines for the category
6. Add any new environment variables to prod.env with 1Password references
## Testing ## Testing
DOCS_TODO.md
-86
View File
@@ -1,86 +0,0 @@
# Documentation TODO
## Plan
Add a `README.md` to each script category folder. Each README should document every script in that folder with:
- What the script does (1-2 sentences)
- Data files required (filename, what it contains, where to put it - top-level `data/`)
- Environment variables required
## Categories to Document
### TypeScript
- `typescript/src/crowdin/README.md`
- `clearHiddenTranslations.ts`
- `reapplyTranslations.ts`
- `writeData.ts`
- `typescript/src/discord/README.md`
- `cycThreads.ts`
- `guildCount.ts`
- `typescript/src/discourse/README.md`
- `bulkUpdateCategories.ts`
- `closeOldTopics.ts`
- `typescript/src/gitea/README.md`
- `deleteFromAllRepos.ts`
- `uploadToAllRepos.ts`
- `uploadToReposConditionally.ts`
- `typescript/src/github/README.md`
- `auditNpmPackages.ts`
- `onboardMentee.ts`
- `postUserStories.ts`
- `typescript/src/music/README.md`
- `id3v2.ts`
- `typescript/src/s3/README.md`
- `bulkUpload.ts`
- `correctContentType.ts`
- `deleteContents.ts`
- `upload.ts`
- `typescript/src/security/README.md`
- `generateReport.ts`
### Python
- `python/cohort/README.md`
- `add_github_team_members.py`
- `analyse_availability.py`
- `assign_cohort_role.py`
- `assign_team_roles.py`
- `catch_up_report.py`
- `check_channel_permissions.py`
- `check_lengths.py`
- `check_member_status.py`
- `create_team_voice_channels.py`
- `discord_activity_checker.py`
- `evaluate_technical_proficiency.py`
- `fetch_roster.py`
- `fix_channel_permissions.py`
- `generate_member_files.py`
- `generate_timeslots.py`
- `get_cohort_members.py`
- `list_all_guild_roles.py`
- `list_discord_roles.py`
- `remove_discord_roles.py`
- `remove_member.py`
- `remove_resigned_members.py`
- `send_activity_report.py`
- `send_checkin.py`
- `send_team_checkin.py`
- `send_team_messages.py`
- `update_cohort_leads_permissions.py`
- `update_roster_messages.py`
- `verify_discord.py`
## Notes
- All data files go in the top-level `data/` directory
- Python scripts resolve `data/` via `DATA_DIR = Path(__file__).parent.parent.parent / "data"`
- TypeScript scripts resolve `data/` via `join(import.meta.dirname, "..", "..", "data")`
- Each README should have a quick "Getting Started" section explaining how to run scripts (via `run.sh` or the Makefile)
README.md
+64 -86
View File
@@ -1,22 +1,31 @@
# Ephemere # Ephemere
A collection of ephemeral scripts for various tasks, written in TypeScript and Python. A collection of ephemeral scripts for various tasks, written in TypeScript, Python, and Bash.
## Project Structure ## Project Structure
``` ```
. .
├── typescript/ # TypeScript project ├── typescript/ # TypeScript scripts
│ ├── src/ # TypeScript source files │ └── src/
│ ├── package.json │ ├── crowdin/ # Crowdin translation management
│ ├── tsconfig.json │ ├── discord/ # Discord bot utilities
│ └── eslint.config.js │ ├── discourse/ # Discourse forum management
├── python/ # Python project │ ├── gitea/ # Gitea bulk repository operations
│ ├── *.py # Python scripts │ ├── github/ # GitHub API interactions
│ ├── pyproject.toml │ ├── music/ # Music file metadata tools
│ └── requirements.txt │ ├── s3/ # S3-compatible object storage
├── Makefile # Build commands for both projects │ ├── security/ # Security analysis and reporting
└── README.md │ └── utils/ # Shared utilities
├── python/
│ └── cohort/ # NHCarrigan cohort programme management
├── bash/
│ ├── cohort/ # GitHub team management for cohorts
│ └── yubikey/ # YubiKey SSH key and permission utilities
├── data/ # Input/output data files (gitignored)
├── prod.env # 1Password vault references (safe to commit)
├── run.sh # Interactive script runner
└── Makefile # Build and utility commands
``` ```
## Setup ## Setup
@@ -25,105 +34,74 @@ A collection of ephemeral scripts for various tasks, written in TypeScript and P
- Node.js (v24+) with nvm - Node.js (v24+) with nvm
- Python 3.10+ - Python 3.10+
- pnpm 10.15.0 - pnpm 10.15.0+
- uv (Python package manager) - uv (Python package manager)
- 1Password CLI (for secrets management) - 1Password CLI (`op`) — for secret injection into TypeScript and Python scripts
- `gh` CLI — for Bash scripts that manage GitHub teams
- `ykman` and `yubico-piv-tool` — for YubiKey Bash scripts
### Installation ### Installation
Install all dependencies (TypeScript and Python): Install all dependencies (TypeScript and Python):
```bash ```bash
make install make install
``` ```
Or install individually: Or install individually:
```bash ```bash
make install-ts # TypeScript dependencies only make install-ts # TypeScript dependencies only
make install-py # Python dependencies only make install-py # Python dependencies only
``` ```
## Running Scripts
The recommended way to run any script is the interactive runner:
```bash
make run
```
This launches a menu to select a language (TypeScript, Python, or Bash), then a category, then a specific script. TypeScript and Python scripts have secrets injected automatically from `prod.env` via 1Password CLI. Bash scripts are run directly.
To run a script directly without the interactive runner:
```bash
# TypeScript
cd typescript && op run --env-file=../prod.env -- pnpm tsx src/<category>/<script>.ts
# Python
cd python && op run --env-file=../prod.env -- uv run python cohort/<script>.py
# Bash
bash bash/<category>/<script>.sh
```
Each category has its own `README.md` with details on every script, its required environment variables, and data file formats.
## Development ## Development
### TypeScript Scripts ### Linting and Formatting
TypeScript scripts are located in the `typescript/src/` directory. To run a TypeScript script with environment variables:
```bash ```bash
# From the root directory make lint # Run all linters (TypeScript and Python)
make run-ts src/s3/upload.ts make lint-ts # TypeScript linter only
make lint-py # Python linter only
# Or manually from typescript directory make build # TypeScript type check
cd typescript make format # Format Python code
pnpm start path/to/script.ts make format-check # Check Python formatting without modifying
make test # Run tests
make clean # Clean build artifacts and caches
make help # Show all available commands
``` ```
### Python Scripts
Python scripts are located in the `python/` directory. To run a Python script with environment variables:
```bash
# From the root directory
make run-py analyse_availability.py
# Or manually from python directory
cd python
uv run python script_name.py
```
## Linting and Formatting
```bash
# Run all linters (TypeScript and Python)
make lint
# Run linters individually
make lint-ts # TypeScript linter
make lint-py # Python linter
# Build TypeScript (type check)
make build
# Format Python code
make format
# Check Python formatting without modifying
make format-check
# Run tests
make test
# Clean build artifacts and caches
make clean
# Show all available commands
make help
```
## CI/CD
The GitHub Actions workflow runs the following checks:
1. **Dependency pin check** - Ensures all dependencies are pinned to exact versions
2. **TypeScript checks**:
- ESLint
- TypeScript build (type checking)
- Tests
3. **Python checks**:
- Ruff linting
- Ruff format checking
## Secrets Management ## Secrets Management
This project uses 1Password CLI for secrets management. Environment variables are stored in `prod.env` as 1Password vault references. This project uses 1Password CLI for secrets management. The `prod.env` file contains 1Password vault references (e.g. `op://Private/Discord/token`) rather than real values, making it safe to commit.
The `make run-ts` and `make run-py` commands automatically inject secrets from 1Password: The interactive runner (`make run`) handles secret injection automatically. To run a script manually with secrets:
```bash
# These commands include 1Password integration
make run-ts src/discord/bot.ts
make run-py evaluate_technical_proficiency.py
```
To manually run scripts with secrets:
```bash ```bash
op run --env-file=prod.env -- <command> op run --env-file=prod.env -- <command>
``` ```
bash/cohort/README.md
+75
View File
@@ -0,0 +1,75 @@
# Cohort Bash Scripts
Shell scripts for managing GitHub team membership during the NHCarrigan spring cohort programme. These scripts handle one-off team changes that are too complex or bulk-oriented to do manually through the GitHub web interface.
All scripts use the `gh` CLI for GitHub API calls. Run `gh auth login` before using them.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: Bash → cohort → <script>
```
Or run directly:
```bash
bash bash/cohort/<script-name>.sh
```
## Table of Contents
- [remove\_github\_members.sh](#remove_github_memberssh)
- [update\_github\_teams.sh](#update_github_teamssh)
---
## remove_github_members.sh
Removes a hardcoded list of inactive members from their GitHub organisation teams in the `nhcarrigan-spring-2026-cohort` organisation. Covers both standard team membership and `-leaders` sub-team membership where applicable.
### Usage
```bash
bash bash/cohort/remove_github_members.sh
```
### Environment Variables
None. Uses `gh` CLI authentication — run `gh auth login` first.
### Data Files
None. Member usernames and team slugs are hardcoded in the script.
### Notes
- The member list and team assignments are specific to a point-in-time removal event. Update the script with the correct usernames before each use.
- Each removal command uses `|| true` so a single failure (e.g. member already removed) does not abort the entire script.
---
## update_github_teams.sh
Orchestrates a multi-step GitHub team restructure: removes all members from a dissolved team, clears its leaders sub-team, then adds each member to their new team. Also promotes a member to leader in their new team.
### Usage
```bash
bash bash/cohort/update_github_teams.sh
```
### Environment Variables
None. Uses `gh` CLI authentication — run `gh auth login` first.
### Data Files
None. All member usernames, team slugs, and role assignments are hardcoded in the script.
### Notes
- This script is specific to a one-off team restructure (Jade Jasmine dissolution). Update the member list and team assignments before each use.
- The script exits immediately on any error (`set -e`). If a step fails, check whether the member or team already exists in the target state.
bash/yubikey/README.md
+109
View File
@@ -0,0 +1,109 @@
# YubiKey Scripts
Shell scripts for managing YubiKey hardware security keys on WSL (Windows Subsystem for Linux). Covers SSH key extraction, Git signing key configuration, and fixing USB permission issues that commonly arise in WSL environments.
All scripts require a YubiKey to be attached and forwarded to WSL via `usbipd`. The `ykman` and `yubico-piv-tool` packages must be installed.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: Bash → yubikey → <script>
```
Or run directly:
```bash
bash bash/yubikey/<script-name>.sh
```
## Table of Contents
- [add-keys-to-git.sh](#add-keys-to-gitsh)
- [fix-yubikey-perms.sh](#fix-yubikey-permssh)
- [list-yubikey-ssh-keys.sh](#list-yubikey-ssh-keyssh)
---
## add-keys-to-git.sh
Extracts the SSH public keys from three YubiKey PIV slots and writes them as Git commit signing keys to the corresponding per-context Git config files. Run this after replacing or re-provisioning a YubiKey.
| Slot | Context | Config file |
|---|---|---|
| 9a | Personal | `~/.git-naomi` |
| 9c | Deepgram | `~/.git-dg` |
| 9e | FreeCodeCamp | `~/.git-fcc` |
### Usage
```bash
bash bash/yubikey/add-keys-to-git.sh
```
### Environment Variables
None.
### Data Files
None.
### Notes
- After running, you must upload the new public keys to GitHub (and any other services that verify commit signatures) manually.
- Requires `ykman` and `ssh-keygen` to be available in your PATH.
---
## fix-yubikey-perms.sh
Repairs YubiKey connectivity in WSL by fixing USB device permissions, restarting smart card services, and applying a polkit policy override that allows smart card access in WSL's "inactive" session context.
### Usage
```bash
bash bash/yubikey/fix-yubikey-perms.sh
```
### Environment Variables
None.
### Data Files
None.
### Notes
- Run this script when `ykman` or `yubico-piv-tool` fail with "Failed to connect" or similar errors after attaching the YubiKey via `usbipd`.
- The polkit fix modifies `/usr/share/polkit-1/actions/org.debian.pcsc-lite.policy` (a backup is created automatically on first run).
- Requires `sudo` access. Several steps use `sudo` to modify system files and restart services.
- Requires `lsusb`, `yubico-piv-tool`, `systemctl`, and `gpgconf` to be available.
---
## list-yubikey-ssh-keys.sh
Scans PIV slots 9a, 9c, 9d, and 9e on the connected YubiKey and prints any SSH public keys found, along with the certificate subject label if one is present.
### Usage
```bash
bash bash/yubikey/list-yubikey-ssh-keys.sh
```
### Environment Variables
None.
### Data Files
None.
### Notes
- Requires `ykman`, `ssh-keygen`, and `openssl` to be available.
- Writes a temporary file to `/tmp/yubi_tmp.pem` during execution; it is cleaned up automatically after each slot is processed.
python/cohort/README.md
+883
View File
@@ -0,0 +1,883 @@
# Cohort Scripts
Scripts for managing the NHCarrigan spring cohort programme. Covers the full lifecycle: applicant evaluation, Discord server setup, team assignment, member onboarding, activity tracking, and member removal.
Most scripts interact with the Discord API and require a `DISCORD_BOT_TOKEN`. Several also use the `gh` CLI for GitHub operations.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: Python → cohort → <script>
```
Or run directly:
```bash
cd python && op run --env-file=../prod.env -- uv run python cohort/<script>.py
```
**Prerequisites:**
- Run `make install-py` to set up the Python virtual environment.
- Most scripts require `DISCORD_BOT_TOKEN` set in `prod.env`.
- Scripts that manage GitHub teams require `gh auth login` to be run first.
## Table of Contents
### Applicant Evaluation
- [verify_discord.py](#verify_discordpy)
- [evaluate_technical_proficiency.py](#evaluate_technical_proficiencypy)
- [analyse_availability.py](#analyse_availabilitypy)
- [generate_member_files.py](#generate_member_filespy)
- [generate_timeslots.py](#generate_timeslotspy)
### Server Setup
- [create_team_voice_channels.py](#create_team_voice_channelspy)
- [fix_channel_permissions.py](#fix_channel_permissionspy)
- [update_cohort_leads_permissions.py](#update_cohort_leads_permissionspy)
- [list_all_guild_roles.py](#list_all_guild_rolespy)
- [list_discord_roles.py](#list_discord_rolespy)
- [check_channel_permissions.py](#check_channel_permissionspy)
### Member Onboarding
- [send_team_messages.py](#send_team_messagespy)
- [assign_cohort_role.py](#assign_cohort_rolepy)
- [assign_team_roles.py](#assign_team_rolespy)
- [add_github_team_members.py](#add_github_team_memberspy)
### Ongoing Management
- [get_cohort_members.py](#get_cohort_memberspy)
- [check_member_status.py](#check_member_statuspy)
- [fetch_roster.py](#fetch_rosterpy)
- [update_roster_messages.py](#update_roster_messagespy)
- [send_checkin.py](#send_checkinpy)
- [send_team_checkin.py](#send_team_checkinpy)
- [send_team_messages.py](#send_team_messagespy)
### Activity Tracking
- [discord_activity_checker.py](#discord_activity_checkerpy)
- [catch_up_report.py](#catch_up_reportpy)
- [check_lengths.py](#check_lengthspy)
- [send_activity_report.py](#send_activity_reportpy)
### Member Removal
- [remove_member.py](#remove_memberpy)
- [remove_resigned_members.py](#remove_resigned_memberspy)
- [remove_discord_roles.py](#remove_discord_rolespy)
---
## verify_discord.py
Reads Discord user IDs from a markdown table and verifies each one against the Discord API, checking whether the user is a member of the freeCodeCamp guild. Handles rate limits and retries automatically.
### Usage
```bash
make run
# Select: Python → cohort → verify_discord.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `table.md` | Markdown table | Applicant data; the script reads Discord IDs from the table rows |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `discord_verification.json` | JSON object | Results grouped as `verified` (with username), `missing`, and `errors` lists |
---
## evaluate_technical_proficiency.py
Evaluates each applicant's technical proficiency by analysing their GitHub profile and self-described expertise. Scores applicants as Beginner, Intermediate, or Advanced based on public repository count, follower count, language variety, and keywords in their self-assessment text.
### Usage
```bash
make run
# Select: Python → cohort → evaluate_technical_proficiency.py
```
### Environment Variables
None. Uses the public GitHub API (unauthenticated, subject to rate limiting).
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `applicants_to_evaluate.json` | JSON array | Applicant records, each with a GitHub profile URL and a self-description field |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `proficiency_evaluations.json` | JSON array | Proficiency scores (`Beginner`/`Intermediate`/`Advanced`) and detected tech stacks per applicant |
### Notes
- Unauthenticated GitHub API requests are rate-limited to 60 per hour. For large batches of applicants, consider adding a GitHub token or running the script in smaller chunks.
---
## analyse_availability.py
Parses a markdown table of applicant availability responses with timezone information, converts local timeslot selections to UTC, and produces a JSON summary of UTC coverage across morning, afternoon, evening, and night blocks per applicant.
### Usage
```bash
make run
# Select: Python → cohort → analyse_availability.py
```
### Environment Variables
None.
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `table.md` | Markdown table | Availability responses with timezone column |
| `discord_verification.json` | JSON object | Output of `verify_discord.py` — used to link applicants to Discord usernames |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `availability_analysis.json` | JSON object | UTC block distribution (morning/afternoon/evening/night) per applicant |
---
## generate_member_files.py
Consolidates all evaluation data into two markdown files: one for participants and one for team leaders. Each entry includes the member's tech stack, availability, proficiency level, and (for leaders) their leadership assessment.
### Usage
```bash
make run
# Select: Python → cohort → generate_member_files.py
```
### Environment Variables
None.
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `discord_verification.json` | JSON object | Verified applicant Discord info |
| `proficiency_evaluations.json` | JSON array | Technical proficiency scores |
| `availability_analysis.json` | JSON object | UTC availability blocks |
| `leadership_candidates.json` | JSON array | Applicants being considered for leadership roles |
| `leadership_evaluations.json` | JSON array | Leadership assessment results |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `participants.md` | Markdown | Profile summary for each participant |
| `leaders.md` | Markdown | Profile summary for each team leader candidate |
---
## generate_timeslots.py
Generates a list of hourly timeslots for use with the [Crabfit](https://crab.fit/) scheduling tool, covering a date range at Los Angeles timezone.
### Usage
```bash
make run
# Select: Python → cohort → generate_timeslots.py
```
### Environment Variables
None.
### Data Files
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `crabfit_timeslots.json` | JSON array of ISO-format strings | One entry per hour across the configured date range |
### Notes
- The date range and timezone are hardcoded in the script. Update the start/end date constants before running.
---
## create_team_voice_channels.py
Creates private voice channels for each cohort team in a specified Discord category. Each channel is visible and joinable only by members who have that team's role.
### Usage
```bash
make run
# Select: Python → cohort → create_team_voice_channels.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Team role IDs and the target category ID are hardcoded in the script.
### Notes
- Update the team role IDs, channel names, and category ID constants in the script before running.
---
## fix_channel_permissions.py
Denies `Send Messages` and `Send Messages in Threads` permissions for the `@everyone` and `@cohort` roles on a specific Discord channel. Used to lock down a channel so only designated roles can post.
### Usage
```bash
make run
# Select: Python → cohort → fix_channel_permissions.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. The target channel ID is hardcoded in the script.
### Notes
- Update the `CHANNEL_ID` constant before running.
---
## update_cohort_leads_permissions.py
Grants the Cohort Leads role `MENTION_EVERYONE` and `PIN_MESSAGES` permissions across all 14 team channels.
### Usage
```bash
make run
# Select: Python → cohort → update_cohort_leads_permissions.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Team channel IDs are hardcoded in the script.
---
## list_all_guild_roles.py
Lists all roles in the freeCodeCamp Discord guild, highlighting team and cohort-related roles. Useful for finding role IDs to use in other scripts.
### Usage
```bash
make run
# Select: Python → cohort → list_all_guild_roles.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Output is printed to stdout.
---
## list_discord_roles.py
Lists Discord roles in the freeCodeCamp server, filtering for cohort-, leader-, and 2026-related roles.
### Usage
```bash
make run
# Select: Python → cohort → list_discord_roles.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Output is printed to stdout.
---
## check_channel_permissions.py
Audits `cohort-team-*` Discord channels to identify incorrect `Send Messages` or `Send Messages in Threads` permissions on the `@everyone` or `@cohort` roles.
### Usage
```bash
make run
# Select: Python → cohort → check_channel_permissions.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Results are printed to stdout.
---
## send_team_messages.py
Sends initial welcome and roster messages to all 14 team Discord channels, pins them, and saves the resulting message IDs, channel IDs, and role IDs to `data/team_message_ids.json`. This file is required by several other scripts.
### Usage
```bash
make run
# Select: Python → cohort → send_team_messages.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Team rosters (leaders and participants per team) |
| `applicants_to_evaluate.json` | JSON array | Applicant data used to populate roster messages |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `team_message_ids.json` | JSON object | Channel ID, pinned message ID, and role ID per team name |
### Notes
- Run this **once** at the start of the cohort. Other scripts (`send_checkin.py`, `update_roster_messages.py`, etc.) depend on `team_message_ids.json`.
---
## assign_cohort_role.py
Assigns the Cohort Discord role to all cohort participants. Reads the full list of unique member Discord IDs from `team_assignments.json` and assigns the role to each one, with exponential backoff to handle rate limits.
### Usage
```bash
make run
# Select: Python → cohort → assign_cohort_role.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Team rosters; all unique user IDs are extracted from here |
---
## assign_team_roles.py
Assigns team-specific Discord roles to all cohort participants based on their team membership in `team_assignments.json`. Uses exponential backoff for rate limit handling.
### Usage
```bash
make run
# Select: Python → cohort → assign_team_roles.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Team rosters with Discord IDs and corresponding role IDs |
---
## add_github_team_members.py
Adds cohort members to their corresponding GitHub teams in the `nhcarrigan-spring-2026-cohort` organisation. Leaders are added to both the main team and the corresponding `-leaders` sub-team; participants are added to the main team only.
### Usage
```bash
make run
# Select: Python → cohort → add_github_team_members.py
```
### Environment Variables
None. Uses the `gh` CLI for authentication (run `gh auth login` first).
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Team rosters with leader/participant distinction |
| `discord_to_github.json` | JSON object | Maps Discord user IDs to GitHub usernames |
---
## get_cohort_members.py
Fetches all Discord guild members with the Cohort role and writes their IDs, usernames, and display names to a JSON file. Also prints a formatted list to stdout.
### Usage
```bash
make run
# Select: Python → cohort → get_cohort_members.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `active_cohort_members.json` | JSON array | Objects with `id`, `username`, and `display_name` for each Cohort role member |
---
## check_member_status.py
Verifies whether specific Discord member IDs are still in the freeCodeCamp guild by querying the Discord API. Used to confirm that removed members have been successfully ejected.
### Usage
```bash
make run
# Select: Python → cohort → check_member_status.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. The member IDs to check are hardcoded in the script. Update the list before running.
---
## fetch_roster.py
Fetches pinned messages from a specific team channel to retrieve the current roster.
### Usage
```bash
make run
# Select: Python → cohort → fetch_roster.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Output is printed to stdout as JSON.
### Notes
- The target channel ID is hardcoded. Update it before running.
---
## update_roster_messages.py
Edits the pinned team roster messages in each Discord channel with the latest data from `team_assignments.json` and `discord_to_github.json`. Run this after any membership changes to keep the pinned rosters up to date.
### Usage
```bash
make run
# Select: Python → cohort → update_roster_messages.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_message_ids.json` | JSON object | Channel and message IDs per team (output of `send_team_messages.py`) |
| `team_assignments.json` | JSON object | Current team rosters |
| `discord_to_github.json` | JSON object | Discord ID → GitHub username mapping |
---
## send_checkin.py
Sends biweekly check-in prompts to all team Discord channels (except Jade Jasmine), automatically creating threads for responses. Members who do not respond face removal for inactivity.
### Usage
```bash
make run
# Select: Python → cohort → send_checkin.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_message_ids.json` | JSON object | Channel IDs per team |
---
## send_team_checkin.py
Sends a capacity check-in message to each team channel asking whether the team feels able to complete their project with their current member count, and inviting them to request support if needed.
### Usage
```bash
make run
# Select: Python → cohort → send_team_checkin.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_message_ids.json` | JSON object | Channel IDs per team |
| `team_assignments.json` | JSON object | Current team rosters (used to mention team members) |
---
## discord_activity_checker.py
Scans each team's Discord channel and threads to identify members who have not sent a message within the last 36 hours. Can optionally send notification messages directly to inactive members via the `--send` CLI flag.
### Usage
```bash
# Check only (no messages sent)
make run
# Select: Python → cohort → discord_activity_checker.py
# Check and notify inactive members
cd python && op run --env-file=../prod.env -- uv run python cohort/discord_activity_checker.py --send
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Member lists per team |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `discord_activity_report.json` | JSON object | Inactive members per team with their last message timestamp |
---
## catch_up_report.py
Generates a detailed markdown activity report covering Discord messages (in team channels and their threads) and GitHub activity (PRs, issues, comments, reviews, commits) since a configured start date. Uses async API calls for efficiency.
### Usage
```bash
make run
# Select: Python → cohort → catch_up_report.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Team rosters |
| `discord_to_github.json` | JSON object | Discord ID → GitHub username mapping |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `catch_up_report.md` | Markdown table | Activity counts per member (Discord messages + GitHub contributions) per team |
### Notes
- The report start date is hardcoded in the script. Update it before each use.
- GitHub activity is fetched via the unauthenticated public API; rate limiting may apply for large cohorts.
---
## check_lengths.py
Dry-run validation that parses `catch_up_report.md` and formats each team's data into Discord monospace table strings, checking whether any would exceed Discord's 2,000-character message limit before actually sending them.
### Usage
```bash
make run
# Select: Python → cohort → check_lengths.py
```
### Environment Variables
None.
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `catch_up_report.md` | Markdown table | Output of `catch_up_report.py` |
### Notes
- Run this after `catch_up_report.py` and before `send_activity_report.py` to catch any messages that would be rejected by Discord.
---
## send_activity_report.py
Parses `catch_up_report.md` and sends the formatted activity table for each team to its Discord channel as a monospace code block.
### Usage
```bash
make run
# Select: Python → cohort → send_activity_report.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `catch_up_report.md` | Markdown table | Output of `catch_up_report.py` |
### Notes
- Run `check_lengths.py` first to verify that no messages exceed Discord's character limit.
---
## remove_member.py
Comprehensive member removal script. Given a Discord ID as a CLI argument, it:
1. Removes the member from `team_assignments.json`.
2. Removes the member from `discord_to_github.json`.
3. Prints GitHub removal instructions.
4. Removes the member's Discord Cohort and team roles.
5. Sends an announcement message to the team's Discord channel.
6. Outputs markdown notes suitable for pasting into `COHORT_NOTES.md`.
### Usage
```bash
cd python && op run --env-file=../prod.env -- uv run python cohort/remove_member.py <discord_id>
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input/Output** (read and updated in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Updated: member removed from their team |
| `discord_to_github.json` | JSON object | Updated: Discord→GitHub mapping removed |
| `team_message_ids.json` | JSON object | Read: used to find the team's Discord channel ID |
---
## remove_resigned_members.py
Removes a list of resigned member Discord IDs from `team_assignments.json`, updating team rosters and reporting which teams were affected. Does not interact with Discord or GitHub.
### Usage
```bash
make run
# Select: Python → cohort → remove_resigned_members.py
```
### Environment Variables
None.
### Data Files
**Input/Output** (read and updated in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Updated in place; affected teams are reported to stdout |
### Notes
- The list of Discord IDs to remove is hardcoded in the script. Update it before running.
- This script only updates the local JSON file. Run `update_roster_messages.py` and handle GitHub/Discord role removal separately.
---
## remove_discord_roles.py
Removes the Cohort and team-specific Discord roles from a hardcoded list of inactive members.
### Usage
```bash
make run
# Select: Python → cohort → remove_discord_roles.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Member IDs and their team-to-role mappings are hardcoded in the script. Update them before running.
typescript/src/crowdin/README.md
+125
View File
@@ -0,0 +1,125 @@
# Crowdin Scripts
Scripts for managing translations in a Crowdin project.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → crowdin → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [writeData.ts](#writedatats)
- [clearHiddenTranslations.ts](#clearhiddentranslationsts)
- [reapplyTranslations.ts](#reapplytranslationsts)
> **Typical workflow:** Run `writeData.ts` first to fetch and cache the project data locally, then use `clearHiddenTranslations.ts` and/or `reapplyTranslations.ts` as needed.
---
## writeData.ts
Fetches all file IDs and string data from a Crowdin project and writes them to local JSON files in `data/`. These files are used as inputs by the other Crowdin scripts.
### Usage
```bash
make run
# Select: TypeScript → crowdin → writeData.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `CROWDIN_PROJECT_ID` | Crowdin project numeric ID |
| `CROWDIN_API_URL` | Base URL of the Crowdin API (e.g. `https://api.crowdin.com/api/v2`) |
| `CROWDIN_TOKEN` | Crowdin personal access token |
### Data Files
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `crowdin-files.json` | JSON array of file IDs | All file IDs in the project |
| `crowdin-strings.json` | JSON array of string objects | All strings in the project, including `id`, `isHidden`, and other metadata |
---
## clearHiddenTranslations.ts
Deletes existing translations for all hidden (suppressed) strings in a Crowdin project, across all active languages, in parallel. Keeps a log file to avoid re-processing strings on subsequent runs.
### Usage
```bash
make run
# Select: TypeScript → crowdin → clearHiddenTranslations.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `CROWDIN_PROJECT_ID` | Crowdin project numeric ID |
| `CROWDIN_API_URL` | Base URL of the Crowdin API |
| `CROWDIN_TOKEN` | Crowdin personal access token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `crowdin-strings.json` | JSON array of string objects | Output of `writeData.ts` — each object must have `id` (number) and `isHidden` (boolean) fields |
| `crowdin-strings-hidden.txt` | Plain text, one ID per line | Log of already-processed string IDs; create an empty file if starting fresh |
**Output** (updated in `data/`):
| File | Description |
|---|---|
| `crowdin-strings-hidden.txt` | Appended with the ID of each string processed in this run |
### Notes
- Run `writeData.ts` first to generate `crowdin-strings.json`.
- Create an empty `crowdin-strings-hidden.txt` in `data/` before the first run. The script will append to it as strings are processed, so re-runs skip already-cleared strings.
- Translations are deleted in parallel across all languages for each string, then the string ID is appended to the log before moving on to the next.
---
## reapplyTranslations.ts
Triggers a pre-translation run on a Crowdin project using Translation Memory (TM), applying perfect matches only, and polls for completion every 5 seconds until the job reaches 100%.
### Usage
```bash
make run
# Select: TypeScript → crowdin → reapplyTranslations.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `CROWDIN_PROJECT_ID` | Crowdin project numeric ID |
| `CROWDIN_API_URL` | Base URL of the Crowdin API |
| `CROWDIN_TOKEN` | Crowdin personal access token |
### Data Files
None. The script fetches the current file and language lists from Crowdin at runtime.
### Notes
- Pre-translation is configured with `translateWithPerfectMatchOnly: true` and `autoApproveOption: "perfectMatchOnly"`, so only exact TM matches are applied and automatically approved.
- Already-approved translations are skipped (`skipApprovedTranslations: true`).
- The script polls progress every 5 seconds and prints the percentage until the job is complete.
typescript/src/discord/README.md
+87
View File
@@ -0,0 +1,87 @@
# Discord Scripts
Scripts for Discord bot utilities and server management.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → discord → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [cycThreads.ts](#cycthreadsts)
- [guildCount.ts](#guildcountts)
---
## cycThreads.ts
Creates Discord forum threads for talk submissions in a conference channel. Iterates over a hardcoded list of talk titles and speakers, creating one public forum thread per talk with a standard discussion prompt message. Talks with missing titles or speakers are automatically filtered out.
### Usage
```bash
make run
# Select: TypeScript → discord → cycThreads.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_TOKEN` | Discord bot token |
### Data Files
None. The list of talks (title + speaker) is hardcoded in the script. The target forum channel ID is also hardcoded.
### Notes
- **Before running**, update the `data` array in the script with the current conference talk submissions and update `CHANNEL_ID` to point to the correct Discord forum channel.
- Uses the `backoffAndRetry` utility to handle Discord API rate limits automatically.
- Entries with an empty `title` or `speaker` field are silently filtered before processing.
- Threads are created with a 24-hour auto-archive duration (`auto_archive_duration: 1440`).
---
## guildCount.ts
Counts and categorises all Discord servers a user belongs to. Uses the Discord OAuth2 PKCE flow (no user token ever stored) to authenticate, fetches the user's guild list, and categorises each server as owned, admin, moderating, partnered, verified, community, or discoverable. Results are printed to the console and displayed as an HTML dashboard served on a local port (opened automatically in your browser).
### Usage
```bash
make run
# Select: TypeScript → discord → guildCount.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_CLIENT_ID` | OAuth2 application client ID (required) |
| `DISCORD_CLIENT_SECRET` | OAuth2 application client secret (optional, improves security) |
| `DISCORD_REDIRECT_URI` | OAuth2 redirect URI (default: `http://127.0.0.1:8721/callback`) |
| `DISCORD_SCOPES` | OAuth2 scopes (default: `identify guilds`) |
### Data Files
None.
### Notes
- **Setup required before first run** (per machine):
1. Create a Discord application in the [Developer Portal](https://discord.com/developers/applications) and note the Client ID.
2. Under OAuth2 → Redirects, add `http://127.0.0.1:8721/callback` (or supply your own via `DISCORD_REDIRECT_URI`).
3. Optionally generate a Client Secret and set it as `DISCORD_CLIENT_SECRET`.
4. Export `DISCORD_CLIENT_ID` (and secret if used) before running the script.
- The script prompts you to confirm setup is complete before launching the OAuth flow.
- The OAuth flow uses PKCE (Proof Key for Code Exchange) — no user tokens are ever logged or stored; this approach is fully within Discord's Terms of Service.
- Discord allows a maximum of 200 servers per user (with Nitro); the script fetches up to 200 at once.
- The HTML dashboard is served on a random available local port and opened automatically; the server closes once the page is loaded.
typescript/src/discourse/README.md
+84
View File
@@ -0,0 +1,84 @@
# Discourse Scripts
Scripts for Discourse forum management.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → discourse → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [bulkUpdateCategories.ts](#bulkupdatecategoriests)
- [closeOldTopics.ts](#closeoldtopicsts)
---
## bulkUpdateCategories.ts
Enables auto-close on all categories and subcategories in a Discourse forum. Fetches the full category list (including subcategories), then updates each one to auto-close based on last post after 672 hours (28 days).
### Usage
```bash
make run
# Select: TypeScript → discourse → bulkUpdateCategories.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCOURSE_URL` | Base URL of the Discourse instance (e.g. `https://forum.example.com`) |
| `DISCOURSE_API_KEY` | Discourse API key |
| `DISCOURSE_API_USERNAME` | Discourse API username |
### Data Files
None.
### Notes
- Auto-close is configured to trigger based on **last post** (not topic creation date), so active topics remain open.
- The auto-close threshold is hardcoded to `672` hours (28 days). Update the `auto_close_hours` value in the script to change this.
- Uses the `backoffAndRetry` utility to handle Discourse API rate limits automatically.
- Categories are processed sequentially; subcategories are fetched individually and deduplicated.
- Failures for individual categories are logged but do not stop the rest of the run.
---
## closeOldTopics.ts
Closes inactive Discourse topics that have had no activity for 28 or more days, skipping any topics older than 6 months (which are assumed to be already archived or otherwise handled). Already-closed topics are also skipped.
### Usage
```bash
make run
# Select: TypeScript → discourse → closeOldTopics.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCOURSE_URL` | Base URL of the Discourse instance (e.g. `https://forum.example.com`) |
| `DISCOURSE_API_KEY` | Discourse API key |
| `DISCOURSE_API_USERNAME` | Discourse API username |
### Data Files
None.
### Notes
- Topics are fetched from the `/latest.json` endpoint in ascending age order. Pagination stops automatically once a topic older than 6 months is encountered.
- The inactivity threshold is **28 days** and the age cutoff is **180 days** (approximately 6 months); both are defined as constants in the script.
- A 500 ms delay is added between close requests to avoid hitting Discourse rate limits. Rate-limit (HTTP 429) responses trigger an automatic 5-second wait and retry.
- At the end of the run, a summary of closed/failed counts is printed.
typescript/src/gitea/README.md
+124
View File
@@ -0,0 +1,124 @@
# Gitea Scripts
Scripts for bulk file management across Gitea repositories.
All scripts operate across three NHCarrigan Gitea organisations: `nhcarrigan`, `nhcarrigan-private`, and `nhcarrigan-games`.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → gitea → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [deleteFromAllRepos.ts](#deletefromallreposets)
- [uploadToAllRepos.ts](#uploadtoallreposets)
- [uploadToReposConditionally.ts](#uploadtoreposconditionallyts)
---
## deleteFromAllRepos.ts
Deletes a specified file from every repository across all NHCarrigan Gitea organisations. Checks each repository first; if the file exists it is deleted, otherwise the repository is skipped.
### Usage
```bash
make run
# Select: TypeScript → gitea → deleteFromAllRepos.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITEA_TOKEN` | Gitea personal access token with repository write permissions |
### Data Files
None. The file path to delete is entered interactively when the script runs.
### Notes
- You will be prompted for the file path to delete (e.g. `.gitea/workflows/security.yml`). Do **not** include a leading slash.
- The deletion commit message is automatically set to `feat: automated delete of {path}`.
- Repositories are fetched 100 at a time with automatic pagination via `paginatedFetch`.
- Failures for individual repositories are logged but do not stop the rest of the run.
---
## uploadToAllRepos.ts
Uploads a file from the local `data/` directory to every repository across all NHCarrigan Gitea organisations. If the file already exists in a repository it is updated; otherwise it is created.
### Usage
```bash
make run
# Select: TypeScript → gitea → uploadToAllRepos.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITEA_TOKEN` | Gitea personal access token with repository write permissions |
### Data Files
**Input** (place in `data/`):
| File | Format | Description |
|---|---|---|
| *(any file)* | Any | The file to upload; name and destination path are entered interactively |
### Notes
- You will be prompted for:
- **Local filename** — path relative to `data/` (e.g. `actions.yml` or `gitea/actions.yml`). The file must exist in `data/`.
- **Destination path** — path in each repository (e.g. `.gitea/workflows/actions.yml`). Do **not** include a leading slash.
- Commit messages are automatically set to `feat: automated upload of {path}`.
- A summary of processed / succeeded / failed counts is printed at the end.
---
## uploadToReposConditionally.ts
Uploads a file to Gitea repositories only if a condition file does (or does not) exist in each repository. Useful for targeting only repositories that have a specific workflow file, language marker, or configuration already in place.
### Usage
```bash
make run
# Select: TypeScript → gitea → uploadToReposConditionally.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITEA_TOKEN` | Gitea personal access token with repository write permissions |
### Data Files
**Input** (place in `data/`):
| File | Format | Description |
|---|---|---|
| *(any file)* | Any | The file to upload; name and destination path are entered interactively |
### Notes
- You will be prompted for:
- **Local filename** — path relative to `data/` (same as `uploadToAllRepos.ts`).
- **Destination path** — path in each repository.
- **Condition file path** — path to check in each repository (e.g. `package.json` or `.gitea/workflows/ci.yml`). Do **not** include a leading slash.
- **Upload condition** — whether to upload when the condition file **exists** or when it **does not exist**.
- If the condition is not met for a repository, it is skipped.
- A summary of processed / succeeded / failed / skipped counts is printed at the end.
typescript/src/github/README.md
+124
View File
@@ -0,0 +1,124 @@
# GitHub Scripts
Scripts for GitHub API interactions and organisation management.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → github → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [auditNpmPackages.ts](#auditnpmpackagets)
- [onboardMentee.ts](#onboardmenteeets)
- [postUserStories.ts](#postuserstoriests)
---
## auditNpmPackages.ts
Audits npm packages across one or more GitHub organisations for known vulnerable package versions. For each repository, it fetches `package.json` (if present) and checks `dependencies` and `devDependencies` against a hardcoded list of vulnerable packages and their specific vulnerable versions. Results are written to a text file in `data/`.
### Usage
```bash
make run
# Select: TypeScript → github → auditNpmPackages.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITHUB_TOKEN` | GitHub personal access token with organisation read permissions |
### Data Files
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `npm-vulnerabilities.txt` | Plain text, one finding per line | Repositories with vulnerable or potentially affected packages |
### Notes
- **Before running**, update the `orgsToCheck` array and the `vulnerablePackages` list in the script to match the organisations and vulnerabilities you want to audit.
- The output file is **overwritten** at the start of each run.
- Repositories without a `package.json` are skipped silently.
- The script distinguishes between finding a package at the exact vulnerable version (marked `!! FOUND VULNERABLE !!`) and finding the package at a different version (noted for awareness).
- Repositories are fetched 100 at a time with automatic pagination.
---
## onboardMentee.ts
Onboards a new mentee to the `nhcarrigan-mentorship` GitHub organisation. Interactively prompts for the mentee's Discord ID, full name, and GitHub username, then:
1. Creates a public repository in `nhcarrigan-mentorship` named after the mentee (kebab-case).
2. Adds the mentee as a collaborator with `maintain` permissions.
3. Sends a welcome message to the mentorship Discord channel tagging the mentee with their repository URL.
### Usage
```bash
make run
# Select: TypeScript → github → onboardMentee.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITHUB_TOKEN` | GitHub personal access token with org and repo write permissions |
| `DISCORD_TOKEN` | Discord bot token |
### Data Files
None.
### Notes
- The target GitHub organisation (`nhcarrigan-mentorship`) and Discord channel ID are hardcoded in the script.
- Full name is converted to kebab-case for the repository name (e.g. `Jane Doe` → `jane-doe`). Special characters other than letters, digits, spaces, and hyphens are stripped.
- Discord ID must be a numeric string.
- A summary of the onboarded mentee is printed to the console on success.
---
## postUserStories.ts
Posts markdown files as GitHub issue body content. Reads all `.md` files from `data/stories/`, parses the filename to extract the repository name and issue number, then updates the corresponding GitHub issue with the file content as its description.
### Usage
```bash
make run
# Select: TypeScript → github → postUserStories.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITHUB_TOKEN` | GitHub personal access token with repo write permissions |
### Data Files
**Input** (expected in `data/stories/`):
| File pattern | Format | Description |
|---|---|---|
| `{repo-name}-{issue-number}.md` | Markdown | User story content for a specific issue; filename determines the target repo and issue number |
### Notes
- **Before running**, update the `orgName` constant in the script to the target GitHub organisation.
- Filenames must follow the exact pattern `{repo-name}-{issue-number}.md` (e.g. `my-repo-42.md`). Files that don't match are skipped with an error message.
- The script updates the issue **description** (body), not a comment.
- A summary of successful and failed updates is printed at the end.
typescript/src/music/README.md
+48
View File
@@ -0,0 +1,48 @@
# Music Scripts
Scripts for working with music files and metadata.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → music → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [id3v2.ts](#id3v2ts)
---
## id3v2.ts
Tags a batch of MP3 files with ID3v2 metadata and album art. Designed for tagging downloaded Neuro-sama tracks: reads all `.mp3` files from a local directory, extracts the track title from the filename, sets the artist to `"Neuro-sama"`, and applies a specified cover image using the `eyeD3` and `id3v2` CLI tools. Progress is displayed via a terminal progress bar.
### Usage
```bash
make run
# Select: TypeScript → music → id3v2.ts
```
### Environment Variables
None.
### Data Files
None. The input directory and cover image path are defined as constants at the top of the script.
### Notes
- **Before running**, update the two constants at the top of the script:
- `directory` — path to the folder containing your MP3 files (default: `/home/naomi/down`)
- `cover` — path to the cover image to embed (default: `/home/naomi/neuro.png`)
- Requires the [`eyeD3`](https://eyed3.readthedocs.io/) and [`id3v2`](https://id3v2.sourceforge.net/) CLI tools to be installed on your system.
- Title is extracted from the filename by looking for text wrapped in `"..."` or `＂...＂` (both ASCII and fullwidth double quotes). If no quoted text is found, the filename (minus `.mp3`) is used as the title.
- Non-MP3 files in the directory are silently skipped.
typescript/src/s3/README.md
+163
View File
@@ -0,0 +1,163 @@
# S3 Scripts
Scripts for managing objects in an S3-compatible bucket (e.g. Hetzner Object Storage).
All scripts use the AWS SDK and share the same three environment variables.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → s3 → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [upload.ts](#uploadts)
- [bulkUpload.ts](#bulkuploAdts)
- [correctContentType.ts](#correctcontenttypets)
- [deleteContents.ts](#deletecontentsts)
---
## upload.ts
Uploads a single file from the local `data/` directory to the S3 bucket. Prompts for the local filename and the destination key (path) in the bucket. The MIME type is detected automatically from the file extension.
### Usage
```bash
make run
# Select: TypeScript → s3 → upload.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
| `S3_ENDPOINT` | S3-compatible endpoint URL |
### Data Files
**Input** (place in `data/`):
| File | Format | Description |
|---|---|---|
| *(any file)* | Any | The file to upload; name and destination path are entered interactively |
### Notes
- You will be prompted for:
- **Local filename** — path relative to `data/` (e.g. `naomi.png` or `img/naomi.png`). The file must exist in `data/`.
- **Destination path** — the key to use in the bucket (e.g. `img/naomi.png`). Do **not** include a leading slash.
- If the file extension is unrecognised, a warning is printed and the object is uploaded without a `Content-Type` header.
- The bucket name is hardcoded as `nhcarrigan`. Update the `Bucket` constant in the script to target a different bucket.
---
## bulkUpload.ts
Uploads all files in the `data/` directory (recursively) to the S3 bucket. Before uploading, it displays a tree view of all files to be uploaded and asks for confirmation. A progress bar tracks upload progress.
### Usage
```bash
make run
# Select: TypeScript → s3 → bulkUpload.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
| `S3_ENDPOINT` | S3-compatible endpoint URL |
### Data Files
**Input** (place in `data/`):
| File | Format | Description |
|---|---|---|
| *(all files)* | Any | Every file under `data/` is uploaded, preserving the relative directory structure as the S3 key |
### Notes
- The destination key for each file mirrors its path relative to `data/` (e.g. `data/img/naomi.png` → S3 key `img/naomi.png`).
- MIME types are detected from file extensions. Files with unrecognised extensions are uploaded without a `Content-Type` header.
- You must confirm the upload by typing `y` at the prompt; the default is `n` (cancel).
- Errors for individual files are logged but do not stop the rest of the run. A final summary of succeeded/failed counts is printed.
- The bucket name is hardcoded as `nhcarrigan`.
---
## correctContentType.ts
Audits all objects in the S3 bucket and interactively corrects their `Content-Type` metadata where it is missing, set to `application/octet-stream`, or does not match the expected MIME type for the file extension. Correction is performed by copying the object over itself with the new metadata (no data transfer, metadata-only update).
### Usage
```bash
make run
# Select: TypeScript → s3 → correctContentType.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
| `S3_ENDPOINT` | S3-compatible endpoint URL |
### Data Files
None.
### Notes
- For each object with an incorrect or missing `Content-Type`, you will be asked (with a default of `yes`) whether to update it. This allows skipping specific files if needed.
- Directory marker objects (keys ending in `/`) are automatically skipped.
- Objects with unrecognised extensions are also skipped (no known expected MIME type).
- A final summary of corrected / skipped / errored counts is printed.
- The bucket name is hardcoded as `nhcarrigan`.
---
## deleteContents.ts
Deletes **all** objects from a specified S3 bucket. Requires double confirmation before proceeding: first you must type the exact bucket name, then confirm with a yes/no prompt. Objects are deleted in batches of 1,000 with a progress bar.
### Usage
```bash
make run
# Select: TypeScript → s3 → deleteContents.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
| `S3_ENDPOINT` | S3-compatible endpoint URL |
### Data Files
None.
### Notes
- **This operation is irreversible.** All objects in the bucket will be permanently deleted.
- The script prompts for the bucket name (with basic validation), then requires you to type the bucket name again exactly as a first confirmation, followed by a yes/no prompt defaulting to `no`.
- If the bucket is already empty the script exits immediately without prompting for confirmation.
- Deletion uses the S3 batch delete API (up to 1,000 objects per request) for efficiency.
- A final summary of succeeded/failed counts is printed.
typescript/src/security/README.md
+53
View File
@@ -0,0 +1,53 @@
# Security Scripts
Scripts for security analysis and reporting.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → security → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [generateReport.ts](#generatereportts)
---
## generateReport.ts
Generates a public HTML security transparency dashboard from [DefectDojo](https://www.defectdojo.org/) findings. Fetches all active, verified findings via the DefectDojo API (handling pagination), maps each finding to its product, aggregates counts by severity (Critical / High / Medium / Low), and writes a styled HTML report to `data/public_security_report.html`.
### Usage
```bash
make run
# Select: TypeScript → security → generateReport.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `DOJO_TOKEN` | DefectDojo personal API token |
### Data Files
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `public_security_report.html` | HTML | Styled security transparency dashboard, grouped by project with severity counts |
### Notes
- The DefectDojo instance URL is hardcoded to `https://security.nhcarrigan.com`. Update the `dojoUrl` constant in the script to point to a different instance.
- Both findings and products are fetched with pagination (up to 1,000 per page); the script handles multiple pages automatically.
- Findings without a product assigned are skipped with a warning.
- Project names are stripped of their org prefix (`nhcarrigan/website-headers` → `Website Headers`) and formatted to title case for display.
- The output HTML is self-contained and references the NHCarrigan global headers script (`https://cdn.nhcarrigan.com/headers/index.js`) for theming.