Compare commits

6 Commits

Author SHA1 Message Date
naomi 892decb654 fix: broken .env vals
CI / dependency-pin-check-typescript (push) Successful in 5s
CI / dependency-pin-check-python (push) Successful in 5s
Security Scan and Upload / Security & DefectDojo Upload (push) Successful in 1m6s
CI / typescript (push) Failing after 4m50s
CI / python (push) Failing after 4m51s
2026-05-18 10:49:58 -07:00
naomi 3aa90fa316 feat: add script for if
CI / dependency-pin-check-typescript (push) Successful in 4s
CI / dependency-pin-check-python (push) Successful in 4s
Security Scan and Upload / Security & DefectDojo Upload (push) Successful in 1m7s
CI / typescript (push) Failing after 4m48s
CI / python (push) Failing after 4m54s
2026-05-07 15:52:18 -07:00
hikari 1617b8599d feat(bash): add env-to-1pass script for importing .env files to 1Password
CI / dependency-pin-check-python (push) Successful in 4s
CI / dependency-pin-check-typescript (push) Successful in 4s
CI / typescript (push) Failing after 4m46s
CI / python (push) Successful in 9m33s
Security Scan and Upload / Security & DefectDojo Upload (push) Successful in 2m45s
2026-03-24 17:20:46 -07:00
hikari 1fb18f95b8 chore: replace .npmrc with pnpm-workspace.yaml
CI / dependency-pin-check-typescript (push) Successful in 7s
CI / dependency-pin-check-python (push) Successful in 8s
CI / typescript (push) Failing after 4m49s
CI / python (push) Successful in 9m31s
Security Scan and Upload / Security & DefectDojo Upload (push) Successful in 2m42s
2026-03-02 16:28:32 -08:00
hikari ec58c9c843 feat: reorganise bash scripts and add comprehensive documentation (#6)
CI / dependency-pin-check-typescript (push) Successful in 5s
CI / dependency-pin-check-python (push) Successful in 4s
CI / python (push) Successful in 9m28s
CI / typescript (push) Successful in 9m42s
Security Scan and Upload / Security & DefectDojo Upload (push) Successful in 1m39s
## Summary

This PR completes the bash script restructuring and adds comprehensive documentation across all script categories.

### Bash Restructuring

- Moved cohort shell scripts (`remove_github_members.sh`, `update_github_teams.sh`) from `python/cohort/` into a new `bash/cohort/` directory
- Moved existing bash utilities (`add-keys-to-git.sh`, `fix-yubikey-perms.sh`, `list-yubikey-ssh-keys.sh`) into a new `bash/yubikey/` subdirectory
- Updated `run.sh` to support **Bash** as a third language option alongside TypeScript and Python
  - Bash scripts are run directly (no 1Password secret injection needed)
  - Category discovery and script listing works the same as for TS/Python
  - Removed dead "Root Scripts" logic that was no longer needed

### Documentation

Added `README.md` files for all script categories that were missing them:

- `bash/cohort/README.md` — cohort GitHub team management scripts
- `bash/yubikey/README.md` — YubiKey SSH key and permission utilities
- `typescript/src/crowdin/README.md` — Crowdin translation management scripts
- `typescript/src/discord/README.md` — Discord bot utility scripts
- `typescript/src/discourse/README.md` — Discourse forum management scripts
- `typescript/src/gitea/README.md` — Gitea bulk repository operation scripts
- `typescript/src/github/README.md` — GitHub API interaction scripts
- `typescript/src/music/README.md` — Music file metadata tools
- `typescript/src/s3/README.md` — S3-compatible object storage scripts
- `typescript/src/security/README.md` — Security analysis and reporting scripts
- `python/cohort/README.md` — Updated to remove moved shell scripts, fix usage commands

Also updated project-level docs:

- **`README.md`** — Corrected project structure, fixed running instructions (removed references to non-existent `make run-ts`/`make run-py` targets), added Bash prerequisites
- **`CLAUDE.md`** — Updated project overview, structure, development standards, and script-adding guides to reflect the current state of the project

 This PR was created with help from Hikari~ 🌸

Co-authored-by: Naomi Carrigan <commits@nhcarrigan.com>
Reviewed-on: #6
Co-authored-by: Hikari <hikari@nhcarrigan.com>
Co-committed-by: Hikari <hikari@nhcarrigan.com>
2026-02-23 20:18:41 -08:00
naomi e481823e06 feat: add mentorship onboarding (#5)
CI / dependency-pin-check-typescript (push) Successful in 4s
CI / dependency-pin-check-python (push) Successful in 4s
CI / python (push) Successful in 9m23s
CI / typescript (push) Successful in 9m48s
Security Scan and Upload / Security & DefectDojo Upload (push) Successful in 1m51s
### Explanation

_No response_

### Issue

_No response_

### Attestations

- [ ] I have read and agree to the [Code of Conduct](https://docs.nhcarrigan.com/community/coc/)
- [ ] I have read and agree to the [Community Guidelines](https://docs.nhcarrigan.com/community/guide/).
- [ ] My contribution complies with the [Contributor Covenant](https://docs.nhcarrigan.com/dev/covenant/).

### Dependencies

- [ ] I have pinned the dependencies to a specific patch version.

### Style

- [ ] I have run the linter and resolved any errors.
- [ ] My pull request uses an appropriate title, matching the conventional commit standards.
- [ ] My scope of feat/fix/chore/etc. correctly matches the nature of changes in my pull request.

### Tests

- [ ] My contribution adds new code, and I have added tests to cover it.
- [ ] My contribution modifies existing code, and I have updated the tests to reflect these changes.
- [ ] All new and existing tests pass locally with my changes.
- [ ] Code coverage remains at or above the configured threshold.

### Documentation

_No response_

### Versioning

_No response_

Reviewed-on: #5
Co-authored-by: Naomi Carrigan <commits@nhcarrigan.com>
Co-committed-by: Naomi Carrigan <commits@nhcarrigan.com>
2026-02-03 11:03:18 -08:00
70 changed files with 5553 additions and 188 deletions
-25
View File
@@ -1,25 +0,0 @@
# Package Manager Configuration
# Force pnpm usage - breaks npm/yarn intentionally
node-linker=pnpm
# Security: Disable all lifecycle scripts
ignore-scripts=true
enable-pre-post-scripts=false
# Security: Require packages to be 10+ days old before installation
minimum-release-age=14400
# Security: Verify package integrity hashes
verify-store-integrity=true
# Security: Enforce strict trust policies
trust-policy=strict
# Security: Strict peer dependency resolution
strict-peer-dependencies=true
# Performance: Use symlinks for node_modules
symlink=true
# Lockfile: Ensure lockfile is not modified during install
frozen-lockfile=false
+45 -30
View File
@@ -4,31 +4,36 @@ This document contains project-specific instructions for working with the Epheme
## Project Overview ## Project Overview
Ephemere is a collection of ephemeral scripts for various tasks, written in TypeScript and Python. It contains utilities for: Ephemere is a collection of ephemeral scripts for various tasks, written in TypeScript, Python, and Bash. It contains utilities for:
- S3 operations (upload, bulk upload, delete, content type correction) - S3 operations (upload, bulk upload, delete, content type correction)
- Discord bot utilities - Discord bot utilities
- Discourse forum management - Discourse forum management
- Gitea/GitHub operations - Gitea/GitHub operations
- Security analysis tools - Security analysis tools
- Music-related scripts - Music-related scripts
- Various utility functions - Cohort programme management (Python + Bash)
- YubiKey SSH key and permission utilities (Bash)
## Project Structure ## Project Structure
``` ```
ephemere/ ephemere/
├── typescript/ # TypeScript scripts ├── typescript/ # TypeScript scripts
── src/ ── src/
├── s3/ # S3 operations (upload, delete, bulk operations) ├── s3/ # S3 operations (upload, delete, bulk operations)
├── discord/ # Discord bot and utilities ├── discord/ # Discord bot and utilities
├── discourse/ # Discourse forum management ├── discourse/ # Discourse forum management
├── gitea/ # Gitea API interactions ├── gitea/ # Gitea API interactions
├── github/ # GitHub API interactions ├── github/ # GitHub API interactions
├── security/ # Security analysis tools ├── security/ # Security analysis tools
├── music/ # Music-related utilities ├── music/ # Music-related utilities
└── utils/ # Shared utilities └── utils/ # Shared utilities
│ └── data/ # Data files for S3 uploads ├── python/
├── python/ # Python scripts │ └── cohort/ # Cohort programme management scripts
├── bash/
│ ├── cohort/ # GitHub team management for cohorts
│ └── yubikey/ # YubiKey SSH key and permission utilities
├── data/ # Input/output data files (gitignored)
└── prod.env # 1Password vault references (safe to commit) └── prod.env # 1Password vault references (safe to commit)
``` ```
@@ -37,13 +42,20 @@ ephemere/
### TypeScript Scripts ### TypeScript Scripts
- All TypeScript scripts must follow Naomi's Node.js project standards - All TypeScript scripts must follow Naomi's Node.js project standards
- Use `@nhcarrigan/typescript-config` and `@nhcarrigan/eslint-config` - Use `@nhcarrigan/typescript-config` and `@nhcarrigan/eslint-config`
- Run scripts using the Makefile: `make run-ts src/path/to/script.ts` - Run scripts using the interactive runner: `make run` (select TypeScript → category → script)
- Interactive scripts should use `@inquirer/prompts` for user input - Interactive scripts should use `@inquirer/prompts` for user input
### Python Scripts ### Python Scripts
- Use `uv` for package management - Use `uv` for package management
- Linting and formatting with `ruff` - Linting and formatting with `ruff`
- Run scripts using the Makefile: `make run-py script_name.py` - Scripts live in subdirectories of `python/` (e.g. `python/cohort/`)
- Run scripts using the interactive runner: `make run` (select Python → category → script)
### Bash Scripts
- Scripts live in subdirectories of `bash/` (e.g. `bash/cohort/`, `bash/yubikey/`)
- Run scripts using the interactive runner: `make run` (select Bash → category → script)
- Or run directly: `bash bash/<category>/<script>.sh`
- Bash scripts do not use 1Password injection (they use `gh` CLI auth or system tools)
### S3 Scripts Specifics ### S3 Scripts Specifics
All S3 scripts in `typescript/src/s3/` follow these patterns: All S3 scripts in `typescript/src/s3/` follow these patterns:
@@ -56,22 +68,24 @@ All S3 scripts in `typescript/src/s3/` follow these patterns:
## Running Scripts ## Running Scripts
Always use the Makefile commands to run scripts (they handle 1Password integration): Always use the interactive runner to run scripts (it handles 1Password integration for TypeScript and Python):
```bash
# TypeScript scripts
make run-ts src/s3/deleteContents.ts
make run-ts src/discord/bot.ts
# Python scripts
make run-py analyse_availability.py
```
Or use the interactive runner:
```bash ```bash
make run # Interactive menu to select language, category, and script make run # Interactive menu to select language, category, and script
``` ```
Or run directly:
```bash
# TypeScript (from project root)
cd typescript && op run --env-file=../prod.env -- pnpm tsx src/<category>/<script>.ts
# Python (from project root)
cd python && op run --env-file=../prod.env -- uv run python <category>/<script>.py
# Bash (no 1Password needed)
bash bash/<category>/<script>.sh
```
## Script Patterns ## Script Patterns
### TypeScript Script Template ### TypeScript Script Template
@@ -232,7 +246,7 @@ The `prod.env` file contains 1Password vault references (like `op://Private/Hetz
5. Consider adding audit logs for security operations 5. Consider adding audit logs for security operations
### Adding a new Python script ### Adding a new Python script
1. Create the script in `python/` 1. Create the script in the appropriate subdirectory of `python/` (e.g. `python/cohort/`)
2. Use type hints for all functions and variables 2. Use type hints for all functions and variables
3. Follow PEP 8 style guide (enforced by ruff) 3. Follow PEP 8 style guide (enforced by ruff)
4. Add the script to `requirements.txt` if it needs new dependencies 4. Add the script to `requirements.txt` if it needs new dependencies
@@ -247,11 +261,12 @@ The `prod.env` file contains 1Password vault references (like `op://Private/Hetz
5. Add unit tests if the utility is complex 5. Add unit tests if the utility is complex
### Adding a new script category ### Adding a new script category
1. Create a new directory under `typescript/src/` or in `python/` 1. Create a new directory under `typescript/src/`, `python/`, or `bash/` as appropriate
2. Follow the naming convention (lowercase, descriptive) 2. Follow the naming convention (lowercase, descriptive)
3. Create at least one example script showing the pattern 3. Create at least one example script showing the pattern
4. Update this CLAUDE.md with specific guidelines for the category 4. Create a `README.md` in the new directory documenting each script
5. Add any new environment variables to prod.env with 1Password references 5. Update this CLAUDE.md with specific guidelines for the category
6. Add any new environment variables to prod.env with 1Password references
## Testing ## Testing
+64 -86
View File
@@ -1,22 +1,31 @@
# Ephemere # Ephemere
A collection of ephemeral scripts for various tasks, written in TypeScript and Python. A collection of ephemeral scripts for various tasks, written in TypeScript, Python, and Bash.
## Project Structure ## Project Structure
``` ```
. .
├── typescript/ # TypeScript project ├── typescript/ # TypeScript scripts
── src/ # TypeScript source files ── src/
├── package.json ├── crowdin/ # Crowdin translation management
│ ├── tsconfig.json ├── discord/ # Discord bot utilities
└── eslint.config.js ├── discourse/ # Discourse forum management
├── python/ # Python project │ ├── gitea/ # Gitea bulk repository operations
├── *.py # Python scripts ├── github/ # GitHub API interactions
├── pyproject.toml ├── music/ # Music file metadata tools
└── requirements.txt ├── s3/ # S3-compatible object storage
├── Makefile # Build commands for both projects │ ├── security/ # Security analysis and reporting
└── README.md │ └── utils/ # Shared utilities
├── python/
│ └── cohort/ # NHCarrigan cohort programme management
├── bash/
│ ├── cohort/ # GitHub team management for cohorts
│ └── yubikey/ # YubiKey SSH key and permission utilities
├── data/ # Input/output data files (gitignored)
├── prod.env # 1Password vault references (safe to commit)
├── run.sh # Interactive script runner
└── Makefile # Build and utility commands
``` ```
## Setup ## Setup
@@ -25,105 +34,74 @@ A collection of ephemeral scripts for various tasks, written in TypeScript and P
- Node.js (v24+) with nvm - Node.js (v24+) with nvm
- Python 3.10+ - Python 3.10+
- pnpm 10.15.0 - pnpm 10.15.0+
- uv (Python package manager) - uv (Python package manager)
- 1Password CLI (for secrets management) - 1Password CLI (`op`) — for secret injection into TypeScript and Python scripts
- `gh` CLI — for Bash scripts that manage GitHub teams
- `ykman` and `yubico-piv-tool` — for YubiKey Bash scripts
### Installation ### Installation
Install all dependencies (TypeScript and Python): Install all dependencies (TypeScript and Python):
```bash ```bash
make install make install
``` ```
Or install individually: Or install individually:
```bash ```bash
make install-ts # TypeScript dependencies only make install-ts # TypeScript dependencies only
make install-py # Python dependencies only make install-py # Python dependencies only
``` ```
## Running Scripts
The recommended way to run any script is the interactive runner:
```bash
make run
```
This launches a menu to select a language (TypeScript, Python, or Bash), then a category, then a specific script. TypeScript and Python scripts have secrets injected automatically from `prod.env` via 1Password CLI. Bash scripts are run directly.
To run a script directly without the interactive runner:
```bash
# TypeScript
cd typescript && op run --env-file=../prod.env -- pnpm tsx src/<category>/<script>.ts
# Python
cd python && op run --env-file=../prod.env -- uv run python cohort/<script>.py
# Bash
bash bash/<category>/<script>.sh
```
Each category has its own `README.md` with details on every script, its required environment variables, and data file formats.
## Development ## Development
### TypeScript Scripts ### Linting and Formatting
TypeScript scripts are located in the `typescript/src/` directory. To run a TypeScript script with environment variables:
```bash ```bash
# From the root directory make lint # Run all linters (TypeScript and Python)
make run-ts src/s3/upload.ts make lint-ts # TypeScript linter only
make lint-py # Python linter only
# Or manually from typescript directory make build # TypeScript type check
cd typescript make format # Format Python code
pnpm start path/to/script.ts make format-check # Check Python formatting without modifying
make test # Run tests
make clean # Clean build artifacts and caches
make help # Show all available commands
``` ```
### Python Scripts
Python scripts are located in the `python/` directory. To run a Python script with environment variables:
```bash
# From the root directory
make run-py analyse_availability.py
# Or manually from python directory
cd python
uv run python script_name.py
```
## Linting and Formatting
```bash
# Run all linters (TypeScript and Python)
make lint
# Run linters individually
make lint-ts # TypeScript linter
make lint-py # Python linter
# Build TypeScript (type check)
make build
# Format Python code
make format
# Check Python formatting without modifying
make format-check
# Run tests
make test
# Clean build artifacts and caches
make clean
# Show all available commands
make help
```
## CI/CD
The GitHub Actions workflow runs the following checks:
1. **Dependency pin check** - Ensures all dependencies are pinned to exact versions
2. **TypeScript checks**:
- ESLint
- TypeScript build (type checking)
- Tests
3. **Python checks**:
- Ruff linting
- Ruff format checking
## Secrets Management ## Secrets Management
This project uses 1Password CLI for secrets management. Environment variables are stored in `prod.env` as 1Password vault references. This project uses 1Password CLI for secrets management. The `prod.env` file contains 1Password vault references (e.g. `op://Private/Discord/token`) rather than real values, making it safe to commit.
The `make run-ts` and `make run-py` commands automatically inject secrets from 1Password: The interactive runner (`make run`) handles secret injection automatically. To run a script manually with secrets:
```bash
# These commands include 1Password integration
make run-ts src/discord/bot.ts
make run-py evaluate_technical_proficiency.py
```
To manually run scripts with secrets:
```bash ```bash
op run --env-file=prod.env -- <command> op run --env-file=prod.env -- <command>
``` ```
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env bash
# env-to-1pass.sh: Create a 1Password item from a .env file
# Each KEY=value line becomes a password-type field labelled KEY
#
# Usage: bash bash/1password/env-to-1pass.sh
# Requires: 1Password CLI (op) — https://developer.1password.com/docs/cli
set -euo pipefail
# Prompt for item name
read -rp "Item name: " item_name
[[ -z "$item_name" ]] && { echo "Item name cannot be empty."; exit 1; }
# Prompt for .env file path
read -rp ".env file path: " env_file
env_file="${env_file/#\~/$HOME}" # expand ~ if present
[[ ! -f "$env_file" ]] && { echo "File not found: $env_file"; exit 1; }
# Build field arguments from the .env file
fields=()
while IFS= read -r line || [[ -n "$line" ]]; do
# Skip blank lines and comments
[[ -z "$line" || "$line" =~ ^[[:space:]]*# ]] && continue
# Skip lines without =
[[ "$line" != *"="* ]] && continue
key="${line%%=*}"
value="${line#*=}"
# Strip surrounding quotes from value if present
value="${value#\"}" ; value="${value%\"}"
value="${value#\'}" ; value="${value%\'}"
fields+=("${key}[password]=${value}")
done < "$env_file"
[[ ${#fields[@]} -eq 0 ]] && { echo "No KEY=value pairs found in $env_file"; exit 1; }
echo "Creating \"$item_name\" with ${#fields[@]} field(s)..."
op item create \
--category "Secure Note" \
--title "$item_name" \
"${fields[@]}"
echo "Done! ✓"
+75
View File
@@ -0,0 +1,75 @@
# Cohort Bash Scripts
Shell scripts for managing GitHub team membership during the NHCarrigan spring cohort programme. These scripts handle one-off team changes that are too complex or bulk-oriented to do manually through the GitHub web interface.
All scripts use the `gh` CLI for GitHub API calls. Run `gh auth login` before using them.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: Bash → cohort → <script>
```
Or run directly:
```bash
bash bash/cohort/<script-name>.sh
```
## Table of Contents
- [remove\_github\_members.sh](#remove_github_memberssh)
- [update\_github\_teams.sh](#update_github_teamssh)
---
## remove_github_members.sh
Removes a hardcoded list of inactive members from their GitHub organisation teams in the `nhcarrigan-spring-2026-cohort` organisation. Covers both standard team membership and `-leaders` sub-team membership where applicable.
### Usage
```bash
bash bash/cohort/remove_github_members.sh
```
### Environment Variables
None. Uses `gh` CLI authentication — run `gh auth login` first.
### Data Files
None. Member usernames and team slugs are hardcoded in the script.
### Notes
- The member list and team assignments are specific to a point-in-time removal event. Update the script with the correct usernames before each use.
- Each removal command uses `|| true` so a single failure (e.g. member already removed) does not abort the entire script.
---
## update_github_teams.sh
Orchestrates a multi-step GitHub team restructure: removes all members from a dissolved team, clears its leaders sub-team, then adds each member to their new team. Also promotes a member to leader in their new team.
### Usage
```bash
bash bash/cohort/update_github_teams.sh
```
### Environment Variables
None. Uses `gh` CLI authentication — run `gh auth login` first.
### Data Files
None. All member usernames, team slugs, and role assignments are hardcoded in the script.
### Notes
- This script is specific to a one-off team restructure (Jade Jasmine dissolution). Update the member list and team assignments before each use.
- The script exits immediately on any error (`set -e`). If a step fails, check whether the member or team already exists in the target state.
+52
View File
@@ -0,0 +1,52 @@
#!/usr/bin/env bash
# Script to remove inactive members from GitHub organization teams
# Date: 2026-02-12
ORG="nhcarrigan-spring-2026-cohort"
# Team 1 (Jade Jasmine) - Remove leader and participant
echo "Removing from Jade Jasmine..."
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine-leaders/memberships/Mista-Log" || true
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine/memberships/A-normal-programmer" || true
# Team 2 (Crimson Dahlia) - Remove 2 participants
echo "Removing from Crimson Dahlia..."
gh api --method DELETE "/orgs/$ORG/teams/crimson-dahlia/memberships/1s-crypto" || true
gh api --method DELETE "/orgs/$ORG/teams/crimson-dahlia/memberships/emlanis" || true
# Team 3 (Rose Camellia) - Remove leader
echo "Removing from Rose Camellia..."
gh api --method DELETE "/orgs/$ORG/teams/rose-camellia-leaders/memberships/michaelboateng1" || true
# Team 4 (Amber Wisteria) - Remove leader and participant
echo "Removing from Amber Wisteria..."
gh api --method DELETE "/orgs/$ORG/teams/amber-wisteria-leaders/memberships/neonbit101" || true
gh api --method DELETE "/orgs/$ORG/teams/amber-wisteria/memberships/avanishchandra" || true
# Team 5 (Ivory Orchid) - Remove participant
echo "Removing from Ivory Orchid..."
gh api --method DELETE "/orgs/$ORG/teams/ivory-orchid/memberships/VuBui217" || true
# Team 7 (Peach Gardenia) - Remove participant
echo "Removing from Peach Gardenia..."
gh api --method DELETE "/orgs/$ORG/teams/peach-gardenia/memberships/TabsOO7" || true
# Team 8 (Violet Carnation) - Remove 2 participants
echo "Removing from Violet Carnation..."
gh api --method DELETE "/orgs/$ORG/teams/violet-carnation/memberships/masudulalam" || true
gh api --method DELETE "/orgs/$ORG/teams/violet-carnation/memberships/urmilbhatt" || true
# Team 10 (Coral Sunflower) - Remove leader and participant
echo "Removing from Coral Sunflower..."
gh api --method DELETE "/orgs/$ORG/teams/coral-sunflower-leaders/memberships/AjayTheWizard" || true
gh api --method DELETE "/orgs/$ORG/teams/coral-sunflower/memberships/Hritikhh" || true
# Team 11 (Indigo Tulip) - Remove participant
echo "Removing from Indigo Tulip..."
gh api --method DELETE "/orgs/$ORG/teams/indigo-tulip/memberships/SiAust" || true
# Team 13 (Mint Narcissus) - Remove participant
echo "Removing from Mint Narcissus..."
gh api --method DELETE "/orgs/$ORG/teams/mint-narcissus/memberships/SergioPardoSanchez" || true
echo "Done removing members from GitHub teams!"
+73
View File
@@ -0,0 +1,73 @@
#!/bin/bash
# Update GitHub teams for Jade Jasmine dissolution
set -e # Exit on error
ORG="nhcarrigan-spring-2026-cohort"
echo "=== Phase 2: GitHub Team Changes ==="
echo ""
# Step 1: Remove all members from jade-jasmine team
echo "Step 1: Removing members from jade-jasmine team..."
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine/memberships/Nikil-D-Gr8" || echo " - Nikil-D-Gr8 already removed or not found"
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine/memberships/amandaw800" || echo " - amandaw800 already removed or not found"
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine/memberships/hayden-html" || echo " - hayden-html already removed or not found"
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine/memberships/hbar1st" || echo " - hbar1st already removed or not found"
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine/memberships/rosacabrerac" || echo " - rosacabrerac already removed or not found"
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine/memberships/ArbitraryPie" || echo " - ArbitraryPie already removed or not found"
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine/memberships/builtbykabir" || echo " - builtbykabir already removed or not found"
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine/memberships/Mista-Log" || echo " - Mista-Log already removed or not found"
echo "✅ jade-jasmine team cleared"
echo ""
# Step 2: Remove leaders from jade-jasmine-leaders team
echo "Step 2: Removing leaders from jade-jasmine-leaders team..."
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine-leaders/memberships/hayden-html" || echo " - hayden-html already removed or not found"
gh api --method DELETE "/orgs/$ORG/teams/jade-jasmine-leaders/memberships/Mista-Log" || echo " - Mista-Log already removed or not found"
echo "✅ jade-jasmine-leaders team cleared"
echo ""
# Step 3: Add members to new teams
echo "Step 3: Adding members to new teams..."
echo " - Adding Nikil-D-Gr8 to crimson-dahlia..."
gh api --method PUT "/orgs/$ORG/teams/crimson-dahlia/memberships/Nikil-D-Gr8" -f role=member
echo " - Adding amandaw800 to violet-carnation..."
gh api --method PUT "/orgs/$ORG/teams/violet-carnation/memberships/amandaw800" -f role=member
echo " - Adding hayden-html to teal-iris..."
gh api --method PUT "/orgs/$ORG/teams/teal-iris/memberships/hayden-html" -f role=member
echo " - Adding hbar1st to indigo-tulip..."
gh api --method PUT "/orgs/$ORG/teams/indigo-tulip/memberships/hbar1st" -f role=member
echo " - Adding rosacabrerac to scarlet-hydrangea..."
gh api --method PUT "/orgs/$ORG/teams/scarlet-hydrangea/memberships/rosacabrerac" -f role=member
echo " - Adding ArbitraryPie to peach-gardenia..."
gh api --method PUT "/orgs/$ORG/teams/peach-gardenia/memberships/ArbitraryPie" -f role=member
echo " - Adding builtbykabir to azure-lotus..."
gh api --method PUT "/orgs/$ORG/teams/azure-lotus/memberships/builtbykabir" -f role=member
echo " - Adding Mista-Log to ivory-orchid..."
gh api --method PUT "/orgs/$ORG/teams/ivory-orchid/memberships/Mista-Log" -f role=member
echo "✅ All members added to new teams"
echo ""
# Step 4: Add Mista-Log to ivory-orchid-leaders
echo "Step 4: Adding Mista-Log to ivory-orchid-leaders..."
gh api --method PUT "/orgs/$ORG/teams/ivory-orchid-leaders/memberships/Mista-Log" -f role=member
echo "✅ Mista-Log promoted to leader in Ivory Orchid"
echo ""
echo "=== Phase 2 Complete! ==="
echo ""
echo "Summary:"
echo "- ✅ jade-jasmine team cleared (8 members removed)"
echo "- ✅ jade-jasmine-leaders team cleared (2 leaders removed)"
echo "- ✅ 8 members added to their new teams"
echo "- ✅ Mista-Log promoted to leader in ivory-orchid"
+109
View File
@@ -0,0 +1,109 @@
# YubiKey Scripts
Shell scripts for managing YubiKey hardware security keys on WSL (Windows Subsystem for Linux). Covers SSH key extraction, Git signing key configuration, and fixing USB permission issues that commonly arise in WSL environments.
All scripts require a YubiKey to be attached and forwarded to WSL via `usbipd`. The `ykman` and `yubico-piv-tool` packages must be installed.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: Bash → yubikey → <script>
```
Or run directly:
```bash
bash bash/yubikey/<script-name>.sh
```
## Table of Contents
- [add-keys-to-git.sh](#add-keys-to-gitsh)
- [fix-yubikey-perms.sh](#fix-yubikey-permssh)
- [list-yubikey-ssh-keys.sh](#list-yubikey-ssh-keyssh)
---
## add-keys-to-git.sh
Extracts the SSH public keys from three YubiKey PIV slots and writes them as Git commit signing keys to the corresponding per-context Git config files. Run this after replacing or re-provisioning a YubiKey.
| Slot | Context | Config file |
|---|---|---|
| 9a | Personal | `~/.git-naomi` |
| 9c | Deepgram | `~/.git-dg` |
| 9e | FreeCodeCamp | `~/.git-fcc` |
### Usage
```bash
bash bash/yubikey/add-keys-to-git.sh
```
### Environment Variables
None.
### Data Files
None.
### Notes
- After running, you must upload the new public keys to GitHub (and any other services that verify commit signatures) manually.
- Requires `ykman` and `ssh-keygen` to be available in your PATH.
---
## fix-yubikey-perms.sh
Repairs YubiKey connectivity in WSL by fixing USB device permissions, restarting smart card services, and applying a polkit policy override that allows smart card access in WSL's "inactive" session context.
### Usage
```bash
bash bash/yubikey/fix-yubikey-perms.sh
```
### Environment Variables
None.
### Data Files
None.
### Notes
- Run this script when `ykman` or `yubico-piv-tool` fail with "Failed to connect" or similar errors after attaching the YubiKey via `usbipd`.
- The polkit fix modifies `/usr/share/polkit-1/actions/org.debian.pcsc-lite.policy` (a backup is created automatically on first run).
- Requires `sudo` access. Several steps use `sudo` to modify system files and restart services.
- Requires `lsusb`, `yubico-piv-tool`, `systemctl`, and `gpgconf` to be available.
---
## list-yubikey-ssh-keys.sh
Scans PIV slots 9a, 9c, 9d, and 9e on the connected YubiKey and prints any SSH public keys found, along with the certificate subject label if one is present.
### Usage
```bash
bash bash/yubikey/list-yubikey-ssh-keys.sh
```
### Environment Variables
None.
### Data Files
None.
### Notes
- Requires `ykman`, `ssh-keygen`, and `openssl` to be available.
- Writes a temporary file to `/tmp/yubi_tmp.pem` during execution; it is cleaned up automatically after each slot is processed.
+21
View File
@@ -0,0 +1,21 @@
# Security
# Do not execute any scripts of installed packages (project scripts still run)
ignoreDepScripts: true
# Do not automatically run pre/post scripts (e.g. preinstall, postbuild)
enablePrePostScripts: false
# Only allow packages published at least 10 days ago (reduces risk of compromised packages)
minimumReleaseAge: 14400
# Fail if a package's trust level has decreased compared to previous releases
trustPolicy: no-downgrade
# Ignore trust policy for packages published more than 1 year ago (predates provenance signing)
trustPolicyIgnoreAfter: 525960
# Fail if there are missing or invalid peer dependencies
strictPeerDependencies: true
# Prevent transitive dependencies from using exotic sources (git repos, direct tarball URLs)
blockExoticSubdeps: true
# Lockfile
# Allow the lockfile to be updated during install (set to true in CI for stricter reproducibility)
preferFrozenLockfile: false
+6 -6
View File
@@ -7,15 +7,15 @@ CROWDIN_TOKEN="op://Environment Variables - Development/Ephemere/Crowdin Token"
GITHUB_TOKEN="op://Environment Variables - Development/Ephemere/GitHub Token" GITHUB_TOKEN="op://Environment Variables - Development/Ephemere/GitHub Token"
# Discord # Discord
DISCORD_TOKEN="op://Environment Variables - Development/Ephemere/Discord Token" DISCORD_TOKEN="op://Environment Variables - Naomi/Hikari/discord_token"
DISCORD_CLIENT_ID="op://Private/Guild Counter/client id" DISCORD_CLIENT_ID="op://Environment Variables - Development/Guild Counter/client id"
DISCORD_CLIENT_SECRET="op://Private/Guild Counter/client secret" DISCORD_CLIENT_SECRET="op://Environment Variables - Development/Guild Counter/client secret"
DISCORD_BOT_TOKEN="op://Environment Variables - Naomi/Amari/bot token" DISCORD_BOT_TOKEN="op://Environment Variables - Naomi/Amari/bot token"
# AWS # AWS
AWS_ACCESS_KEY_ID="op://Private/Hetzner/S3 Access Key ID" AWS_ACCESS_KEY_ID="op://Environment Variables - Development/Cloudflare R2/ID"
AWS_SECRET_ACCESS_KEY="op://Private/Hetzner/S3 Secret Access Key" AWS_SECRET_ACCESS_KEY="op://Environment Variables - Development/Cloudflare R2/Secret"
S3_ENDPOINT="op://Private/Hetzner/S3 Endpoint" S3_ENDPOINT="op://Environment Variables - Development/Cloudflare R2/Endpoint"
# Gitea # Gitea
GITEA_TOKEN="op://Private/Gitea/token" GITEA_TOKEN="op://Private/Gitea/token"
+883
View File
@@ -0,0 +1,883 @@
# Cohort Scripts
Scripts for managing the NHCarrigan spring cohort programme. Covers the full lifecycle: applicant evaluation, Discord server setup, team assignment, member onboarding, activity tracking, and member removal.
Most scripts interact with the Discord API and require a `DISCORD_BOT_TOKEN`. Several also use the `gh` CLI for GitHub operations.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: Python → cohort → <script>
```
Or run directly:
```bash
cd python && op run --env-file=../prod.env -- uv run python cohort/<script>.py
```
**Prerequisites:**
- Run `make install-py` to set up the Python virtual environment.
- Most scripts require `DISCORD_BOT_TOKEN` set in `prod.env`.
- Scripts that manage GitHub teams require `gh auth login` to be run first.
## Table of Contents
### Applicant Evaluation
- [verify_discord.py](#verify_discordpy)
- [evaluate_technical_proficiency.py](#evaluate_technical_proficiencypy)
- [analyse_availability.py](#analyse_availabilitypy)
- [generate_member_files.py](#generate_member_filespy)
- [generate_timeslots.py](#generate_timeslotspy)
### Server Setup
- [create_team_voice_channels.py](#create_team_voice_channelspy)
- [fix_channel_permissions.py](#fix_channel_permissionspy)
- [update_cohort_leads_permissions.py](#update_cohort_leads_permissionspy)
- [list_all_guild_roles.py](#list_all_guild_rolespy)
- [list_discord_roles.py](#list_discord_rolespy)
- [check_channel_permissions.py](#check_channel_permissionspy)
### Member Onboarding
- [send_team_messages.py](#send_team_messagespy)
- [assign_cohort_role.py](#assign_cohort_rolepy)
- [assign_team_roles.py](#assign_team_rolespy)
- [add_github_team_members.py](#add_github_team_memberspy)
### Ongoing Management
- [get_cohort_members.py](#get_cohort_memberspy)
- [check_member_status.py](#check_member_statuspy)
- [fetch_roster.py](#fetch_rosterpy)
- [update_roster_messages.py](#update_roster_messagespy)
- [send_checkin.py](#send_checkinpy)
- [send_team_checkin.py](#send_team_checkinpy)
- [send_team_messages.py](#send_team_messagespy)
### Activity Tracking
- [discord_activity_checker.py](#discord_activity_checkerpy)
- [catch_up_report.py](#catch_up_reportpy)
- [check_lengths.py](#check_lengthspy)
- [send_activity_report.py](#send_activity_reportpy)
### Member Removal
- [remove_member.py](#remove_memberpy)
- [remove_resigned_members.py](#remove_resigned_memberspy)
- [remove_discord_roles.py](#remove_discord_rolespy)
---
## verify_discord.py
Reads Discord user IDs from a markdown table and verifies each one against the Discord API, checking whether the user is a member of the freeCodeCamp guild. Handles rate limits and retries automatically.
### Usage
```bash
make run
# Select: Python → cohort → verify_discord.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `table.md` | Markdown table | Applicant data; the script reads Discord IDs from the table rows |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `discord_verification.json` | JSON object | Results grouped as `verified` (with username), `missing`, and `errors` lists |
---
## evaluate_technical_proficiency.py
Evaluates each applicant's technical proficiency by analysing their GitHub profile and self-described expertise. Scores applicants as Beginner, Intermediate, or Advanced based on public repository count, follower count, language variety, and keywords in their self-assessment text.
### Usage
```bash
make run
# Select: Python → cohort → evaluate_technical_proficiency.py
```
### Environment Variables
None. Uses the public GitHub API (unauthenticated, subject to rate limiting).
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `applicants_to_evaluate.json` | JSON array | Applicant records, each with a GitHub profile URL and a self-description field |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `proficiency_evaluations.json` | JSON array | Proficiency scores (`Beginner`/`Intermediate`/`Advanced`) and detected tech stacks per applicant |
### Notes
- Unauthenticated GitHub API requests are rate-limited to 60 per hour. For large batches of applicants, consider adding a GitHub token or running the script in smaller chunks.
---
## analyse_availability.py
Parses a markdown table of applicant availability responses with timezone information, converts local timeslot selections to UTC, and produces a JSON summary of UTC coverage across morning, afternoon, evening, and night blocks per applicant.
### Usage
```bash
make run
# Select: Python → cohort → analyse_availability.py
```
### Environment Variables
None.
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `table.md` | Markdown table | Availability responses with timezone column |
| `discord_verification.json` | JSON object | Output of `verify_discord.py` — used to link applicants to Discord usernames |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `availability_analysis.json` | JSON object | UTC block distribution (morning/afternoon/evening/night) per applicant |
---
## generate_member_files.py
Consolidates all evaluation data into two markdown files: one for participants and one for team leaders. Each entry includes the member's tech stack, availability, proficiency level, and (for leaders) their leadership assessment.
### Usage
```bash
make run
# Select: Python → cohort → generate_member_files.py
```
### Environment Variables
None.
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `discord_verification.json` | JSON object | Verified applicant Discord info |
| `proficiency_evaluations.json` | JSON array | Technical proficiency scores |
| `availability_analysis.json` | JSON object | UTC availability blocks |
| `leadership_candidates.json` | JSON array | Applicants being considered for leadership roles |
| `leadership_evaluations.json` | JSON array | Leadership assessment results |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `participants.md` | Markdown | Profile summary for each participant |
| `leaders.md` | Markdown | Profile summary for each team leader candidate |
---
## generate_timeslots.py
Generates a list of hourly timeslots for use with the [Crabfit](https://crab.fit/) scheduling tool, covering a date range at Los Angeles timezone.
### Usage
```bash
make run
# Select: Python → cohort → generate_timeslots.py
```
### Environment Variables
None.
### Data Files
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `crabfit_timeslots.json` | JSON array of ISO-format strings | One entry per hour across the configured date range |
### Notes
- The date range and timezone are hardcoded in the script. Update the start/end date constants before running.
---
## create_team_voice_channels.py
Creates private voice channels for each cohort team in a specified Discord category. Each channel is visible and joinable only by members who have that team's role.
### Usage
```bash
make run
# Select: Python → cohort → create_team_voice_channels.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Team role IDs and the target category ID are hardcoded in the script.
### Notes
- Update the team role IDs, channel names, and category ID constants in the script before running.
---
## fix_channel_permissions.py
Denies `Send Messages` and `Send Messages in Threads` permissions for the `@everyone` and `@cohort` roles on a specific Discord channel. Used to lock down a channel so only designated roles can post.
### Usage
```bash
make run
# Select: Python → cohort → fix_channel_permissions.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. The target channel ID is hardcoded in the script.
### Notes
- Update the `CHANNEL_ID` constant before running.
---
## update_cohort_leads_permissions.py
Grants the Cohort Leads role `MENTION_EVERYONE` and `PIN_MESSAGES` permissions across all 14 team channels.
### Usage
```bash
make run
# Select: Python → cohort → update_cohort_leads_permissions.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Team channel IDs are hardcoded in the script.
---
## list_all_guild_roles.py
Lists all roles in the freeCodeCamp Discord guild, highlighting team and cohort-related roles. Useful for finding role IDs to use in other scripts.
### Usage
```bash
make run
# Select: Python → cohort → list_all_guild_roles.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Output is printed to stdout.
---
## list_discord_roles.py
Lists Discord roles in the freeCodeCamp server, filtering for cohort-, leader-, and 2026-related roles.
### Usage
```bash
make run
# Select: Python → cohort → list_discord_roles.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Output is printed to stdout.
---
## check_channel_permissions.py
Audits `cohort-team-*` Discord channels to identify incorrect `Send Messages` or `Send Messages in Threads` permissions on the `@everyone` or `@cohort` roles.
### Usage
```bash
make run
# Select: Python → cohort → check_channel_permissions.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Results are printed to stdout.
---
## send_team_messages.py
Sends initial welcome and roster messages to all 14 team Discord channels, pins them, and saves the resulting message IDs, channel IDs, and role IDs to `data/team_message_ids.json`. This file is required by several other scripts.
### Usage
```bash
make run
# Select: Python → cohort → send_team_messages.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Team rosters (leaders and participants per team) |
| `applicants_to_evaluate.json` | JSON array | Applicant data used to populate roster messages |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `team_message_ids.json` | JSON object | Channel ID, pinned message ID, and role ID per team name |
### Notes
- Run this **once** at the start of the cohort. Other scripts (`send_checkin.py`, `update_roster_messages.py`, etc.) depend on `team_message_ids.json`.
---
## assign_cohort_role.py
Assigns the Cohort Discord role to all cohort participants. Reads the full list of unique member Discord IDs from `team_assignments.json` and assigns the role to each one, with exponential backoff to handle rate limits.
### Usage
```bash
make run
# Select: Python → cohort → assign_cohort_role.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Team rosters; all unique user IDs are extracted from here |
---
## assign_team_roles.py
Assigns team-specific Discord roles to all cohort participants based on their team membership in `team_assignments.json`. Uses exponential backoff for rate limit handling.
### Usage
```bash
make run
# Select: Python → cohort → assign_team_roles.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Team rosters with Discord IDs and corresponding role IDs |
---
## add_github_team_members.py
Adds cohort members to their corresponding GitHub teams in the `nhcarrigan-spring-2026-cohort` organisation. Leaders are added to both the main team and the corresponding `-leaders` sub-team; participants are added to the main team only.
### Usage
```bash
make run
# Select: Python → cohort → add_github_team_members.py
```
### Environment Variables
None. Uses the `gh` CLI for authentication (run `gh auth login` first).
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Team rosters with leader/participant distinction |
| `discord_to_github.json` | JSON object | Maps Discord user IDs to GitHub usernames |
---
## get_cohort_members.py
Fetches all Discord guild members with the Cohort role and writes their IDs, usernames, and display names to a JSON file. Also prints a formatted list to stdout.
### Usage
```bash
make run
# Select: Python → cohort → get_cohort_members.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `active_cohort_members.json` | JSON array | Objects with `id`, `username`, and `display_name` for each Cohort role member |
---
## check_member_status.py
Verifies whether specific Discord member IDs are still in the freeCodeCamp guild by querying the Discord API. Used to confirm that removed members have been successfully ejected.
### Usage
```bash
make run
# Select: Python → cohort → check_member_status.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. The member IDs to check are hardcoded in the script. Update the list before running.
---
## fetch_roster.py
Fetches pinned messages from a specific team channel to retrieve the current roster.
### Usage
```bash
make run
# Select: Python → cohort → fetch_roster.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Output is printed to stdout as JSON.
### Notes
- The target channel ID is hardcoded. Update it before running.
---
## update_roster_messages.py
Edits the pinned team roster messages in each Discord channel with the latest data from `team_assignments.json` and `discord_to_github.json`. Run this after any membership changes to keep the pinned rosters up to date.
### Usage
```bash
make run
# Select: Python → cohort → update_roster_messages.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_message_ids.json` | JSON object | Channel and message IDs per team (output of `send_team_messages.py`) |
| `team_assignments.json` | JSON object | Current team rosters |
| `discord_to_github.json` | JSON object | Discord ID → GitHub username mapping |
---
## send_checkin.py
Sends biweekly check-in prompts to all team Discord channels (except Jade Jasmine), automatically creating threads for responses. Members who do not respond face removal for inactivity.
### Usage
```bash
make run
# Select: Python → cohort → send_checkin.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_message_ids.json` | JSON object | Channel IDs per team |
---
## send_team_checkin.py
Sends a capacity check-in message to each team channel asking whether the team feels able to complete their project with their current member count, and inviting them to request support if needed.
### Usage
```bash
make run
# Select: Python → cohort → send_team_checkin.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_message_ids.json` | JSON object | Channel IDs per team |
| `team_assignments.json` | JSON object | Current team rosters (used to mention team members) |
---
## discord_activity_checker.py
Scans each team's Discord channel and threads to identify members who have not sent a message within the last 36 hours. Can optionally send notification messages directly to inactive members via the `--send` CLI flag.
### Usage
```bash
# Check only (no messages sent)
make run
# Select: Python → cohort → discord_activity_checker.py
# Check and notify inactive members
cd python && op run --env-file=../prod.env -- uv run python cohort/discord_activity_checker.py --send
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Member lists per team |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `discord_activity_report.json` | JSON object | Inactive members per team with their last message timestamp |
---
## catch_up_report.py
Generates a detailed markdown activity report covering Discord messages (in team channels and their threads) and GitHub activity (PRs, issues, comments, reviews, commits) since a configured start date. Uses async API calls for efficiency.
### Usage
```bash
make run
# Select: Python → cohort → catch_up_report.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Team rosters |
| `discord_to_github.json` | JSON object | Discord ID → GitHub username mapping |
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `catch_up_report.md` | Markdown table | Activity counts per member (Discord messages + GitHub contributions) per team |
### Notes
- The report start date is hardcoded in the script. Update it before each use.
- GitHub activity is fetched via the unauthenticated public API; rate limiting may apply for large cohorts.
---
## check_lengths.py
Dry-run validation that parses `catch_up_report.md` and formats each team's data into Discord monospace table strings, checking whether any would exceed Discord's 2,000-character message limit before actually sending them.
### Usage
```bash
make run
# Select: Python → cohort → check_lengths.py
```
### Environment Variables
None.
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `catch_up_report.md` | Markdown table | Output of `catch_up_report.py` |
### Notes
- Run this after `catch_up_report.py` and before `send_activity_report.py` to catch any messages that would be rejected by Discord.
---
## send_activity_report.py
Parses `catch_up_report.md` and sends the formatted activity table for each team to its Discord channel as a monospace code block.
### Usage
```bash
make run
# Select: Python → cohort → send_activity_report.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `catch_up_report.md` | Markdown table | Output of `catch_up_report.py` |
### Notes
- Run `check_lengths.py` first to verify that no messages exceed Discord's character limit.
---
## remove_member.py
Comprehensive member removal script. Given a Discord ID as a CLI argument, it:
1. Removes the member from `team_assignments.json`.
2. Removes the member from `discord_to_github.json`.
3. Prints GitHub removal instructions.
4. Removes the member's Discord Cohort and team roles.
5. Sends an announcement message to the team's Discord channel.
6. Outputs markdown notes suitable for pasting into `COHORT_NOTES.md`.
### Usage
```bash
cd python && op run --env-file=../prod.env -- uv run python cohort/remove_member.py <discord_id>
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
**Input/Output** (read and updated in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Updated: member removed from their team |
| `discord_to_github.json` | JSON object | Updated: Discord→GitHub mapping removed |
| `team_message_ids.json` | JSON object | Read: used to find the team's Discord channel ID |
---
## remove_resigned_members.py
Removes a list of resigned member Discord IDs from `team_assignments.json`, updating team rosters and reporting which teams were affected. Does not interact with Discord or GitHub.
### Usage
```bash
make run
# Select: Python → cohort → remove_resigned_members.py
```
### Environment Variables
None.
### Data Files
**Input/Output** (read and updated in `data/`):
| File | Format | Description |
|---|---|---|
| `team_assignments.json` | JSON object | Updated in place; affected teams are reported to stdout |
### Notes
- The list of Discord IDs to remove is hardcoded in the script. Update it before running.
- This script only updates the local JSON file. Run `update_roster_messages.py` and handle GitHub/Discord role removal separately.
---
## remove_discord_roles.py
Removes the Cohort and team-specific Discord roles from a hardcoded list of inactive members.
### Usage
```bash
make run
# Select: Python → cohort → remove_discord_roles.py
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_BOT_TOKEN` | Discord bot token |
### Data Files
None. Member IDs and their team-to-role mappings are hardcoded in the script. Update them before running.
+17 -3
View File
@@ -1,15 +1,29 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Add GitHub users to their appropriate teams in nhcarrigan-spring-2026-cohort org""" """Add GitHub users to their appropriate teams in the cohort GitHub organisation.
Uses the GitHub CLI to add each member to their corresponding team and, for
leaders, to the team's leaders sub-team.
Data files (place in data/):
- team_assignments.json Team rosters with leaders and participants per team
- discord_to_github.json Mapping of Discord IDs to GitHub usernames
Env vars:
- None (uses `gh` CLI for authentication)
"""
import json import json
import subprocess import subprocess
import time import time
from pathlib import Path
DATA_DIR = Path(__file__).parent.parent.parent / "data"
# Load team assignments and Discord to GitHub mappings # Load team assignments and Discord to GitHub mappings
with open("team_assignments.json") as f: with open(DATA_DIR / "team_assignments.json") as f:
teams = json.load(f) teams = json.load(f)
with open("discord_to_github.json") as f: with open(DATA_DIR / "discord_to_github.json") as f:
discord_to_github = json.load(f) discord_to_github = json.load(f)
# Map team names to GitHub team slugs # Map team names to GitHub team slugs
+23 -3
View File
@@ -1,6 +1,26 @@
"""Analyse applicant availability from a markdown table and produce UTC block stats.
Reads a markdown table of availability responses and a Discord verification file,
then produces a JSON summary of coverage across morning/afternoon/evening UTC blocks
for each day of the week.
Data files (place in data/):
- table.md Markdown table of applicant availability responses
- discord_verification.json Discord ID verification results (from verify_discord.py)
Outputs (written to data/):
- availability_analysis.json UTC block distribution per applicant
Env vars:
- None
"""
import json import json
import re import re
from collections import defaultdict from collections import defaultdict
from pathlib import Path
DATA_DIR = Path(__file__).parent.parent.parent / "data"
DAYS = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"] DAYS = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"]
@@ -99,7 +119,7 @@ def analyze_applicant_availability(timezone_str: str, day_slots: dict) -> dict:
def parse_table_md() -> list[dict]: def parse_table_md() -> list[dict]:
"""Parse table.md and extract availability data""" """Parse table.md and extract availability data"""
with open("table.md") as f: with open(DATA_DIR / "table.md") as f:
content = f.read() content = f.read()
lines = content.strip().split("\n") lines = content.strip().split("\n")
@@ -131,7 +151,7 @@ def parse_table_md() -> list[dict]:
def main(): def main():
with open("discord_verification.json") as f: with open(DATA_DIR / "discord_verification.json") as f:
verification = json.load(f) verification = json.load(f)
verified_ids = {v[0] for v in verification["verified"]} verified_ids = {v[0] for v in verification["verified"]}
@@ -167,7 +187,7 @@ def main():
} }
) )
with open("availability_analysis.json", "w") as f: with open(DATA_DIR / "availability_analysis.json", "w") as f:
json.dump(availability_results, f, indent=2) json.dump(availability_results, f, indent=2)
block_distribution = defaultdict(int) block_distribution = defaultdict(int)
+4 -1
View File
@@ -6,9 +6,12 @@ Respects Discord rate limits with proper backoff and retry logic.
import json import json
import os import os
import time import time
from pathlib import Path
import requests import requests
DATA_DIR = Path(__file__).parent.parent.parent / "data"
BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"] BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
GUILD_ID = "692816967895220344" GUILD_ID = "692816967895220344"
COHORT_ROLE_ID = "1464314780935258112" COHORT_ROLE_ID = "1464314780935258112"
@@ -48,7 +51,7 @@ def assign_role_with_retry(user_id: str, role_id: str, max_retries: int = 5) ->
def main(): def main():
with open("team_assignments.json") as f: with open(DATA_DIR / "team_assignments.json") as f:
teams = json.load(f) teams = json.load(f)
all_users = [] all_users = []
+4 -1
View File
@@ -6,9 +6,12 @@ Respects Discord rate limits with proper backoff and retry logic.
import json import json
import os import os
import time import time
from pathlib import Path
import requests import requests
DATA_DIR = Path(__file__).parent.parent.parent / "data"
BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"] BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
GUILD_ID = "692816967895220344" GUILD_ID = "692816967895220344"
@@ -64,7 +67,7 @@ def assign_role_with_retry(user_id: str, role_id: str, max_retries: int = 5) ->
def main(): def main():
with open("team_assignments.json") as f: with open(DATA_DIR / "team_assignments.json") as f:
teams = json.load(f) teams = json.load(f)
print(f"Assigning team roles to {len(teams)} teams...") print(f"Assigning team roles to {len(teams)} teams...")
+519
View File
@@ -0,0 +1,519 @@
#!/usr/bin/env python3
"""Catch-Up Activity Report.
Generates a markdown report of Discord and GitHub activity since Feb 15, 2026.
Covers Discord messages in team channels (+ threads) and GitHub activity
(PRs opened, issues opened, issue comments, PR comments, PR reviews, commits).
"""
import asyncio
import json
import os
import subprocess
from datetime import datetime, timezone
from pathlib import Path
import aiohttp
DATA_DIR = Path(__file__).parent.parent.parent / "data"
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
DISCORD_API_BASE = "https://discord.com/api/v10"
GITHUB_API_BASE = "https://api.github.com"
GITHUB_ORG = "nhcarrigan-spring-2026-cohort"
CUTOFF = datetime(2026, 2, 15, 0, 0, 0, tzinfo=timezone.utc)
CUTOFF_ISO = CUTOFF.isoformat().replace("+00:00", "Z")
OUTPUT_FILE = "catch_up_report.md"
TEXT_CHANNEL_IDS: dict[str, str] = {
"Crimson Dahlia": "1464316744909852682",
"Rose Camellia": "1464316751268286611",
"Amber Wisteria": "1464316761410113641",
"Ivory Orchid": "1464316770889240730",
"Teal Iris": "1464316776459407448",
"Peach Gardenia": "1464316785040953543",
"Violet Carnation": "1464316805261824032",
"Azure Lotus": "1464316814455472139",
"Coral Sunflower": "1464316819711066263",
"Indigo Tulip": "1464316826384072925",
"Scarlet Hydrangea": "1464316839306985506",
"Mint Narcissus": "1464316844251807952",
"Sage Marigold": "1464316850669093040",
}
def team_repo_slug(team_name: str) -> str:
"""Convert a team name to its repository slug."""
return team_name.lower().replace(" ", "-")
def get_github_token() -> str:
"""Retrieve the GitHub token via the gh CLI."""
result = subprocess.run(
["gh", "auth", "token"], capture_output=True, text=True, check=True
)
return result.stdout.strip()
class ActivityCollector:
"""Collects Discord and GitHub activity for the catch-up report."""
def __init__(self, discord_token: str, github_token: str) -> None:
self.discord_headers = {
"Authorization": f"Bot {discord_token}",
"Content-Type": "application/json",
}
self.github_headers = {
"Authorization": f"Bearer {github_token}",
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
}
self.session: aiohttp.ClientSession | None = None
async def __aenter__(self) -> "ActivityCollector":
self.session = aiohttp.ClientSession()
return self
async def __aexit__(
self, exc_type: object, exc_val: object, exc_tb: object
) -> None:
if self.session:
await self.session.close()
async def get_discord_username(self, user_id: str) -> str:
"""Fetch a Discord user's display name or username."""
url = f"{DISCORD_API_BASE}/users/{user_id}"
async with self.session.get(url, headers=self.discord_headers) as response:
if response.status == 429:
retry_after = float((await response.json()).get("retry_after", 1))
await asyncio.sleep(retry_after)
return await self.get_discord_username(user_id)
if response.status != 200:
return "*(unknown)*"
data = await response.json()
return data.get("global_name") or data.get("username") or "*(unknown)*"
async def _get_discord_thread_ids(self, channel_id: str) -> list[str]:
"""Return IDs of all active and archived threads in a channel."""
thread_ids: list[str] = []
url = f"{DISCORD_API_BASE}/channels/{channel_id}/threads/active"
async with self.session.get(url, headers=self.discord_headers) as response:
if response.status == 200:
data = await response.json()
thread_ids.extend(t["id"] for t in data.get("threads", []))
for archive_type in ("public", "private"):
url = (
f"{DISCORD_API_BASE}/channels/{channel_id}"
f"/threads/archived/{archive_type}"
)
async with self.session.get(url, headers=self.discord_headers) as response:
if response.status == 200:
data = await response.json()
thread_ids.extend(t["id"] for t in data.get("threads", []))
return thread_ids
async def _count_messages_in_channel(
self, channel_id: str, label: str = ""
) -> dict[str, int]:
"""Count messages per Discord user ID since CUTOFF."""
counts: dict[str, int] = {}
before_id: str | None = None
page = 0
while True:
url = f"{DISCORD_API_BASE}/channels/{channel_id}/messages?limit=100"
if before_id:
url += f"&before={before_id}"
async with self.session.get(url, headers=self.discord_headers) as response:
if response.status == 429:
retry_after = float((await response.json()).get("retry_after", 1))
print(f" [Discord] rate limited, waiting {retry_after:.1f}s...")
await asyncio.sleep(retry_after)
continue
if response.status != 200:
print(f" [Discord] channel {channel_id} → HTTP {response.status}")
break
messages: list[dict] = await response.json()
if not messages:
break
page += 1
prefix = f" ({label})" if label else ""
print(
f" [Discord]{prefix} page {page}{len(messages)} messages fetched", # noqa: E501
end="\r",
)
reached_cutoff = False
for message in messages:
ts = datetime.fromisoformat(
message["timestamp"].replace("Z", "+00:00")
)
if ts < CUTOFF:
reached_cutoff = True
break
if message["author"].get("bot", False):
continue
author_id = message["author"]["id"]
counts[author_id] = counts.get(author_id, 0) + 1
if reached_cutoff or len(messages) < 100:
print()
break
before_id = messages[-1]["id"]
await asyncio.sleep(0.5)
return counts
async def collect_discord_counts(
self, team_name: str, channel_id: str, member_ids: list[str]
) -> dict[str, int]:
"""Return message counts per member for a team's channel and threads."""
print(" [Discord] Scanning main channel...")
totals: dict[str, int] = await self._count_messages_in_channel(
channel_id, label="main"
)
thread_ids = await self._get_discord_thread_ids(channel_id)
total_threads = len(thread_ids)
for i, thread_id in enumerate(thread_ids, start=1):
print(f" [Discord] Scanning thread {i}/{total_threads}...")
thread_counts = await self._count_messages_in_channel(
thread_id, label=f"thread {i}/{total_threads}"
)
for user_id, count in thread_counts.items():
totals[user_id] = totals.get(user_id, 0) + count
await asyncio.sleep(0.3)
if total_threads == 0:
print(" [Discord] No threads found.")
return {member_id: totals.get(member_id, 0) for member_id in member_ids}
async def _github_get_all_pages(self, url: str, params: dict) -> list[dict]:
"""Fetch all pages from a paginated GitHub REST API endpoint."""
results: list[dict] = []
page = 1
while True:
paged_params = {**params, "per_page": 100, "page": page}
async with self.session.get(
url, headers=self.github_headers, params=paged_params
) as response:
if response.status in (404, 422):
break
if response.status == 403:
print(f" [GitHub] rate limited on {url}, waiting 60s...")
await asyncio.sleep(60)
continue
if response.status != 200:
print(f" [GitHub] {url} → HTTP {response.status}")
break
data: list[dict] = await response.json()
if not data:
break
results.extend(data)
if len(data) < 100:
break
page += 1
await asyncio.sleep(0.2)
return results
async def collect_github_counts(
self, team_name: str, github_usernames: list[str]
) -> dict[str, dict[str, int]]:
"""Return activity counts per member for a team's GitHub repository."""
repo_slug = team_repo_slug(team_name)
repo = f"{GITHUB_ORG}/{repo_slug}"
print(f" [GitHub] repo: {repo}")
counts: dict[str, dict[str, int]] = {
username: {
"prs_opened": 0,
"issues_opened": 0,
"issue_comments": 0,
"pr_comments": 0,
"pr_reviews": 0,
"commits": 0,
}
for username in github_usernames
if username
}
def resolve_username(login: str) -> str | None:
lower = login.lower()
for u in github_usernames:
if u and u.lower() == lower:
return u
return None
print(" [GitHub] Fetching PRs...")
prs = await self._github_get_all_pages(
f"{GITHUB_API_BASE}/repos/{repo}/pulls",
{"state": "all", "sort": "created", "direction": "desc"},
)
print(f" [GitHub] {len(prs)} PRs fetched — counting opens since cutoff...")
for pr in prs:
created_at = datetime.fromisoformat(pr["created_at"].replace("Z", "+00:00"))
if created_at < CUTOFF:
break
login = pr["user"]["login"]
username = resolve_username(login)
if username:
counts[username]["prs_opened"] += 1
print(" [GitHub] Fetching issues...")
issues = await self._github_get_all_pages(
f"{GITHUB_API_BASE}/repos/{repo}/issues",
{
"state": "all",
"sort": "created",
"direction": "desc",
"since": CUTOFF_ISO,
},
)
print(f" [GitHub] {len(issues)} issues/PRs fetched — counting issue opens...")
for issue in issues:
if "pull_request" in issue:
continue
created_at = datetime.fromisoformat(
issue["created_at"].replace("Z", "+00:00")
)
if created_at < CUTOFF:
continue
login = issue["user"]["login"]
username = resolve_username(login)
if username:
counts[username]["issues_opened"] += 1
print(" [GitHub] Fetching issue comments...")
issue_comments = await self._github_get_all_pages(
f"{GITHUB_API_BASE}/repos/{repo}/issues/comments",
{"sort": "created", "direction": "desc", "since": CUTOFF_ISO},
)
print(f" [GitHub] {len(issue_comments)} issue comments fetched.")
for comment in issue_comments:
created_at = datetime.fromisoformat(
comment["created_at"].replace("Z", "+00:00")
)
if created_at < CUTOFF:
continue
login = comment["user"]["login"]
username = resolve_username(login)
if username:
counts[username]["issue_comments"] += 1
print(" [GitHub] Fetching PR review comments...")
pr_comments = await self._github_get_all_pages(
f"{GITHUB_API_BASE}/repos/{repo}/pulls/comments",
{"sort": "created", "direction": "desc", "since": CUTOFF_ISO},
)
print(f" [GitHub] {len(pr_comments)} PR review comments fetched.")
for comment in pr_comments:
created_at = datetime.fromisoformat(
comment["created_at"].replace("Z", "+00:00")
)
if created_at < CUTOFF:
continue
login = comment["user"]["login"]
username = resolve_username(login)
if username:
counts[username]["pr_comments"] += 1
all_pr_numbers = [pr["number"] for pr in prs]
total_prs = len(all_pr_numbers)
print(f" [GitHub] Fetching reviews for {total_prs} PRs...")
for i, pr_number in enumerate(all_pr_numbers, start=1):
print(f" [GitHub] PR reviews: {i}/{total_prs}", end="\r")
reviews = await self._github_get_all_pages(
f"{GITHUB_API_BASE}/repos/{repo}/pulls/{pr_number}/reviews",
{},
)
for review in reviews:
submitted_at_raw = review.get("submitted_at")
if not submitted_at_raw:
continue
submitted_at = datetime.fromisoformat(
submitted_at_raw.replace("Z", "+00:00")
)
if submitted_at < CUTOFF:
continue
login = review["user"]["login"]
username = resolve_username(login)
if username:
counts[username]["pr_reviews"] += 1
await asyncio.sleep(0.1)
if total_prs > 0:
print()
member_list = list(counts.keys())
total_members = len(member_list)
print(f" [GitHub] Fetching commits for {total_members} members...")
for i, username in enumerate(member_list, start=1):
print(f" [GitHub] Commits: {i}/{total_members} ({username})", end="\r")
commits = await self._github_get_all_pages(
f"{GITHUB_API_BASE}/repos/{repo}/commits",
{"author": username, "since": CUTOFF_ISO},
)
counts[username]["commits"] = len(commits)
await asyncio.sleep(0.2)
if total_members > 0:
print()
return counts
def build_report(
team_data: list[dict],
discord_to_github: dict[str, str],
discord_usernames: dict[str, str],
discord_results: dict[str, dict[str, int]],
github_results: dict[str, dict[str, dict[str, int]]],
) -> str:
"""Build the markdown activity report."""
lines = [
"# Catch-Up Activity Report",
"",
f"**Period:** 2026-02-15 00:00 UTC → "
f"{datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M')} UTC",
"",
"## Activity by Team",
"",
"| Discord ID | Discord Username | GitHub Username | Team | "
"Discord Messages | PRs Opened | Issues Opened | Issue Comments | "
"PR Comments | PR Reviews | Commits |",
"|------------|-----------------|-----------------|------|"
"-----------------|------------|---------------|----------------|"
"-------------|------------|---------|",
]
for team in team_data:
team_name = team["name"]
if team_name == "Jade Jasmine":
continue
member_ids = team["leaders"] + team["participants"]
team_discord_counts = discord_results.get(team_name, {})
team_github_counts = github_results.get(team_name, {})
for member_id in member_ids:
github_username = discord_to_github.get(member_id, "")
discord_username = discord_usernames.get(member_id, "*(unknown)*")
discord_msg_count = team_discord_counts.get(member_id, 0)
if github_username:
gh = team_github_counts.get(github_username, {})
prs = gh.get("prs_opened", 0)
issues = gh.get("issues_opened", 0)
issue_comments = gh.get("issue_comments", 0)
pr_comments = gh.get("pr_comments", 0)
pr_reviews = gh.get("pr_reviews", 0)
commits = gh.get("commits", 0)
else:
prs = issues = issue_comments = pr_comments = pr_reviews = commits = (
"N/A"
)
lines.append(
f"| {member_id} | {discord_username} | {github_username or 'N/A'} "
f"| {team_name} | {discord_msg_count} | {prs} | {issues} "
f"| {issue_comments} | {pr_comments} | {pr_reviews} | {commits} |"
)
lines.append("")
lines.append(
f"*Generated at {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S')} UTC*"
)
return "\n".join(lines)
async def main() -> None:
"""Run the catch-up activity report."""
print("Loading data files...")
with open(DATA_DIR / "team_assignments.json") as f:
team_data: list[dict] = json.load(f)
with open(DATA_DIR / "discord_to_github.json") as f:
discord_to_github: dict[str, str] = json.load(f)
print("Getting GitHub token via gh CLI...")
github_token = get_github_token()
print(f"\nCollecting activity since {CUTOFF.isoformat()}...\n")
discord_results: dict[str, dict[str, int]] = {}
github_results: dict[str, dict[str, dict[str, int]]] = {}
discord_usernames: dict[str, str] = {}
async with ActivityCollector(DISCORD_BOT_TOKEN, github_token) as collector:
all_member_ids: list[str] = []
for team in team_data:
if team["name"] == "Jade Jasmine":
continue
all_member_ids.extend(team["leaders"] + team["participants"])
unique_member_ids = list(dict.fromkeys(all_member_ids))
total_members = len(unique_member_ids)
print(f"Fetching Discord usernames for {total_members} members...")
for i, member_id in enumerate(unique_member_ids, start=1):
if member_id not in discord_usernames:
print(f" username {i}/{total_members}...", end="\r")
discord_usernames[member_id] = await collector.get_discord_username(
member_id
)
await asyncio.sleep(0.3)
print(f" Done — {total_members} usernames fetched. ")
for team in team_data:
team_name = team["name"]
if team_name == "Jade Jasmine":
continue
print(f"\n=== {team_name} ===")
channel_id = TEXT_CHANNEL_IDS[team_name]
member_ids = team["leaders"] + team["participants"]
discord_results[team_name] = await collector.collect_discord_counts(
team_name, channel_id, member_ids
)
github_usernames_for_team = [
discord_to_github[mid]
for mid in member_ids
if mid in discord_to_github and discord_to_github[mid]
]
github_results[team_name] = await collector.collect_github_counts(
team_name, github_usernames_for_team
)
print("\nBuilding report...")
report = build_report(
team_data,
discord_to_github,
discord_usernames,
discord_results,
github_results,
)
with open(OUTPUT_FILE, "w") as f:
f.write(report)
print(f"\n✅ Report saved to {OUTPUT_FILE}")
if __name__ == "__main__":
asyncio.run(main())
+142
View File
@@ -0,0 +1,142 @@
#!/usr/bin/env python3
"""Check cohort-team-* channels for incorrect @everyone or @cohort permissions.
Find channels where @everyone or @cohort has Send Messages or
Send Messages in Threads enabled.
"""
import asyncio
import os
import aiohttp
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
BASE_URL = "https://discord.com/api/v10"
GUILD_ID = "739845668582981683"
SEND_MESSAGES = 0x0000000000000800
SEND_MESSAGES_IN_THREADS = 0x0000004000000000
async def check_permissions() -> None:
"""Check all cohort-team-* channels for permission issues."""
headers = {"Authorization": f"Bot {DISCORD_BOT_TOKEN}"}
async with aiohttp.ClientSession() as session:
print("Fetching channels...")
async with session.get(
f"{BASE_URL}/guilds/{GUILD_ID}/channels", headers=headers
) as resp:
if resp.status != 200:
error = await resp.text()
print(f"Error fetching channels: {resp.status} - {error}")
return
channels = await resp.json()
print("Fetching roles...")
async with session.get(
f"{BASE_URL}/guilds/{GUILD_ID}/roles", headers=headers
) as resp:
if resp.status != 200:
error = await resp.text()
print(f"Error fetching roles: {resp.status} - {error}")
return
roles = await resp.json()
everyone_role_id = GUILD_ID
cohort_role_id = None
for role in roles:
if "cohort" in role["name"].lower():
cohort_role_id = role["id"]
print(f"Found cohort role: {role['name']} ({role['id']})")
break
if not cohort_role_id:
print("Warning: Could not find @cohort role!")
cohort_channels = [
ch
for ch in channels
if ch["name"].startswith("cohort-team-") and ch["type"] == 0
]
print(f"\nFound {len(cohort_channels)} cohort-team-* channels\n")
problematic_channels = []
for channel in sorted(cohort_channels, key=lambda x: x["name"]):
channel_name = channel["name"]
channel_id = channel["id"]
permission_overwrites = channel.get("permission_overwrites", [])
everyone_perms = None
cohort_perms = None
for overwrite in permission_overwrites:
if overwrite["id"] == everyone_role_id:
everyone_perms = overwrite
elif cohort_role_id and overwrite["id"] == cohort_role_id:
cohort_perms = overwrite
issues = []
if everyone_perms:
deny = int(everyone_perms.get("deny", "0"))
allow = int(everyone_perms.get("allow", "0"))
if (allow & SEND_MESSAGES) or not (deny & SEND_MESSAGES):
issues.append("@everyone can send messages")
if (allow & SEND_MESSAGES_IN_THREADS) or not (
deny & SEND_MESSAGES_IN_THREADS
):
issues.append("@everyone can send messages in threads")
else:
issues.append(
"@everyone has no permission overwrite (inheriting server perms)"
)
if cohort_perms and cohort_role_id:
deny = int(cohort_perms.get("deny", "0"))
allow = int(cohort_perms.get("allow", "0"))
if (allow & SEND_MESSAGES) or not (deny & SEND_MESSAGES):
issues.append("@cohort can send messages")
if (allow & SEND_MESSAGES_IN_THREADS) or not (
deny & SEND_MESSAGES_IN_THREADS
):
issues.append("@cohort can send messages in threads")
elif cohort_role_id:
issues.append(
"@cohort has no permission overwrite (inheriting server perms)"
)
if issues:
problematic_channels.append(
{"name": channel_name, "id": channel_id, "issues": issues}
)
print(f"{channel_name}")
for issue in issues:
print(f" - {issue}")
else:
print(f"{channel_name}")
print("\n" + "=" * 60)
print(
f"\nSummary: {len(problematic_channels)} channels with permission issues\n"
)
if problematic_channels:
print("Problematic channels:")
for ch in problematic_channels:
print(f"\n{ch['name']} (ID: {ch['id']})")
for issue in ch["issues"]:
print(f"{issue}")
else:
print("All channels have correct permissions! 🎉")
if __name__ == "__main__":
asyncio.run(check_permissions())
+86
View File
@@ -0,0 +1,86 @@
"""Dry-run check of Discord message lengths before sending the activity report.
Parses the catch_up_report.md table, formats each team's data into a monospace
Discord table, and reports whether any message would exceed Discord's 2000-char limit.
Run this before send_activity_report.py to catch length issues early.
Data files (place in data/):
- catch_up_report.md Activity report generated by catch_up_report.py
Env vars:
- None
"""
FIELDS = [
("Discord Username", "Name", 18),
("Discord Messages", "Msgs", 5),
("PRs Opened", "PRs", 4),
("Issues Opened", "Issues", 6),
("Issue Comments", "Issue♟", 7),
("PR Comments", "PR♟", 5),
("PR Reviews", "Reviews", 7),
("Commits", "Commits", 7),
]
REPORT_PATH = "data/catch_up_report.md"
def parse_report(path: str) -> dict[str, list[dict]]:
"""Parse the markdown table from catch_up_report.md into team → rows."""
teams: dict[str, list[dict]] = {}
with open(path, encoding="utf-8") as f:
lines = f.readlines()
header_line = None
for i, line in enumerate(lines):
if line.startswith("| Discord ID |"):
header_line = i
break
headers = [h.strip() for h in lines[header_line].strip().strip("|").split("|")]
for line in lines[header_line + 2 :]:
line = line.strip()
if not line.startswith("|"):
break
vals = [v.strip() for v in line.strip().strip("|").split("|")]
row = dict(zip(headers, vals))
teams.setdefault(row["Team"], []).append(row)
return teams
def format_table(members: list[dict]) -> str:
"""Format a team's member list as a monospace table for Discord."""
members = sorted(members, key=lambda r: int(r["Discord Messages"]), reverse=True)
col_widths = [w for _, _, w in FIELDS]
col_headers = [h for _, h, _ in FIELDS]
max_name = max(len(m["Discord Username"]) for m in members)
col_widths[0] = max(col_widths[0], max_name)
def pad(val: str, width: int, right_align: bool = False) -> str:
return val.rjust(width) if right_align else val.ljust(width)
header_row = " ".join(
pad(col_headers[i], col_widths[i], right_align=(i > 0))
for i in range(len(FIELDS))
)
separator = " ".join("-" * w for w in col_widths)
rows = []
for m in members:
vals = [m[key] for key, _, _ in FIELDS]
row = " ".join(
pad(vals[i], col_widths[i], right_align=(i > 0)) for i in range(len(FIELDS))
)
rows.append(row)
return "\n".join([header_row, separator] + rows)
def main() -> None:
"""Check Discord message lengths for all teams."""
teams = parse_report(REPORT_PATH)
for team, members in teams.items():
table = format_table(members)
msg = f"**{team} — Activity Report (Feb 1523)**\n```\n{table}\n```"
status = "OK" if len(msg) <= 2000 else f"OVER by {len(msg) - 2000}"
print(f"{team}: {len(msg)} chars — {status}")
if __name__ == "__main__":
main()
+49
View File
@@ -0,0 +1,49 @@
#!/usr/bin/env python3
"""Check if removed members are still in the Discord server."""
import asyncio
import os
import aiohttp
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
BASE_URL = "https://discord.com/api/v10"
GUILD_ID = "1354624415861833870"
SAMPLE_MEMBERS = [
"899092786802987069",
"1318882254365397032",
"799293680799711273",
"237793557992308736",
]
async def check_member(session: aiohttp.ClientSession, user_id: str) -> bool | None:
"""Check if a member is in the server."""
url = f"{BASE_URL}/guilds/{GUILD_ID}/members/{user_id}"
headers = {"Authorization": f"Bot {DISCORD_BOT_TOKEN}"}
async with session.get(url, headers=headers) as resp:
if resp.status == 200:
data = await resp.json()
roles = data.get("roles", [])
print(f"✅ User {user_id} IS in server - has {len(roles)} roles: {roles}")
return True
if resp.status == 404:
print(f"❌ User {user_id} NOT in server (left or was kicked)")
return False
error = await resp.text()
print(f"⚠️ Error checking {user_id}: {resp.status} - {error}")
return None
async def main() -> None:
"""Check if sample members are still in the server."""
async with aiohttp.ClientSession() as session:
for user_id in SAMPLE_MEMBERS:
await check_member(session, user_id)
await asyncio.sleep(1)
if __name__ == "__main__":
asyncio.run(main())
+18 -4
View File
@@ -1,6 +1,17 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Discord Team Activity Checker """Check for team members who have not sent a message in their channel within 36 hours.
Checks for team members who haven't sent messages in their channels within 36 hours
Scans each team's Discord channel and flags members with no recent activity.
Optionally sends a direct mention message to inactive members.
Data files (place in data/):
- team_assignments.json Team rosters with leaders and participants per team
Outputs (written to data/):
- discord_activity_report.json Inactive members per team with last-seen timestamps
Env vars:
- DISCORD_BOT_TOKEN Bot token for the Discord API
""" """
import asyncio import asyncio
@@ -8,16 +19,19 @@ import json
import os import os
import sys import sys
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from pathlib import Path
import aiohttp import aiohttp
DATA_DIR = Path(__file__).parent.parent.parent / "data"
# Configuration # Configuration
DISCORD_TOKEN = os.environ["DISCORD_BOT_TOKEN"] DISCORD_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
DISCORD_API_BASE = "https://discord.com/api/v10" DISCORD_API_BASE = "https://discord.com/api/v10"
INACTIVE_THRESHOLD_HOURS = 36 INACTIVE_THRESHOLD_HOURS = 36
# Load team assignments from file # Load team assignments from file
with open("team_assignments.json") as f: with open(DATA_DIR / "team_assignments.json") as f:
team_data = json.load(f) team_data = json.load(f)
# Build TEAMS dictionary with channel IDs and member lists # Build TEAMS dictionary with channel IDs and member lists
@@ -233,7 +247,7 @@ async def main():
print("\n" + "=" * 80) print("\n" + "=" * 80)
# Save results to JSON # Save results to JSON
with open("discord_activity_report.json", "w") as f: with open(DATA_DIR / "discord_activity_report.json", "w") as f:
json.dump( json.dump(
{ {
"generated_at": datetime.now(timezone.utc).isoformat(), "generated_at": datetime.now(timezone.utc).isoformat(),
@@ -1,8 +1,27 @@
"""Evaluate the technical proficiency of cohort applicants using their GitHub profiles.
Fetches each applicant's public GitHub repositories and scores their proficiency as
Beginner, Intermediate, or Advanced based on language variety, repo count, commit
activity, and presence of certain technologies.
Data files (place in data/):
- applicants_to_evaluate.json List of applicants with GitHub usernames
Outputs (written to data/):
- proficiency_evaluations.json Proficiency scores and tech stacks per applicant
Env vars:
- None (uses public GitHub API; may be rate-limited without authentication)
"""
import json import json
import re import re
import time import time
import urllib.error import urllib.error
import urllib.request import urllib.request
from pathlib import Path
DATA_DIR = Path(__file__).parent.parent.parent / "data"
# GitHub API (no auth needed for public repos, but rate limited) # GitHub API (no auth needed for public repos, but rate limited)
GITHUB_API = "https://api.github.com" GITHUB_API = "https://api.github.com"
@@ -234,7 +253,7 @@ def evaluate_applicant(applicant: dict, index: int, total: int) -> dict:
def main(): def main():
# Load applicants # Load applicants
with open("applicants_to_evaluate.json") as f: with open(DATA_DIR / "applicants_to_evaluate.json") as f:
applicants = json.load(f) applicants = json.load(f)
print(f"Evaluating {len(applicants)} applicants...\n") print(f"Evaluating {len(applicants)} applicants...\n")
@@ -249,7 +268,7 @@ def main():
print(f" Progress: {i + 1}/{len(applicants)} complete") print(f" Progress: {i + 1}/{len(applicants)} complete")
# Save results # Save results
with open("proficiency_evaluations.json", "w") as f: with open(DATA_DIR / "proficiency_evaluations.json", "w") as f:
json.dump(evaluations, f, indent=2) json.dump(evaluations, f, indent=2)
# Summary # Summary
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env python3
"""Fetch pinned messages from Ivory Orchid channel to find the roster."""
import asyncio
import json
import os
import aiohttp
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
CHANNEL_ID = "1464316770889240730"
async def main() -> None:
"""Fetch and print pinned messages from the Ivory Orchid channel."""
headers = {
"Authorization": f"Bot {DISCORD_BOT_TOKEN}",
"Content-Type": "application/json",
}
async with (
aiohttp.ClientSession() as session,
session.get(
f"https://discord.com/api/v10/channels/{CHANNEL_ID}/pins",
headers=headers,
) as response,
):
pins = await response.json()
print(json.dumps(pins, indent=2))
if __name__ == "__main__":
asyncio.run(main())
+129
View File
@@ -0,0 +1,129 @@
#!/usr/bin/env python3
"""Fix Send Messages / Send Messages in Threads permissions for a Discord channel.
Denies Send Messages and Send Messages in Threads for both @everyone and the
@cohort role on the target channel. Update CHANNEL_ID and COHORT_ROLE_ID before
running.
Data files (place in data/):
- None
Env vars:
- DISCORD_BOT_TOKEN Bot token for the Discord API
"""
import asyncio
import os
import aiohttp
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
BASE_URL = "https://discord.com/api/v10"
GUILD_ID = "739845668582981683"
CHANNEL_ID = "1467964405646885237"
SEND_MESSAGES = 0x0000000000000800
SEND_MESSAGES_IN_THREADS = 0x0000004000000000
EVERYONE_ROLE_ID = GUILD_ID
COHORT_ROLE_ID = "1390925253102010521"
async def fix_permissions() -> None:
"""Fix the channel permissions."""
headers = {
"Authorization": f"Bot {DISCORD_BOT_TOKEN}",
"Content-Type": "application/json",
}
async with aiohttp.ClientSession() as session:
print("Fetching current channel permissions...")
async with session.get(
f"{BASE_URL}/channels/{CHANNEL_ID}", headers=headers
) as resp:
if resp.status != 200:
error = await resp.text()
print(f"Error fetching channel: {resp.status} - {error}")
return
channel = await resp.json()
print(f"Channel: {channel['name']}")
permission_overwrites = channel.get("permission_overwrites", [])
everyone_overwrite = None
cohort_overwrite = None
for overwrite in permission_overwrites:
if overwrite["id"] == EVERYONE_ROLE_ID:
everyone_overwrite = overwrite
elif overwrite["id"] == COHORT_ROLE_ID:
cohort_overwrite = overwrite
print("\nFixing @everyone permissions...")
if everyone_overwrite:
current_deny = int(everyone_overwrite.get("deny", "0"))
current_allow = int(everyone_overwrite.get("allow", "0"))
new_deny = current_deny | SEND_MESSAGES | SEND_MESSAGES_IN_THREADS
new_allow = current_allow & ~SEND_MESSAGES & ~SEND_MESSAGES_IN_THREADS
payload = {
"type": 0,
"deny": str(new_deny),
"allow": str(new_allow),
}
else:
payload = {
"type": 0,
"deny": str(SEND_MESSAGES | SEND_MESSAGES_IN_THREADS),
"allow": "0",
}
async with session.put(
f"{BASE_URL}/channels/{CHANNEL_ID}/permissions/{EVERYONE_ROLE_ID}",
headers=headers,
json=payload,
) as resp:
if resp.status == 204:
print("✅ @everyone permissions fixed!")
else:
error = await resp.text()
print(f"❌ Error fixing @everyone: {resp.status} - {error}")
print("\nFixing @cohort permissions...")
if cohort_overwrite:
current_deny = int(cohort_overwrite.get("deny", "0"))
current_allow = int(cohort_overwrite.get("allow", "0"))
new_deny = current_deny | SEND_MESSAGES | SEND_MESSAGES_IN_THREADS
new_allow = current_allow & ~SEND_MESSAGES & ~SEND_MESSAGES_IN_THREADS
payload = {
"type": 0,
"deny": str(new_deny),
"allow": str(new_allow),
}
else:
payload = {
"type": 0,
"deny": str(SEND_MESSAGES | SEND_MESSAGES_IN_THREADS),
"allow": "0",
}
async with session.put(
f"{BASE_URL}/channels/{CHANNEL_ID}/permissions/{COHORT_ROLE_ID}",
headers=headers,
json=payload,
) as resp:
if resp.status == 204:
print("✅ @cohort permissions fixed!")
else:
error = await resp.text()
print(f"❌ Error fixing @cohort: {resp.status} - {error}")
print("\n✨ Done! Permissions have been fixed.")
if __name__ == "__main__":
asyncio.run(fix_permissions())
+30 -7
View File
@@ -1,23 +1,46 @@
"""Generate markdown participant and leader profile files for the cohort.
Reads all evaluation data files and produces two markdown files summarising
each member's tech stack, availability, proficiency, and leadership assessment.
Data files (place in data/):
- discord_verification.json Discord ID verification results (from verify_discord.py)
- proficiency_evaluations.json Proficiency scores (from evaluate_technical_proficiency.py)
- availability_analysis.json Availability UTC blocks (from analyse_availability.py)
- leadership_candidates.json List of applicants who expressed interest in leading
- leadership_evaluations.json Leadership assessment scores
Outputs (written to data/):
- participants.md Markdown profile for each participant
- leaders.md Markdown profile for each leader candidate
Env vars:
- None
"""
import json import json
from pathlib import Path
DATA_DIR = Path(__file__).parent.parent.parent / "data"
BLOCK_EMOJIS = {"mornings": "🌅", "afternoons": "☀️", "evenings": "🌆", "nights": "🌙"} BLOCK_EMOJIS = {"mornings": "🌅", "afternoons": "☀️", "evenings": "🌆", "nights": "🌙"}
def load_all_data(): def load_all_data():
"""Load all evaluation data files""" """Load all evaluation data files"""
with open("discord_verification.json") as f: with open(DATA_DIR / "discord_verification.json") as f:
verification = json.load(f) verification = json.load(f)
with open("proficiency_evaluations.json") as f: with open(DATA_DIR / "proficiency_evaluations.json") as f:
proficiency = json.load(f) proficiency = json.load(f)
with open("availability_analysis.json") as f: with open(DATA_DIR / "availability_analysis.json") as f:
availability = json.load(f) availability = json.load(f)
with open("leadership_candidates.json") as f: with open(DATA_DIR / "leadership_candidates.json") as f:
candidates = json.load(f) candidates = json.load(f)
with open("leadership_evaluations.json") as f: with open(DATA_DIR / "leadership_evaluations.json") as f:
leadership = json.load(f) leadership = json.load(f)
return verification, proficiency, availability, candidates, leadership return verification, proficiency, availability, candidates, leadership
@@ -230,14 +253,14 @@ def main():
participants_md = generate_participants_md( participants_md = generate_participants_md(
non_leader_ids, verified_usernames, prof_by_id, avail_by_id non_leader_ids, verified_usernames, prof_by_id, avail_by_id
) )
with open("participants.md", "w") as f: with open(DATA_DIR / "participants.md", "w") as f:
f.write(participants_md) f.write(participants_md)
print(f"Generated participants.md with {len(non_leader_ids)} participants") print(f"Generated participants.md with {len(non_leader_ids)} participants")
leaders_md = generate_leaders_md( leaders_md = generate_leaders_md(
leader_ids, verified_usernames, prof_by_id, avail_by_id, lead_by_id leader_ids, verified_usernames, prof_by_id, avail_by_id, lead_by_id
) )
with open("leaders.md", "w") as f: with open(DATA_DIR / "leaders.md", "w") as f:
f.write(leaders_md) f.write(leaders_md)
print(f"Generated leaders.md with {len(leader_ids)} leaders") print(f"Generated leaders.md with {len(leader_ids)} leaders")
+16 -1
View File
@@ -1,5 +1,20 @@
"""Generate hourly timeslot JSON for use with Crabfit scheduling tool.
Produces a list of ISO-format datetime strings covering every hour across the
scheduling window. Update the start_date and end_date constants before running.
Outputs (written to data/):
- crabfit_timeslots.json List of hourly timeslot strings
Env vars:
- None
"""
import json import json
from datetime import datetime, timedelta from datetime import datetime, timedelta
from pathlib import Path
DATA_DIR = Path(__file__).parent.parent.parent / "data"
# Generate hourly time slots from Feb 1 to March 3, 2026 # Generate hourly time slots from Feb 1 to March 3, 2026
# 24 hours a day, America/Los_Angeles timezone # 24 hours a day, America/Los_Angeles timezone
@@ -18,7 +33,7 @@ print(f"First: {times[0]}")
print(f"Last: {times[-1]}") print(f"Last: {times[-1]}")
# Save to file for use # Save to file for use
with open("/home/naomi/docs/cohort/crabfit_timeslots.json", "w") as f: with open(DATA_DIR / "crabfit_timeslots.json", "w") as f:
json.dump(times, f) json.dump(times, f)
print("Saved to crabfit_timeslots.json") print("Saved to crabfit_timeslots.json")
+88
View File
@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""Get all members who currently have the Cohort role."""
import asyncio
import json
import os
from pathlib import Path
import aiohttp
DATA_DIR = Path(__file__).parent.parent.parent / "data"
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
GUILD_ID = "692816967895220344"
COHORT_ROLE_ID = "1464314780935258112"
async def get_all_members_with_role(
session: aiohttp.ClientSession,
) -> list[dict[str, str]]:
"""Get all members who have the Cohort role."""
members = []
headers = {"Authorization": f"Bot {DISCORD_BOT_TOKEN}"}
after = None
while True:
url = f"https://discord.com/api/v10/guilds/{GUILD_ID}/members"
params: dict[str, str | int] = {"limit": 1000}
if after:
params["after"] = after
async with session.get(url, headers=headers, params=params) as resp:
if resp.status != 200:
error_text = await resp.text()
raise RuntimeError(
f"Failed to fetch members: {resp.status} - {error_text}"
)
data = await resp.json()
if not data:
break
for member in data:
if COHORT_ROLE_ID in member.get("roles", []):
members.append(
{
"id": member["user"]["id"],
"username": member["user"]["username"],
"display_name": member.get("nick")
or member["user"]["username"],
}
)
if len(data) < 1000:
break
after = data[-1]["user"]["id"]
return members
async def main() -> None:
"""Get all cohort members with the Cohort role."""
async with aiohttp.ClientSession() as session:
print("Fetching all members with Cohort role...")
cohort_members = await get_all_members_with_role(session)
print(f"\n✨ Found {len(cohort_members)} members with the Cohort role:\n")
for i, member in enumerate(cohort_members, 1):
print(
f"{i}. {member['display_name']} (@{member['username']}) - ID: {member['id']}" # noqa: E501
)
with open(DATA_DIR / "active_cohort_members.json", "w") as f:
json.dump(cohort_members, f, indent=2)
print("\nSaved to active_cohort_members.json")
print("\nMember IDs only:")
print(json.dumps([m["id"] for m in cohort_members], indent=2))
if __name__ == "__main__":
asyncio.run(main())
+61
View File
@@ -0,0 +1,61 @@
#!/usr/bin/env python3
"""List all roles in the guild to find the correct team role IDs."""
import asyncio
import os
import aiohttp
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
BASE_URL = "https://discord.com/api/v10"
GUILD_ID = "1354624415861833870"
async def list_guild_roles() -> None:
"""List all roles in the guild, highlighting team-related ones."""
url = f"{BASE_URL}/guilds/{GUILD_ID}/roles"
headers = {"Authorization": f"Bot {DISCORD_BOT_TOKEN}"}
async with (
aiohttp.ClientSession() as session,
session.get(url, headers=headers) as resp,
):
if resp.status == 200:
roles = await resp.json()
print(f"Found {len(roles)} roles in the server:\n")
team_names = [
"Jade Jasmine",
"Crimson Dahlia",
"Rose Camellia",
"Amber Wisteria",
"Ivory Orchid",
"Teal Iris",
"Peach Gardenia",
"Violet Carnation",
"Azure Lotus",
"Coral Sunflower",
"Indigo Tulip",
"Scarlet Hydrangea",
"Mint Narcissus",
"Sage Marigold",
"Cohort",
]
for role in sorted(roles, key=lambda r: r["name"].lower()):
name = role["name"]
is_team = any(team in name for team in team_names)
if is_team or "spring" in name.lower() or "2026" in name:
print(f"{role['id']}: {name}")
print("\n\nAll roles (sorted by name):")
for role in sorted(roles, key=lambda r: r["name"].lower()):
print(f"{role['id']}: {role['name']}")
else:
error = await resp.text()
print(f"Error: {resp.status} - {error}")
if __name__ == "__main__":
asyncio.run(list_guild_roles())
+48
View File
@@ -0,0 +1,48 @@
"""List all Discord roles in the freeCodeCamp server."""
import asyncio
import os
import aiohttp
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
GUILD_ID = "692816967895220344"
API_BASE = "https://discord.com/api/v10"
async def get_guild_roles(session: aiohttp.ClientSession) -> list[dict]:
"""Get all guild roles."""
url = f"{API_BASE}/guilds/{GUILD_ID}/roles"
headers = {"Authorization": f"Bot {DISCORD_BOT_TOKEN}"}
async with session.get(url, headers=headers) as resp:
if resp.status == 200:
return await resp.json()
print(f"Failed to get roles: {resp.status}")
text = await resp.text()
print(text)
return []
async def main() -> None:
"""List all roles from freeCodeCamp Discord, highlighting cohort-related ones."""
async with aiohttp.ClientSession() as session:
print("Fetching all roles from freeCodeCamp Discord...\n")
roles = await get_guild_roles(session)
print(f"Found {len(roles)} roles:\n")
cohort_roles = [
r
for r in roles
if "2026" in r["name"] or "Cohort" in r["name"] or "Leader" in r["name"]
]
print("=== Cohort/Leader Roles ===")
for role in cohort_roles:
print(f" {role['name']}: {role['id']}")
if __name__ == "__main__":
asyncio.run(main())
+116
View File
@@ -0,0 +1,116 @@
#!/usr/bin/env python3
"""Remove the Cohort and team-specific Discord roles from a list of members.
Update INACTIVE_MEMBERS and MEMBER_TO_TEAM before running to target the correct
members. Removes both the cohort-wide role and the member's team role.
Data files (place in data/):
- None (member IDs and team mappings are defined as constants in the script)
Env vars:
- DISCORD_BOT_TOKEN Bot token for the Discord API
"""
import asyncio
import os
import aiohttp
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
GUILD_ID = "1354624415861833870"
BASE_URL = "https://discord.com/api/v10"
INACTIVE_MEMBERS = [
"1177481351490981889",
"899092786802987069",
"1318882254365397032",
"680429718511943770",
"1195902319976521748",
"1424001797072359576",
"1333378962289590365",
"799293680799711273",
"1183395404293869662",
"1325958873831440566",
"717028253633871965",
"847789364217905182",
"746703502369095700",
"192541018908786690",
"1017761694514163712",
]
COHORT_ROLE_ID = "1464314780935258112"
TEAM_ROLE_IDS = {
1: "1464314923780931677",
2: "1464315093402784015",
3: "1464315098452726106",
4: "1464315105264275600",
5: "1464315109873684593",
6: "1464315114378498152",
7: "1464315118904152107",
8: "1464315124251754559",
9: "1464315128437801177",
10: "1464315132896088168",
11: "1464315138428633241",
12: "1464315142710890520",
13: "1464315149203804405",
14: "1464315153599299803",
}
MEMBER_TO_TEAM = {
"1177481351490981889": 1,
"899092786802987069": 1,
"1318882254365397032": 2,
"680429718511943770": 2,
"1195902319976521748": 3,
"1424001797072359576": 4,
"1333378962289590365": 4,
"799293680799711273": 5,
"1183395404293869662": 7,
"1325958873831440566": 8,
"717028253633871965": 8,
"847789364217905182": 10,
"746703502369095700": 10,
"192541018908786690": 11,
"1017761694514163712": 13,
}
async def remove_role(
session: aiohttp.ClientSession, user_id: str, role_id: str
) -> bool:
"""Remove a role from a user."""
url = f"{BASE_URL}/guilds/{GUILD_ID}/members/{user_id}/roles/{role_id}"
headers = {"Authorization": f"Bot {DISCORD_BOT_TOKEN}"}
async with session.delete(url, headers=headers) as resp:
if resp.status == 204:
return True
text = await resp.text()
print(f" Error removing role {role_id} from {user_id}: {resp.status} - {text}")
return False
async def main() -> None:
"""Remove roles from all inactive members."""
async with aiohttp.ClientSession() as session:
print("Removing roles from inactive members...\n")
for member_id in INACTIVE_MEMBERS:
print(f"Processing {member_id}...")
if await remove_role(session, member_id, COHORT_ROLE_ID):
print(" ✓ Removed cohort role")
if member_id in MEMBER_TO_TEAM:
team_id = MEMBER_TO_TEAM[member_id]
team_role_id = TEAM_ROLE_IDS[team_id]
if await remove_role(session, member_id, team_role_id):
print(f" ✓ Removed Team {team_id} role")
await asyncio.sleep(1.5)
print("\nDone!")
if __name__ == "__main__":
asyncio.run(main())
+247
View File
@@ -0,0 +1,247 @@
#!/usr/bin/env python3
"""Remove a member from the Spring 2026 Cohort.
This script:
1. Removes from team_assignments.json (so activity checker stops tracking them)
2. Removes from discord_to_github.json
3. Removes from GitHub teams
4. Removes Discord roles
5. Sends a message to the team channel announcing the removal
6. Outputs notes to add to COHORT_NOTES.md
Usage:
python remove_member.py <discord_id>
"""
import asyncio
import json
import os
import sys
from datetime import datetime, timezone
from pathlib import Path
import aiohttp
DATA_DIR = Path(__file__).parent.parent.parent / "data"
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
DISCORD_API_BASE = "https://discord.com/api/v10"
DISCORD_GUILD_ID = "692816967895220344"
COHORT_ROLE_ID = "1464316447591985194"
TEXT_CHANNEL_IDS = {
"Jade Jasmine": "1464316501573107886",
"Crimson Dahlia": "1464316744909852682",
"Rose Camellia": "1464316751268286611",
"Amber Wisteria": "1464316761410113641",
"Ivory Orchid": "1464316770889240730",
"Teal Iris": "1464316776459407448",
"Peach Gardenia": "1464316785040953543",
"Violet Carnation": "1464316805261824032",
"Azure Lotus": "1464316814455472139",
"Coral Sunflower": "1464316819711066263",
"Indigo Tulip": "1464316826384072925",
"Scarlet Hydrangea": "1464316839306985506",
"Mint Narcissus": "1464316844251807952",
"Sage Marigold": "1464316850669093040",
}
class MemberRemover:
"""Handles the complete removal of a cohort member."""
def __init__(self, discord_id: str) -> None:
self.discord_id = discord_id
self.headers = {
"Authorization": f"Bot {DISCORD_BOT_TOKEN}",
"Content-Type": "application/json",
}
self.session: aiohttp.ClientSession | None = None
self.github_username: str | None = None
self.team_name: str | None = None
self.role: str | None = None
self.team_role_id: str | None = None
async def __aenter__(self) -> "MemberRemover":
self.session = aiohttp.ClientSession()
return self
async def __aexit__(
self, exc_type: object, exc_val: object, exc_tb: object
) -> None:
if self.session:
await self.session.close()
def find_member_info(self) -> bool:
"""Find which team the member is on and their role."""
with open(DATA_DIR / "team_assignments.json") as f:
teams = json.load(f)
for team in teams:
if self.discord_id in team["leaders"]:
self.team_name = team["name"]
self.role = "leader"
return True
if self.discord_id in team["participants"]:
self.team_name = team["name"]
self.role = "participant"
return True
return False
def remove_from_team_assignments(self) -> None:
"""Remove member from team_assignments.json."""
with open(DATA_DIR / "team_assignments.json") as f:
teams = json.load(f)
for team in teams:
if self.discord_id in team["leaders"]:
team["leaders"].remove(self.discord_id)
print(
f"✅ Removed from {team['name']} leaders in team_assignments.json"
)
elif self.discord_id in team["participants"]:
team["participants"].remove(self.discord_id)
print(
f"✅ Removed from {team['name']} participants in team_assignments.json"
)
with open(DATA_DIR / "team_assignments.json", "w") as f:
json.dump(teams, f, indent=2)
def remove_from_discord_to_github(self) -> None:
"""Remove member from discord_to_github.json."""
with open(DATA_DIR / "discord_to_github.json") as f:
mappings = json.load(f)
if self.discord_id in mappings:
self.github_username = mappings[self.discord_id]
del mappings[self.discord_id]
with open(DATA_DIR / "discord_to_github.json", "w") as f:
json.dump(mappings, f, indent=2)
print(f"✅ Removed {self.github_username} from discord_to_github.json")
else:
print("⚠️ No GitHub username found in discord_to_github.json")
async def remove_from_github_teams(self) -> None:
"""Print instructions to remove member from GitHub organization teams."""
if not self.github_username:
print("⚠️ Skipping GitHub removal (no username)")
return
print(f"️ Please manually remove {self.github_username} from GitHub teams:")
print(f" - Team: {self.team_name}")
if self.role == "leader":
print(f" - Team: {self.team_name} Leaders")
async def remove_discord_roles(self) -> None:
"""Remove Discord roles from the member."""
with open(DATA_DIR / "team_message_ids.json") as f:
team_message_data = json.load(f)
if self.team_name not in team_message_data:
print(f"⚠️ Could not find role ID for team {self.team_name}")
return
self.team_role_id = team_message_data[self.team_name]["role_id"]
url = (
f"{DISCORD_API_BASE}/guilds/{DISCORD_GUILD_ID}"
f"/members/{self.discord_id}/roles/{self.team_role_id}"
)
async with self.session.delete(url, headers=self.headers) as response:
if response.status == 204:
print(f"✅ Removed {self.team_name} Discord role")
else:
print(f"❌ Failed to remove team role: {response.status}")
url = (
f"{DISCORD_API_BASE}/guilds/{DISCORD_GUILD_ID}"
f"/members/{self.discord_id}/roles/{COHORT_ROLE_ID}"
)
async with self.session.delete(url, headers=self.headers) as response:
if response.status == 204:
print("✅ Removed Spring 2026 Cohort Discord role")
else:
print(f"❌ Failed to remove cohort role: {response.status}")
async def send_team_message(self) -> None:
"""Send a message to the team channel announcing the removal."""
channel_id = TEXT_CHANNEL_IDS[self.team_name]
message = (
f"<@{self.discord_id}> has been removed from the cohort for inactivity."
)
url = f"{DISCORD_API_BASE}/channels/{channel_id}/messages"
data = {"content": message}
async with self.session.post(url, headers=self.headers, json=data) as response:
if response.status == 200:
print(f"✅ Sent removal message to {self.team_name} channel")
else:
print(f"❌ Failed to send message: {response.status}")
def generate_cohort_notes(self) -> str:
"""Generate text to add to COHORT_NOTES.md."""
today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
notes = f"\n## {today} - Member Removal\n\n"
notes += "| Discord ID | GitHub Username | Team | Role |\n"
notes += "|------------|-----------------|------|------|\n"
notes += (
f"| {self.discord_id} | {self.github_username or '-'} "
f"| {self.team_name} | {self.role.capitalize()} |\n"
)
return notes
async def main() -> None:
"""Remove a member from the cohort."""
if len(sys.argv) != 2:
print("Usage: python remove_member.py <discord_id>")
sys.exit(1)
discord_id = sys.argv[1]
async with MemberRemover(discord_id) as remover:
if not remover.find_member_info():
print(f"❌ Discord ID {discord_id} not found in any team")
sys.exit(1)
print("\n📋 Found member:")
print(f" Discord ID: {discord_id}")
print(f" Team: {remover.team_name}")
print(f" Role: {remover.role}")
print()
confirm = input("Proceed with removal? (yes/no): ")
if confirm.lower() != "yes":
print("❌ Removal cancelled")
sys.exit(0)
print("\n🚀 Starting removal process...\n")
remover.remove_from_team_assignments()
remover.remove_from_discord_to_github()
await remover.remove_from_github_teams()
await remover.remove_discord_roles()
await remover.send_team_message()
print("\n" + "=" * 80)
print("📝 Add this to COHORT_NOTES.md:")
print("=" * 80)
print(remover.generate_cohort_notes())
print("=" * 80)
print("\n✅ Member removal complete!")
if __name__ == "__main__":
asyncio.run(main())
+63
View File
@@ -0,0 +1,63 @@
#!/usr/bin/env python3
"""Remove resigned members from team_assignments.json.
Update RESIGNED_IDS with the Discord IDs of members who have resigned
before running this script.
"""
import json
from pathlib import Path
DATA_DIR = Path(__file__).parent.parent.parent / "data"
# Discord IDs of members who have resigned - update before running
RESIGNED_IDS: list[str] = []
def main() -> None:
"""Remove resigned members from team assignments."""
with open(DATA_DIR / "team_assignments.json") as f:
teams = json.load(f)
changes_made = []
for team in teams:
team_name = team["name"]
original_leaders = len(team["leaders"])
team["leaders"] = [lid for lid in team["leaders"] if lid not in RESIGNED_IDS]
if len(team["leaders"]) < original_leaders:
removed = original_leaders - len(team["leaders"])
changes_made.append(f"Removed {removed} leader(s) from {team_name}")
original_participants = len(team["participants"])
team["participants"] = [
pid for pid in team["participants"] if pid not in RESIGNED_IDS
]
if len(team["participants"]) < original_participants:
removed = original_participants - len(team["participants"])
changes_made.append(f"Removed {removed} participant(s) from {team_name}")
print(f"⚠️ {team_name}: Proficiency distribution may need updating")
with open(DATA_DIR / "team_assignments.json", "w") as f:
json.dump(teams, f, indent=2)
if changes_made:
print("✅ Successfully removed resigned members from team_assignments.json")
print("\nChanges made:")
for change in changes_made:
print(f" - {change}")
else:
print("No changes needed - resigned members were not found in team assignments")
print("\nUpdated team sizes:")
for team in teams:
total_members = len(team["leaders"]) + len(team["participants"])
print(
f" {team['name']}: {total_members} members "
f"({len(team['leaders'])} leaders, {len(team['participants'])} participants)" # noqa: E501
)
if __name__ == "__main__":
main()
+178
View File
@@ -0,0 +1,178 @@
"""Send formatted activity report tables to each team's Discord channel.
Parses catch_up_report.md and posts a monospace table of each member's Discord
and GitHub activity stats to their respective team channel.
Data files (place in data/):
- catch_up_report.md Activity report generated by catch_up_report.py
Env vars:
- DISCORD_BOT_TOKEN Bot token for the Discord API
"""
import asyncio
import os
import aiohttp
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
GUILD_ID = "692816967895220344"
API_BASE = "https://discord.com/api/v10"
CHANNEL_IDS = {
"Crimson Dahlia": "1464316744909852682",
"Rose Camellia": "1464316751268286611",
"Amber Wisteria": "1464316761410113641",
"Ivory Orchid": "1464316770889240730",
"Teal Iris": "1464316776459407448",
"Peach Gardenia": "1464316785040953543",
"Violet Carnation": "1464316805261824032",
"Azure Lotus": "1464316814455472139",
"Coral Sunflower": "1464316819711066263",
"Indigo Tulip": "1464316826384072925",
"Scarlet Hydrangea": "1464316839306985506",
"Mint Narcissus": "1464316844251807952",
"Sage Marigold": "1464316850669093040",
}
REPORT_PATH = os.path.join(
os.path.dirname(__file__), "..", "..", "data", "catch_up_report.md"
)
FIELDS = [
("Discord Username", "Name", 18),
("Discord Messages", "Msgs", 5),
("PRs Opened", "PRs", 4),
("Issues Opened", "Issues", 6),
("Issue Comments", "Issue♟", 7),
("PR Comments", "PR♟", 5),
("PR Reviews", "Reviews", 7),
("Commits", "Commits", 7),
]
def parse_report(path: str) -> dict[str, list[dict]]:
"""Parse the markdown table from catch_up_report.md into team → rows."""
teams: dict[str, list[dict]] = {}
with open(path, encoding="utf-8") as f:
lines = f.readlines()
header_line = None
for i, line in enumerate(lines):
if line.startswith("| Discord ID |"):
header_line = i
break
if header_line is None:
raise ValueError("Could not find table header in report")
headers = [h.strip() for h in lines[header_line].strip().strip("|").split("|")]
for line in lines[header_line + 2 :]:
line = line.strip()
if not line.startswith("|"):
break
row_values = [v.strip() for v in line.strip().strip("|").split("|")]
row = dict(zip(headers, row_values))
team = row["Team"]
teams.setdefault(team, []).append(row)
return teams
def format_table(members: list[dict]) -> str:
"""Format a team's member list as a monospace table for Discord."""
members = sorted(members, key=lambda r: int(r["Discord Messages"]), reverse=True)
col_widths = [width for _, _, width in FIELDS]
col_headers = [header for _, header, _ in FIELDS]
name_col_index = 0
max_name = max(len(m["Discord Username"]) for m in members)
col_widths[name_col_index] = max(col_widths[name_col_index], max_name)
def pad(val: str, width: int, right_align: bool = False) -> str:
return val.rjust(width) if right_align else val.ljust(width)
header_row = " ".join(
pad(col_headers[i], col_widths[i], right_align=(i > 0))
for i in range(len(FIELDS))
)
separator = " ".join("-" * w for w in col_widths)
rows = []
for m in members:
source_keys = [key for key, _, _ in FIELDS]
values = [m[key] for key in source_keys]
row = " ".join(
pad(values[i], col_widths[i], right_align=(i > 0))
for i in range(len(FIELDS))
)
rows.append(row)
return "\n".join([header_row, separator] + rows)
async def send_message(
session: aiohttp.ClientSession, channel_id: str, content: str
) -> None:
"""Send a message to a Discord channel."""
headers = {
"Authorization": f"Bot {DISCORD_BOT_TOKEN}",
"Content-Type": "application/json",
}
url = f"{API_BASE}/channels/{channel_id}/messages"
while True:
async with session.post(
url, json={"content": content}, headers=headers
) as resp:
if resp.status == 429:
data = await resp.json()
retry_after = data.get("retry_after", 5)
print(f" Rate limited — sleeping {retry_after}s...")
await asyncio.sleep(retry_after)
continue
if resp.status not in (200, 201):
text = await resp.text()
print(f" ERROR {resp.status}: {text}")
return
async def main() -> None:
"""Send activity tables to all team channels."""
teams = parse_report(REPORT_PATH)
team_names = list(CHANNEL_IDS.keys())
print(f"Sending activity tables to {len(team_names)} channels...\n")
async with aiohttp.ClientSession() as session:
for i, team_name in enumerate(team_names, 1):
channel_id = CHANNEL_IDS[team_name]
members = teams.get(team_name, [])
if not members:
print(f" [{i}/{len(team_names)}] {team_name} — no data, skipping")
continue
table = format_table(members)
message = (
f"**{team_name} — Activity Report (Feb 1523)**\n```\n{table}\n```"
)
if len(message) > 2000:
print(
f" [{i}/{len(team_names)}] {team_name} — WARNING: "
f"message is {len(message)} chars (over 2000!)"
)
print(
f" [{i}/{len(team_names)}] Sending to {team_name}... ",
end="",
flush=True,
)
await send_message(session, channel_id, message)
print("sent!")
await asyncio.sleep(1)
print("\nAll done! 🌸")
if __name__ == "__main__":
asyncio.run(main())
+97
View File
@@ -0,0 +1,97 @@
"""Send biweekly check-in messages to all team Discord channels.
Posts a check-in prompt to each team channel and automatically creates a thread
for responses. Members who do not respond by the deadline may be removed for
inactivity.
Data files (place in data/):
- team_message_ids.json Channel IDs per team (generated by send_team_messages.py)
Env vars:
- DISCORD_BOT_TOKEN Bot token for the Discord API
"""
import asyncio
import json
import os
from pathlib import Path
import aiohttp
DATA_DIR = Path(__file__).parent.parent.parent / "data"
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
DISCORD_API = "https://discord.com/api/v10"
GUILD_ID = "692816967895220344"
CHECK_IN_MESSAGE = """@everyone it is time for Naomi to do a check in. I need **each and every one of you** to respond to this message **in the thread** by Monday, or you **will be removed** for inactivity.
1. What have you achieved over the last two weeks?
2. What is your focus for the next two weeks?
3. How much time can you **reliably** commit to your team over the next two weeks?
4. Are you facing any **blockers** or **challenges** that are preventing you from contributing?
5. Do you need **help** from your team leaders or from Naomi to get unstuck?
"""
async def send_checkin_to_teams() -> None:
"""Send check-in message to all team channels (except Jade Jasmine)."""
with open(DATA_DIR / "team_message_ids.json") as f:
team_data = json.load(f)
headers = {
"Authorization": f"Bot {DISCORD_BOT_TOKEN}",
"Content-Type": "application/json",
}
async with aiohttp.ClientSession() as session:
for team_name, data in team_data.items():
if team_name == "jade-jasmine":
print(f"⏭️ Skipping {team_name} (dissolved)")
continue
channel_id = data["channel_id"]
print(f"📤 Sending check-in to {team_name} (channel {channel_id})...")
async with session.post(
f"{DISCORD_API}/channels/{channel_id}/messages",
headers=headers,
json={"content": CHECK_IN_MESSAGE},
) as resp:
if resp.status == 200:
message_data = await resp.json()
message_id = message_data["id"]
print(f" ✅ Message sent! Message ID: {message_id}")
thread_name = (
f"Check-in Responses - {team_name.replace('-', ' ').title()}"
)
async with session.post(
f"{DISCORD_API}/channels/{channel_id}/messages/{message_id}/threads",
headers=headers,
json={
"name": thread_name,
"auto_archive_duration": 10080,
},
) as thread_resp:
if thread_resp.status == 201:
thread_data = await thread_resp.json()
thread_id = thread_data["id"]
print(f" 🧵 Thread created! Thread ID: {thread_id}")
else:
error_text = await thread_resp.text()
print(
f" ❌ Failed to create thread: "
f"{thread_resp.status} - {error_text}"
)
else:
error_text = await resp.text()
print(f" ❌ Failed to send message: {resp.status} - {error_text}")
await asyncio.sleep(1)
print("\n✅ Check-in messages sent to all teams!")
if __name__ == "__main__":
asyncio.run(send_checkin_to_teams())
+104
View File
@@ -0,0 +1,104 @@
#!/usr/bin/env python3
"""Send a capacity check-in message to each team channel.
Posts a message asking whether the team feels able to complete their project
given their current member count, and invites them to request support if needed.
Data files (place in data/):
- team_message_ids.json Channel and role IDs per team (from send_team_messages.py)
- team_assignments.json Team rosters used to report current member counts
Env vars:
- DISCORD_BOT_TOKEN Bot token for the Discord API
"""
import asyncio
import json
import os
from pathlib import Path
import aiohttp
DATA_DIR = Path(__file__).parent.parent.parent / "data"
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
NAOMI_ID = "465650873650118659"
def build_checkin_message(team_name: str, team: dict, role_id: str) -> str:
"""Build the team check-in message."""
total_members = len(team["leaders"]) + len(team["participants"])
num_leaders = len(team["leaders"])
num_participants = len(team["participants"])
leader_text = "leader" if num_leaders == 1 else "leaders"
participant_text = "participant" if num_participants == 1 else "participants"
return f"""## Team Check-In
Your team currently has **{total_members} members** ({num_leaders} {leader_text} + {num_participants} {participant_text}).
Given the recent changes to team size, we want to make sure you feel confident moving forward with your project. Please discuss as a team and let us know:
**Do you feel you can still complete your project with your current team size?**
If you have concerns about capacity, need additional support, or would like to discuss options (such as combining with another team or adjusting project scope), please ping <@{NAOMI_ID}> and we'll work together to find a solution.
Your success is the priority here - we want to make sure every team has what they need to build something amazing! 💜
-# <@&{role_id}>"""
async def send_message(
session: aiohttp.ClientSession, channel_id: str, content: str
) -> dict | None:
"""Send a message to a Discord channel."""
url = f"https://discord.com/api/v10/channels/{channel_id}/messages"
headers = {
"Authorization": f"Bot {DISCORD_BOT_TOKEN}",
"Content-Type": "application/json",
}
payload = {"content": content}
async with session.post(url, headers=headers, json=payload) as resp:
if resp.status in [200, 201]:
return await resp.json()
error_text = await resp.text()
print(
f"❌ Failed to send message to channel {channel_id}: {resp.status} - {error_text}" # noqa: E501
)
return None
async def main() -> None:
"""Send check-in messages to all teams."""
with open(DATA_DIR / "team_message_ids.json") as f:
team_data = json.load(f)
with open(DATA_DIR / "team_assignments.json") as f:
teams = json.load(f)
async with aiohttp.ClientSession() as session:
print("📢 Sending team check-in messages...\n")
for team in teams:
team_name = team["name"]
channel_id = team_data[team_name]["channel_id"]
role_id = team_data[team_name]["role_id"]
print(f"Processing {team_name}...")
checkin_msg = build_checkin_message(team_name, team, role_id)
result = await send_message(session, channel_id, checkin_msg)
if result:
total = len(team["leaders"]) + len(team["participants"])
print(f" ✅ Sent check-in ({total} members)")
await asyncio.sleep(2)
print("\n✨ Done sending all team check-in messages!")
if __name__ == "__main__":
asyncio.run(main())
+22 -3
View File
@@ -1,10 +1,29 @@
"""Send initial welcome and roster messages to all team Discord channels.
Creates a pinned roster message in each team channel and stores the resulting
message ID, channel ID, and role ID in team_message_ids.json for use by
other scripts (send_checkin.py, update_roster_messages.py, etc.).
Data files (place in data/):
- team_assignments.json Team rosters with leaders and participants per team
- applicants_to_evaluate.json Applicant data including Discord channel/role IDs
Outputs (written to data/):
- team_message_ids.json Channel ID, message ID, and role ID per team
Env vars:
- DISCORD_BOT_TOKEN Bot token for the Discord API
"""
import json import json
import os import os
import time import time
from pathlib import Path
import requests import requests
# Amari's bot token DATA_DIR = Path(__file__).parent.parent.parent / "data"
TOKEN = os.environ["DISCORD_BOT_TOKEN"] TOKEN = os.environ["DISCORD_BOT_TOKEN"]
GUILD_ID = "692816967895220344" GUILD_ID = "692816967895220344"
@@ -72,12 +91,12 @@ TEAMS = {
} }
# Load team assignments and convert to dict by team name # Load team assignments and convert to dict by team name
with open("team_assignments.json") as f: with open(DATA_DIR / "team_assignments.json") as f:
team_list = json.load(f) team_list = json.load(f)
team_data = {team["name"]: team for team in team_list} team_data = {team["name"]: team for team in team_list}
# Load applicants to get project_url by discord_id # Load applicants to get project_url by discord_id
with open("applicants_to_evaluate.json") as f: with open(DATA_DIR / "applicants_to_evaluate.json") as f:
applicants = json.load(f) applicants = json.load(f)
applicant_lookup = {str(a["discord_id"]): a for a in applicants} applicant_lookup = {str(a["discord_id"]): a for a in applicants}
+103
View File
@@ -0,0 +1,103 @@
"""Update team roster messages in Discord from team_assignments.json."""
import asyncio
import json
import os
from pathlib import Path
import aiohttp
DATA_DIR = Path(__file__).parent.parent.parent / "data"
DISCORD_BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
GUILD_ID = "692816967895220344"
API_BASE = "https://discord.com/api/v10"
async def edit_message(
session: aiohttp.ClientSession,
channel_id: str,
message_id: str,
content: str,
) -> bool:
"""Edit a Discord message."""
url = f"{API_BASE}/channels/{channel_id}/messages/{message_id}"
headers = {
"Authorization": f"Bot {DISCORD_BOT_TOKEN}",
"Content-Type": "application/json",
}
payload = {"content": content}
async with session.patch(url, headers=headers, json=payload) as resp:
if resp.status == 200:
return True
text = await resp.text()
print(f" ❌ Failed to edit message: {resp.status} - {text}")
return False
def generate_roster(team: dict, discord_to_github: dict) -> str:
"""Generate roster message for a team."""
team_name = team["name"]
leader_lines = [
f"- <@{discord_id}> ({discord_to_github.get(discord_id, 'Unknown')})"
for discord_id in team["leaders"]
]
participant_lines = [
f"- <@{discord_id}> ({discord_to_github.get(discord_id, 'Unknown')})"
for discord_id in team["participants"]
]
leaders_text = "\n".join(leader_lines) if leader_lines else "None"
participants_text = "\n".join(participant_lines) if participant_lines else "None"
return f"""# Team {team_name}
**Leaders:**
{leaders_text}
**Participants:**
{participants_text}"""
async def main() -> None:
"""Update roster messages for all teams."""
with open(DATA_DIR / "team_message_ids.json") as f:
team_data = json.load(f)
with open(DATA_DIR / "team_assignments.json") as f:
teams = json.load(f)
with open(DATA_DIR / "discord_to_github.json") as f:
discord_to_github = json.load(f)
async with aiohttp.ClientSession() as session:
for team in teams:
team_name = team["name"]
print(f"Updating roster for {team_name}...")
if team_name not in team_data:
print(" ⚠️ Team not found in team_message_ids.json")
continue
channel_id = team_data[team_name]["channel_id"]
message_id = team_data[team_name]["message_id"]
roster_content = generate_roster(team, discord_to_github)
success = await edit_message(
session, channel_id, message_id, roster_content
)
if success:
print(f" ✅ Updated (Message ID: {message_id})")
await asyncio.sleep(0.5)
print("\n✅ All roster messages updated!")
if __name__ == "__main__":
asyncio.run(main())
+20 -2
View File
@@ -1,8 +1,26 @@
"""Verify Discord user IDs from a markdown table of applicant data.
Reads a markdown table containing Discord IDs and checks each one against the
Discord API to confirm the user exists. Handles rate limits automatically.
Data files (place in data/):
- table.md Markdown table of applicants including a Discord ID column
Outputs (written to data/):
- discord_verification.json Verification result (valid/invalid) per Discord ID
Env vars:
- DISCORD_BOT_TOKEN Bot token for the Discord API
"""
import json import json
import os import os
import time import time
import urllib.error import urllib.error
import urllib.request import urllib.request
from pathlib import Path
DATA_DIR = Path(__file__).parent.parent.parent / "data"
# Configuration # Configuration
BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"] BOT_TOKEN = os.environ["DISCORD_BOT_TOKEN"]
@@ -10,7 +28,7 @@ GUILD_ID = "692816967895220344"
BASE_URL = "https://discord.com/api/v10" BASE_URL = "https://discord.com/api/v10"
# Read Discord IDs from table.md # Read Discord IDs from table.md
with open("table.md") as f: with open(DATA_DIR / "table.md") as f:
content = f.read() content = f.read()
lines = content.strip().split("\n") lines = content.strip().split("\n")
@@ -104,7 +122,7 @@ print(f"Missing: {len(missing)}")
print(f"Errors: {len(errors)}") print(f"Errors: {len(errors)}")
# Save results # Save results
with open("discord_verification.json", "w") as f: with open(DATA_DIR / "discord_verification.json", "w") as f:
json.dump({"verified": verified, "missing": missing, "errors": errors}, f, indent=2) json.dump({"verified": verified, "missing": missing, "errors": errors}, f, indent=2)
print("\nResults saved to discord_verification.json") print("\nResults saved to discord_verification.json")
+764
View File
@@ -0,0 +1,764 @@
#!/usr/bin/env python3
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "requests==2.32.3",
# "tqdm==4.67.1",
# "questionary==2.0.1",
# "rich==13.9.4",
# ]
# ///
"""
Interactive TUI wizard for downloading Gargoyle-compatible interactive fiction games from IFDB.
Fetches the IFDB SQL dump, builds an in-memory index, then walks the user through
filtering by format, rating, and genre before downloading the matching games.
Usage:
uv run download.py
"""
from __future__ import annotations
import io
import re
import sys
import zipfile
from collections import defaultdict
from pathlib import Path
from urllib.parse import unquote, urljoin, urlparse
import questionary
import requests
from rich.console import Console
from rich.table import Table
from tqdm import tqdm
console = Console()
# ---------------------------------------------------------------------------
# Format families — groups every Gargoyle-compatible extension by interpreter
# ---------------------------------------------------------------------------
FORMAT_FAMILIES: dict[str, frozenset[str]] = {
"Z-machine": frozenset({".z1", ".z2", ".z3", ".z4", ".z5", ".z6", ".z7", ".z8", ".zblorb", ".zlb"}),
"Glulx": frozenset({".ulx", ".gblorb", ".glb", ".blorb", ".blb"}),
"TADS 2": frozenset({".gam"}),
"TADS 3": frozenset({".t3"}),
"Hugo": frozenset({".hex"}),
"ADRIFT": frozenset({".taf"}),
"Alan": frozenset({".acd", ".a2c", ".a3c"}),
"Level 9": frozenset({".l9", ".sna"}),
"Magnetic Scrolls": frozenset({".mag"}),
"AGT": frozenset({".agx"}),
"JACL": frozenset({".jacl", ".j2"}),
"Scott Adams": frozenset({".saga"}),
}
GARGOYLE_EXTENSIONS: frozenset[str] = frozenset().union(*FORMAT_FAMILIES.values())
DUMP_URL_CANDIDATES: list[str] = [
"https://ifarchive.org/if-archive/info/ifdb/ifdb-archive-20260301.zip",
"https://ifarchive.org/if-archive/info/ifdb/ifdb-archive-20251201.zip",
"https://ifarchive.org/if-archive/info/ifdb/ifdb-archive-20250901.zip",
"https://ifarchive.org/if-archive/info/ifdb/ifdb-archive-20250601.zip",
"https://ifarchive.org/if-archive/info/ifdb/ifdb-archive-20250301.zip",
"https://ifarchive.org/if-archive/info/ifdb/ifdb-archive-20241201.zip",
"https://ifarchive.org/if-archive/info/ifdb/ifdb-archive-20240901.zip",
"https://ifarchive.org/if-archive/info/ifdb/ifdb-archive-20240601.zip",
"https://ifarchive.org/if-archive/info/ifdb/ifdb-archive-20240301.zip",
]
IFARCHIVE_BASE = "https://ifarchive.org"
BAYESIAN_WEIGHT = 10
BROWSER_HEADERS = {
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
),
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
}
# ---------------------------------------------------------------------------
# Welcome screen
# ---------------------------------------------------------------------------
def show_welcome() -> None:
console.print()
console.print("[bold cyan]━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[/bold cyan]")
console.print("[bold white] IFDB Interactive Fiction Downloader[/bold white]")
console.print("[bold cyan]━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[/bold cyan]")
console.print()
console.print(
"This tool downloads interactive fiction games from the [bold]IF Database (IFDB)[/bold],\n"
"filtered to only include files playable in [bold]Gargoyle[/bold] — a multi-interpreter\n"
"IF player supporting Z-machine, Glulx, TADS, Hugo, ADRIFT, and more.\n"
)
console.print("[bold]Here's how it works:[/bold]")
console.print(" 1. Download and parse the IFDB SQL database dump (~50 MB compressed)")
console.print(" 2. Build an in-memory index of all games, ratings, and download links")
console.print(" 3. Walk you through three filters: format, rating, and genre")
console.print(" 4. Show you a summary of how many games match before you commit")
console.print(" 5. Download everything to a directory of your choice")
console.print()
console.print("[dim]Files are saved with their original names — no renaming.[/dim]")
console.print()
# ---------------------------------------------------------------------------
# Dump fetch
# ---------------------------------------------------------------------------
def find_dump_url() -> str:
console.print("[bold]Searching for the latest IFDB dump on IF Archive...[/bold]")
for url in DUMP_URL_CANDIDATES:
try:
response = requests.head(url, timeout=15, allow_redirects=True)
if response.status_code == 200:
console.print(f" [green]✓[/green] Found: {url}")
return url
console.print(f" [dim]{response.status_code}: {url}[/dim]")
except requests.RequestException as exc:
console.print(f" [red]✗[/red] {exc}: {url}")
raise SystemExit(
"\nCould not auto-detect the IFDB dump URL. "
"Please check your internet connection and try again."
)
def download_bytes(url: str, label: str) -> bytes:
response = requests.get(url, stream=True, timeout=120)
response.raise_for_status()
total = int(response.headers.get("content-length", 0))
buffer = io.BytesIO()
with tqdm(total=total or None, unit="B", unit_scale=True, desc=label) as bar:
for chunk in response.iter_content(chunk_size=65_536):
buffer.write(chunk)
bar.update(len(chunk))
return buffer.getvalue()
def extract_sql_from_zip(zip_data: bytes) -> str:
with zipfile.ZipFile(io.BytesIO(zip_data)) as archive:
sql_names = [n for n in archive.namelist() if n.endswith(".sql")]
if not sql_names:
raise SystemExit("No .sql file found inside the IFDB dump zip.")
main = max(sql_names, key=lambda n: archive.getinfo(n).file_size)
console.print(f"Extracting [bold]{main}[/bold] ({archive.getinfo(main).file_size:,} bytes)...")
return archive.read(main).decode("utf-8", errors="replace")
# ---------------------------------------------------------------------------
# MySQL dump parser
# ---------------------------------------------------------------------------
def _parse_sql_value(raw: str) -> str | None:
stripped = raw.strip()
return None if stripped.upper() == "NULL" else stripped
def parse_mysql_values(values_str: str) -> list[tuple[str | None, ...]]:
rows: list[tuple[str | None, ...]] = []
current_row: list[str | None] = []
token_chars: list[str] = []
in_string = False
depth = 0
i = 0
length = len(values_str)
while i < length:
char = values_str[i]
if in_string:
if char == "\\":
if i + 1 < length:
token_chars.append(values_str[i + 1])
i += 2
else:
i += 1
continue
if char == "'":
if i + 1 < length and values_str[i + 1] == "'":
token_chars.append("'")
i += 2
continue
in_string = False
i += 1
continue
token_chars.append(char)
i += 1
continue
if char == "'":
in_string = True
i += 1
continue
if char == "(":
depth += 1
if depth == 1:
current_row = []
token_chars = []
else:
token_chars.append(char)
i += 1
continue
if char == ")":
depth -= 1
if depth == 0:
current_row.append(_parse_sql_value("".join(token_chars)))
rows.append(tuple(current_row))
current_row = []
token_chars = []
else:
token_chars.append(char)
i += 1
continue
if char == "," and depth == 1:
current_row.append(_parse_sql_value("".join(token_chars)))
token_chars = []
i += 1
continue
if depth > 0:
token_chars.append(char)
i += 1
return rows
def _extract_column_names(create_body: str) -> list[str]:
columns: list[str] = []
for match in re.finditer(r"^\s*`(\w+)`\s+\w", create_body, re.MULTILINE):
columns.append(match.group(1))
return columns
def parse_dump(sql: str, tables_wanted: set[str]) -> dict[str, list[dict]]:
table_columns: dict[str, list[str]] = {}
table_data: dict[str, list[dict]] = {t: [] for t in tables_wanted}
console.print("Splitting dump into statements...")
statements = sql.split(";\n")
console.print(f" {len(statements):,} statements found")
create_re = re.compile(r"CREATE\s+TABLE\s+`(\w+)`\s*\((.+)\)", re.DOTALL | re.IGNORECASE)
insert_re = re.compile(
r"INSERT\s+INTO\s+`(\w+)`(?:\s*\(([^)]+)\))?\s+VALUES\s*(.+)",
re.DOTALL | re.IGNORECASE,
)
for statement in tqdm(statements, desc="Parsing statements", unit="stmt"):
upper = statement.lstrip()[:20].upper()
if upper.startswith("CREATE"):
match = create_re.search(statement)
if match:
name = match.group(1)
if name in tables_wanted:
table_columns[name] = _extract_column_names(match.group(2))
elif upper.startswith("INSERT"):
match = insert_re.search(statement)
if not match:
continue
name = match.group(1)
if name not in tables_wanted:
continue
if match.group(2):
columns = [c.strip().strip("`").strip('"') for c in match.group(2).split(",")]
else:
columns = table_columns.get(name, [])
if not columns:
continue
for row in parse_mysql_values(match.group(3)):
if len(row) == len(columns):
table_data[name].append(dict(zip(columns, row)))
for table in tables_wanted:
cols = table_columns.get(table, [])
console.print(
f" [bold]{table}[/bold]: {len(table_data[table]):,} rows "
f"({', '.join(cols[:6])}{'...' if len(cols) > 6 else ''})"
)
return table_data
# ---------------------------------------------------------------------------
# URL utilities
# ---------------------------------------------------------------------------
def is_gargoyle_url(url: str) -> bool:
return Path(urlparse(url).path.lower()).suffix in GARGOYLE_EXTENSIONS
def resolve_url(url: str) -> str:
return url if urlparse(url).scheme else urljoin(IFARCHIVE_BASE, url)
def get_format_family(url: str) -> str | None:
ext = Path(urlparse(url).path.lower()).suffix
for family, extensions in FORMAT_FAMILIES.items():
if ext in extensions:
return family
return None
def best_link(links: list[dict]) -> dict | None:
uncompressed = [
lnk for lnk in links
if is_gargoyle_url(lnk["url"])
and lnk.get("compression") in (None, "", "0", "false", "FALSE")
]
if uncompressed:
return uncompressed[0]
compatible = [lnk for lnk in links if is_gargoyle_url(lnk["url"])]
return compatible[0] if compatible else None
# ---------------------------------------------------------------------------
# Index building
# ---------------------------------------------------------------------------
def build_indices(data: dict[str, list[dict]]) -> dict:
console.print("\n[bold]Building indices...[/bold]")
game_title: dict[str, str] = {}
game_author: dict[str, str] = {}
game_genre: dict[str, str] = {}
for row in data["games"]:
gid = row.get("id")
if not gid:
continue
game_title[gid] = row.get("title") or f"game_{gid}"
game_author[gid] = row.get("author") or ""
genre = (row.get("genre") or "").strip()
game_genre[gid] = genre if genre else "Uncategorised"
ratings_by_game: dict[str, list[float]] = defaultdict(list)
for row in data["reviews"]:
gid = row.get("gameid")
raw = row.get("rating")
if not gid or raw in (None, "0", "NULL", ""):
continue
try:
ratings_by_game[gid].append(float(raw))
except ValueError:
pass
raw_avg: dict[str, float] = {
gid: sum(rs) / len(rs) for gid, rs in ratings_by_game.items()
}
links_by_game: dict[str, list[dict]] = defaultdict(list)
for row in data["gamelinks"]:
gid = row.get("gameid")
url = row.get("url", "")
if not gid or not url:
continue
full_url = resolve_url(url)
if is_gargoyle_url(full_url):
links_by_game[gid].append({**row, "url": full_url})
all_gargoyle_ids: set[str] = set(links_by_game.keys())
game_family: dict[str, str] = {}
for gid in all_gargoyle_ids:
link = best_link(links_by_game[gid])
if link:
game_family[gid] = get_format_family(link["url"]) or "Unknown"
console.print(f" Games in DB: {len(game_title):,}")
console.print(f" Games with ratings: {len(ratings_by_game):,}")
console.print(f" Games with Gargoyle links: {len(all_gargoyle_ids):,}")
return {
"game_title": game_title,
"game_author": game_author,
"game_genre": game_genre,
"raw_avg": raw_avg,
"links_by_game": links_by_game,
"all_gargoyle_ids": all_gargoyle_ids,
"game_family": game_family,
}
# ---------------------------------------------------------------------------
# Filter helpers
# ---------------------------------------------------------------------------
RATING_KEYS = ["all", "rated", "≥ 2", "≥ 3", "≥ 4", "≥ 5"]
RATING_LABELS: dict[str, str] = {
"all": "All (including unrated)",
"rated": "Any rated game (≥ 1 star)",
"≥ 2": "≥ 2 stars",
"≥ 3": "≥ 3 stars",
"≥ 4": "≥ 4 stars",
"≥ 5": "≥ 5 stars (perfect scores only)",
}
def count_by_format(indices: dict) -> dict[str, int]:
counts: dict[str, int] = defaultdict(int)
for gid in indices["all_gargoyle_ids"]:
family = indices["game_family"].get(gid, "Unknown")
counts[family] += 1
return dict(sorted(counts.items(), key=lambda kv: kv[1], reverse=True))
def count_by_rating(indices: dict) -> dict[str, int]:
all_ids = indices["all_gargoyle_ids"]
raw_avg = indices["raw_avg"]
return {
"all": len(all_ids),
"rated": sum(1 for gid in all_ids if gid in raw_avg),
"≥ 2": sum(1 for gid in all_ids if raw_avg.get(gid, 0) >= 2),
"≥ 3": sum(1 for gid in all_ids if raw_avg.get(gid, 0) >= 3),
"≥ 4": sum(1 for gid in all_ids if raw_avg.get(gid, 0) >= 4),
"≥ 5": sum(1 for gid in all_ids if raw_avg.get(gid, 0) >= 5),
}
def count_by_genre(indices: dict) -> dict[str, int]:
counts: dict[str, int] = defaultdict(int)
for gid in indices["all_gargoyle_ids"]:
genre = indices["game_genre"].get(gid, "Uncategorised")
counts[genre] += 1
return dict(sorted(counts.items(), key=lambda kv: kv[1], reverse=True))
def _passes_rating_filter(gid: str, raw_avg: dict[str, float], rating_key: str) -> bool:
if rating_key == "all":
return True
if rating_key == "rated":
return gid in raw_avg
threshold = float(rating_key.replace("", ""))
return raw_avg.get(gid, 0) >= threshold
def apply_filters(
indices: dict,
selected_families: set[str],
rating_key: str,
selected_genres: set[str],
) -> list[str]:
raw_avg = indices["raw_avg"]
result: list[str] = []
for gid in indices["all_gargoyle_ids"]:
if indices["game_family"].get(gid, "Unknown") not in selected_families:
continue
if not _passes_rating_filter(gid, raw_avg, rating_key):
continue
if indices["game_genre"].get(gid, "Uncategorised") not in selected_genres:
continue
result.append(gid)
return result
# ---------------------------------------------------------------------------
# TUI wizard steps
# ---------------------------------------------------------------------------
def ask_formats(indices: dict) -> set[str]:
format_counts = count_by_format(indices)
console.print()
console.print("[bold cyan]Step 1 of 3 — File Formats[/bold cyan]")
console.print(
"Select the formats you want to include. "
"[dim]All are pre-selected — uncheck any you don't want.[/dim]"
)
console.print()
choices = [
questionary.Choice(
title=f"{family} ({count:,} games)",
value=family,
checked=True,
)
for family, count in format_counts.items()
if count > 0
]
selected = questionary.checkbox("Formats to include:", choices=choices).ask()
if selected is None:
sys.exit(0)
if not selected:
console.print("[yellow]Nothing selected — defaulting to all formats.[/yellow]")
return set(format_counts.keys())
return set(selected)
def ask_rating(indices: dict) -> str:
rating_counts = count_by_rating(indices)
console.print()
console.print("[bold cyan]Step 2 of 3 — Minimum Rating[/bold cyan]")
console.print(
"Choose the minimum average rating a game must have to be included.\n"
"[dim]Counts are independent of your format selection.[/dim]"
)
console.print()
choices = [
questionary.Choice(
title=f"{RATING_LABELS[key]} ({rating_counts[key]:,} games)",
value=key,
)
for key in RATING_KEYS
]
selected = questionary.select("Minimum rating:", choices=choices).ask()
if selected is None:
sys.exit(0)
return selected
def ask_genres(indices: dict) -> set[str]:
genre_counts = count_by_genre(indices)
console.print()
console.print("[bold cyan]Step 3 of 3 — Genres[/bold cyan]")
console.print(
"Select the genres you want to include. "
"[dim]All are pre-selected — uncheck any you don't want.\n"
"Counts are independent of your format and rating selections.[/dim]"
)
console.print()
choices = [
questionary.Choice(
title=f"{genre} ({count:,} games)",
value=genre,
checked=True,
)
for genre, count in genre_counts.items()
if count > 0
]
selected = questionary.checkbox("Genres to include:", choices=choices).ask()
if selected is None:
sys.exit(0)
if not selected:
console.print("[yellow]Nothing selected — defaulting to all genres.[/yellow]")
return set(genre_counts.keys())
return set(selected)
def show_filter_summary(
indices: dict,
selected_families: set[str],
rating_key: str,
selected_genres: set[str],
) -> int:
format_counts = count_by_format(indices)
rating_counts = count_by_rating(indices)
genre_counts = count_by_genre(indices)
format_total = sum(format_counts.get(f, 0) for f in selected_families)
rating_total = rating_counts[rating_key]
genre_total = sum(genre_counts.get(g, 0) for g in selected_genres)
combined = apply_filters(indices, selected_families, rating_key, selected_genres)
if len(selected_families) <= 4:
families_label = ", ".join(sorted(selected_families))
else:
families_label = f"{len(selected_families)} formats selected"
if len(selected_genres) <= 3:
genres_label = ", ".join(sorted(selected_genres))
else:
genres_label = f"{len(selected_genres)} genres selected"
table = Table(title="Filter Summary", show_header=True, header_style="bold cyan")
table.add_column("Filter", style="bold")
table.add_column("Selection")
table.add_column("Matching games", justify="right")
table.add_row("Format", families_label, f"{format_total:,}")
table.add_row("Rating", RATING_LABELS[rating_key], f"{rating_total:,}")
table.add_row("Genre", genres_label, f"{genre_total:,}")
table.add_section()
table.add_row(
"[bold]Combined[/bold]",
"[dim]all three filters applied[/dim]",
f"[bold green]{len(combined):,}[/bold green]",
)
console.print()
console.print(table)
console.print()
return len(combined)
def ask_output_path() -> Path:
console.print()
def validate_path(raw: str) -> bool | str:
if not raw.strip():
return "Please enter a path."
p = Path(raw.strip()).expanduser()
if p.exists() and not p.is_dir():
return f"{raw!r} exists and is not a directory."
return True
path_str = questionary.text(
"Where should the games be saved? (absolute path to a directory)",
validate=validate_path,
).ask()
if path_str is None:
sys.exit(0)
output_dir = Path(path_str.strip()).expanduser().resolve()
output_dir.mkdir(parents=True, exist_ok=True)
return output_dir
# ---------------------------------------------------------------------------
# Download
# ---------------------------------------------------------------------------
def download_games(indices: dict, matching_ids: list[str], output_dir: Path) -> None:
console.print(f"\n[bold]Downloading {len(matching_ids):,} games to:[/bold] {output_dir}")
console.print()
errors: list[str] = []
skipped = 0
downloaded = 0
for gid in tqdm(matching_ids, desc="Downloading", unit="game"):
title = indices["game_title"].get(gid, f"game_{gid}")
link = best_link(indices["links_by_game"].get(gid, []))
if not link:
errors.append(f"{title}: no suitable download link")
continue
url = link["url"]
filename = unquote(Path(urlparse(url).path).name)
if not filename:
filename = f"game_{gid}" + Path(urlparse(url).path).suffix
filepath = output_dir / filename
if filepath.exists():
skipped += 1
continue
try:
response = requests.get(url, timeout=60, stream=True, headers=BROWSER_HEADERS)
response.raise_for_status()
with filepath.open("wb") as fh:
for chunk in response.iter_content(chunk_size=65_536):
fh.write(chunk)
downloaded += 1
except requests.RequestException as exc:
errors.append(f"{title}: {exc}")
if filepath.exists():
filepath.unlink()
console.print()
console.print("[bold]━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[/bold]")
console.print(f"[green]Downloaded:[/green] {downloaded:,}")
console.print(f"[dim]Skipped (already present):[/dim] {skipped:,}")
console.print(f"[red]Errors:[/red] {len(errors):,}")
console.print(f"[bold]Saved to:[/bold] {output_dir}")
if errors:
console.print(f"\n[red]First {min(20, len(errors))} errors:[/red]")
for msg in errors[:20]:
console.print(f" {msg}")
if len(errors) > 20:
console.print(f" ... and {len(errors) - 20} more")
error_log = output_dir / "download_errors.txt"
error_log.write_text("\n".join(errors), encoding="utf-8")
console.print(f"\n[dim]Full error log:[/dim] {error_log}")
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def main() -> None:
show_welcome()
confirmed = questionary.confirm(
"Ready to fetch the IFDB database and get started?"
).ask()
if not confirmed:
console.print("[dim]Bye! 👋[/dim]")
sys.exit(0)
console.print()
dump_url = find_dump_url()
console.print(f"\n[bold]Downloading:[/bold] {dump_url}")
zip_data = download_bytes(dump_url, "IFDB dump")
console.print("\n[bold]Extracting SQL from archive...[/bold]")
sql = extract_sql_from_zip(zip_data)
console.print(f"SQL text: {len(sql):,} characters")
del zip_data
console.print("\n[bold]Parsing database tables (this may take a minute)...[/bold]")
data = parse_dump(sql, {"games", "gamelinks", "reviews"})
del sql
indices = build_indices(data)
del data
# Filter wizard — loops if the user wants to edit
selected_families: set[str] = set()
rating_key: str = "all"
selected_genres: set[str] = set()
first_run = True
while True:
selected_families = ask_formats(indices)
rating_key = ask_rating(indices)
selected_genres = ask_genres(indices)
match_count = show_filter_summary(indices, selected_families, rating_key, selected_genres)
if match_count == 0:
console.print("[yellow]No games match your current filters — please adjust them.[/yellow]")
action = questionary.select(
"What would you like to do?",
choices=["Edit filters", "Quit"],
).ask()
if action != "Edit filters":
sys.exit(0)
continue
action = questionary.select(
f"Download {match_count:,} matching games?",
choices=[
questionary.Choice(f"Yes — download all {match_count:,} games", value="download"),
questionary.Choice("Edit filters", value="edit"),
questionary.Choice("Quit", value="quit"),
],
).ask()
if action is None or action == "quit":
sys.exit(0)
if action == "edit":
continue
break
output_dir = ask_output_path()
matching_ids = apply_filters(indices, selected_families, rating_key, selected_genres)
download_games(indices, matching_ids, output_dir)
if __name__ == "__main__":
main()
+2
View File
@@ -79,6 +79,8 @@ ignore = [
"DTZ001", "DTZ001",
# Ambiguous variable names - context makes it clear # Ambiguous variable names - context makes it clear
"E741", "E741",
# Long lines in string literals (Discord messages, URLs)
"E501",
] ]
[tool.ruff.lint.pydocstyle] [tool.ruff.lint.pydocstyle]
+1
View File
@@ -4,3 +4,4 @@ ruff==0.14.14
# Runtime dependencies # Runtime dependencies
requests==2.32.3 requests==2.32.3
aiohttp==3.11.12 aiohttp==3.11.12
pandas==3.0.1
+16 -12
View File
@@ -96,7 +96,7 @@ select_option() {
# Step 1: Select Language # Step 1: Select Language
echo "" echo ""
languages=("TypeScript" "Python") languages=("TypeScript" "Python" "Bash")
select_option "Select a language:" "${languages[@]}" select_option "Select a language:" "${languages[@]}"
lang_index=$? lang_index=$?
language="${languages[$lang_index]}" language="${languages[$lang_index]}"
@@ -109,15 +109,16 @@ if [ "$language" == "TypeScript" ]; then
runner="pnpm tsx" runner="pnpm tsx"
# Get subdirectories as categories (excluding utils and interfaces) # Get subdirectories as categories (excluding utils and interfaces)
mapfile -t categories < <(find "$script_dir" -mindepth 1 -maxdepth 1 -type d ! -name 'utils' ! -name 'interfaces' -exec basename {} \; | sort) mapfile -t categories < <(find "$script_dir" -mindepth 1 -maxdepth 1 -type d ! -name 'utils' ! -name 'interfaces' -exec basename {} \; | sort)
else elif [ "$language" == "Python" ]; then
script_dir="python" script_dir="python"
runner="uv run python" runner="uv run python"
# Get subdirectories as categories (excluding __pycache__ and .venv) # Get subdirectories as categories (excluding __pycache__ and .venv)
mapfile -t categories < <(find "$script_dir" -mindepth 1 -maxdepth 1 -type d ! -name '__pycache__' ! -name '.venv' ! -name '*.egg-info' -exec basename {} \; | sort) mapfile -t categories < <(find "$script_dir" -mindepth 1 -maxdepth 1 -type d ! -name '__pycache__' ! -name '.venv' ! -name '*.egg-info' -exec basename {} \; | sort)
# Add "Root Scripts" option for Python files in root else
if ls "$script_dir"/*.py &>/dev/null 2>&1; then script_dir="bash"
categories=("Root Scripts" "${categories[@]}") runner="bash"
fi # Get subdirectories as categories
mapfile -t categories < <(find "$script_dir" -mindepth 1 -maxdepth 1 -type d -exec basename {} \; | sort)
fi fi
if [ ${#categories[@]} -eq 0 ]; then if [ ${#categories[@]} -eq 0 ]; then
@@ -132,12 +133,12 @@ category="${categories[$cat_index]}"
echo -e "\n ${GREEN}$STAR Selected: ${WHITE}$category${RESET}\n" echo -e "\n ${GREEN}$STAR Selected: ${WHITE}$category${RESET}\n"
# Step 3: Get scripts in category # Step 3: Get scripts in category
if [ "$category" == "Root Scripts" ]; then if [ "$language" == "TypeScript" ]; then
search_dir="$script_dir"
mapfile -t scripts < <(find "$search_dir" -maxdepth 1 -name "*.py" -exec basename {} \; | sort)
elif [ "$language" == "TypeScript" ]; then
search_dir="$script_dir/$category" search_dir="$script_dir/$category"
mapfile -t scripts < <(find "$search_dir" -name "*.ts" -exec basename {} \; | sort) mapfile -t scripts < <(find "$search_dir" -name "*.ts" -exec basename {} \; | sort)
elif [ "$language" == "Bash" ]; then
search_dir="$script_dir/$category"
mapfile -t scripts < <(find "$search_dir" -name "*.sh" -exec basename {} \; | sort)
else else
search_dir="$script_dir/$category" search_dir="$script_dir/$category"
mapfile -t scripts < <(find "$search_dir" -name "*.py" ! -name "__init__.py" -exec basename {} \; | sort) mapfile -t scripts < <(find "$search_dir" -name "*.py" ! -name "__init__.py" -exec basename {} \; | sort)
@@ -155,8 +156,8 @@ script="${scripts[$script_index]}"
echo -e "\n ${GREEN}$STAR Selected: ${WHITE}$script${RESET}\n" echo -e "\n ${GREEN}$STAR Selected: ${WHITE}$script${RESET}\n"
# Build the full script path # Build the full script path
if [ "$category" == "Root Scripts" ]; then if [ "$language" == "Bash" ]; then
script_path="$script" script_path="bash/$category/$script"
elif [ "$language" == "TypeScript" ]; then elif [ "$language" == "TypeScript" ]; then
script_path="src/$category/$script" script_path="src/$category/$script"
else else
@@ -178,6 +179,9 @@ if [ "$language" == "TypeScript" ]; then
cd typescript cd typescript
echo -e " ${DIM}$ op run --env-file=../prod.env -- $runner $script_path${RESET}\n" echo -e " ${DIM}$ op run --env-file=../prod.env -- $runner $script_path${RESET}\n"
op run --env-file=../prod.env --no-masking -- $runner "$script_path" op run --env-file=../prod.env --no-masking -- $runner "$script_path"
elif [ "$language" == "Bash" ]; then
echo -e " ${DIM}$ $runner $script_path${RESET}\n"
$runner "$script_path"
else else
cd python cd python
echo -e " ${DIM}$ op run --env-file=../prod.env -- $runner $script_path${RESET}\n" echo -e " ${DIM}$ op run --env-file=../prod.env -- $runner $script_path${RESET}\n"
+125
View File
@@ -0,0 +1,125 @@
# Crowdin Scripts
Scripts for managing translations in a Crowdin project.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → crowdin → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [writeData.ts](#writedatats)
- [clearHiddenTranslations.ts](#clearhiddentranslationsts)
- [reapplyTranslations.ts](#reapplytranslationsts)
> **Typical workflow:** Run `writeData.ts` first to fetch and cache the project data locally, then use `clearHiddenTranslations.ts` and/or `reapplyTranslations.ts` as needed.
---
## writeData.ts
Fetches all file IDs and string data from a Crowdin project and writes them to local JSON files in `data/`. These files are used as inputs by the other Crowdin scripts.
### Usage
```bash
make run
# Select: TypeScript → crowdin → writeData.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `CROWDIN_PROJECT_ID` | Crowdin project numeric ID |
| `CROWDIN_API_URL` | Base URL of the Crowdin API (e.g. `https://api.crowdin.com/api/v2`) |
| `CROWDIN_TOKEN` | Crowdin personal access token |
### Data Files
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `crowdin-files.json` | JSON array of file IDs | All file IDs in the project |
| `crowdin-strings.json` | JSON array of string objects | All strings in the project, including `id`, `isHidden`, and other metadata |
---
## clearHiddenTranslations.ts
Deletes existing translations for all hidden (suppressed) strings in a Crowdin project, across all active languages, in parallel. Keeps a log file to avoid re-processing strings on subsequent runs.
### Usage
```bash
make run
# Select: TypeScript → crowdin → clearHiddenTranslations.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `CROWDIN_PROJECT_ID` | Crowdin project numeric ID |
| `CROWDIN_API_URL` | Base URL of the Crowdin API |
| `CROWDIN_TOKEN` | Crowdin personal access token |
### Data Files
**Input** (expected in `data/`):
| File | Format | Description |
|---|---|---|
| `crowdin-strings.json` | JSON array of string objects | Output of `writeData.ts` — each object must have `id` (number) and `isHidden` (boolean) fields |
| `crowdin-strings-hidden.txt` | Plain text, one ID per line | Log of already-processed string IDs; create an empty file if starting fresh |
**Output** (updated in `data/`):
| File | Description |
|---|---|
| `crowdin-strings-hidden.txt` | Appended with the ID of each string processed in this run |
### Notes
- Run `writeData.ts` first to generate `crowdin-strings.json`.
- Create an empty `crowdin-strings-hidden.txt` in `data/` before the first run. The script will append to it as strings are processed, so re-runs skip already-cleared strings.
- Translations are deleted in parallel across all languages for each string, then the string ID is appended to the log before moving on to the next.
---
## reapplyTranslations.ts
Triggers a pre-translation run on a Crowdin project using Translation Memory (TM), applying perfect matches only, and polls for completion every 5 seconds until the job reaches 100%.
### Usage
```bash
make run
# Select: TypeScript → crowdin → reapplyTranslations.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `CROWDIN_PROJECT_ID` | Crowdin project numeric ID |
| `CROWDIN_API_URL` | Base URL of the Crowdin API |
| `CROWDIN_TOKEN` | Crowdin personal access token |
### Data Files
None. The script fetches the current file and language lists from Crowdin at runtime.
### Notes
- Pre-translation is configured with `translateWithPerfectMatchOnly: true` and `autoApproveOption: "perfectMatchOnly"`, so only exact TM matches are applied and automatically approved.
- Already-approved translations are skipped (`skipApprovedTranslations: true`).
- The script polls progress every 5 seconds and prints the percentage until the job is complete.
@@ -1,4 +1,15 @@
/** /**
* @file Delete hidden/outdated translations from a Crowdin project.
* Reads a list of string IDs from data/crowdin-strings.json, checks which
* have already been processed via the log file, and deletes translations
* for unprocessed strings across all active languages in parallel.
* Data files (place in data/):
* - crowdin-strings.json String IDs to clear (from crowdin/writeData.ts)
* - crowdin-strings-hidden.txt Log of already-processed string IDs
* Env vars:
* CROWDIN_PROJECT_ID - Crowdin project numeric ID
* CROWDIN_API_URL - Base URL of the Crowdin API
* CROWDIN_TOKEN - Crowdin personal access token.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
@@ -1,4 +1,12 @@
/** /**
* @file Trigger a pre-translation run on a Crowdin project and wait for completion.
* Fetches all active languages and files, submits a pre-translation request,
* and polls for progress every 5 seconds until it reaches 100%%.
* Data files: None
* Env vars:
* CROWDIN_PROJECT_ID - Crowdin project numeric ID
* CROWDIN_API_URL - Base URL of the Crowdin API
* CROWDIN_TOKEN - Crowdin personal access token.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+87
View File
@@ -0,0 +1,87 @@
# Discord Scripts
Scripts for Discord bot utilities and server management.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → discord → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [cycThreads.ts](#cycthreadsts)
- [guildCount.ts](#guildcountts)
---
## cycThreads.ts
Creates Discord forum threads for talk submissions in a conference channel. Iterates over a hardcoded list of talk titles and speakers, creating one public forum thread per talk with a standard discussion prompt message. Talks with missing titles or speakers are automatically filtered out.
### Usage
```bash
make run
# Select: TypeScript → discord → cycThreads.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_TOKEN` | Discord bot token |
### Data Files
None. The list of talks (title + speaker) is hardcoded in the script. The target forum channel ID is also hardcoded.
### Notes
- **Before running**, update the `data` array in the script with the current conference talk submissions and update `CHANNEL_ID` to point to the correct Discord forum channel.
- Uses the `backoffAndRetry` utility to handle Discord API rate limits automatically.
- Entries with an empty `title` or `speaker` field are silently filtered before processing.
- Threads are created with a 24-hour auto-archive duration (`auto_archive_duration: 1440`).
---
## guildCount.ts
Counts and categorises all Discord servers a user belongs to. Uses the Discord OAuth2 PKCE flow (no user token ever stored) to authenticate, fetches the user's guild list, and categorises each server as owned, admin, moderating, partnered, verified, community, or discoverable. Results are printed to the console and displayed as an HTML dashboard served on a local port (opened automatically in your browser).
### Usage
```bash
make run
# Select: TypeScript → discord → guildCount.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCORD_CLIENT_ID` | OAuth2 application client ID (required) |
| `DISCORD_CLIENT_SECRET` | OAuth2 application client secret (optional, improves security) |
| `DISCORD_REDIRECT_URI` | OAuth2 redirect URI (default: `http://127.0.0.1:8721/callback`) |
| `DISCORD_SCOPES` | OAuth2 scopes (default: `identify guilds`) |
### Data Files
None.
### Notes
- **Setup required before first run** (per machine):
1. Create a Discord application in the [Developer Portal](https://discord.com/developers/applications) and note the Client ID.
2. Under OAuth2 → Redirects, add `http://127.0.0.1:8721/callback` (or supply your own via `DISCORD_REDIRECT_URI`).
3. Optionally generate a Client Secret and set it as `DISCORD_CLIENT_SECRET`.
4. Export `DISCORD_CLIENT_ID` (and secret if used) before running the script.
- The script prompts you to confirm setup is complete before launching the OAuth flow.
- The OAuth flow uses PKCE (Proof Key for Code Exchange) — no user tokens are ever logged or stored; this approach is fully within Discord's Terms of Service.
- Discord allows a maximum of 200 servers per user (with Nitro); the script fetches up to 200 at once.
- The HTML dashboard is served on a random available local port and opened automatically; the server closes once the page is loaded.
+6
View File
@@ -1,4 +1,10 @@
/** /**
* @file Create discussion threads for CYC talk submissions in a Discord channel.
* Iterates over a hardcoded list of talk titles and speakers and creates a
* Discord forum thread for each one. Update the data array and CHANNEL_ID before running.
* Data files: None (talk data is hardcoded in the script)
* Env vars:
* DISCORD_TOKEN - Bot token for the Discord API.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+10
View File
@@ -1,4 +1,14 @@
/** /**
* @file Count and categorise all Discord servers a user belongs to.
* Uses the Discord OAuth2 PKCE flow with a local callback server to authenticate,
* then fetches and categorises all guilds (owned, admin, moderated, partnered,
* verified, community, discoverable) and serves an HTML dashboard on localhost.
* Data files: None
* Env vars:
* DISCORD_CLIENT_ID - OAuth2 application client ID
* DISCORD_CLIENT_SECRET - OAuth2 application client secret (optional)
* DISCORD_REDIRECT_URI - OAuth2 redirect URI (default: http://127.0.0.1:8721/callback)
* DISCORD_SCOPES - OAuth2 scopes (default: identify guilds).
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+84
View File
@@ -0,0 +1,84 @@
# Discourse Scripts
Scripts for Discourse forum management.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → discourse → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [bulkUpdateCategories.ts](#bulkupdatecategoriests)
- [closeOldTopics.ts](#closeoldtopicsts)
---
## bulkUpdateCategories.ts
Enables auto-close on all categories and subcategories in a Discourse forum. Fetches the full category list (including subcategories), then updates each one to auto-close based on last post after 672 hours (28 days).
### Usage
```bash
make run
# Select: TypeScript → discourse → bulkUpdateCategories.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCOURSE_URL` | Base URL of the Discourse instance (e.g. `https://forum.example.com`) |
| `DISCOURSE_API_KEY` | Discourse API key |
| `DISCOURSE_API_USERNAME` | Discourse API username |
### Data Files
None.
### Notes
- Auto-close is configured to trigger based on **last post** (not topic creation date), so active topics remain open.
- The auto-close threshold is hardcoded to `672` hours (28 days). Update the `auto_close_hours` value in the script to change this.
- Uses the `backoffAndRetry` utility to handle Discourse API rate limits automatically.
- Categories are processed sequentially; subcategories are fetched individually and deduplicated.
- Failures for individual categories are logged but do not stop the rest of the run.
---
## closeOldTopics.ts
Closes inactive Discourse topics that have had no activity for 28 or more days, skipping any topics older than 6 months (which are assumed to be already archived or otherwise handled). Already-closed topics are also skipped.
### Usage
```bash
make run
# Select: TypeScript → discourse → closeOldTopics.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `DISCOURSE_URL` | Base URL of the Discourse instance (e.g. `https://forum.example.com`) |
| `DISCOURSE_API_KEY` | Discourse API key |
| `DISCOURSE_API_USERNAME` | Discourse API username |
### Data Files
None.
### Notes
- Topics are fetched from the `/latest.json` endpoint in ascending age order. Pagination stops automatically once a topic older than 6 months is encountered.
- The inactivity threshold is **28 days** and the age cutoff is **180 days** (approximately 6 months); both are defined as constants in the script.
- A 500 ms delay is added between close requests to avoid hitting Discourse rate limits. Rate-limit (HTTP 429) responses trigger an automatic 5-second wait and retry.
- At the end of the run, a summary of closed/failed counts is printed.
@@ -1,4 +1,12 @@
/** /**
* @file Set auto-close settings on all Discourse forum categories.
* Fetches all categories and subcategories and enables auto-close based on
* last post (672 hours / 28 days) for each one.
* Data files: None
* Env vars:
* DISCOURSE_URL - Base URL of the Discourse instance
* DISCOURSE_API_KEY - Discourse API key
* DISCOURSE_API_USERNAME - Discourse API username.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
@@ -1,4 +1,11 @@
/** /**
* @file Close inactive Discourse topics that have had no activity for 28+ days.
* Skips topics older than 6 months (already archived) and any that are already closed.
* Data files: None
* Env vars:
* DISCOURSE_URL - Base URL of the Discourse instance
* DISCOURSE_API_KEY - Discourse API key
* DISCOURSE_API_USERNAME - Discourse API username.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+124
View File
@@ -0,0 +1,124 @@
# Gitea Scripts
Scripts for bulk file management across Gitea repositories.
All scripts operate across three NHCarrigan Gitea organisations: `nhcarrigan`, `nhcarrigan-private`, and `nhcarrigan-games`.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → gitea → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [deleteFromAllRepos.ts](#deletefromallreposets)
- [uploadToAllRepos.ts](#uploadtoallreposets)
- [uploadToReposConditionally.ts](#uploadtoreposconditionallyts)
---
## deleteFromAllRepos.ts
Deletes a specified file from every repository across all NHCarrigan Gitea organisations. Checks each repository first; if the file exists it is deleted, otherwise the repository is skipped.
### Usage
```bash
make run
# Select: TypeScript → gitea → deleteFromAllRepos.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITEA_TOKEN` | Gitea personal access token with repository write permissions |
### Data Files
None. The file path to delete is entered interactively when the script runs.
### Notes
- You will be prompted for the file path to delete (e.g. `.gitea/workflows/security.yml`). Do **not** include a leading slash.
- The deletion commit message is automatically set to `feat: automated delete of {path}`.
- Repositories are fetched 100 at a time with automatic pagination via `paginatedFetch`.
- Failures for individual repositories are logged but do not stop the rest of the run.
---
## uploadToAllRepos.ts
Uploads a file from the local `data/` directory to every repository across all NHCarrigan Gitea organisations. If the file already exists in a repository it is updated; otherwise it is created.
### Usage
```bash
make run
# Select: TypeScript → gitea → uploadToAllRepos.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITEA_TOKEN` | Gitea personal access token with repository write permissions |
### Data Files
**Input** (place in `data/`):
| File | Format | Description |
|---|---|---|
| *(any file)* | Any | The file to upload; name and destination path are entered interactively |
### Notes
- You will be prompted for:
- **Local filename** — path relative to `data/` (e.g. `actions.yml` or `gitea/actions.yml`). The file must exist in `data/`.
- **Destination path** — path in each repository (e.g. `.gitea/workflows/actions.yml`). Do **not** include a leading slash.
- Commit messages are automatically set to `feat: automated upload of {path}`.
- A summary of processed / succeeded / failed counts is printed at the end.
---
## uploadToReposConditionally.ts
Uploads a file to Gitea repositories only if a condition file does (or does not) exist in each repository. Useful for targeting only repositories that have a specific workflow file, language marker, or configuration already in place.
### Usage
```bash
make run
# Select: TypeScript → gitea → uploadToReposConditionally.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITEA_TOKEN` | Gitea personal access token with repository write permissions |
### Data Files
**Input** (place in `data/`):
| File | Format | Description |
|---|---|---|
| *(any file)* | Any | The file to upload; name and destination path are entered interactively |
### Notes
- You will be prompted for:
- **Local filename** — path relative to `data/` (same as `uploadToAllRepos.ts`).
- **Destination path** — path in each repository.
- **Condition file path** — path to check in each repository (e.g. `package.json` or `.gitea/workflows/ci.yml`). Do **not** include a leading slash.
- **Upload condition** — whether to upload when the condition file **exists** or when it **does not exist**.
- If the condition is not met for a repository, it is skipped.
- A summary of processed / succeeded / failed / skipped counts is printed at the end.
@@ -1,4 +1,10 @@
/** /**
* @file Delete a file from every repository across all nhcarrigan Gitea orgs.
* Prompts for the file path to delete, then removes it from every repo
* across nhcarrigan, nhcarrigan-private, and nhcarrigan-games.
* Data files: None
* Env vars:
* GITEA_TOKEN - Gitea personal access token with repo write permissions.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+7
View File
@@ -1,4 +1,11 @@
/** /**
* @file Upload a file from data/ to every repository across all nhcarrigan Gitea orgs.
* Prompts for the local filename and destination path, then creates or updates
* the file in every repo across nhcarrigan, nhcarrigan-private, and nhcarrigan-games.
* Data files (place in data/):
* - Any file to upload (prompted interactively)
* Env vars:
* GITEA_TOKEN - Gitea personal access token with repo write permissions.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
@@ -1,4 +1,11 @@
/** /**
* @file Conditionally upload a file to Gitea repos based on whether another file exists.
* Prompts for the file to upload, a condition file path, and whether to upload
* when the condition file exists or does not exist.
* Data files (place in data/):
* - Any file to upload (prompted interactively)
* Env vars:
* GITEA_TOKEN - Gitea personal access token with repo write permissions.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+124
View File
@@ -0,0 +1,124 @@
# GitHub Scripts
Scripts for GitHub API interactions and organisation management.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → github → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [auditNpmPackages.ts](#auditnpmpackagets)
- [onboardMentee.ts](#onboardmenteeets)
- [postUserStories.ts](#postuserstoriests)
---
## auditNpmPackages.ts
Audits npm packages across one or more GitHub organisations for known vulnerable package versions. For each repository, it fetches `package.json` (if present) and checks `dependencies` and `devDependencies` against a hardcoded list of vulnerable packages and their specific vulnerable versions. Results are written to a text file in `data/`.
### Usage
```bash
make run
# Select: TypeScript → github → auditNpmPackages.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITHUB_TOKEN` | GitHub personal access token with organisation read permissions |
### Data Files
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `npm-vulnerabilities.txt` | Plain text, one finding per line | Repositories with vulnerable or potentially affected packages |
### Notes
- **Before running**, update the `orgsToCheck` array and the `vulnerablePackages` list in the script to match the organisations and vulnerabilities you want to audit.
- The output file is **overwritten** at the start of each run.
- Repositories without a `package.json` are skipped silently.
- The script distinguishes between finding a package at the exact vulnerable version (marked `!! FOUND VULNERABLE !!`) and finding the package at a different version (noted for awareness).
- Repositories are fetched 100 at a time with automatic pagination.
---
## onboardMentee.ts
Onboards a new mentee to the `nhcarrigan-mentorship` GitHub organisation. Interactively prompts for the mentee's Discord ID, full name, and GitHub username, then:
1. Creates a public repository in `nhcarrigan-mentorship` named after the mentee (kebab-case).
2. Adds the mentee as a collaborator with `maintain` permissions.
3. Sends a welcome message to the mentorship Discord channel tagging the mentee with their repository URL.
### Usage
```bash
make run
# Select: TypeScript → github → onboardMentee.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITHUB_TOKEN` | GitHub personal access token with org and repo write permissions |
| `DISCORD_TOKEN` | Discord bot token |
### Data Files
None.
### Notes
- The target GitHub organisation (`nhcarrigan-mentorship`) and Discord channel ID are hardcoded in the script.
- Full name is converted to kebab-case for the repository name (e.g. `Jane Doe``jane-doe`). Special characters other than letters, digits, spaces, and hyphens are stripped.
- Discord ID must be a numeric string.
- A summary of the onboarded mentee is printed to the console on success.
---
## postUserStories.ts
Posts markdown files as GitHub issue body content. Reads all `.md` files from `data/stories/`, parses the filename to extract the repository name and issue number, then updates the corresponding GitHub issue with the file content as its description.
### Usage
```bash
make run
# Select: TypeScript → github → postUserStories.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `GITHUB_TOKEN` | GitHub personal access token with repo write permissions |
### Data Files
**Input** (expected in `data/stories/`):
| File pattern | Format | Description |
|---|---|---|
| `{repo-name}-{issue-number}.md` | Markdown | User story content for a specific issue; filename determines the target repo and issue number |
### Notes
- **Before running**, update the `orgName` constant in the script to the target GitHub organisation.
- Filenames must follow the exact pattern `{repo-name}-{issue-number}.md` (e.g. `my-repo-42.md`). Files that don't match are skipped with an error message.
- The script updates the issue **description** (body), not a comment.
- A summary of successful and failed updates is printed at the end.
@@ -1,4 +1,12 @@
/** /**
* @file Audit npm packages across a GitHub organisation for known vulnerabilities.
* Lists all repositories in the target org, fetches each package.json, and checks
* for usage of packages in the hardcoded vulnerable package list.
* Update the vulnerable package list and orgName constant before running.
* Outputs (written to data/):
* - npm-vulnerabilities.txt Repos and packages with vulnerability findings
* Env vars:
* GITHUB_TOKEN - GitHub personal access token with org read permissions.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+217
View File
@@ -0,0 +1,217 @@
/**
* @file Onboard a new mentee to the nhcarrigan-mentorship GitHub organisation.
* Prompts for the mentee's Discord ID, full name, and GitHub username, creates
* a personal repository in the org, adds the mentee as a collaborator with
* maintain permissions, and posts a welcome message to the Discord channel.
* Data files: None
* Env vars:
* GITHUB_TOKEN - GitHub personal access token with org and repo permissions
* DISCORD_TOKEN - Bot token for the Discord API.
* @copyright NHCarrigan
* @license Naomi's Public License
* @author Naomi Carrigan
*/
import { input } from "@inquirer/prompts";
import { Octokit } from "@octokit/rest";
// Environment variable checks
const githubToken = process.env.GITHUB_TOKEN;
const discordToken = process.env.DISCORD_TOKEN;
if (githubToken === undefined || githubToken === "") {
throw new Error("GITHUB_TOKEN is not set");
}
if (discordToken === undefined || discordToken === "") {
throw new Error("DISCORD_TOKEN is not set");
}
const octokit = new Octokit({ auth: githubToken });
/**
* Converts a full name to kebab-case.
* @param fullName - The full name to convert.
* @returns The kebab-case version.
*/
function toKebabCase(fullName: string): string {
return fullName.trim().toLowerCase().
replaceAll(/[^\d\sa-z-]/g, "").
replaceAll(/\s+/g, "-");
}
/**
* Prompts for mentee information.
* @returns The mentee information.
*/
async function getMenteeInfo(): Promise<{
discordId: string;
fullName: string;
githubUsername: string;
}> {
const discordId = await input({
message: "Enter the mentee's Discord ID:",
validate: (value) => {
const trimmed = value.trim();
if (trimmed === "") {
return "Discord ID cannot be empty";
}
if (!/^\d+$/.test(trimmed)) {
return "Discord ID must be numeric";
}
return true;
},
});
const fullName = await input({
message: "Enter the mentee's full name:",
validate: (value) => {
if (value.trim() === "") {
return "Full name cannot be empty";
}
return true;
},
});
const githubUsername = await input({
message: "Enter the mentee's GitHub username:",
validate: (value) => {
if (value.trim() === "") {
return "GitHub username cannot be empty";
}
return true;
},
});
return { discordId, fullName, githubUsername };
}
interface RepoData {
/**
* Using camelCase interface for internal consistency.
*/
htmlUrl: string;
}
/**
* Creates a public GitHub repository in the nhcarrigan-mentorship organization with auto-init enabled.
* @param repoName - The kebab-case repository name derived from mentee's name.
* @param fullName - The mentee's full name used in repository description.
* @returns The created repository data with HTML URL.
*/
async function createRepository(
repoName: string,
fullName: string,
): Promise<RepoData> {
console.log("\n1️⃣ Creating repository...");
const { data: repo } = await octokit.rest.repos.createInOrg({
// eslint-disable-next-line @typescript-eslint/naming-convention -- GitHub API
auto_init: true,
description: `Mentorship repository for ${fullName}`,
name: repoName,
org: "nhcarrigan-mentorship",
private: false,
});
console.log("✅ Repository created successfully!");
return { htmlUrl: repo.html_url };
}
/**
* Adds the mentee as a collaborator.
* @param repoName - The repository name.
* @param githubUsername - The mentee's GitHub username.
*/
async function addCollaborator(
repoName: string,
githubUsername: string,
): Promise<void> {
console.log("\n2️⃣ Adding collaborator...");
await octokit.rest.repos.addCollaborator({
owner: "nhcarrigan-mentorship",
permission: "maintain",
repo: repoName,
username: githubUsername,
});
console.log("✅ Collaborator added with maintain permissions!");
}
/**
* Sends a welcome message to Discord.
* @param discordId - The mentee's Discord ID.
* @param repoUrl - The repository URL.
*/
async function sendDiscordMessage(
discordId: string,
repoUrl: string,
): Promise<void> {
console.log("\n3️⃣ Sending Discord welcome message...");
const channelId = "1400589073613062204";
const welcomeMessage = {
content: `<@${discordId}> Welcome to the mentorship programme! 🎉\n\nYour personal repository has been created: ${repoUrl}\n\nYou have been added as a collaborator with maintain permissions. Feel free to use this space to practice, experiment, and work on projects. I'm here to help guide you on your journey!\n\nLooking forward to working with you! 💖`,
};
const discordResponse = await fetch(
`https://discord.com/api/v10/channels/${channelId}/messages`,
{
body: JSON.stringify(welcomeMessage),
headers: {
// eslint-disable-next-line @typescript-eslint/naming-convention -- Discord API
"Authorization": `Bot ${String(discordToken)}`,
// eslint-disable-next-line @typescript-eslint/naming-convention -- HTTP headers
"Content-Type": "application/json",
},
method: "POST",
},
);
if (!discordResponse.ok) {
const error = await discordResponse.text();
throw new Error(`Discord API error: ${error}`);
}
console.log("✅ Discord welcome message sent!");
}
/**
* Main function to onboard a new mentee.
*/
async function main(): Promise<void> {
console.log("🚀 Starting mentorship onboarding process...\n");
try {
const { discordId, fullName, githubUsername } = await getMenteeInfo();
const repoName = toKebabCase(fullName);
console.log(`\n📦 Repository will be created as: nhcarrigan-mentorship/${repoName}`);
const repo = await createRepository(repoName, fullName);
await addCollaborator(repoName, githubUsername);
await sendDiscordMessage(discordId, repo.htmlUrl);
// Success summary
console.log("\n🎊 Onboarding completed successfully!");
console.log(`\n📋 Summary:`);
console.log(` Mentee: ${fullName} (@${githubUsername})`);
console.log(` Discord ID: ${discordId}`);
console.log(` Repository: ${repo.htmlUrl}`);
console.log(` Permissions: Maintain`);
console.log(` Welcome message sent to Discord channel`);
} catch (error) {
console.error("\n❌ Error during onboarding:");
if (error instanceof Error) {
console.error(error.message);
} else {
console.error(error);
}
process.exit(1);
}
}
// Run the script
await main().catch((error: unknown) => {
console.error("❌ Unexpected error:", error);
process.exit(1);
});
+7
View File
@@ -1,4 +1,11 @@
/** /**
* @file Post user story markdown files as GitHub issue descriptions.
* Reads all markdown files from data/stories/ and updates the corresponding
* GitHub issue with that content. Filename format: {repo-name}-{issue-number}.md.
* Data files (place in data/stories/):
* - {repo-name}-{issue-number}.md User story content for each issue
* Env vars:
* GITHUB_TOKEN - GitHub personal access token with repo write permissions.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+48
View File
@@ -0,0 +1,48 @@
# Music Scripts
Scripts for working with music files and metadata.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → music → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [id3v2.ts](#id3v2ts)
---
## id3v2.ts
Tags a batch of MP3 files with ID3v2 metadata and album art. Designed for tagging downloaded Neuro-sama tracks: reads all `.mp3` files from a local directory, extracts the track title from the filename, sets the artist to `"Neuro-sama"`, and applies a specified cover image using the `eyeD3` and `id3v2` CLI tools. Progress is displayed via a terminal progress bar.
### Usage
```bash
make run
# Select: TypeScript → music → id3v2.ts
```
### Environment Variables
None.
### Data Files
None. The input directory and cover image path are defined as constants at the top of the script.
### Notes
- **Before running**, update the two constants at the top of the script:
- `directory` — path to the folder containing your MP3 files (default: `/home/naomi/down`)
- `cover` — path to the cover image to embed (default: `/home/naomi/neuro.png`)
- Requires the [`eyeD3`](https://eyed3.readthedocs.io/) and [`id3v2`](https://id3v2.sourceforge.net/) CLI tools to be installed on your system.
- Title is extracted from the filename by looking for text wrapped in `"..."` or `...` (both ASCII and fullwidth double quotes). If no quoted text is found, the filename (minus `.mp3`) is used as the title.
- Non-MP3 files in the directory are silently skipped.
+6
View File
@@ -1,4 +1,10 @@
/** /**
* @file Tag MP3 files with ID3v2 metadata and album art for Neuro-sama tracks.
* Reads all MP3 files from the download directory, extracts the title from
* the filename, sets the artist to "Neuro-sama", and applies the cover image.
* Update the directory and cover constants before running.
* Data files: None (reads from a local directory path defined in the script)
* Env vars: None (uses eyeD3 and id3v2 CLI tools).
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+163
View File
@@ -0,0 +1,163 @@
# S3 Scripts
Scripts for managing objects in an S3-compatible bucket (e.g. Hetzner Object Storage).
All scripts use the AWS SDK and share the same three environment variables.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → s3 → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [upload.ts](#uploadts)
- [bulkUpload.ts](#bulkuploAdts)
- [correctContentType.ts](#correctcontenttypets)
- [deleteContents.ts](#deletecontentsts)
---
## upload.ts
Uploads a single file from the local `data/` directory to the S3 bucket. Prompts for the local filename and the destination key (path) in the bucket. The MIME type is detected automatically from the file extension.
### Usage
```bash
make run
# Select: TypeScript → s3 → upload.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
| `S3_ENDPOINT` | S3-compatible endpoint URL |
### Data Files
**Input** (place in `data/`):
| File | Format | Description |
|---|---|---|
| *(any file)* | Any | The file to upload; name and destination path are entered interactively |
### Notes
- You will be prompted for:
- **Local filename** — path relative to `data/` (e.g. `naomi.png` or `img/naomi.png`). The file must exist in `data/`.
- **Destination path** — the key to use in the bucket (e.g. `img/naomi.png`). Do **not** include a leading slash.
- If the file extension is unrecognised, a warning is printed and the object is uploaded without a `Content-Type` header.
- The bucket name is hardcoded as `nhcarrigan`. Update the `Bucket` constant in the script to target a different bucket.
---
## bulkUpload.ts
Uploads all files in the `data/` directory (recursively) to the S3 bucket. Before uploading, it displays a tree view of all files to be uploaded and asks for confirmation. A progress bar tracks upload progress.
### Usage
```bash
make run
# Select: TypeScript → s3 → bulkUpload.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
| `S3_ENDPOINT` | S3-compatible endpoint URL |
### Data Files
**Input** (place in `data/`):
| File | Format | Description |
|---|---|---|
| *(all files)* | Any | Every file under `data/` is uploaded, preserving the relative directory structure as the S3 key |
### Notes
- The destination key for each file mirrors its path relative to `data/` (e.g. `data/img/naomi.png` → S3 key `img/naomi.png`).
- MIME types are detected from file extensions. Files with unrecognised extensions are uploaded without a `Content-Type` header.
- You must confirm the upload by typing `y` at the prompt; the default is `n` (cancel).
- Errors for individual files are logged but do not stop the rest of the run. A final summary of succeeded/failed counts is printed.
- The bucket name is hardcoded as `nhcarrigan`.
---
## correctContentType.ts
Audits all objects in the S3 bucket and interactively corrects their `Content-Type` metadata where it is missing, set to `application/octet-stream`, or does not match the expected MIME type for the file extension. Correction is performed by copying the object over itself with the new metadata (no data transfer, metadata-only update).
### Usage
```bash
make run
# Select: TypeScript → s3 → correctContentType.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
| `S3_ENDPOINT` | S3-compatible endpoint URL |
### Data Files
None.
### Notes
- For each object with an incorrect or missing `Content-Type`, you will be asked (with a default of `yes`) whether to update it. This allows skipping specific files if needed.
- Directory marker objects (keys ending in `/`) are automatically skipped.
- Objects with unrecognised extensions are also skipped (no known expected MIME type).
- A final summary of corrected / skipped / errored counts is printed.
- The bucket name is hardcoded as `nhcarrigan`.
---
## deleteContents.ts
Deletes **all** objects from a specified S3 bucket. Requires double confirmation before proceeding: first you must type the exact bucket name, then confirm with a yes/no prompt. Objects are deleted in batches of 1,000 with a progress bar.
### Usage
```bash
make run
# Select: TypeScript → s3 → deleteContents.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
| `S3_ENDPOINT` | S3-compatible endpoint URL |
### Data Files
None.
### Notes
- **This operation is irreversible.** All objects in the bucket will be permanently deleted.
- The script prompts for the bucket name (with basic validation), then requires you to type the bucket name again exactly as a first confirmation, followed by a yes/no prompt defaulting to `no`.
- If the bucket is already empty the script exits immediately without prompting for confirmation.
- Deletion uses the S3 batch delete API (up to 1,000 objects per request) for efficiency.
- A final summary of succeeded/failed counts is printed.
+9
View File
@@ -1,4 +1,13 @@
/** /**
* @file Upload all files in the data/ directory to an S3-compatible bucket.
* Displays a tree of files to upload, prompts for confirmation, then uploads
* with a progress bar showing per-file status.
* Data files (place in data/):
* - All files to upload are read from the data/ directory recursively
* Env vars:
* AWS_ACCESS_KEY_ID - S3 access key
* AWS_SECRET_ACCESS_KEY - S3 secret key
* S3_ENDPOINT - S3-compatible endpoint URL.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+8
View File
@@ -1,4 +1,12 @@
/** /**
* @file Fix incorrect Content-Type metadata on objects in an S3-compatible bucket.
* Lists all objects, detects MIME type from extension, and re-uploads metadata
* for any file whose stored Content-Type does not match the expected type.
* Data files: None
* Env vars:
* AWS_ACCESS_KEY_ID - S3 access key
* AWS_SECRET_ACCESS_KEY - S3 secret key
* S3_ENDPOINT - S3-compatible endpoint URL.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+8
View File
@@ -1,4 +1,12 @@
/** /**
* @file Delete all objects from an S3-compatible bucket.
* Prompts for the bucket name, requires double confirmation (type name + yes/no),
* then lists and deletes all objects in batches with a progress bar.
* Data files: None
* Env vars:
* AWS_ACCESS_KEY_ID - S3 access key
* AWS_SECRET_ACCESS_KEY - S3 secret key
* S3_ENDPOINT - S3-compatible endpoint URL.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+8
View File
@@ -1,4 +1,12 @@
/** /**
* @file Upload a single file from the data/ directory to an S3-compatible bucket.
* Prompts for the local filename and the destination path in the bucket.
* Data files (place in data/):
* - Any file to upload (prompted interactively)
* Env vars:
* AWS_ACCESS_KEY_ID - S3 access key
* AWS_SECRET_ACCESS_KEY - S3 secret key
* S3_ENDPOINT - S3-compatible endpoint URL.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan
+53
View File
@@ -0,0 +1,53 @@
# Security Scripts
Scripts for security analysis and reporting.
## Getting Started
Run scripts via the interactive runner from the project root:
```bash
make run
# Select: TypeScript → security → <script>
```
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
## Table of Contents
- [generateReport.ts](#generatereportts)
---
## generateReport.ts
Generates a public HTML security transparency dashboard from [DefectDojo](https://www.defectdojo.org/) findings. Fetches all active, verified findings via the DefectDojo API (handling pagination), maps each finding to its product, aggregates counts by severity (Critical / High / Medium / Low), and writes a styled HTML report to `data/public_security_report.html`.
### Usage
```bash
make run
# Select: TypeScript → security → generateReport.ts
```
### Environment Variables
| Variable | Description |
|---|---|
| `DOJO_TOKEN` | DefectDojo personal API token |
### Data Files
**Output** (written to `data/`):
| File | Format | Description |
|---|---|---|
| `public_security_report.html` | HTML | Styled security transparency dashboard, grouped by project with severity counts |
### Notes
- The DefectDojo instance URL is hardcoded to `https://security.nhcarrigan.com`. Update the `dojoUrl` constant in the script to point to a different instance.
- Both findings and products are fetched with pagination (up to 1,000 per page); the script handles multiple pages automatically.
- Findings without a product assigned are skipped with a warning.
- Project names are stripped of their org prefix (`nhcarrigan/website-headers``Website Headers`) and formatted to title case for display.
- The output HTML is self-contained and references the NHCarrigan global headers script (`https://cdn.nhcarrigan.com/headers/index.js`) for theming.
@@ -1,5 +1,12 @@
/* eslint-disable max-lines -- Necessary for all of the HTML templating. */ /* eslint-disable max-lines -- Necessary for all of the HTML templating. */
/** /**
* @file Generate a public HTML security report from DefectDojo findings.
* Fetches all active, verified findings from DefectDojo and produces a styled
* HTML dashboard grouped by project with severity counts.
* Outputs (written to data/):
* - public_security_report.html HTML security report
* Env vars:
* DOJO_TOKEN - DefectDojo API token.
* @copyright NHCarrigan * @copyright NHCarrigan
* @license Naomi's Public License * @license Naomi's Public License
* @author Naomi Carrigan * @author Naomi Carrigan