generated from nhcarrigan/template
feat: reorganise bash scripts and add comprehensive documentation (#6)
CI / dependency-pin-check-typescript (push) Successful in 5s
CI / dependency-pin-check-python (push) Successful in 4s
CI / python (push) Successful in 9m28s
CI / typescript (push) Successful in 9m42s
Security Scan and Upload / Security & DefectDojo Upload (push) Successful in 1m39s
CI / dependency-pin-check-typescript (push) Successful in 5s
CI / dependency-pin-check-python (push) Successful in 4s
CI / python (push) Successful in 9m28s
CI / typescript (push) Successful in 9m42s
Security Scan and Upload / Security & DefectDojo Upload (push) Successful in 1m39s
## Summary This PR completes the bash script restructuring and adds comprehensive documentation across all script categories. ### Bash Restructuring - Moved cohort shell scripts (`remove_github_members.sh`, `update_github_teams.sh`) from `python/cohort/` into a new `bash/cohort/` directory - Moved existing bash utilities (`add-keys-to-git.sh`, `fix-yubikey-perms.sh`, `list-yubikey-ssh-keys.sh`) into a new `bash/yubikey/` subdirectory - Updated `run.sh` to support **Bash** as a third language option alongside TypeScript and Python - Bash scripts are run directly (no 1Password secret injection needed) - Category discovery and script listing works the same as for TS/Python - Removed dead "Root Scripts" logic that was no longer needed ### Documentation Added `README.md` files for all script categories that were missing them: - `bash/cohort/README.md` — cohort GitHub team management scripts - `bash/yubikey/README.md` — YubiKey SSH key and permission utilities - `typescript/src/crowdin/README.md` — Crowdin translation management scripts - `typescript/src/discord/README.md` — Discord bot utility scripts - `typescript/src/discourse/README.md` — Discourse forum management scripts - `typescript/src/gitea/README.md` — Gitea bulk repository operation scripts - `typescript/src/github/README.md` — GitHub API interaction scripts - `typescript/src/music/README.md` — Music file metadata tools - `typescript/src/s3/README.md` — S3-compatible object storage scripts - `typescript/src/security/README.md` — Security analysis and reporting scripts - `python/cohort/README.md` — Updated to remove moved shell scripts, fix usage commands Also updated project-level docs: - **`README.md`** — Corrected project structure, fixed running instructions (removed references to non-existent `make run-ts`/`make run-py` targets), added Bash prerequisites - **`CLAUDE.md`** — Updated project overview, structure, development standards, and script-adding guides to reflect the current state of the project ✨ This PR was created with help from Hikari~ 🌸 Co-authored-by: Naomi Carrigan <commits@nhcarrigan.com> Reviewed-on: #6 Co-authored-by: Hikari <hikari@nhcarrigan.com> Co-committed-by: Hikari <hikari@nhcarrigan.com>
This commit was merged in pull request #6.
This commit is contained in:
@@ -1,22 +1,31 @@
|
||||
# Ephemere
|
||||
|
||||
A collection of ephemeral scripts for various tasks, written in TypeScript and Python.
|
||||
A collection of ephemeral scripts for various tasks, written in TypeScript, Python, and Bash.
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
.
|
||||
├── typescript/ # TypeScript project
|
||||
│ ├── src/ # TypeScript source files
|
||||
│ ├── package.json
|
||||
│ ├── tsconfig.json
|
||||
│ └── eslint.config.js
|
||||
├── python/ # Python project
|
||||
│ ├── *.py # Python scripts
|
||||
│ ├── pyproject.toml
|
||||
│ └── requirements.txt
|
||||
├── Makefile # Build commands for both projects
|
||||
└── README.md
|
||||
├── typescript/ # TypeScript scripts
|
||||
│ └── src/
|
||||
│ ├── crowdin/ # Crowdin translation management
|
||||
│ ├── discord/ # Discord bot utilities
|
||||
│ ├── discourse/ # Discourse forum management
|
||||
│ ├── gitea/ # Gitea bulk repository operations
|
||||
│ ├── github/ # GitHub API interactions
|
||||
│ ├── music/ # Music file metadata tools
|
||||
│ ├── s3/ # S3-compatible object storage
|
||||
│ ├── security/ # Security analysis and reporting
|
||||
│ └── utils/ # Shared utilities
|
||||
├── python/
|
||||
│ └── cohort/ # NHCarrigan cohort programme management
|
||||
├── bash/
|
||||
│ ├── cohort/ # GitHub team management for cohorts
|
||||
│ └── yubikey/ # YubiKey SSH key and permission utilities
|
||||
├── data/ # Input/output data files (gitignored)
|
||||
├── prod.env # 1Password vault references (safe to commit)
|
||||
├── run.sh # Interactive script runner
|
||||
└── Makefile # Build and utility commands
|
||||
```
|
||||
|
||||
## Setup
|
||||
@@ -25,105 +34,74 @@ A collection of ephemeral scripts for various tasks, written in TypeScript and P
|
||||
|
||||
- Node.js (v24+) with nvm
|
||||
- Python 3.10+
|
||||
- pnpm 10.15.0
|
||||
- pnpm 10.15.0+
|
||||
- uv (Python package manager)
|
||||
- 1Password CLI (for secrets management)
|
||||
- 1Password CLI (`op`) — for secret injection into TypeScript and Python scripts
|
||||
- `gh` CLI — for Bash scripts that manage GitHub teams
|
||||
- `ykman` and `yubico-piv-tool` — for YubiKey Bash scripts
|
||||
|
||||
### Installation
|
||||
|
||||
Install all dependencies (TypeScript and Python):
|
||||
|
||||
```bash
|
||||
make install
|
||||
```
|
||||
|
||||
Or install individually:
|
||||
|
||||
```bash
|
||||
make install-ts # TypeScript dependencies only
|
||||
make install-py # Python dependencies only
|
||||
```
|
||||
|
||||
## Running Scripts
|
||||
|
||||
The recommended way to run any script is the interactive runner:
|
||||
|
||||
```bash
|
||||
make run
|
||||
```
|
||||
|
||||
This launches a menu to select a language (TypeScript, Python, or Bash), then a category, then a specific script. TypeScript and Python scripts have secrets injected automatically from `prod.env` via 1Password CLI. Bash scripts are run directly.
|
||||
|
||||
To run a script directly without the interactive runner:
|
||||
|
||||
```bash
|
||||
# TypeScript
|
||||
cd typescript && op run --env-file=../prod.env -- pnpm tsx src/<category>/<script>.ts
|
||||
|
||||
# Python
|
||||
cd python && op run --env-file=../prod.env -- uv run python cohort/<script>.py
|
||||
|
||||
# Bash
|
||||
bash bash/<category>/<script>.sh
|
||||
```
|
||||
|
||||
Each category has its own `README.md` with details on every script, its required environment variables, and data file formats.
|
||||
|
||||
## Development
|
||||
|
||||
### TypeScript Scripts
|
||||
|
||||
TypeScript scripts are located in the `typescript/src/` directory. To run a TypeScript script with environment variables:
|
||||
### Linting and Formatting
|
||||
|
||||
```bash
|
||||
# From the root directory
|
||||
make run-ts src/s3/upload.ts
|
||||
|
||||
# Or manually from typescript directory
|
||||
cd typescript
|
||||
pnpm start path/to/script.ts
|
||||
make lint # Run all linters (TypeScript and Python)
|
||||
make lint-ts # TypeScript linter only
|
||||
make lint-py # Python linter only
|
||||
make build # TypeScript type check
|
||||
make format # Format Python code
|
||||
make format-check # Check Python formatting without modifying
|
||||
make test # Run tests
|
||||
make clean # Clean build artifacts and caches
|
||||
make help # Show all available commands
|
||||
```
|
||||
|
||||
### Python Scripts
|
||||
|
||||
Python scripts are located in the `python/` directory. To run a Python script with environment variables:
|
||||
|
||||
```bash
|
||||
# From the root directory
|
||||
make run-py analyse_availability.py
|
||||
|
||||
# Or manually from python directory
|
||||
cd python
|
||||
uv run python script_name.py
|
||||
```
|
||||
|
||||
## Linting and Formatting
|
||||
|
||||
```bash
|
||||
# Run all linters (TypeScript and Python)
|
||||
make lint
|
||||
|
||||
# Run linters individually
|
||||
make lint-ts # TypeScript linter
|
||||
make lint-py # Python linter
|
||||
|
||||
# Build TypeScript (type check)
|
||||
make build
|
||||
|
||||
# Format Python code
|
||||
make format
|
||||
|
||||
# Check Python formatting without modifying
|
||||
make format-check
|
||||
|
||||
# Run tests
|
||||
make test
|
||||
|
||||
# Clean build artifacts and caches
|
||||
make clean
|
||||
|
||||
# Show all available commands
|
||||
make help
|
||||
```
|
||||
|
||||
## CI/CD
|
||||
|
||||
The GitHub Actions workflow runs the following checks:
|
||||
|
||||
1. **Dependency pin check** - Ensures all dependencies are pinned to exact versions
|
||||
2. **TypeScript checks**:
|
||||
- ESLint
|
||||
- TypeScript build (type checking)
|
||||
- Tests
|
||||
3. **Python checks**:
|
||||
- Ruff linting
|
||||
- Ruff format checking
|
||||
|
||||
## Secrets Management
|
||||
|
||||
This project uses 1Password CLI for secrets management. Environment variables are stored in `prod.env` as 1Password vault references.
|
||||
This project uses 1Password CLI for secrets management. The `prod.env` file contains 1Password vault references (e.g. `op://Private/Discord/token`) rather than real values, making it safe to commit.
|
||||
|
||||
The `make run-ts` and `make run-py` commands automatically inject secrets from 1Password:
|
||||
```bash
|
||||
# These commands include 1Password integration
|
||||
make run-ts src/discord/bot.ts
|
||||
make run-py evaluate_technical_proficiency.py
|
||||
```
|
||||
The interactive runner (`make run`) handles secret injection automatically. To run a script manually with secrets:
|
||||
|
||||
To manually run scripts with secrets:
|
||||
```bash
|
||||
op run --env-file=prod.env -- <command>
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user