generated from nhcarrigan/template
docs: add README files for all script categories and update project docs
CI / dependency-pin-check-typescript (pull_request) Successful in 5s
CI / dependency-pin-check-python (pull_request) Successful in 4s
Security Scan and Upload / Security & DefectDojo Upload (pull_request) Successful in 57s
CI / python (pull_request) Successful in 9m33s
CI / typescript (pull_request) Successful in 9m41s
CI / dependency-pin-check-typescript (pull_request) Successful in 5s
CI / dependency-pin-check-python (pull_request) Successful in 4s
Security Scan and Upload / Security & DefectDojo Upload (pull_request) Successful in 57s
CI / python (pull_request) Successful in 9m33s
CI / typescript (pull_request) Successful in 9m41s
Add Getting Started sections and correct usage commands to all category READMEs (TypeScript, Python, Bash). Update top-level README.md and CLAUDE.md to reflect the Bash language, correct project structure, and accurate make run instructions. Remove completed DOCS_TODO.md.
This commit is contained in:
@@ -0,0 +1,125 @@
|
||||
# Crowdin Scripts
|
||||
|
||||
Scripts for managing translations in a Crowdin project.
|
||||
|
||||
## Getting Started
|
||||
|
||||
Run scripts via the interactive runner from the project root:
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → crowdin → <script>
|
||||
```
|
||||
|
||||
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [writeData.ts](#writedatats)
|
||||
- [clearHiddenTranslations.ts](#clearhiddentranslationsts)
|
||||
- [reapplyTranslations.ts](#reapplytranslationsts)
|
||||
|
||||
> **Typical workflow:** Run `writeData.ts` first to fetch and cache the project data locally, then use `clearHiddenTranslations.ts` and/or `reapplyTranslations.ts` as needed.
|
||||
|
||||
---
|
||||
|
||||
## writeData.ts
|
||||
|
||||
Fetches all file IDs and string data from a Crowdin project and writes them to local JSON files in `data/`. These files are used as inputs by the other Crowdin scripts.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → crowdin → writeData.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `CROWDIN_PROJECT_ID` | Crowdin project numeric ID |
|
||||
| `CROWDIN_API_URL` | Base URL of the Crowdin API (e.g. `https://api.crowdin.com/api/v2`) |
|
||||
| `CROWDIN_TOKEN` | Crowdin personal access token |
|
||||
|
||||
### Data Files
|
||||
|
||||
**Output** (written to `data/`):
|
||||
|
||||
| File | Format | Description |
|
||||
|---|---|---|
|
||||
| `crowdin-files.json` | JSON array of file IDs | All file IDs in the project |
|
||||
| `crowdin-strings.json` | JSON array of string objects | All strings in the project, including `id`, `isHidden`, and other metadata |
|
||||
|
||||
---
|
||||
|
||||
## clearHiddenTranslations.ts
|
||||
|
||||
Deletes existing translations for all hidden (suppressed) strings in a Crowdin project, across all active languages, in parallel. Keeps a log file to avoid re-processing strings on subsequent runs.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → crowdin → clearHiddenTranslations.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `CROWDIN_PROJECT_ID` | Crowdin project numeric ID |
|
||||
| `CROWDIN_API_URL` | Base URL of the Crowdin API |
|
||||
| `CROWDIN_TOKEN` | Crowdin personal access token |
|
||||
|
||||
### Data Files
|
||||
|
||||
**Input** (expected in `data/`):
|
||||
|
||||
| File | Format | Description |
|
||||
|---|---|---|
|
||||
| `crowdin-strings.json` | JSON array of string objects | Output of `writeData.ts` — each object must have `id` (number) and `isHidden` (boolean) fields |
|
||||
| `crowdin-strings-hidden.txt` | Plain text, one ID per line | Log of already-processed string IDs; create an empty file if starting fresh |
|
||||
|
||||
**Output** (updated in `data/`):
|
||||
|
||||
| File | Description |
|
||||
|---|---|
|
||||
| `crowdin-strings-hidden.txt` | Appended with the ID of each string processed in this run |
|
||||
|
||||
### Notes
|
||||
|
||||
- Run `writeData.ts` first to generate `crowdin-strings.json`.
|
||||
- Create an empty `crowdin-strings-hidden.txt` in `data/` before the first run. The script will append to it as strings are processed, so re-runs skip already-cleared strings.
|
||||
- Translations are deleted in parallel across all languages for each string, then the string ID is appended to the log before moving on to the next.
|
||||
|
||||
---
|
||||
|
||||
## reapplyTranslations.ts
|
||||
|
||||
Triggers a pre-translation run on a Crowdin project using Translation Memory (TM), applying perfect matches only, and polls for completion every 5 seconds until the job reaches 100%.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → crowdin → reapplyTranslations.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `CROWDIN_PROJECT_ID` | Crowdin project numeric ID |
|
||||
| `CROWDIN_API_URL` | Base URL of the Crowdin API |
|
||||
| `CROWDIN_TOKEN` | Crowdin personal access token |
|
||||
|
||||
### Data Files
|
||||
|
||||
None. The script fetches the current file and language lists from Crowdin at runtime.
|
||||
|
||||
### Notes
|
||||
|
||||
- Pre-translation is configured with `translateWithPerfectMatchOnly: true` and `autoApproveOption: "perfectMatchOnly"`, so only exact TM matches are applied and automatically approved.
|
||||
- Already-approved translations are skipped (`skipApprovedTranslations: true`).
|
||||
- The script polls progress every 5 seconds and prints the percentage until the job is complete.
|
||||
@@ -0,0 +1,87 @@
|
||||
# Discord Scripts
|
||||
|
||||
Scripts for Discord bot utilities and server management.
|
||||
|
||||
## Getting Started
|
||||
|
||||
Run scripts via the interactive runner from the project root:
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → discord → <script>
|
||||
```
|
||||
|
||||
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [cycThreads.ts](#cycthreadsts)
|
||||
- [guildCount.ts](#guildcountts)
|
||||
|
||||
---
|
||||
|
||||
## cycThreads.ts
|
||||
|
||||
Creates Discord forum threads for talk submissions in a conference channel. Iterates over a hardcoded list of talk titles and speakers, creating one public forum thread per talk with a standard discussion prompt message. Talks with missing titles or speakers are automatically filtered out.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → discord → cycThreads.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `DISCORD_TOKEN` | Discord bot token |
|
||||
|
||||
### Data Files
|
||||
|
||||
None. The list of talks (title + speaker) is hardcoded in the script. The target forum channel ID is also hardcoded.
|
||||
|
||||
### Notes
|
||||
|
||||
- **Before running**, update the `data` array in the script with the current conference talk submissions and update `CHANNEL_ID` to point to the correct Discord forum channel.
|
||||
- Uses the `backoffAndRetry` utility to handle Discord API rate limits automatically.
|
||||
- Entries with an empty `title` or `speaker` field are silently filtered before processing.
|
||||
- Threads are created with a 24-hour auto-archive duration (`auto_archive_duration: 1440`).
|
||||
|
||||
---
|
||||
|
||||
## guildCount.ts
|
||||
|
||||
Counts and categorises all Discord servers a user belongs to. Uses the Discord OAuth2 PKCE flow (no user token ever stored) to authenticate, fetches the user's guild list, and categorises each server as owned, admin, moderating, partnered, verified, community, or discoverable. Results are printed to the console and displayed as an HTML dashboard served on a local port (opened automatically in your browser).
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → discord → guildCount.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `DISCORD_CLIENT_ID` | OAuth2 application client ID (required) |
|
||||
| `DISCORD_CLIENT_SECRET` | OAuth2 application client secret (optional, improves security) |
|
||||
| `DISCORD_REDIRECT_URI` | OAuth2 redirect URI (default: `http://127.0.0.1:8721/callback`) |
|
||||
| `DISCORD_SCOPES` | OAuth2 scopes (default: `identify guilds`) |
|
||||
|
||||
### Data Files
|
||||
|
||||
None.
|
||||
|
||||
### Notes
|
||||
|
||||
- **Setup required before first run** (per machine):
|
||||
1. Create a Discord application in the [Developer Portal](https://discord.com/developers/applications) and note the Client ID.
|
||||
2. Under OAuth2 → Redirects, add `http://127.0.0.1:8721/callback` (or supply your own via `DISCORD_REDIRECT_URI`).
|
||||
3. Optionally generate a Client Secret and set it as `DISCORD_CLIENT_SECRET`.
|
||||
4. Export `DISCORD_CLIENT_ID` (and secret if used) before running the script.
|
||||
- The script prompts you to confirm setup is complete before launching the OAuth flow.
|
||||
- The OAuth flow uses PKCE (Proof Key for Code Exchange) — no user tokens are ever logged or stored; this approach is fully within Discord's Terms of Service.
|
||||
- Discord allows a maximum of 200 servers per user (with Nitro); the script fetches up to 200 at once.
|
||||
- The HTML dashboard is served on a random available local port and opened automatically; the server closes once the page is loaded.
|
||||
@@ -0,0 +1,84 @@
|
||||
# Discourse Scripts
|
||||
|
||||
Scripts for Discourse forum management.
|
||||
|
||||
## Getting Started
|
||||
|
||||
Run scripts via the interactive runner from the project root:
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → discourse → <script>
|
||||
```
|
||||
|
||||
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [bulkUpdateCategories.ts](#bulkupdatecategoriests)
|
||||
- [closeOldTopics.ts](#closeoldtopicsts)
|
||||
|
||||
---
|
||||
|
||||
## bulkUpdateCategories.ts
|
||||
|
||||
Enables auto-close on all categories and subcategories in a Discourse forum. Fetches the full category list (including subcategories), then updates each one to auto-close based on last post after 672 hours (28 days).
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → discourse → bulkUpdateCategories.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `DISCOURSE_URL` | Base URL of the Discourse instance (e.g. `https://forum.example.com`) |
|
||||
| `DISCOURSE_API_KEY` | Discourse API key |
|
||||
| `DISCOURSE_API_USERNAME` | Discourse API username |
|
||||
|
||||
### Data Files
|
||||
|
||||
None.
|
||||
|
||||
### Notes
|
||||
|
||||
- Auto-close is configured to trigger based on **last post** (not topic creation date), so active topics remain open.
|
||||
- The auto-close threshold is hardcoded to `672` hours (28 days). Update the `auto_close_hours` value in the script to change this.
|
||||
- Uses the `backoffAndRetry` utility to handle Discourse API rate limits automatically.
|
||||
- Categories are processed sequentially; subcategories are fetched individually and deduplicated.
|
||||
- Failures for individual categories are logged but do not stop the rest of the run.
|
||||
|
||||
---
|
||||
|
||||
## closeOldTopics.ts
|
||||
|
||||
Closes inactive Discourse topics that have had no activity for 28 or more days, skipping any topics older than 6 months (which are assumed to be already archived or otherwise handled). Already-closed topics are also skipped.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → discourse → closeOldTopics.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `DISCOURSE_URL` | Base URL of the Discourse instance (e.g. `https://forum.example.com`) |
|
||||
| `DISCOURSE_API_KEY` | Discourse API key |
|
||||
| `DISCOURSE_API_USERNAME` | Discourse API username |
|
||||
|
||||
### Data Files
|
||||
|
||||
None.
|
||||
|
||||
### Notes
|
||||
|
||||
- Topics are fetched from the `/latest.json` endpoint in ascending age order. Pagination stops automatically once a topic older than 6 months is encountered.
|
||||
- The inactivity threshold is **28 days** and the age cutoff is **180 days** (approximately 6 months); both are defined as constants in the script.
|
||||
- A 500 ms delay is added between close requests to avoid hitting Discourse rate limits. Rate-limit (HTTP 429) responses trigger an automatic 5-second wait and retry.
|
||||
- At the end of the run, a summary of closed/failed counts is printed.
|
||||
@@ -0,0 +1,124 @@
|
||||
# Gitea Scripts
|
||||
|
||||
Scripts for bulk file management across Gitea repositories.
|
||||
|
||||
All scripts operate across three NHCarrigan Gitea organisations: `nhcarrigan`, `nhcarrigan-private`, and `nhcarrigan-games`.
|
||||
|
||||
## Getting Started
|
||||
|
||||
Run scripts via the interactive runner from the project root:
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → gitea → <script>
|
||||
```
|
||||
|
||||
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [deleteFromAllRepos.ts](#deletefromallreposets)
|
||||
- [uploadToAllRepos.ts](#uploadtoallreposets)
|
||||
- [uploadToReposConditionally.ts](#uploadtoreposconditionallyts)
|
||||
|
||||
---
|
||||
|
||||
## deleteFromAllRepos.ts
|
||||
|
||||
Deletes a specified file from every repository across all NHCarrigan Gitea organisations. Checks each repository first; if the file exists it is deleted, otherwise the repository is skipped.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → gitea → deleteFromAllRepos.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `GITEA_TOKEN` | Gitea personal access token with repository write permissions |
|
||||
|
||||
### Data Files
|
||||
|
||||
None. The file path to delete is entered interactively when the script runs.
|
||||
|
||||
### Notes
|
||||
|
||||
- You will be prompted for the file path to delete (e.g. `.gitea/workflows/security.yml`). Do **not** include a leading slash.
|
||||
- The deletion commit message is automatically set to `feat: automated delete of {path}`.
|
||||
- Repositories are fetched 100 at a time with automatic pagination via `paginatedFetch`.
|
||||
- Failures for individual repositories are logged but do not stop the rest of the run.
|
||||
|
||||
---
|
||||
|
||||
## uploadToAllRepos.ts
|
||||
|
||||
Uploads a file from the local `data/` directory to every repository across all NHCarrigan Gitea organisations. If the file already exists in a repository it is updated; otherwise it is created.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → gitea → uploadToAllRepos.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `GITEA_TOKEN` | Gitea personal access token with repository write permissions |
|
||||
|
||||
### Data Files
|
||||
|
||||
**Input** (place in `data/`):
|
||||
|
||||
| File | Format | Description |
|
||||
|---|---|---|
|
||||
| *(any file)* | Any | The file to upload; name and destination path are entered interactively |
|
||||
|
||||
### Notes
|
||||
|
||||
- You will be prompted for:
|
||||
- **Local filename** — path relative to `data/` (e.g. `actions.yml` or `gitea/actions.yml`). The file must exist in `data/`.
|
||||
- **Destination path** — path in each repository (e.g. `.gitea/workflows/actions.yml`). Do **not** include a leading slash.
|
||||
- Commit messages are automatically set to `feat: automated upload of {path}`.
|
||||
- A summary of processed / succeeded / failed counts is printed at the end.
|
||||
|
||||
---
|
||||
|
||||
## uploadToReposConditionally.ts
|
||||
|
||||
Uploads a file to Gitea repositories only if a condition file does (or does not) exist in each repository. Useful for targeting only repositories that have a specific workflow file, language marker, or configuration already in place.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → gitea → uploadToReposConditionally.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `GITEA_TOKEN` | Gitea personal access token with repository write permissions |
|
||||
|
||||
### Data Files
|
||||
|
||||
**Input** (place in `data/`):
|
||||
|
||||
| File | Format | Description |
|
||||
|---|---|---|
|
||||
| *(any file)* | Any | The file to upload; name and destination path are entered interactively |
|
||||
|
||||
### Notes
|
||||
|
||||
- You will be prompted for:
|
||||
- **Local filename** — path relative to `data/` (same as `uploadToAllRepos.ts`).
|
||||
- **Destination path** — path in each repository.
|
||||
- **Condition file path** — path to check in each repository (e.g. `package.json` or `.gitea/workflows/ci.yml`). Do **not** include a leading slash.
|
||||
- **Upload condition** — whether to upload when the condition file **exists** or when it **does not exist**.
|
||||
- If the condition is not met for a repository, it is skipped.
|
||||
- A summary of processed / succeeded / failed / skipped counts is printed at the end.
|
||||
@@ -0,0 +1,124 @@
|
||||
# GitHub Scripts
|
||||
|
||||
Scripts for GitHub API interactions and organisation management.
|
||||
|
||||
## Getting Started
|
||||
|
||||
Run scripts via the interactive runner from the project root:
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → github → <script>
|
||||
```
|
||||
|
||||
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [auditNpmPackages.ts](#auditnpmpackagets)
|
||||
- [onboardMentee.ts](#onboardmenteeets)
|
||||
- [postUserStories.ts](#postuserstoriests)
|
||||
|
||||
---
|
||||
|
||||
## auditNpmPackages.ts
|
||||
|
||||
Audits npm packages across one or more GitHub organisations for known vulnerable package versions. For each repository, it fetches `package.json` (if present) and checks `dependencies` and `devDependencies` against a hardcoded list of vulnerable packages and their specific vulnerable versions. Results are written to a text file in `data/`.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → github → auditNpmPackages.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `GITHUB_TOKEN` | GitHub personal access token with organisation read permissions |
|
||||
|
||||
### Data Files
|
||||
|
||||
**Output** (written to `data/`):
|
||||
|
||||
| File | Format | Description |
|
||||
|---|---|---|
|
||||
| `npm-vulnerabilities.txt` | Plain text, one finding per line | Repositories with vulnerable or potentially affected packages |
|
||||
|
||||
### Notes
|
||||
|
||||
- **Before running**, update the `orgsToCheck` array and the `vulnerablePackages` list in the script to match the organisations and vulnerabilities you want to audit.
|
||||
- The output file is **overwritten** at the start of each run.
|
||||
- Repositories without a `package.json` are skipped silently.
|
||||
- The script distinguishes between finding a package at the exact vulnerable version (marked `!! FOUND VULNERABLE !!`) and finding the package at a different version (noted for awareness).
|
||||
- Repositories are fetched 100 at a time with automatic pagination.
|
||||
|
||||
---
|
||||
|
||||
## onboardMentee.ts
|
||||
|
||||
Onboards a new mentee to the `nhcarrigan-mentorship` GitHub organisation. Interactively prompts for the mentee's Discord ID, full name, and GitHub username, then:
|
||||
|
||||
1. Creates a public repository in `nhcarrigan-mentorship` named after the mentee (kebab-case).
|
||||
2. Adds the mentee as a collaborator with `maintain` permissions.
|
||||
3. Sends a welcome message to the mentorship Discord channel tagging the mentee with their repository URL.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → github → onboardMentee.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `GITHUB_TOKEN` | GitHub personal access token with org and repo write permissions |
|
||||
| `DISCORD_TOKEN` | Discord bot token |
|
||||
|
||||
### Data Files
|
||||
|
||||
None.
|
||||
|
||||
### Notes
|
||||
|
||||
- The target GitHub organisation (`nhcarrigan-mentorship`) and Discord channel ID are hardcoded in the script.
|
||||
- Full name is converted to kebab-case for the repository name (e.g. `Jane Doe` → `jane-doe`). Special characters other than letters, digits, spaces, and hyphens are stripped.
|
||||
- Discord ID must be a numeric string.
|
||||
- A summary of the onboarded mentee is printed to the console on success.
|
||||
|
||||
---
|
||||
|
||||
## postUserStories.ts
|
||||
|
||||
Posts markdown files as GitHub issue body content. Reads all `.md` files from `data/stories/`, parses the filename to extract the repository name and issue number, then updates the corresponding GitHub issue with the file content as its description.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → github → postUserStories.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `GITHUB_TOKEN` | GitHub personal access token with repo write permissions |
|
||||
|
||||
### Data Files
|
||||
|
||||
**Input** (expected in `data/stories/`):
|
||||
|
||||
| File pattern | Format | Description |
|
||||
|---|---|---|
|
||||
| `{repo-name}-{issue-number}.md` | Markdown | User story content for a specific issue; filename determines the target repo and issue number |
|
||||
|
||||
### Notes
|
||||
|
||||
- **Before running**, update the `orgName` constant in the script to the target GitHub organisation.
|
||||
- Filenames must follow the exact pattern `{repo-name}-{issue-number}.md` (e.g. `my-repo-42.md`). Files that don't match are skipped with an error message.
|
||||
- The script updates the issue **description** (body), not a comment.
|
||||
- A summary of successful and failed updates is printed at the end.
|
||||
@@ -0,0 +1,48 @@
|
||||
# Music Scripts
|
||||
|
||||
Scripts for working with music files and metadata.
|
||||
|
||||
## Getting Started
|
||||
|
||||
Run scripts via the interactive runner from the project root:
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → music → <script>
|
||||
```
|
||||
|
||||
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [id3v2.ts](#id3v2ts)
|
||||
|
||||
---
|
||||
|
||||
## id3v2.ts
|
||||
|
||||
Tags a batch of MP3 files with ID3v2 metadata and album art. Designed for tagging downloaded Neuro-sama tracks: reads all `.mp3` files from a local directory, extracts the track title from the filename, sets the artist to `"Neuro-sama"`, and applies a specified cover image using the `eyeD3` and `id3v2` CLI tools. Progress is displayed via a terminal progress bar.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → music → id3v2.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
None.
|
||||
|
||||
### Data Files
|
||||
|
||||
None. The input directory and cover image path are defined as constants at the top of the script.
|
||||
|
||||
### Notes
|
||||
|
||||
- **Before running**, update the two constants at the top of the script:
|
||||
- `directory` — path to the folder containing your MP3 files (default: `/home/naomi/down`)
|
||||
- `cover` — path to the cover image to embed (default: `/home/naomi/neuro.png`)
|
||||
- Requires the [`eyeD3`](https://eyed3.readthedocs.io/) and [`id3v2`](https://id3v2.sourceforge.net/) CLI tools to be installed on your system.
|
||||
- Title is extracted from the filename by looking for text wrapped in `"..."` or `"..."` (both ASCII and fullwidth double quotes). If no quoted text is found, the filename (minus `.mp3`) is used as the title.
|
||||
- Non-MP3 files in the directory are silently skipped.
|
||||
@@ -0,0 +1,163 @@
|
||||
# S3 Scripts
|
||||
|
||||
Scripts for managing objects in an S3-compatible bucket (e.g. Hetzner Object Storage).
|
||||
|
||||
All scripts use the AWS SDK and share the same three environment variables.
|
||||
|
||||
## Getting Started
|
||||
|
||||
Run scripts via the interactive runner from the project root:
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → s3 → <script>
|
||||
```
|
||||
|
||||
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [upload.ts](#uploadts)
|
||||
- [bulkUpload.ts](#bulkuploAdts)
|
||||
- [correctContentType.ts](#correctcontenttypets)
|
||||
- [deleteContents.ts](#deletecontentsts)
|
||||
|
||||
---
|
||||
|
||||
## upload.ts
|
||||
|
||||
Uploads a single file from the local `data/` directory to the S3 bucket. Prompts for the local filename and the destination key (path) in the bucket. The MIME type is detected automatically from the file extension.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → s3 → upload.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
|
||||
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
|
||||
| `S3_ENDPOINT` | S3-compatible endpoint URL |
|
||||
|
||||
### Data Files
|
||||
|
||||
**Input** (place in `data/`):
|
||||
|
||||
| File | Format | Description |
|
||||
|---|---|---|
|
||||
| *(any file)* | Any | The file to upload; name and destination path are entered interactively |
|
||||
|
||||
### Notes
|
||||
|
||||
- You will be prompted for:
|
||||
- **Local filename** — path relative to `data/` (e.g. `naomi.png` or `img/naomi.png`). The file must exist in `data/`.
|
||||
- **Destination path** — the key to use in the bucket (e.g. `img/naomi.png`). Do **not** include a leading slash.
|
||||
- If the file extension is unrecognised, a warning is printed and the object is uploaded without a `Content-Type` header.
|
||||
- The bucket name is hardcoded as `nhcarrigan`. Update the `Bucket` constant in the script to target a different bucket.
|
||||
|
||||
---
|
||||
|
||||
## bulkUpload.ts
|
||||
|
||||
Uploads all files in the `data/` directory (recursively) to the S3 bucket. Before uploading, it displays a tree view of all files to be uploaded and asks for confirmation. A progress bar tracks upload progress.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → s3 → bulkUpload.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
|
||||
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
|
||||
| `S3_ENDPOINT` | S3-compatible endpoint URL |
|
||||
|
||||
### Data Files
|
||||
|
||||
**Input** (place in `data/`):
|
||||
|
||||
| File | Format | Description |
|
||||
|---|---|---|
|
||||
| *(all files)* | Any | Every file under `data/` is uploaded, preserving the relative directory structure as the S3 key |
|
||||
|
||||
### Notes
|
||||
|
||||
- The destination key for each file mirrors its path relative to `data/` (e.g. `data/img/naomi.png` → S3 key `img/naomi.png`).
|
||||
- MIME types are detected from file extensions. Files with unrecognised extensions are uploaded without a `Content-Type` header.
|
||||
- You must confirm the upload by typing `y` at the prompt; the default is `n` (cancel).
|
||||
- Errors for individual files are logged but do not stop the rest of the run. A final summary of succeeded/failed counts is printed.
|
||||
- The bucket name is hardcoded as `nhcarrigan`.
|
||||
|
||||
---
|
||||
|
||||
## correctContentType.ts
|
||||
|
||||
Audits all objects in the S3 bucket and interactively corrects their `Content-Type` metadata where it is missing, set to `application/octet-stream`, or does not match the expected MIME type for the file extension. Correction is performed by copying the object over itself with the new metadata (no data transfer, metadata-only update).
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → s3 → correctContentType.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
|
||||
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
|
||||
| `S3_ENDPOINT` | S3-compatible endpoint URL |
|
||||
|
||||
### Data Files
|
||||
|
||||
None.
|
||||
|
||||
### Notes
|
||||
|
||||
- For each object with an incorrect or missing `Content-Type`, you will be asked (with a default of `yes`) whether to update it. This allows skipping specific files if needed.
|
||||
- Directory marker objects (keys ending in `/`) are automatically skipped.
|
||||
- Objects with unrecognised extensions are also skipped (no known expected MIME type).
|
||||
- A final summary of corrected / skipped / errored counts is printed.
|
||||
- The bucket name is hardcoded as `nhcarrigan`.
|
||||
|
||||
---
|
||||
|
||||
## deleteContents.ts
|
||||
|
||||
Deletes **all** objects from a specified S3 bucket. Requires double confirmation before proceeding: first you must type the exact bucket name, then confirm with a yes/no prompt. Objects are deleted in batches of 1,000 with a progress bar.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → s3 → deleteContents.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `AWS_ACCESS_KEY_ID` | S3 access key ID |
|
||||
| `AWS_SECRET_ACCESS_KEY` | S3 secret access key |
|
||||
| `S3_ENDPOINT` | S3-compatible endpoint URL |
|
||||
|
||||
### Data Files
|
||||
|
||||
None.
|
||||
|
||||
### Notes
|
||||
|
||||
- **This operation is irreversible.** All objects in the bucket will be permanently deleted.
|
||||
- The script prompts for the bucket name (with basic validation), then requires you to type the bucket name again exactly as a first confirmation, followed by a yes/no prompt defaulting to `no`.
|
||||
- If the bucket is already empty the script exits immediately without prompting for confirmation.
|
||||
- Deletion uses the S3 batch delete API (up to 1,000 objects per request) for efficiency.
|
||||
- A final summary of succeeded/failed counts is printed.
|
||||
@@ -0,0 +1,53 @@
|
||||
# Security Scripts
|
||||
|
||||
Scripts for security analysis and reporting.
|
||||
|
||||
## Getting Started
|
||||
|
||||
Run scripts via the interactive runner from the project root:
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → security → <script>
|
||||
```
|
||||
|
||||
Scripts that require secrets (API keys, tokens) have them injected automatically via 1Password CLI.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [generateReport.ts](#generatereportts)
|
||||
|
||||
---
|
||||
|
||||
## generateReport.ts
|
||||
|
||||
Generates a public HTML security transparency dashboard from [DefectDojo](https://www.defectdojo.org/) findings. Fetches all active, verified findings via the DefectDojo API (handling pagination), maps each finding to its product, aggregates counts by severity (Critical / High / Medium / Low), and writes a styled HTML report to `data/public_security_report.html`.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
make run
|
||||
# Select: TypeScript → security → generateReport.ts
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `DOJO_TOKEN` | DefectDojo personal API token |
|
||||
|
||||
### Data Files
|
||||
|
||||
**Output** (written to `data/`):
|
||||
|
||||
| File | Format | Description |
|
||||
|---|---|---|
|
||||
| `public_security_report.html` | HTML | Styled security transparency dashboard, grouped by project with severity counts |
|
||||
|
||||
### Notes
|
||||
|
||||
- The DefectDojo instance URL is hardcoded to `https://security.nhcarrigan.com`. Update the `dojoUrl` constant in the script to point to a different instance.
|
||||
- Both findings and products are fetched with pagination (up to 1,000 per page); the script handles multiple pages automatically.
|
||||
- Findings without a product assigned are skipped with a warning.
|
||||
- Project names are stripped of their org prefix (`nhcarrigan/website-headers` → `Website Headers`) and formatted to title case for display.
|
||||
- The output HTML is self-contained and references the NHCarrigan global headers script (`https://cdn.nhcarrigan.com/headers/index.js`) for theming.
|
||||
Reference in New Issue
Block a user