generated from nhcarrigan/template
This commit is contained in:
@@ -591,6 +591,46 @@ export const navigation = [
|
|||||||
label: "TypeScript Config",
|
label: "TypeScript Config",
|
||||||
link: "/projects/typescript-config",
|
link: "/projects/typescript-config",
|
||||||
badge: { text: "v4.0.0", variant: "tip" },
|
badge: { text: "v4.0.0", variant: "tip" },
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Discord Analytics",
|
||||||
|
link: "/projects/discord-analytics",
|
||||||
|
badge: { text: "v1.0.0", variant: "tip" },
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Eclaire",
|
||||||
|
link: "/projects/eclaire",
|
||||||
|
badge: { text: "v1.0.0", variant: "tip" },
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "freeCodeCamp Review Generator",
|
||||||
|
link: "/projects/freecodecamp-review-generator",
|
||||||
|
badge: { text: "v1.0.0", variant: "tip" },
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Keiko",
|
||||||
|
link: "/projects/keiko",
|
||||||
|
badge: { text: "v1.0.0", variant: "tip" },
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Meeting Minutes",
|
||||||
|
link: "/projects/meeting-minutes",
|
||||||
|
badge: { text: "v1.0.0", variant: "tip" },
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Nomena",
|
||||||
|
link: "/projects/nomena",
|
||||||
|
badge: { text: "v1.0.0", variant: "tip" },
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Scripts",
|
||||||
|
link: "/projects/scripts",
|
||||||
|
badge: { text: "v1.0.0", variant: "tip" },
|
||||||
|
},
|
||||||
|
{
|
||||||
|
label: "Sylvara",
|
||||||
|
link: "/projects/sylvara",
|
||||||
|
badge: { text: "unreleased", variant: "danger" },
|
||||||
}
|
}
|
||||||
].sort((a, b) => a.label.localeCompare(b.label)),
|
].sort((a, b) => a.label.localeCompare(b.label)),
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -0,0 +1,240 @@
|
|||||||
|
---
|
||||||
|
title: Discord Analytics
|
||||||
|
---
|
||||||
|
|
||||||
|
Discord Analytics (hereinafter the "Application") is a TypeScript library package designed to integrate with Discord bots to provide automated analytics tracking and metrics collection. The Application pairs with the @nhcarrigan/logger tool to pipe Discord bot analytics and telemetry data.
|
||||||
|
|
||||||
|
## 1. User Documentation
|
||||||
|
|
||||||
|
This section is for those interacting with a live instance of the Application.
|
||||||
|
|
||||||
|
### Overview
|
||||||
|
|
||||||
|
Discord Analytics is distributed as an npm package and is designed to be integrated into Discord.js-based bots. The package is available at [@nhcarrigan/discord-analytics](https://www.npmjs.com/package/@nhcarrigan/discord-analytics).
|
||||||
|
|
||||||
|
### Features
|
||||||
|
|
||||||
|
The Application provides the following capabilities:
|
||||||
|
|
||||||
|
- **Automated Daily Metrics Collection**: Runs a CRON job at midnight (system time) to collect:
|
||||||
|
- Total number of guilds (Discord servers) the bot is a member of
|
||||||
|
- Total member count across all guilds
|
||||||
|
- Approximate number of user installs for the application
|
||||||
|
|
||||||
|
- **Gateway Event Logging**: Allows logging of Discord gateway events with custom metadata for detailed analytics tracking
|
||||||
|
|
||||||
|
### Basic Usage
|
||||||
|
|
||||||
|
1. Install the package via npm:
|
||||||
|
```
|
||||||
|
npm install @nhcarrigan/discord-analytics
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Import and initialize the DiscordAnalytics class with your Discord client and logger instance:
|
||||||
|
```typescript
|
||||||
|
import { DiscordAnalytics } from '@nhcarrigan/discord-analytics';
|
||||||
|
|
||||||
|
const analytics = new DiscordAnalytics(client, logger);
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Start the automated metrics collection:
|
||||||
|
```typescript
|
||||||
|
analytics.startCron();
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Log gateway events from your Discord event listeners:
|
||||||
|
```typescript
|
||||||
|
client.on(Events.MessageCreate, async (message) => {
|
||||||
|
await analytics.logGatewayEvent(Events.MessageCreate, {
|
||||||
|
channelId: message.channelId
|
||||||
|
});
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
5. Stop the CRON job when needed:
|
||||||
|
```typescript
|
||||||
|
analytics.stopCron();
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2. Technical Documentation
|
||||||
|
|
||||||
|
This section is for those interested in running their own instance of the Application.
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- Node.js (compatible with node-schedule 2.1.1)
|
||||||
|
- Discord.js v14.x or higher
|
||||||
|
- @nhcarrigan/logger v1.1.0-hotfix or higher
|
||||||
|
|
||||||
|
### Installation
|
||||||
|
|
||||||
|
The Application is installed as an npm package dependency:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm install @nhcarrigan/discord-analytics
|
||||||
|
```
|
||||||
|
|
||||||
|
Or using pnpm:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm add @nhcarrigan/discord-analytics
|
||||||
|
```
|
||||||
|
|
||||||
|
### Architecture
|
||||||
|
|
||||||
|
#### Core Components
|
||||||
|
|
||||||
|
The Application consists of a single main class located in `src/index.ts`:
|
||||||
|
|
||||||
|
**DiscordAnalytics Class** (src/index.ts:14-96)
|
||||||
|
|
||||||
|
The main class provides three public methods:
|
||||||
|
|
||||||
|
1. **`constructor(client: Client, logger: Logger)`** (src/index.ts:22-25)
|
||||||
|
- Initializes the analytics instance with a Discord.js client and logger
|
||||||
|
- Parameters:
|
||||||
|
- `client`: The Discord.js client instance to monitor
|
||||||
|
- `logger`: Instance of @nhcarrigan/logger for metric logging
|
||||||
|
|
||||||
|
2. **`startCron(): void`** (src/index.ts:33-65)
|
||||||
|
- Starts a CRON job scheduled to run at midnight daily (0 0 * * *)
|
||||||
|
- Collects and logs three metrics:
|
||||||
|
- `user_installs`: Approximate count of user installs
|
||||||
|
- `guilds`: Total number of guilds the bot is in
|
||||||
|
- `members`: Total member count across all guilds
|
||||||
|
- Includes error handling that logs failures via the logger
|
||||||
|
|
||||||
|
3. **`stopCron(): void`** (src/index.ts:70-76)
|
||||||
|
- Cancels the running CRON job if active
|
||||||
|
- Safely handles the case where no job is running
|
||||||
|
|
||||||
|
4. **`logGatewayEvent(event: Events, payload: Record<string, unknown>): Promise<void>`** (src/index.ts:85-95)
|
||||||
|
- Logs Discord gateway events as metrics
|
||||||
|
- Parameters:
|
||||||
|
- `event`: The Discord.js event type (e.g., Events.MessageCreate)
|
||||||
|
- `payload`: Custom metadata to attach to the metric
|
||||||
|
|
||||||
|
#### Dependencies
|
||||||
|
|
||||||
|
- **node-schedule** (2.1.1): Provides CRON job scheduling functionality
|
||||||
|
- **@nhcarrigan/logger** (peer dependency): Handles metric and error logging
|
||||||
|
- **discord.js** (peer dependency v14.x): Discord API wrapper
|
||||||
|
|
||||||
|
#### Build Process
|
||||||
|
|
||||||
|
The Application uses TypeScript and includes the following build scripts:
|
||||||
|
|
||||||
|
- `pnpm run lint`: Runs ESLint with zero warnings tolerance
|
||||||
|
- `pnpm run build`: Removes the `prod` directory and compiles TypeScript
|
||||||
|
- `pnpm prepublish`: Runs linting and build before publishing
|
||||||
|
|
||||||
|
The compiled output is located in the `prod` directory with `prod/index.js` as the main entry point.
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
The CRON schedule is hardcoded to run at midnight daily (system time). The schedule can be identified in the code at src/index.ts:37 as `"0 0 * * *"`.
|
||||||
|
|
||||||
|
### Data Collection
|
||||||
|
|
||||||
|
The Application collects the following metrics:
|
||||||
|
|
||||||
|
- **Guild Count**: Total number of Discord servers (src/index.ts:54)
|
||||||
|
- **Member Count**: Sum of all members across guilds (src/index.ts:55)
|
||||||
|
- **User Installs**: Approximate user installation count (src/index.ts:49-52)
|
||||||
|
- **Gateway Events**: Custom event tracking with metadata (src/index.ts:89-94)
|
||||||
|
|
||||||
|
All metrics are logged using the provided logger instance's `metric()` method.
|
||||||
|
|
||||||
|
### Error Handling
|
||||||
|
|
||||||
|
Errors during the CRON job execution are caught and logged via the logger's `error()` method (src/index.ts:57-62). The Application ensures errors are properly converted to Error instances before logging.
|
||||||
|
|
||||||
|
## 3. Legal Documentation
|
||||||
|
|
||||||
|
This section is for expansions to our legal policies specific to the Application.
|
||||||
|
|
||||||
|
### License
|
||||||
|
|
||||||
|
The Application is licensed under [Naomi's Public License](https://docs.nhcarrigan.com/#/license) (referenced as "global software license").
|
||||||
|
|
||||||
|
Copyright is held by Naomi Carrigan.
|
||||||
|
|
||||||
|
### Privacy Considerations
|
||||||
|
|
||||||
|
The Application collects the following data from Discord:
|
||||||
|
|
||||||
|
- Guild (server) identifiers and counts
|
||||||
|
- Member counts (aggregate numbers, not individual user data)
|
||||||
|
- User installation counts (approximate numbers from Discord API)
|
||||||
|
- Gateway event data with optional custom metadata
|
||||||
|
|
||||||
|
Users integrating this Application should ensure their privacy policies adequately disclose the collection and logging of this telemetry data.
|
||||||
|
|
||||||
|
### Third-Party Services
|
||||||
|
|
||||||
|
The Application interacts with:
|
||||||
|
|
||||||
|
- Discord API (via discord.js)
|
||||||
|
- The configured logging service (@nhcarrigan/logger)
|
||||||
|
|
||||||
|
Data is transmitted to whatever backend service is configured with the logger instance.
|
||||||
|
|
||||||
|
## 4. Contributing Documentation
|
||||||
|
|
||||||
|
This section is for documentation related to contributing to the Application's codebase.
|
||||||
|
|
||||||
|
### Contributing Guidelines
|
||||||
|
|
||||||
|
Full contributing guidelines can be found at: https://docs.nhcarrigan.com/#/contributing
|
||||||
|
|
||||||
|
### Code of Conduct
|
||||||
|
|
||||||
|
Before interacting with the community, please review the Code of Conduct at: https://docs.nhcarrigan.com/#/coc
|
||||||
|
|
||||||
|
### Development Setup
|
||||||
|
|
||||||
|
1. Clone the repository
|
||||||
|
2. Install dependencies:
|
||||||
|
```bash
|
||||||
|
pnpm install
|
||||||
|
```
|
||||||
|
3. Make your changes in the `src` directory
|
||||||
|
4. Run linting:
|
||||||
|
```bash
|
||||||
|
pnpm run lint
|
||||||
|
```
|
||||||
|
5. Build the project:
|
||||||
|
```bash
|
||||||
|
pnpm run build
|
||||||
|
```
|
||||||
|
|
||||||
|
### Code Style
|
||||||
|
|
||||||
|
The project uses:
|
||||||
|
|
||||||
|
- ESLint with @nhcarrigan/eslint-config (v5.2.0)
|
||||||
|
- TypeScript with @nhcarrigan/typescript-config (v4.0.0)
|
||||||
|
- Maximum warnings: 0 (strict mode)
|
||||||
|
|
||||||
|
### Pull Request Process
|
||||||
|
|
||||||
|
1. Create a Pull Request with your proposed changes
|
||||||
|
2. Ensure all linting passes
|
||||||
|
3. The maintainers will review as soon as possible
|
||||||
|
|
||||||
|
### Reporting Issues
|
||||||
|
|
||||||
|
If you have feedback or a bug report, please open a GitHub issue.
|
||||||
|
|
||||||
|
### Contact
|
||||||
|
|
||||||
|
The maintainers may be contacted through:
|
||||||
|
|
||||||
|
- [Chat Server](http://chat.nhcarrigan.com)
|
||||||
|
- Email: contact@nhcarrigan.com
|
||||||
|
|
||||||
|
### Current Version
|
||||||
|
|
||||||
|
Version: 0.0.6
|
||||||
|
|
||||||
|
Recent changes can be viewed in the git commit history.
|
||||||
@@ -0,0 +1,363 @@
|
|||||||
|
---
|
||||||
|
title: Eclaire
|
||||||
|
---
|
||||||
|
|
||||||
|
Eclaire (hereinafter the "Application") is a real-time speech-to-speech translation web application that captures live audio from a user's microphone, transcribes the speech using Deepgram's API, and translates the transcribed text into a target language.
|
||||||
|
|
||||||
|
## 1. User Documentation
|
||||||
|
|
||||||
|
This section is for those interacting with a live instance of the Application.
|
||||||
|
|
||||||
|
### Getting Started
|
||||||
|
|
||||||
|
1. **Initial Setup**: Navigate to the Configuration page to set up your API credentials and language preferences.
|
||||||
|
|
||||||
|
2. **Configuration Options**:
|
||||||
|
- **Deepgram API Key**: Required for speech-to-text transcription. You must provide your own API key from [Deepgram](https://deepgram.com).
|
||||||
|
- **Translation Service API Key**: Required for text translation. A demo key is available with a 10 requests/minute rate limit, or you can provide your own key.
|
||||||
|
- **Source Language**: Select from 12 supported languages (English, German, Dutch, Swedish, Danish, Spanish, French, Portuguese, Italian, Turkish, Norwegian, Indonesian).
|
||||||
|
- **Target Language**: Select from 60+ supported languages for translation.
|
||||||
|
|
||||||
|
3. **Using the Translation Interface**:
|
||||||
|
- Navigate to the Agent page to begin translation.
|
||||||
|
- Click "Start" to begin capturing audio from your microphone.
|
||||||
|
- Speak clearly into your microphone in the configured source language.
|
||||||
|
- The application will display real-time transcription and translations.
|
||||||
|
- Click "Finalize" to manually end the current speech segment if needed.
|
||||||
|
- Click "Stop" to end the session.
|
||||||
|
|
||||||
|
### Features
|
||||||
|
|
||||||
|
- **Real-Time Transcription**: Powered by Deepgram's nova-3-general model with automatic punctuation
|
||||||
|
- **Multi-Language Translation**: Support for 60+ target languages via LibreTranslate
|
||||||
|
- **Conversation History**: View all transcriptions and translations in a message-based format
|
||||||
|
- **Connection Status**: Monitor your connection status and any errors
|
||||||
|
- **Cost Estimation**: Track estimated costs (approximately $0.32/hour)
|
||||||
|
|
||||||
|
### Audio Quality Settings
|
||||||
|
|
||||||
|
The application automatically enables:
|
||||||
|
- Echo cancellation
|
||||||
|
- Noise suppression
|
||||||
|
- Auto gain control
|
||||||
|
|
||||||
|
### Limitations
|
||||||
|
|
||||||
|
- Source language is limited to 12 languages supported by Deepgram
|
||||||
|
- Translation service demo key has a rate limit of 10 requests/minute
|
||||||
|
- Requires modern browser with microphone access
|
||||||
|
- WebSocket connection required for real-time functionality
|
||||||
|
|
||||||
|
## 2. Technical Documentation
|
||||||
|
|
||||||
|
This section is for those interested in running their own instance of the Application.
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- Node.js (version compatible with Angular 19.1.4)
|
||||||
|
- pnpm package manager
|
||||||
|
- Modern web browser with WebRTC support
|
||||||
|
- Deepgram API key
|
||||||
|
- LibreTranslate API key (or access to a LibreTranslate instance)
|
||||||
|
|
||||||
|
### Installation
|
||||||
|
|
||||||
|
1. Clone the repository:
|
||||||
|
```bash
|
||||||
|
git clone <repository-url>
|
||||||
|
cd eclaire
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Install dependencies:
|
||||||
|
```bash
|
||||||
|
npm install
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Start the development server:
|
||||||
|
```bash
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Build for production:
|
||||||
|
```bash
|
||||||
|
npm run build
|
||||||
|
```
|
||||||
|
|
||||||
|
### Technology Stack
|
||||||
|
|
||||||
|
**Frontend Framework:**
|
||||||
|
- Angular 19.1.4 with standalone components
|
||||||
|
- TypeScript 5.7.3
|
||||||
|
- RxJS 7.8.1 for reactive programming
|
||||||
|
- Angular Reactive Forms for configuration management
|
||||||
|
|
||||||
|
**External APIs:**
|
||||||
|
- Deepgram WebSocket API (nova-3-general model) for speech-to-text
|
||||||
|
- LibreTranslate API for text translation
|
||||||
|
|
||||||
|
**Audio Processing:**
|
||||||
|
- Web Audio API for microphone access
|
||||||
|
- ScriptProcessor for audio stream processing (4096 buffer size)
|
||||||
|
- 48kHz sample rate, mono channel, Int16 format
|
||||||
|
|
||||||
|
### Architecture
|
||||||
|
|
||||||
|
#### Components
|
||||||
|
|
||||||
|
1. **HomeComponent** (`src/app/home/home.component.ts`)
|
||||||
|
- Landing page with navigation links
|
||||||
|
- Entry point for users
|
||||||
|
|
||||||
|
2. **ConfigComponent** (`src/app/config/config.component.ts`)
|
||||||
|
- Configuration interface using reactive forms
|
||||||
|
- Manages API keys and language preferences
|
||||||
|
- Form validation for required fields
|
||||||
|
- Stores configuration in ConfigService
|
||||||
|
|
||||||
|
3. **AgentComponent** (`src/app/agent/agent.component.ts`)
|
||||||
|
- Main translation interface
|
||||||
|
- Manages WebSocket lifecycle
|
||||||
|
- Handles microphone capture and audio streaming
|
||||||
|
- Displays transcription and translation results
|
||||||
|
- Tracks connection status and errors
|
||||||
|
|
||||||
|
#### Services
|
||||||
|
|
||||||
|
1. **ConfigService** (`src/app/config.service.ts`)
|
||||||
|
- In-memory storage for application configuration
|
||||||
|
- Provides configuration access across components
|
||||||
|
- Stores: API keys, source language, target language
|
||||||
|
|
||||||
|
2. **SocketService** (`src/app/socket.service.ts`)
|
||||||
|
- Manages WebSocket connection to Deepgram
|
||||||
|
- Handles audio streaming in real-time
|
||||||
|
- Event-based architecture for message handling
|
||||||
|
- Supports keep-alive and finalization
|
||||||
|
- Configuration options:
|
||||||
|
- Model: nova-3-general
|
||||||
|
- Language: configurable source language
|
||||||
|
- Encoding: linear16
|
||||||
|
- Sample rate: 48000
|
||||||
|
- Channels: 1 (mono)
|
||||||
|
- Endpointing: 500ms
|
||||||
|
- Features: punctuate
|
||||||
|
|
||||||
|
#### Routing
|
||||||
|
|
||||||
|
- `/` - Redirects to home
|
||||||
|
- `/home` - Landing page
|
||||||
|
- `/config` - Configuration page
|
||||||
|
- `/agent` - Translation interface
|
||||||
|
|
||||||
|
### API Integration
|
||||||
|
|
||||||
|
#### Deepgram API
|
||||||
|
|
||||||
|
The application uses Deepgram's WebSocket API for real-time transcription:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
wss://api.deepgram.com/v1/listen
|
||||||
|
```
|
||||||
|
|
||||||
|
Query parameters:
|
||||||
|
- `model=nova-3-general`
|
||||||
|
- `language=<source-language>`
|
||||||
|
- `encoding=linear16`
|
||||||
|
- `sample_rate=48000`
|
||||||
|
- `channels=1`
|
||||||
|
- `endpointing=500`
|
||||||
|
- `punctuate=true`
|
||||||
|
|
||||||
|
Authentication: Bearer token (API key) in WebSocket URL
|
||||||
|
|
||||||
|
#### LibreTranslate API
|
||||||
|
|
||||||
|
The application uses the LibreTranslate API for text translation:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
https://trans.nhcarrigan.com/translate
|
||||||
|
```
|
||||||
|
|
||||||
|
POST request body:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"q": "<text-to-translate>",
|
||||||
|
"source": "<source-language-code>",
|
||||||
|
"target": "<target-language-code>",
|
||||||
|
"api_key": "<api-key>"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Development
|
||||||
|
|
||||||
|
#### Available Scripts
|
||||||
|
|
||||||
|
- `npm run dev` - Start development server
|
||||||
|
- `npm run lint` - Run ESLint with zero warnings policy
|
||||||
|
- `npm run build` - Build for production
|
||||||
|
|
||||||
|
#### Code Quality
|
||||||
|
|
||||||
|
- ESLint configuration from `@nhcarrigan/eslint-config`
|
||||||
|
- TypeScript configuration from `@nhcarrigan/typescript-config`
|
||||||
|
- Strict linting with `--max-warnings 0`
|
||||||
|
|
||||||
|
#### Testing
|
||||||
|
|
||||||
|
- Framework: Jasmine ~5.5.0
|
||||||
|
- Test Runner: Karma ~6.4.4
|
||||||
|
- Coverage reporting enabled
|
||||||
|
|
||||||
|
### Deployment Considerations
|
||||||
|
|
||||||
|
1. **Environment Variables**: API keys should be configurable per deployment environment
|
||||||
|
2. **CORS**: Ensure proper CORS configuration for LibreTranslate API
|
||||||
|
3. **HTTPS**: Required for microphone access in production
|
||||||
|
4. **WebSocket**: Ensure WebSocket connections are allowed through firewalls
|
||||||
|
5. **Browser Support**: Requires WebRTC and Web Audio API support
|
||||||
|
|
||||||
|
### Performance
|
||||||
|
|
||||||
|
- Audio streaming at 48kHz with 4096 sample buffer
|
||||||
|
- WebSocket for low-latency real-time communication
|
||||||
|
- Estimated cost: $0.32/hour for Deepgram usage
|
||||||
|
- Efficient audio processing with minimal overhead
|
||||||
|
|
||||||
|
## 3. Legal Documentation
|
||||||
|
|
||||||
|
This section is for expansions to our legal policies specific to the Application.
|
||||||
|
|
||||||
|
### License
|
||||||
|
|
||||||
|
This software is licensed under the [global software license](https://docs.nhcarrigan.com/#/license).
|
||||||
|
|
||||||
|
Copyright held by Naomi Carrigan.
|
||||||
|
|
||||||
|
### Privacy Policy
|
||||||
|
|
||||||
|
The Application's privacy practices are governed by the [global privacy policy](https://docs.nhcarrigan.com/#/privacy).
|
||||||
|
|
||||||
|
#### Application-Specific Privacy Considerations
|
||||||
|
|
||||||
|
1. **Audio Data**: The Application captures audio from your microphone and streams it to Deepgram's servers for transcription. Audio data is not stored locally by the Application.
|
||||||
|
|
||||||
|
2. **API Keys**: API keys are stored in-memory during your session and are not persisted to local storage or transmitted to any servers other than the respective API providers (Deepgram and LibreTranslate).
|
||||||
|
|
||||||
|
3. **Third-Party Services**:
|
||||||
|
- **Deepgram**: Audio data is transmitted to Deepgram for transcription. Refer to [Deepgram's privacy policy](https://deepgram.com/privacy) for information on their data handling practices.
|
||||||
|
- **LibreTranslate**: Transcribed text is transmitted to the LibreTranslate API for translation. The default instance is hosted at trans.nhcarrigan.com.
|
||||||
|
|
||||||
|
4. **Local Storage**: The Application does not persist any user data to browser storage.
|
||||||
|
|
||||||
|
### Terms of Service
|
||||||
|
|
||||||
|
The Application's usage is governed by the [global terms of service](https://docs.nhcarrigan.com/#/terms).
|
||||||
|
|
||||||
|
#### Application-Specific Terms
|
||||||
|
|
||||||
|
1. **API Usage**: Users are responsible for their own Deepgram API usage and associated costs. The Application provides cost estimates but does not guarantee accuracy.
|
||||||
|
|
||||||
|
2. **Demo API Key**: The translation service demo API key is provided for testing purposes only and has rate limits (10 requests/minute). It may be revoked or changed at any time.
|
||||||
|
|
||||||
|
3. **Service Availability**: The Application depends on third-party services (Deepgram and LibreTranslate). Service interruptions may occur due to factors outside of our control.
|
||||||
|
|
||||||
|
4. **Accuracy**: The Application provides translation services based on third-party APIs. Translation accuracy is not guaranteed and should not be relied upon for critical communications.
|
||||||
|
|
||||||
|
### Security Policy
|
||||||
|
|
||||||
|
For security concerns, please refer to the [SECURITY.md](SECURITY.md) file in the repository.
|
||||||
|
|
||||||
|
## 4. Contributing Documentation
|
||||||
|
|
||||||
|
This section is for documentation related to contributing to the Application's codebase.
|
||||||
|
|
||||||
|
### Contributing Guidelines
|
||||||
|
|
||||||
|
General contributing guidelines can be found at: https://docs.nhcarrigan.com/#/contributing
|
||||||
|
|
||||||
|
### Code of Conduct
|
||||||
|
|
||||||
|
Before interacting with our community, please read our [Code of Conduct](CODE_OF_CONDUCT.md).
|
||||||
|
|
||||||
|
### Development Setup
|
||||||
|
|
||||||
|
1. Fork the repository
|
||||||
|
2. Clone your fork: `git clone <your-fork-url>`
|
||||||
|
3. Install dependencies: `npm install`
|
||||||
|
4. Create a feature branch: `git checkout -b feature/your-feature-name`
|
||||||
|
5. Make your changes
|
||||||
|
6. Run linting: `npm run lint`
|
||||||
|
7. Commit your changes with clear commit messages
|
||||||
|
8. Push to your fork: `git push origin feature/your-feature-name`
|
||||||
|
9. Open a Pull Request
|
||||||
|
|
||||||
|
### Code Style
|
||||||
|
|
||||||
|
- Follow the ESLint configuration (`@nhcarrigan/eslint-config`)
|
||||||
|
- Use TypeScript with strict type checking
|
||||||
|
- Follow Angular style guide for component and service structure
|
||||||
|
- Use meaningful variable and function names
|
||||||
|
- Add comments for complex logic
|
||||||
|
|
||||||
|
### Component Guidelines
|
||||||
|
|
||||||
|
- Use standalone components (Angular 19+)
|
||||||
|
- Implement proper lifecycle hooks
|
||||||
|
- Use reactive programming patterns with RxJS
|
||||||
|
- Follow Angular dependency injection best practices
|
||||||
|
- Keep components focused on a single responsibility
|
||||||
|
|
||||||
|
### Service Guidelines
|
||||||
|
|
||||||
|
- Use `@Injectable({ providedIn: 'root' })` for singleton services
|
||||||
|
- Implement proper error handling
|
||||||
|
- Use TypeScript interfaces for type safety
|
||||||
|
- Document public methods with JSDoc comments
|
||||||
|
|
||||||
|
### Testing Guidelines
|
||||||
|
|
||||||
|
- Write unit tests for components and services
|
||||||
|
- Aim for high code coverage
|
||||||
|
- Test user interactions and edge cases
|
||||||
|
- Mock external dependencies (APIs, services)
|
||||||
|
|
||||||
|
### Pull Request Process
|
||||||
|
|
||||||
|
1. Ensure all tests pass
|
||||||
|
2. Ensure linting passes with zero warnings
|
||||||
|
3. Update documentation if needed
|
||||||
|
4. Provide a clear description of changes
|
||||||
|
5. Reference any related issues
|
||||||
|
6. Wait for review from maintainers
|
||||||
|
|
||||||
|
### Areas for Contribution
|
||||||
|
|
||||||
|
- **UI/UX Improvements**: Enhance the user interface and experience
|
||||||
|
- **Additional Languages**: Expand language support where possible
|
||||||
|
- **Performance Optimization**: Improve audio processing efficiency
|
||||||
|
- **Testing**: Increase test coverage
|
||||||
|
- **Documentation**: Improve user and technical documentation
|
||||||
|
- **Accessibility**: Enhance accessibility features
|
||||||
|
- **Error Handling**: Improve error messages and handling
|
||||||
|
- **Configuration**: Add more configuration options
|
||||||
|
|
||||||
|
### Reporting Issues
|
||||||
|
|
||||||
|
If you have feedback or a bug report, please feel free to open a GitHub issue with:
|
||||||
|
- Clear description of the issue
|
||||||
|
- Steps to reproduce (for bugs)
|
||||||
|
- Expected vs actual behavior
|
||||||
|
- Browser and OS information
|
||||||
|
- Screenshots if applicable
|
||||||
|
|
||||||
|
### Contact
|
||||||
|
|
||||||
|
We may be contacted through:
|
||||||
|
- [Chat Server](http://chat.nhcarrigan.com)
|
||||||
|
- Email: contact@nhcarrigan.com
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Version**: 1.1.0
|
||||||
|
**Last Updated**: 2025-10-31
|
||||||
|
**Author**: Naomi Carrigan
|
||||||
@@ -0,0 +1,443 @@
|
|||||||
|
---
|
||||||
|
title: freeCodeCamp Review Generator
|
||||||
|
---
|
||||||
|
|
||||||
|
freeCodeCamp Review Generator (hereinafter the "Application") is a command-line tool that aggregates all review lesson pages from freeCodeCamp's Full Stack Developer curriculum into a single, comprehensive PDF document for offline study and reference.
|
||||||
|
|
||||||
|
## 1. User Documentation
|
||||||
|
|
||||||
|
This section is for those interacting with a live instance of the Application.
|
||||||
|
|
||||||
|
### Overview
|
||||||
|
|
||||||
|
The Application generates a compiled PDF document containing all review pages from freeCodeCamp's Full Stack Developer curriculum. The generated PDF includes:
|
||||||
|
|
||||||
|
- A custom cover page with generation date
|
||||||
|
- An automatically generated table of contents with links to each review section
|
||||||
|
- All review content organized according to the curriculum structure
|
||||||
|
- Custom headers and footers for easy navigation
|
||||||
|
- Page numbers and links to community resources
|
||||||
|
|
||||||
|
### Generated PDF Structure
|
||||||
|
|
||||||
|
The output PDF (`fcc-review-pages.pdf`) follows the freeCodeCamp curriculum structure:
|
||||||
|
|
||||||
|
1. **HTML Topics** - Reviews covering basic HTML, semantic HTML, forms and tables, and accessibility
|
||||||
|
2. **CSS Topics** - Reviews covering basic CSS, layout (Flexbox, Grid), styling, responsive design, animations, and more
|
||||||
|
3. **JavaScript Topics** - Reviews covering variables, functions, arrays, objects, DOM manipulation, asynchronous programming, and more
|
||||||
|
4. **Frontend Libraries** - Reviews covering React fundamentals, state management, routing, and frontend frameworks
|
||||||
|
5. **Python Topics** - Reviews covering Python basics, data structures, algorithms, and object-oriented programming
|
||||||
|
6. **Relational Databases** - Reviews covering SQL, Bash scripting, and Git
|
||||||
|
|
||||||
|
### What to Expect
|
||||||
|
|
||||||
|
Each review page in the generated PDF contains:
|
||||||
|
- **Topic headings** - Clear section markers for each subject
|
||||||
|
- **Code examples** - Syntax-highlighted code snippets demonstrating concepts
|
||||||
|
- **Explanations** - Detailed descriptions of programming concepts and best practices
|
||||||
|
- **Reference material** - Quick-reference information for common patterns and methods
|
||||||
|
|
||||||
|
### Output Format
|
||||||
|
|
||||||
|
- **File name**: `fcc-review-pages.pdf`
|
||||||
|
- **Page format**: Legal size (8.5" x 14")
|
||||||
|
- **Margins**: 0.75 inches on all sides
|
||||||
|
- **Header**: Displays "Naomi's freeCodeCamp Review"
|
||||||
|
- **Footer**: Contains page numbers and link to community Discord server
|
||||||
|
|
||||||
|
### Disclaimer
|
||||||
|
|
||||||
|
This is not an officially sanctioned freeCodeCamp document. The Application is maintained independently and the generated content reflects the curriculum state at the time of generation. For the most up-to-date information, always refer to the official freeCodeCamp website.
|
||||||
|
|
||||||
|
## 2. Technical Documentation
|
||||||
|
|
||||||
|
This section is for those interested in running their own instance of the Application.
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- **Node.js**: Version 18.x or higher recommended
|
||||||
|
- **pnpm**: Package manager (version 10.18.0 or compatible)
|
||||||
|
- **Operating System**: Linux, macOS, or Windows
|
||||||
|
- **Disk Space**: Minimum 500MB for dependencies and output files
|
||||||
|
|
||||||
|
### Installation
|
||||||
|
|
||||||
|
1. Clone the repository:
|
||||||
|
```bash
|
||||||
|
git clone <repository-url>
|
||||||
|
cd fcc-review-generator
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Install dependencies:
|
||||||
|
```bash
|
||||||
|
pnpm install
|
||||||
|
```
|
||||||
|
|
||||||
|
This will automatically install Chrome browser via Puppeteer (required for PDF generation).
|
||||||
|
|
||||||
|
### Directory Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
fcc-review-generator/
|
||||||
|
├── content/ # Review lesson markdown files
|
||||||
|
│ └── review-*/ # Individual review topic directories
|
||||||
|
├── data/
|
||||||
|
│ └── order.json # Curriculum ordering configuration
|
||||||
|
├── src/
|
||||||
|
│ ├── config/
|
||||||
|
│ │ ├── options.ts # PDF generation options
|
||||||
|
│ │ └── text.ts # Cover page text template
|
||||||
|
│ └── index.ts # Main application entry point
|
||||||
|
├── package.json # Project dependencies and scripts
|
||||||
|
└── tsconfig.json # TypeScript configuration
|
||||||
|
```
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
#### Content Source
|
||||||
|
|
||||||
|
1. Copy review lesson markdown files from the freeCodeCamp repository to the `content/` directory
|
||||||
|
2. The Application recursively walks the directory tree, so nested folders are supported
|
||||||
|
3. Only `.md` files are processed; other files are ignored
|
||||||
|
|
||||||
|
#### Curriculum Ordering
|
||||||
|
|
||||||
|
1. Copy the Full Stack curriculum order from `/curriculum/structure/superblocks/fullstack.json` in the freeCodeCamp repository
|
||||||
|
2. Paste into `data/order.json` in this project
|
||||||
|
3. The Application uses this to sort review pages in the correct curriculum order
|
||||||
|
|
||||||
|
#### PDF Customization
|
||||||
|
|
||||||
|
Modify `src/config/options.ts` to customize:
|
||||||
|
- Page size and margins
|
||||||
|
- Header and footer templates
|
||||||
|
- Styling and formatting
|
||||||
|
|
||||||
|
Modify `src/config/text.ts` to customize:
|
||||||
|
- Cover page content
|
||||||
|
- Introduction text
|
||||||
|
- Disclaimer text
|
||||||
|
|
||||||
|
### Usage
|
||||||
|
|
||||||
|
Run the Application:
|
||||||
|
```bash
|
||||||
|
pnpm start
|
||||||
|
```
|
||||||
|
|
||||||
|
The Application will:
|
||||||
|
1. Read all markdown files from `content/` directory
|
||||||
|
2. Sort them according to `data/order.json`
|
||||||
|
3. Generate a table of contents
|
||||||
|
4. Compile all content into a single markdown file (`review.md`)
|
||||||
|
5. Convert the markdown to PDF (`fcc-review-pages.pdf`)
|
||||||
|
6. Clean up the temporary markdown file
|
||||||
|
|
||||||
|
### Output
|
||||||
|
|
||||||
|
- **Success**: Generates `fcc-review-pages.pdf` in the project root
|
||||||
|
- **Console**: Displays progress messages during execution
|
||||||
|
- **Errors**: Logged to console with descriptive messages
|
||||||
|
|
||||||
|
### Development
|
||||||
|
|
||||||
|
#### Build TypeScript
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm build
|
||||||
|
```
|
||||||
|
|
||||||
|
Compiles TypeScript files according to `tsconfig.json`.
|
||||||
|
|
||||||
|
#### Linting
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm lint
|
||||||
|
```
|
||||||
|
|
||||||
|
Runs ESLint with strict settings (max warnings: 0).
|
||||||
|
|
||||||
|
#### Testing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm test
|
||||||
|
```
|
||||||
|
|
||||||
|
Currently outputs a placeholder message (no tests implemented).
|
||||||
|
|
||||||
|
### Technical Architecture
|
||||||
|
|
||||||
|
#### Core Components
|
||||||
|
|
||||||
|
1. **File Reader** (`readDirectoryRecursively`)
|
||||||
|
- Recursively scans the content directory
|
||||||
|
- Returns flat array of file paths
|
||||||
|
- Handles nested directory structures
|
||||||
|
|
||||||
|
2. **File Sorter** (`sortFiles`)
|
||||||
|
- Sorts files based on curriculum order from `order.json`
|
||||||
|
- Matches file directory names to curriculum block names
|
||||||
|
- Maintains proper curriculum sequence
|
||||||
|
|
||||||
|
3. **Content Processor** (`rollupFiles`)
|
||||||
|
- Extracts frontmatter metadata (title, ID, etc.)
|
||||||
|
- Strips frontmatter and freeCodeCamp-specific heading syntax
|
||||||
|
- Generates table of contents with anchor links
|
||||||
|
- Compiles all content into single markdown file
|
||||||
|
|
||||||
|
4. **PDF Generator** (`createPdf`)
|
||||||
|
- Uses `md-to-pdf` library (which uses Puppeteer/Chrome)
|
||||||
|
- Applies custom styling and layout options
|
||||||
|
- Generates headers, footers, and page numbers
|
||||||
|
- Writes final PDF to disk
|
||||||
|
|
||||||
|
#### Dependencies
|
||||||
|
|
||||||
|
- **md-to-pdf** (v5.2.4): Markdown to PDF conversion using headless Chrome
|
||||||
|
- **tsx** (v4.20.6): TypeScript execution for development
|
||||||
|
- **typescript** (v5.9.3): TypeScript compiler
|
||||||
|
- **eslint** (v9.37.0): Code quality and style enforcement
|
||||||
|
|
||||||
|
### Troubleshooting
|
||||||
|
|
||||||
|
#### Common Issues
|
||||||
|
|
||||||
|
**Issue**: Puppeteer/Chrome installation fails
|
||||||
|
- **Solution**: Ensure you have sufficient disk space and network connectivity. Run `pnpx puppeteer browsers install chrome` manually.
|
||||||
|
|
||||||
|
**Issue**: PDF generation fails with "Protocol error"
|
||||||
|
- **Solution**: Chrome may need additional system dependencies. On Linux, install required libraries for headless Chrome.
|
||||||
|
|
||||||
|
**Issue**: Files not appearing in correct order
|
||||||
|
- **Solution**: Verify `data/order.json` matches the structure of your content directory names.
|
||||||
|
|
||||||
|
**Issue**: Out of memory errors
|
||||||
|
- **Solution**: The Application processes all content in memory. For very large datasets, consider increasing Node.js memory limit: `NODE_OPTIONS="--max-old-space-size=4096" pnpm start`
|
||||||
|
|
||||||
|
### Performance Considerations
|
||||||
|
|
||||||
|
- File processing is sequential for consistent ordering
|
||||||
|
- PDF generation uses headless Chrome, which is memory-intensive
|
||||||
|
- Typical generation time: 10-30 seconds depending on content volume
|
||||||
|
- Temporary markdown file is automatically cleaned up after PDF generation
|
||||||
|
|
||||||
|
### System Requirements
|
||||||
|
|
||||||
|
- **Memory**: Minimum 2GB RAM (4GB recommended)
|
||||||
|
- **CPU**: Modern multi-core processor recommended
|
||||||
|
- **Network**: Required for initial dependency installation
|
||||||
|
- **Storage**: 500MB minimum for dependencies and output
|
||||||
|
|
||||||
|
## 3. Legal Documentation
|
||||||
|
|
||||||
|
This section is for expansions to our legal policies specific to the Application.
|
||||||
|
|
||||||
|
### License
|
||||||
|
|
||||||
|
This software is licensed under Naomi's Public License. The full license text can be found at:
|
||||||
|
- Local: [LICENSE.md](LICENSE.md)
|
||||||
|
- Online: https://docs.nhcarrigan.com/#/license
|
||||||
|
|
||||||
|
Copyright is held by Naomi Carrigan.
|
||||||
|
|
||||||
|
### Privacy Policy
|
||||||
|
|
||||||
|
The Application processes markdown files locally and does not collect, store, or transmit any personal data. For information about privacy practices related to the project infrastructure (repository hosting, issue tracking, etc.), please refer to:
|
||||||
|
- Local: [PRIVACY.md](PRIVACY.md)
|
||||||
|
- Online: https://docs.nhcarrigan.com/#/privacy
|
||||||
|
|
||||||
|
### Terms of Service
|
||||||
|
|
||||||
|
By using this Application, you agree to the Terms of Service. The full terms can be found at:
|
||||||
|
- Local: [TERMS.md](TERMS.md)
|
||||||
|
- Online: https://docs.nhcarrigan.com/#/terms
|
||||||
|
|
||||||
|
### Security Policy
|
||||||
|
|
||||||
|
If you discover a security vulnerability in the Application, please follow the responsible disclosure process outlined in our Security Policy:
|
||||||
|
- Local: [SECURITY.md](SECURITY.md)
|
||||||
|
- Online: https://docs.nhcarrigan.com/#/security
|
||||||
|
|
||||||
|
### Content Attribution
|
||||||
|
|
||||||
|
The review content processed by this Application originates from freeCodeCamp.org and is subject to freeCodeCamp's licensing terms. This tool is an unofficial aggregator and is not endorsed by or affiliated with freeCodeCamp.org.
|
||||||
|
|
||||||
|
Users of the generated PDF should respect freeCodeCamp's intellectual property rights and use the content for personal educational purposes only.
|
||||||
|
|
||||||
|
### Disclaimer of Warranty
|
||||||
|
|
||||||
|
THE APPLICATION IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. THE MAINTAINERS MAKE NO WARRANTY THAT:
|
||||||
|
- The generated PDF will be error-free or complete
|
||||||
|
- The content will be current or up-to-date with the latest freeCodeCamp curriculum
|
||||||
|
- The Application will function without interruption
|
||||||
|
|
||||||
|
Users should always refer to the official freeCodeCamp website for the most accurate and current curriculum information.
|
||||||
|
|
||||||
|
## 4. Contributing Documentation
|
||||||
|
|
||||||
|
This section is for documentation related to contributing to the Application's codebase.
|
||||||
|
|
||||||
|
### Getting Started
|
||||||
|
|
||||||
|
We welcome contributions to the freeCodeCamp Review Generator! Before contributing, please review:
|
||||||
|
|
||||||
|
1. **Contributing Guidelines**: https://docs.nhcarrigan.com/#/contributing
|
||||||
|
- Local reference: [CONTRIBUTING.md](CONTRIBUTING.md)
|
||||||
|
2. **Code of Conduct**: https://docs.nhcarrigan.com/#/coc
|
||||||
|
- Local reference: [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)
|
||||||
|
|
||||||
|
### Ways to Contribute
|
||||||
|
|
||||||
|
#### 1. Report Bugs
|
||||||
|
|
||||||
|
If you encounter a bug, please open a GitHub issue with:
|
||||||
|
- Clear description of the problem
|
||||||
|
- Steps to reproduce
|
||||||
|
- Expected vs. actual behavior
|
||||||
|
- Your environment (OS, Node.js version, pnpm version)
|
||||||
|
- Any relevant error messages or logs
|
||||||
|
|
||||||
|
#### 2. Suggest Enhancements
|
||||||
|
|
||||||
|
We appreciate feature requests and enhancement suggestions! Please open a GitHub issue describing:
|
||||||
|
- The problem you're trying to solve
|
||||||
|
- Your proposed solution
|
||||||
|
- Any alternative approaches you've considered
|
||||||
|
- How this would benefit other users
|
||||||
|
|
||||||
|
#### 3. Submit Pull Requests
|
||||||
|
|
||||||
|
Ready to contribute code? Follow these steps:
|
||||||
|
|
||||||
|
1. **Fork and Clone**
|
||||||
|
```bash
|
||||||
|
git clone <your-fork-url>
|
||||||
|
cd fcc-review-generator
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Create a Branch**
|
||||||
|
```bash
|
||||||
|
git checkout -b feature/your-feature-name
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Make Changes**
|
||||||
|
- Follow existing code style and patterns
|
||||||
|
- Update relevant documentation
|
||||||
|
- Ensure TypeScript types are correct
|
||||||
|
- Test your changes thoroughly
|
||||||
|
|
||||||
|
4. **Run Quality Checks**
|
||||||
|
```bash
|
||||||
|
pnpm lint # Check code style
|
||||||
|
pnpm build # Ensure TypeScript compiles
|
||||||
|
pnpm start # Test PDF generation
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Commit Changes**
|
||||||
|
```bash
|
||||||
|
git add .
|
||||||
|
git commit -m "feat: description of your changes"
|
||||||
|
```
|
||||||
|
|
||||||
|
Follow conventional commit message format:
|
||||||
|
- `feat:` for new features
|
||||||
|
- `fix:` for bug fixes
|
||||||
|
- `docs:` for documentation changes
|
||||||
|
- `refactor:` for code refactoring
|
||||||
|
- `chore:` for maintenance tasks
|
||||||
|
|
||||||
|
6. **Push and Create PR**
|
||||||
|
```bash
|
||||||
|
git push origin feature/your-feature-name
|
||||||
|
```
|
||||||
|
Then open a Pull Request on GitHub with a clear description of your changes.
|
||||||
|
|
||||||
|
### Code Style Guidelines
|
||||||
|
|
||||||
|
- **TypeScript**: Use strict type checking; avoid `any` types
|
||||||
|
- **Formatting**: Follow the ESLint configuration (`@nhcarrigan/eslint-config`)
|
||||||
|
- **Naming**: Use descriptive variable and function names
|
||||||
|
- **Comments**: Include JSDoc comments for exported functions
|
||||||
|
- **Error Handling**: Use try-catch blocks and provide meaningful error messages
|
||||||
|
|
||||||
|
### Project Conventions
|
||||||
|
|
||||||
|
#### Copyright Headers
|
||||||
|
|
||||||
|
All source files should include the copyright header:
|
||||||
|
```typescript
|
||||||
|
/**
|
||||||
|
* @copyright NHCarrigan
|
||||||
|
* @license Naomi's Public License
|
||||||
|
* @author Naomi Carrigan
|
||||||
|
*/
|
||||||
|
```
|
||||||
|
|
||||||
|
#### File Organization
|
||||||
|
|
||||||
|
- Place configuration in `src/config/`
|
||||||
|
- Keep utility functions modular and focused
|
||||||
|
- Use ES modules (import/export)
|
||||||
|
- Maintain clear separation of concerns
|
||||||
|
|
||||||
|
#### TypeScript Configuration
|
||||||
|
|
||||||
|
The project uses strict TypeScript settings:
|
||||||
|
- `strict: true`
|
||||||
|
- Target: ES2022
|
||||||
|
- Module: ESNext
|
||||||
|
- No implicit any types
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
|
||||||
|
Currently, the project does not have an automated test suite. Contributions that add testing infrastructure are welcome! Potential testing improvements:
|
||||||
|
|
||||||
|
- Unit tests for file processing functions
|
||||||
|
- Integration tests for PDF generation
|
||||||
|
- Snapshot tests for output consistency
|
||||||
|
- CI/CD pipeline integration
|
||||||
|
|
||||||
|
### Development Workflow
|
||||||
|
|
||||||
|
1. **Install dependencies**: `pnpm install`
|
||||||
|
2. **Make changes** to source files in `src/`
|
||||||
|
3. **Test locally**: `pnpm start` to verify PDF generation
|
||||||
|
4. **Check for errors**: `pnpm lint` and `pnpm build`
|
||||||
|
5. **Commit and push** your changes
|
||||||
|
6. **Create Pull Request** for review
|
||||||
|
|
||||||
|
### Review Process
|
||||||
|
|
||||||
|
Pull requests will be reviewed by project maintainers. The review process includes:
|
||||||
|
|
||||||
|
- Code quality and style verification
|
||||||
|
- Functionality testing
|
||||||
|
- Documentation completeness
|
||||||
|
- Alignment with project goals
|
||||||
|
|
||||||
|
Please be patient during the review process and be open to feedback and requested changes.
|
||||||
|
|
||||||
|
### Communication
|
||||||
|
|
||||||
|
For questions, discussions, or help with contributions:
|
||||||
|
|
||||||
|
- **GitHub Issues**: For bug reports and feature requests
|
||||||
|
- **Pull Requests**: For code contributions and discussions
|
||||||
|
- **Discord**: Join the community at http://chat.nhcarrigan.com
|
||||||
|
- **Email**: contact@nhcarrigan.com for private inquiries
|
||||||
|
|
||||||
|
### Recognition
|
||||||
|
|
||||||
|
All contributors will be recognized for their contributions. Significant contributions may result in being listed in a CONTRIBUTORS file or project documentation.
|
||||||
|
|
||||||
|
### License for Contributions
|
||||||
|
|
||||||
|
By contributing to this project, you agree that your contributions will be licensed under the same license as the project (Naomi's Public License). You confirm that:
|
||||||
|
|
||||||
|
- You have the right to submit the contribution
|
||||||
|
- Your contribution is your original work or properly attributed
|
||||||
|
- You understand the contribution will be publicly available
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
Thank you for your interest in contributing to the freeCodeCamp Review Generator! Your contributions help make this tool better for everyone in the learning community.
|
||||||
@@ -0,0 +1,473 @@
|
|||||||
|
---
|
||||||
|
title: Keiko
|
||||||
|
---
|
||||||
|
|
||||||
|
Keiko (hereinafter the "Application") is a private Discord bot that serves as Naomi's personal AI assistant, providing intelligent conversational assistance powered by Anthropic's Claude AI with integrated web search capabilities.
|
||||||
|
|
||||||
|
## 1. User Documentation
|
||||||
|
|
||||||
|
This section is for those interacting with a live instance of the Application.
|
||||||
|
|
||||||
|
### 1.1. Getting Started
|
||||||
|
|
||||||
|
Keiko is available through Discord and can be interacted with in multiple ways:
|
||||||
|
|
||||||
|
- **Direct Messages (DM)**: Send a direct message to Keiko to start a private conversation
|
||||||
|
- **Guild/Server Messages**: Mention @Keiko in a server where the bot is present
|
||||||
|
- **Thread Messages**: Participate in threads created by Keiko
|
||||||
|
|
||||||
|
### 1.2. Access Requirements
|
||||||
|
|
||||||
|
- **Primary User**: Naomi (User ID: 465650873650118659) has full access to all features
|
||||||
|
- **Subscribers**: Other users require an active Discord entitlement (premium subscription) to interact with Keiko
|
||||||
|
- **Unauthorized Users**: Will receive a prompt to purchase a premium subscription
|
||||||
|
|
||||||
|
### 1.3. Available Commands
|
||||||
|
|
||||||
|
#### `/dm`
|
||||||
|
Reopens or initiates a direct message conversation with Keiko.
|
||||||
|
|
||||||
|
**Usage**: `/dm`
|
||||||
|
|
||||||
|
**Example**:
|
||||||
|
```
|
||||||
|
/dm
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `/clear`
|
||||||
|
Clears the conversation history in the current channel or thread. This inserts a marker message that prevents Keiko from accessing previous messages in the conversation context.
|
||||||
|
|
||||||
|
**Usage**: `/clear`
|
||||||
|
|
||||||
|
**Example**:
|
||||||
|
```
|
||||||
|
/clear
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.4. Conversational Features
|
||||||
|
|
||||||
|
#### Message Handling
|
||||||
|
- Keiko maintains conversation context by reading the last 20 messages in the channel/thread
|
||||||
|
- Responses are automatically chunked into 2000-character segments for readability
|
||||||
|
- Keiko will show typing indicators while processing your request
|
||||||
|
- Multiple message chunks are delivered with a 2.5-second delay between them
|
||||||
|
|
||||||
|
#### Web Search Integration
|
||||||
|
- Keiko can search the web to provide up-to-date information
|
||||||
|
- Web search is automatically invoked when needed (up to 5 searches per request)
|
||||||
|
- Citations and sources are included in responses when available
|
||||||
|
|
||||||
|
#### Cost Display
|
||||||
|
- Each response includes token usage statistics
|
||||||
|
- Input and output token counts are displayed
|
||||||
|
- Estimated API costs are shown for transparency
|
||||||
|
|
||||||
|
### 1.5. Keiko's Personality
|
||||||
|
|
||||||
|
Keiko is designed with a calm, demure, and reserved personality. She conducts herself like a butler - always polite, respectful, and ready to serve. Her primary role is to provide clinical and unbiased information in response to requests, with an emphasis on including sources and links to confirm claims.
|
||||||
|
|
||||||
|
### 1.6. Getting Help
|
||||||
|
|
||||||
|
If you encounter any issues or need support:
|
||||||
|
- Visit the community server: [chat.nhcarrigan.com](https://chat.nhcarrigan.com)
|
||||||
|
- Open an issue on the GitHub repository
|
||||||
|
- Contact via email: contact@nhcarrigan.com
|
||||||
|
|
||||||
|
## 2. Technical Documentation
|
||||||
|
|
||||||
|
This section is for those interested in running their own instance of the Application.
|
||||||
|
|
||||||
|
### 2.1. Prerequisites
|
||||||
|
|
||||||
|
Before setting up Keiko, ensure you have the following:
|
||||||
|
|
||||||
|
- **Node.js**: Latest LTS version recommended
|
||||||
|
- **npm**: Package manager (comes with Node.js)
|
||||||
|
- **Discord Bot**: A registered Discord application with bot token
|
||||||
|
- **Anthropic API Key**: Access to Claude AI API
|
||||||
|
- **1Password CLI** (optional): For secure environment variable management
|
||||||
|
|
||||||
|
### 2.2. Technology Stack
|
||||||
|
|
||||||
|
- **Runtime**: Node.js with ES Modules
|
||||||
|
- **Language**: TypeScript 5.9.3
|
||||||
|
- **Discord Framework**: discord.js v14.23.2
|
||||||
|
- **AI Integration**: @anthropic-ai/sdk v0.65.0
|
||||||
|
- **Web Server**: Fastify v5.6.1
|
||||||
|
- **Logging**: @nhcarrigan/logger v1.1.1
|
||||||
|
- **Analytics**: @nhcarrigan/discord-analytics v0.0.6
|
||||||
|
|
||||||
|
### 2.3. Installation
|
||||||
|
|
||||||
|
1. **Clone the repository**:
|
||||||
|
```bash
|
||||||
|
git clone <repository-url>
|
||||||
|
cd keiko
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Install dependencies**:
|
||||||
|
```bash
|
||||||
|
npm install
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Configure environment variables**:
|
||||||
|
|
||||||
|
Create a `prod.env` file in the root directory with the following variables:
|
||||||
|
|
||||||
|
```env
|
||||||
|
DISCORD_TOKEN=your_discord_bot_token
|
||||||
|
AI_TOKEN=your_anthropic_api_key
|
||||||
|
LOG_TOKEN=your_logging_token (optional)
|
||||||
|
```
|
||||||
|
|
||||||
|
Alternatively, you can set these as system environment variables.
|
||||||
|
|
||||||
|
### 2.4. Configuration
|
||||||
|
|
||||||
|
#### Discord Bot Setup
|
||||||
|
|
||||||
|
1. Create a Discord application at [Discord Developer Portal](https://discord.com/developers/applications)
|
||||||
|
2. Enable the following Gateway Intents:
|
||||||
|
- DirectMessages
|
||||||
|
- MessageContent
|
||||||
|
- GuildMessages
|
||||||
|
- Guilds
|
||||||
|
3. Generate a bot token and add it to your environment configuration
|
||||||
|
4. Invite the bot to your server with appropriate permissions
|
||||||
|
|
||||||
|
#### AI Configuration
|
||||||
|
|
||||||
|
The AI model configuration is set in the codebase (src/modules/makeAiRequest.ts:29):
|
||||||
|
- **Model**: claude-sonnet-4-5-20250929
|
||||||
|
- **Max Tokens**: 3000
|
||||||
|
- **Temperature**: 1
|
||||||
|
- **Web Search**: Enabled (max 5 uses per request)
|
||||||
|
|
||||||
|
#### Personality Customization
|
||||||
|
|
||||||
|
Edit `src/config/personality.ts` to customize Keiko's personality and behavior.
|
||||||
|
|
||||||
|
#### User Authorization
|
||||||
|
|
||||||
|
Update `src/utils/isNaomi.ts:15` to configure authorized user IDs and SKU IDs for subscription management.
|
||||||
|
|
||||||
|
### 2.5. Building the Application
|
||||||
|
|
||||||
|
Compile TypeScript to JavaScript:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run build
|
||||||
|
```
|
||||||
|
|
||||||
|
This will:
|
||||||
|
- Remove the existing `prod` directory
|
||||||
|
- Compile TypeScript files to the `prod` directory
|
||||||
|
|
||||||
|
### 2.6. Running the Application
|
||||||
|
|
||||||
|
#### With 1Password CLI:
|
||||||
|
```bash
|
||||||
|
npm start
|
||||||
|
```
|
||||||
|
|
||||||
|
This uses 1Password CLI to inject environment variables from `prod.env`.
|
||||||
|
|
||||||
|
#### Without 1Password CLI:
|
||||||
|
```bash
|
||||||
|
node prod/index.js
|
||||||
|
```
|
||||||
|
|
||||||
|
Ensure environment variables are set in your system before running.
|
||||||
|
|
||||||
|
### 2.7. Development
|
||||||
|
|
||||||
|
#### Linting
|
||||||
|
|
||||||
|
Run ESLint to check code quality:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run lint
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Testing
|
||||||
|
|
||||||
|
Currently, tests are not implemented:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm test
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.8. Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
keiko/
|
||||||
|
├── src/
|
||||||
|
│ ├── commands/ # Slash command definitions
|
||||||
|
│ │ ├── clear.ts # Clear history command
|
||||||
|
│ │ └── dm.ts # DM command
|
||||||
|
│ ├── config/ # Configuration files
|
||||||
|
│ │ └── personality.ts # AI personality definition
|
||||||
|
│ ├── events/ # Discord event handlers
|
||||||
|
│ │ ├── handleDmMessage.ts
|
||||||
|
│ │ ├── handleGuildMessage.ts
|
||||||
|
│ │ ├── handleThreadMessage.ts
|
||||||
|
│ │ └── onMessage.ts
|
||||||
|
│ ├── modules/ # Core business logic
|
||||||
|
│ │ ├── clear.ts # Clear command logic
|
||||||
|
│ │ ├── dm.ts # DM command logic
|
||||||
|
│ │ ├── makeAiRequest.ts # AI request handler
|
||||||
|
│ │ └── sendAiResponse.ts # Response chunking
|
||||||
|
│ ├── server/ # Web server
|
||||||
|
│ │ └── serve.ts # Fastify health check server
|
||||||
|
│ ├── utils/ # Utility functions
|
||||||
|
│ │ ├── ai.ts # Anthropic client
|
||||||
|
│ │ ├── calculateCost.ts
|
||||||
|
│ │ ├── isNaomi.ts # Authorization
|
||||||
|
│ │ ├── logger.ts
|
||||||
|
│ │ ├── replyToError.ts
|
||||||
|
│ │ └── sleep.ts
|
||||||
|
│ └── index.ts # Main entry point
|
||||||
|
├── prod/ # Compiled JavaScript (generated)
|
||||||
|
├── package.json
|
||||||
|
├── tsconfig.json
|
||||||
|
├── prod.env # Environment variables
|
||||||
|
└── LICENSE.md
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.9. Monitoring and Health Checks
|
||||||
|
|
||||||
|
Keiko runs a Fastify web server on port 3333 that provides:
|
||||||
|
- Health check endpoint at `GET /`
|
||||||
|
- Basic bot information page
|
||||||
|
- Uptime monitoring capability
|
||||||
|
|
||||||
|
### 2.10. Error Handling
|
||||||
|
|
||||||
|
The application includes:
|
||||||
|
- Global unhandled rejection handler
|
||||||
|
- Global uncaught exception handler
|
||||||
|
- User-facing error messages with support links
|
||||||
|
- Comprehensive logging via @nhcarrigan/logger
|
||||||
|
|
||||||
|
### 2.11. Analytics
|
||||||
|
|
||||||
|
Discord analytics are tracked for:
|
||||||
|
- Direct messages
|
||||||
|
- Guild messages
|
||||||
|
- Thread messages
|
||||||
|
- Token usage and API costs per interaction
|
||||||
|
|
||||||
|
### 2.12. Deployment
|
||||||
|
|
||||||
|
**Recommended Environment**:
|
||||||
|
- Linux server (tested on Arch Linux 6.17.5-arch1-1)
|
||||||
|
- Node.js runtime
|
||||||
|
- Process manager (e.g., PM2, systemd)
|
||||||
|
|
||||||
|
**Deployment Steps**:
|
||||||
|
|
||||||
|
1. Clone and configure the repository
|
||||||
|
2. Build the application
|
||||||
|
3. Set up environment variables
|
||||||
|
4. Run with a process manager for reliability
|
||||||
|
5. Configure monitoring for the health check endpoint (port 3333)
|
||||||
|
|
||||||
|
## 3. Legal Documentation
|
||||||
|
|
||||||
|
This section is for expansions to our legal policies specific to the Application.
|
||||||
|
|
||||||
|
### 3.1. License
|
||||||
|
|
||||||
|
This software is licensed under [Naomi's Public License](https://docs.nhcarrigan.com/#/license).
|
||||||
|
|
||||||
|
Copyright held by Naomi Carrigan.
|
||||||
|
|
||||||
|
### 3.2. Privacy and Data Handling
|
||||||
|
|
||||||
|
#### Data Collection
|
||||||
|
|
||||||
|
Keiko collects and processes the following data:
|
||||||
|
- Discord user IDs for authorization
|
||||||
|
- Message content for conversation context (last 20 messages)
|
||||||
|
- Discord entitlement information for subscription management
|
||||||
|
- Analytics data (message counts, token usage, API costs)
|
||||||
|
|
||||||
|
#### Data Retention
|
||||||
|
|
||||||
|
- **Conversation History**: Stored temporarily in memory during active sessions
|
||||||
|
- **Message Context**: Fetched on-demand from Discord's API (last 20 messages)
|
||||||
|
- **No Persistent Storage**: Keiko does not maintain a traditional database
|
||||||
|
|
||||||
|
#### Third-Party Services
|
||||||
|
|
||||||
|
Keiko integrates with:
|
||||||
|
- **Discord**: For bot functionality and message delivery
|
||||||
|
- **Anthropic**: For AI-powered responses (Claude AI)
|
||||||
|
- **Web Search Services**: For retrieving up-to-date information
|
||||||
|
|
||||||
|
User messages and conversation context are sent to these services for processing.
|
||||||
|
|
||||||
|
### 3.3. Terms of Use
|
||||||
|
|
||||||
|
#### Authorized Use
|
||||||
|
|
||||||
|
- Keiko is a private assistant primarily designed for Naomi's personal use
|
||||||
|
- Subscription-based access is available for authorized users
|
||||||
|
- Unauthorized access attempts will be rejected
|
||||||
|
|
||||||
|
#### Service Availability
|
||||||
|
|
||||||
|
- The Application is provided "as is" without warranties
|
||||||
|
- Service availability is not guaranteed
|
||||||
|
- Features and functionality may change without notice
|
||||||
|
|
||||||
|
#### Acceptable Use
|
||||||
|
|
||||||
|
Users must:
|
||||||
|
- Comply with Discord's Terms of Service
|
||||||
|
- Use the bot for lawful purposes only
|
||||||
|
- Respect the bot's intended functionality
|
||||||
|
- Not attempt to abuse, exploit, or reverse engineer the service
|
||||||
|
|
||||||
|
### 3.4. API Usage and Costs
|
||||||
|
|
||||||
|
- AI interactions incur costs via the Anthropic API
|
||||||
|
- Token usage and costs are displayed transparently to users
|
||||||
|
- Excessive usage may be rate-limited or restricted
|
||||||
|
|
||||||
|
## 4. Contributing Documentation
|
||||||
|
|
||||||
|
This section is for documentation related to contributing to the Application's codebase.
|
||||||
|
|
||||||
|
### 4.1. How to Contribute
|
||||||
|
|
||||||
|
We welcome contributions to Keiko! Here's how you can help:
|
||||||
|
|
||||||
|
1. **Fork the repository** on GitHub
|
||||||
|
2. **Create a feature branch** for your changes
|
||||||
|
3. **Make your changes** following our coding standards
|
||||||
|
4. **Test your changes** thoroughly
|
||||||
|
5. **Submit a Pull Request** with a clear description of your changes
|
||||||
|
|
||||||
|
### 4.2. Contributing Guidelines
|
||||||
|
|
||||||
|
Please review our comprehensive contributing guidelines:
|
||||||
|
|
||||||
|
[Contributing Guidelines](https://docs.nhcarrigan.com/#/contributing)
|
||||||
|
|
||||||
|
### 4.3. Code of Conduct
|
||||||
|
|
||||||
|
Before interacting with our community, please read our Code of Conduct:
|
||||||
|
|
||||||
|
[Code of Conduct](https://docs.nhcarrigan.com/#/coc)
|
||||||
|
|
||||||
|
### 4.4. Development Setup
|
||||||
|
|
||||||
|
1. **Clone the repository**:
|
||||||
|
```bash
|
||||||
|
git clone <repository-url>
|
||||||
|
cd keiko
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Install dependencies**:
|
||||||
|
```bash
|
||||||
|
npm install
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Set up environment variables** (see Technical Documentation section)
|
||||||
|
|
||||||
|
4. **Run the linter**:
|
||||||
|
```bash
|
||||||
|
npm run lint
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Build the project**:
|
||||||
|
```bash
|
||||||
|
npm run build
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.5. Coding Standards
|
||||||
|
|
||||||
|
- **Language**: TypeScript
|
||||||
|
- **Module System**: ES Modules
|
||||||
|
- **Linting**: ESLint with @nhcarrigan/eslint-config
|
||||||
|
- **Type Checking**: Strict TypeScript configuration
|
||||||
|
- **Code Style**: Follow existing patterns in the codebase
|
||||||
|
|
||||||
|
### 4.6. Pull Request Guidelines
|
||||||
|
|
||||||
|
When submitting a Pull Request:
|
||||||
|
|
||||||
|
- **Clear Description**: Explain what your changes do and why
|
||||||
|
- **Reference Issues**: Link to any related issues
|
||||||
|
- **Test Your Changes**: Ensure the bot builds and runs correctly
|
||||||
|
- **Follow Conventions**: Match the existing code style
|
||||||
|
- **Small, Focused Changes**: Keep PRs focused on a single feature or fix
|
||||||
|
|
||||||
|
### 4.7. Areas for Contribution
|
||||||
|
|
||||||
|
Potential areas where contributions are welcome:
|
||||||
|
|
||||||
|
- **Testing**: Implement unit and integration tests
|
||||||
|
- **Documentation**: Improve or expand documentation
|
||||||
|
- **Features**: Add new commands or capabilities
|
||||||
|
- **Bug Fixes**: Fix reported issues
|
||||||
|
- **Performance**: Optimize existing functionality
|
||||||
|
- **Error Handling**: Improve error messages and handling
|
||||||
|
|
||||||
|
### 4.8. Reporting Issues
|
||||||
|
|
||||||
|
If you find a bug or have a feature request:
|
||||||
|
|
||||||
|
1. **Check existing issues** to avoid duplicates
|
||||||
|
2. **Create a detailed issue** with:
|
||||||
|
- Clear title
|
||||||
|
- Description of the problem or feature
|
||||||
|
- Steps to reproduce (for bugs)
|
||||||
|
- Expected vs actual behavior
|
||||||
|
- Environment details (if applicable)
|
||||||
|
|
||||||
|
### 4.9. Getting Help
|
||||||
|
|
||||||
|
Need help contributing?
|
||||||
|
|
||||||
|
- Join our community: [chat.nhcarrigan.com](https://chat.nhcarrigan.com)
|
||||||
|
- Email: contact@nhcarrigan.com
|
||||||
|
- Check the documentation: [docs.nhcarrigan.com](https://docs.nhcarrigan.com)
|
||||||
|
|
||||||
|
### 4.10. Recognition
|
||||||
|
|
||||||
|
Contributors will be:
|
||||||
|
- Acknowledged in Pull Request comments
|
||||||
|
- Listed in project contributors (if applicable)
|
||||||
|
- Appreciated for their time and effort!
|
||||||
|
|
||||||
|
## Appendix
|
||||||
|
|
||||||
|
### A.1. Key File References
|
||||||
|
|
||||||
|
- **Main Entry Point**: src/index.ts:1
|
||||||
|
- **AI Request Handler**: src/modules/makeAiRequest.ts:29
|
||||||
|
- **Authorization Logic**: src/utils/isNaomi.ts:15
|
||||||
|
- **Personality Configuration**: src/config/personality.ts:11
|
||||||
|
- **Message Router**: src/events/onMessage.ts:1
|
||||||
|
|
||||||
|
### A.2. Important Constants
|
||||||
|
|
||||||
|
- **Bot ID**: 1425897287065800785
|
||||||
|
- **Naomi's User ID**: 465650873650118659
|
||||||
|
- **SKU ID**: 1425905043244060762
|
||||||
|
- **Health Check Port**: 3333
|
||||||
|
- **Max Context Messages**: 20
|
||||||
|
- **Max AI Tokens**: 3000
|
||||||
|
- **Message Chunk Size**: 2000 characters
|
||||||
|
- **Chunk Delay**: 2.5 seconds
|
||||||
|
|
||||||
|
### A.3. External Resources
|
||||||
|
|
||||||
|
- **Community**: https://chat.nhcarrigan.com
|
||||||
|
- **Documentation**: https://docs.nhcarrigan.com
|
||||||
|
- **License**: https://docs.nhcarrigan.com/#/license
|
||||||
|
- **Contributing**: https://docs.nhcarrigan.com/#/contributing
|
||||||
|
- **Code of Conduct**: https://docs.nhcarrigan.com/#/coc
|
||||||
|
- **Contact Email**: contact@nhcarrigan.com
|
||||||
@@ -0,0 +1,422 @@
|
|||||||
|
---
|
||||||
|
title: Meeting Minutes
|
||||||
|
---
|
||||||
|
|
||||||
|
Meeting Minutes (hereinafter the "Application") is a local, offline meeting recording, transcription, and summarization tool designed to protect user data privacy by processing all audio and text entirely on the user's machine without any external services or cloud processing.
|
||||||
|
|
||||||
|
## 1. User Documentation
|
||||||
|
|
||||||
|
This section is for those interacting with a live instance of the Application.
|
||||||
|
|
||||||
|
### Overview
|
||||||
|
|
||||||
|
The Application automates the process of capturing, transcribing, and summarizing meeting audio. It operates entirely offline to ensure complete data privacy and confidentiality of your meeting discussions.
|
||||||
|
|
||||||
|
### Features
|
||||||
|
|
||||||
|
- **Audio Recording**: Captures meeting audio from your system's default audio input/output mix
|
||||||
|
- **Speech-to-Text Transcription**: Converts recorded audio to text using WhisperX (OpenAI's Whisper model)
|
||||||
|
- **AI Summarization**: Generates concise meeting summaries with key decisions, action items, and discussion points using a local LLM
|
||||||
|
- **Speaker Diarization** (Optional): Identifies and labels different speakers in the meeting
|
||||||
|
- **Complete Privacy**: All processing happens locally on your machine with no data transmission to external services
|
||||||
|
|
||||||
|
### How to Use
|
||||||
|
|
||||||
|
1. **Configure the meeting**: Edit the configuration section in `main.py`:
|
||||||
|
- `WHISPER_MODEL`: Speech recognition model size (default: "small.en")
|
||||||
|
- `DURATION`: Maximum recording duration in seconds (default: 36000 = 10 hours)
|
||||||
|
|
||||||
|
2. **Start recording**: Uncomment the `record_audio()` call in the main execution block to enable live recording, or place a pre-recorded `.wav` file in the project directory
|
||||||
|
|
||||||
|
3. **Run the application**: Execute `uvx python@3.12 main.py`
|
||||||
|
|
||||||
|
4. **Review output**: The Application generates:
|
||||||
|
- A JSON transcript file containing the full transcription with timestamps
|
||||||
|
- A text summary file containing the meeting summary, key decisions, action items, and discussion points
|
||||||
|
|
||||||
|
### Output Format
|
||||||
|
|
||||||
|
**Transcript** (`meeting_audio_[timestamp].json`): Contains:
|
||||||
|
- Full transcript text
|
||||||
|
- Segment-level timing information
|
||||||
|
- Speaker labels (if diarization is enabled)
|
||||||
|
|
||||||
|
**Summary** (`summary_[timestamp].txt`): Contains:
|
||||||
|
- Concise narrative summary of the meeting
|
||||||
|
- Key decisions made
|
||||||
|
- Action items identified
|
||||||
|
- Notable discussion points
|
||||||
|
|
||||||
|
### Keyboard Controls
|
||||||
|
|
||||||
|
- Press `q` during recording to stop early (when using the record_audio function)
|
||||||
|
|
||||||
|
## 2. Technical Documentation
|
||||||
|
|
||||||
|
This section is for those interested in running their own instance of the Application.
|
||||||
|
|
||||||
|
### System Requirements
|
||||||
|
|
||||||
|
- **Operating System**: Linux (default), Windows, or macOS with appropriate audio input configuration
|
||||||
|
- **Python**: Version 3.12 or compatible
|
||||||
|
- **Audio System**: PulseAudio (Linux) or equivalent audio framework
|
||||||
|
- **Disk Space**: Sufficient space for audio files and model storage (~2-5 GB for models)
|
||||||
|
- **Memory**: Minimum 8GB RAM recommended for local LLM processing
|
||||||
|
|
||||||
|
### Dependencies
|
||||||
|
|
||||||
|
The Application requires the following external tools:
|
||||||
|
|
||||||
|
1. **FFmpeg**: For audio recording and processing
|
||||||
|
2. **WhisperX**: For speech-to-text transcription
|
||||||
|
3. **Ollama**: For local LLM-based summarization with llama3:8b model
|
||||||
|
4. **uv**: Python package and environment manager
|
||||||
|
|
||||||
|
### Installation
|
||||||
|
|
||||||
|
#### Step 1: Install Ollama
|
||||||
|
|
||||||
|
Download and install Ollama from https://ollama.ai, then pull the required model:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ollama pull llama3:8b
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Step 2: Install uv
|
||||||
|
|
||||||
|
Follow the installation instructions at https://docs.astral.sh/uv/
|
||||||
|
|
||||||
|
#### Step 3: Install WhisperX
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uvx python@3.12 whisperx
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Step 4: Install FFmpeg
|
||||||
|
|
||||||
|
**Linux (Debian/Ubuntu):**
|
||||||
|
```bash
|
||||||
|
sudo apt-get install ffmpeg
|
||||||
|
```
|
||||||
|
|
||||||
|
**macOS:**
|
||||||
|
```bash
|
||||||
|
brew install ffmpeg
|
||||||
|
```
|
||||||
|
|
||||||
|
**Windows:**
|
||||||
|
Download from https://ffmpeg.org/download.html
|
||||||
|
|
||||||
|
#### Step 5: Clone the Repository
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone [repository-url]
|
||||||
|
cd meeting-minutes
|
||||||
|
```
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
Edit `main.py` to configure the Application (lines 5-13):
|
||||||
|
|
||||||
|
```python
|
||||||
|
TIMESTAMP = datetime.datetime.now().strftime("%Y%m%d_%H%M%S") # Uncomment for dynamic timestamps
|
||||||
|
MEETING_FILE = f"meeting_audio_{TIMESTAMP}.wav"
|
||||||
|
WHISPER_MODEL = "small.en" # Options: tiny.en, base.en, small.en, medium.en, large
|
||||||
|
DURATION = 36000 # Recording duration in seconds
|
||||||
|
OUTPUT_TRANSCRIPT = f"meeting_audio_{TIMESTAMP}.json"
|
||||||
|
OUTPUT_SUMMARY = f"summary_{TIMESTAMP}.txt"
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Audio Input Configuration
|
||||||
|
|
||||||
|
By default, the Application uses PulseAudio (Linux). For other platforms:
|
||||||
|
|
||||||
|
**macOS:**
|
||||||
|
```python
|
||||||
|
"-f", "avfoundation", # Replace line 19
|
||||||
|
"-i", ":0", # Replace line 20 (use appropriate device index)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Windows:**
|
||||||
|
```python
|
||||||
|
"-f", "dshow", # Replace line 19
|
||||||
|
"-i", "audio=DEVICE_NAME", # Replace line 20 (use your audio device name)
|
||||||
|
```
|
||||||
|
|
||||||
|
List available devices:
|
||||||
|
```bash
|
||||||
|
ffmpeg -list_devices true -f [dshow|avfoundation] -i dummy
|
||||||
|
```
|
||||||
|
|
||||||
|
### Optional: Speaker Diarization
|
||||||
|
|
||||||
|
Speaker diarization allows the Application to identify and label different speakers in the meeting.
|
||||||
|
|
||||||
|
#### Setup
|
||||||
|
|
||||||
|
1. Create a Hugging Face account at https://huggingface.co
|
||||||
|
2. Accept terms for these models:
|
||||||
|
- https://huggingface.co/pyannote/segmentation-3.0
|
||||||
|
- https://huggingface.co/pyannote/speaker-diarization-3.1
|
||||||
|
3. Generate a Hugging Face token with read permissions
|
||||||
|
4. Add the token to the WhisperX command in `transcribe_audio()` function (main.py:27-37):
|
||||||
|
|
||||||
|
```python
|
||||||
|
subprocess.run([
|
||||||
|
"whisper",
|
||||||
|
MEETING_FILE,
|
||||||
|
"--device", "cpu",
|
||||||
|
"--language", "en",
|
||||||
|
"--model", WHISPER_MODEL,
|
||||||
|
"--output_format", "json",
|
||||||
|
"--output_dir", ".",
|
||||||
|
"--hf_token", "YOUR_HF_TOKEN_HERE", # Add this line
|
||||||
|
], check=True)
|
||||||
|
```
|
||||||
|
|
||||||
|
After the first successful run, you can remove the token from the code.
|
||||||
|
|
||||||
|
### Running the Application
|
||||||
|
|
||||||
|
#### With Live Recording
|
||||||
|
|
||||||
|
1. Uncomment line 84 in `main.py`:
|
||||||
|
```python
|
||||||
|
record_audio()
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Run the application:
|
||||||
|
```bash
|
||||||
|
uvx python@3.12 main.py
|
||||||
|
```
|
||||||
|
|
||||||
|
#### With Pre-recorded Audio
|
||||||
|
|
||||||
|
1. Ensure line 84 remains commented out
|
||||||
|
2. Place your `.wav` file in the project directory
|
||||||
|
3. Update the `TIMESTAMP` variable to match your file name
|
||||||
|
4. Run:
|
||||||
|
```bash
|
||||||
|
uvx python@3.12 main.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### Architecture
|
||||||
|
|
||||||
|
The Application follows a linear pipeline architecture:
|
||||||
|
|
||||||
|
```
|
||||||
|
Audio Input → Recording (FFmpeg) → Transcription (WhisperX) → Summarization (Ollama) → Cleanup
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Component Details
|
||||||
|
|
||||||
|
**Recording Module** (`record_audio()` - main.py:15-25):
|
||||||
|
- Uses FFmpeg to capture system audio
|
||||||
|
- Supports configurable duration and audio quality
|
||||||
|
- Outputs WAV format for compatibility
|
||||||
|
|
||||||
|
**Transcription Module** (`transcribe_audio()` - main.py:27-37):
|
||||||
|
- Leverages WhisperX for accurate speech recognition
|
||||||
|
- Supports multiple model sizes (trade-off between speed and accuracy)
|
||||||
|
- Generates timestamped JSON transcripts
|
||||||
|
- Optional speaker diarization support
|
||||||
|
|
||||||
|
**Summarization Module** (`summarize_with_local_model()` - main.py:39-74):
|
||||||
|
- Uses Ollama with llama3:8b for local AI processing
|
||||||
|
- Extracts key decisions, action items, and discussion points
|
||||||
|
- Generates natural language summaries
|
||||||
|
- No external API calls or data transmission
|
||||||
|
|
||||||
|
**Cleanup Module** (`cleanup()` - main.py:76-81):
|
||||||
|
- Removes temporary audio and transcript files
|
||||||
|
- Retains only the final summary
|
||||||
|
|
||||||
|
### Customization
|
||||||
|
|
||||||
|
#### Changing the Whisper Model
|
||||||
|
|
||||||
|
Available models with trade-offs:
|
||||||
|
|
||||||
|
- `tiny.en`: Fastest, least accurate (~1GB RAM)
|
||||||
|
- `base.en`: Fast, moderate accuracy (~1GB RAM)
|
||||||
|
- `small.en`: Balanced (default) (~2GB RAM)
|
||||||
|
- `medium.en`: Slower, more accurate (~5GB RAM)
|
||||||
|
- `large`: Slowest, most accurate (~10GB RAM)
|
||||||
|
|
||||||
|
Change the model on line 10 of `main.py`.
|
||||||
|
|
||||||
|
#### Changing the LLM Model
|
||||||
|
|
||||||
|
To use a different Ollama model:
|
||||||
|
|
||||||
|
1. Pull the desired model: `ollama pull [model-name]`
|
||||||
|
2. Update line 63 in `main.py`:
|
||||||
|
```python
|
||||||
|
["ollama", "run", "[model-name]"],
|
||||||
|
```
|
||||||
|
|
||||||
|
Popular alternatives:
|
||||||
|
- `llama3.1:8b`: Newer version of llama3
|
||||||
|
- `mistral`: Smaller, faster model
|
||||||
|
- `mixtral`: More powerful model (requires more RAM)
|
||||||
|
|
||||||
|
#### Customizing the Summary Prompt
|
||||||
|
|
||||||
|
Modify the prompt in `summarize_with_local_model()` (lines 49-60) to adjust summary style and content.
|
||||||
|
|
||||||
|
### Troubleshooting
|
||||||
|
|
||||||
|
**Issue**: "ffmpeg: command not found"
|
||||||
|
**Solution**: Install FFmpeg for your operating system
|
||||||
|
|
||||||
|
**Issue**: "ollama: command not found"
|
||||||
|
**Solution**: Install Ollama and ensure it's in your system PATH
|
||||||
|
|
||||||
|
**Issue**: WhisperX fails to transcribe
|
||||||
|
**Solution**: Ensure the audio file is valid WAV format and the Whisper model is downloaded
|
||||||
|
|
||||||
|
**Issue**: Summarization takes too long
|
||||||
|
**Solution**: Use a smaller/faster Ollama model or switch to GPU processing
|
||||||
|
|
||||||
|
**Issue**: No audio captured during recording
|
||||||
|
**Solution**: Check your system's default audio input/output settings and adjust FFmpeg device parameters
|
||||||
|
|
||||||
|
### Performance Optimization
|
||||||
|
|
||||||
|
- **GPU Acceleration**: Change `--device` from "cpu" to "cuda" in the transcribe_audio() function if you have a compatible NVIDIA GPU
|
||||||
|
- **Smaller Models**: Use smaller Whisper models for faster processing
|
||||||
|
- **Parallel Processing**: For multiple meetings, run separate instances in different directories
|
||||||
|
|
||||||
|
## 3. Legal Documentation
|
||||||
|
|
||||||
|
This section is for expansions to our legal policies specific to the Application.
|
||||||
|
|
||||||
|
### Privacy Considerations
|
||||||
|
|
||||||
|
The Application is designed with privacy as a core principle:
|
||||||
|
|
||||||
|
- **No External Data Transmission**: All audio recording, transcription, and summarization occurs locally on the user's machine
|
||||||
|
- **No Cloud Services**: No APIs, cloud services, or external servers are contacted during operation
|
||||||
|
- **User Control**: Users have complete control over all recorded data, transcripts, and summaries
|
||||||
|
- **Data Retention**: The Application automatically cleans up temporary files, retaining only the user-requested summary
|
||||||
|
|
||||||
|
### User Responsibilities
|
||||||
|
|
||||||
|
Users of the Application must:
|
||||||
|
|
||||||
|
1. **Obtain Consent**: Ensure all meeting participants have consented to recording before using the Application
|
||||||
|
2. **Comply with Laws**: Follow all applicable local, state, and federal laws regarding audio recording and consent
|
||||||
|
3. **Protect Data**: Securely store and handle meeting summaries and transcripts containing sensitive or confidential information
|
||||||
|
4. **Respect Confidentiality**: Honor any confidentiality agreements or expectations of meeting participants
|
||||||
|
|
||||||
|
### Compliance Recommendations
|
||||||
|
|
||||||
|
- Check local "one-party" vs "all-party" consent laws before recording
|
||||||
|
- Announce at the beginning of meetings that recording is in progress
|
||||||
|
- Store output files in encrypted storage if they contain sensitive information
|
||||||
|
- Implement appropriate data retention policies for meeting summaries
|
||||||
|
|
||||||
|
### License
|
||||||
|
|
||||||
|
This software is licensed under the [global software license](https://docs.nhcarrigan.com/#/license).
|
||||||
|
|
||||||
|
Copyright held by Naomi Carrigan.
|
||||||
|
|
||||||
|
For additional legal documentation, refer to:
|
||||||
|
- [Terms of Service](TERMS.md)
|
||||||
|
- [Privacy Policy](PRIVACY.md)
|
||||||
|
- [Security Policy](SECURITY.md)
|
||||||
|
|
||||||
|
## 4. Contributing Documentation
|
||||||
|
|
||||||
|
This section is for documentation related to contributing to the Application's codebase.
|
||||||
|
|
||||||
|
### How to Contribute
|
||||||
|
|
||||||
|
We welcome contributions to the Meeting Minutes Application! Please review our [contributing guidelines](https://docs.nhcarrigan.com/#/contributing) before getting started.
|
||||||
|
|
||||||
|
### Contribution Process
|
||||||
|
|
||||||
|
1. **Fork the Repository**: Create your own fork of the project
|
||||||
|
2. **Create a Branch**: Make a feature branch for your changes
|
||||||
|
3. **Make Changes**: Implement your feature or bug fix
|
||||||
|
4. **Test Thoroughly**: Ensure your changes work as expected
|
||||||
|
5. **Submit a Pull Request**: Describe your changes and submit for review
|
||||||
|
|
||||||
|
### Code of Conduct
|
||||||
|
|
||||||
|
Before interacting with our community, please read our [Code of Conduct](CODE_OF_CONDUCT.md). We are committed to providing a welcoming and inclusive environment for all contributors.
|
||||||
|
|
||||||
|
### Development Setup
|
||||||
|
|
||||||
|
To set up a development environment:
|
||||||
|
|
||||||
|
1. Follow the installation instructions in Section 2
|
||||||
|
2. Create a test meeting audio file or use the sample files
|
||||||
|
3. Make changes to `main.py`
|
||||||
|
4. Test your changes with various audio inputs and configurations
|
||||||
|
|
||||||
|
### Contribution Ideas
|
||||||
|
|
||||||
|
We welcome contributions in the following areas:
|
||||||
|
|
||||||
|
**Features**:
|
||||||
|
- GUI interface for easier configuration and operation
|
||||||
|
- Support for additional audio formats (MP3, M4A, etc.)
|
||||||
|
- Real-time transcription display during recording
|
||||||
|
- Integration with calendar systems for automatic meeting detection
|
||||||
|
- Multi-language support beyond English
|
||||||
|
- Custom vocabulary support for domain-specific terminology
|
||||||
|
- Export to additional formats (PDF, Markdown, etc.)
|
||||||
|
- Meeting analytics and insights
|
||||||
|
|
||||||
|
**Improvements**:
|
||||||
|
- Better error handling and user feedback
|
||||||
|
- Configuration file support (YAML/JSON) instead of hardcoded values
|
||||||
|
- Automated testing suite
|
||||||
|
- Performance optimizations
|
||||||
|
- Documentation improvements
|
||||||
|
- Cross-platform compatibility testing
|
||||||
|
|
||||||
|
**Bug Fixes**:
|
||||||
|
- Any identified issues in audio recording, transcription, or summarization
|
||||||
|
- Platform-specific bugs
|
||||||
|
- Edge cases in file handling
|
||||||
|
|
||||||
|
### Testing Guidelines
|
||||||
|
|
||||||
|
When contributing, please test:
|
||||||
|
|
||||||
|
1. **Audio Recording**: Verify recording works with various audio sources
|
||||||
|
2. **Transcription**: Test with different accents, speaking speeds, and audio quality
|
||||||
|
3. **Summarization**: Ensure summaries are coherent and capture key information
|
||||||
|
4. **Cross-platform**: Test on different operating systems if possible
|
||||||
|
5. **Error Handling**: Test edge cases and error conditions
|
||||||
|
|
||||||
|
### Code Style
|
||||||
|
|
||||||
|
- Follow PEP 8 Python style guidelines
|
||||||
|
- Use meaningful variable and function names
|
||||||
|
- Add comments for complex logic
|
||||||
|
- Keep functions focused on single responsibilities
|
||||||
|
|
||||||
|
### Reporting Issues
|
||||||
|
|
||||||
|
If you encounter bugs or have feature requests:
|
||||||
|
|
||||||
|
1. Check existing GitHub issues to avoid duplicates
|
||||||
|
2. Open a new issue with a clear title and description
|
||||||
|
3. Include steps to reproduce (for bugs)
|
||||||
|
4. Provide system information (OS, Python version, etc.)
|
||||||
|
|
||||||
|
### Getting Help
|
||||||
|
|
||||||
|
If you need assistance:
|
||||||
|
|
||||||
|
- Open a GitHub issue with the "question" label
|
||||||
|
- Join our [Chat Server](http://chat.nhcarrigan.com)
|
||||||
|
- Email us at `contact@nhcarrigan.com`
|
||||||
|
|
||||||
|
### Recognition
|
||||||
|
|
||||||
|
All contributors will be acknowledged in the project. Thank you for helping make Meeting Minutes better!
|
||||||
@@ -0,0 +1,781 @@
|
|||||||
|
---
|
||||||
|
title: Nomena
|
||||||
|
---
|
||||||
|
|
||||||
|
Nomena (hereinafter the "Application") is a Discord bot that generates creative project ideas by providing AI-powered project names with explanations and custom anime mascot images. The Application leverages Claude AI for intelligent text generation and Google Gemini for custom avatar creation.
|
||||||
|
|
||||||
|
## 1. User Documentation
|
||||||
|
|
||||||
|
This section is for those interacting with a live instance of the Application.
|
||||||
|
|
||||||
|
### 1.1 Overview
|
||||||
|
|
||||||
|
Nomena is a Discord bot designed to help generate creative project ideas. When mentioned in Discord, the bot generates fitting project names based on your description and creates a unique anime mascot image to serve as the project's avatar.
|
||||||
|
|
||||||
|
### 1.2 Access Requirements
|
||||||
|
|
||||||
|
Currently, the Application is configured for single-user access. Users must be authorized by the bot owner to interact with Nomena. If you attempt to use the bot without authorization, you will receive a message indicating that the bot can only generate project ideas for authorized users.
|
||||||
|
|
||||||
|
### 1.3 How to Use
|
||||||
|
|
||||||
|
#### Basic Usage
|
||||||
|
|
||||||
|
To generate a project idea, mention the bot in a Discord channel followed by your project description:
|
||||||
|
|
||||||
|
```
|
||||||
|
@Nomena <your project description>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example:**
|
||||||
|
```
|
||||||
|
@Nomena A web application for tracking daily water intake and sending reminders
|
||||||
|
```
|
||||||
|
|
||||||
|
#### What You'll Receive
|
||||||
|
|
||||||
|
After mentioning the bot, you will receive a response containing:
|
||||||
|
|
||||||
|
1. **Project Names**: 1-5 fitting project names generated based on your description
|
||||||
|
2. **Name Explanations**: Reasoning for why each name was chosen
|
||||||
|
3. **Project Description**: Your original input, echoed back for reference
|
||||||
|
4. **Mascot Image**: A full-body anime character image (3:4 aspect ratio, white background) attached as `avatar.png`
|
||||||
|
|
||||||
|
The bot will display a typing indicator while processing your request, which typically takes a few seconds.
|
||||||
|
|
||||||
|
#### Unique Names
|
||||||
|
|
||||||
|
The Application automatically checks against a database of existing project names to ensure that generated names are unique and don't conflict with previously created projects.
|
||||||
|
|
||||||
|
#### Image Generation Limitations
|
||||||
|
|
||||||
|
In rare cases, image generation may fail due to API limitations or content restrictions. When this occurs, you will still receive the project names and descriptions, with a message indicating that the image could not be generated.
|
||||||
|
|
||||||
|
### 1.4 Best Practices
|
||||||
|
|
||||||
|
- **Be Descriptive**: Provide clear, detailed descriptions of your project idea for better name suggestions
|
||||||
|
- **Include Context**: Mention the project's purpose, target audience, or key features
|
||||||
|
- **One Request at a Time**: Wait for the bot to respond before sending additional requests
|
||||||
|
- **Respect Rate Limits**: Avoid excessive requests in rapid succession
|
||||||
|
|
||||||
|
### 1.5 Example Interactions
|
||||||
|
|
||||||
|
**Request:**
|
||||||
|
```
|
||||||
|
@Nomena A mobile app for learning Japanese vocabulary through spaced repetition
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```
|
||||||
|
Project Name: KanjiFlow, NihongoLoop, or MemoryMichi
|
||||||
|
|
||||||
|
Explanations:
|
||||||
|
- KanjiFlow: Combines "Kanji" (Japanese characters) with "Flow" (smooth progression)
|
||||||
|
- NihongoLoop: Uses "Nihongo" (Japanese language) with "Loop" (repetition)
|
||||||
|
- MemoryMichi: Blends "Memory" with "Michi" (path in Japanese)
|
||||||
|
|
||||||
|
Project Description: A mobile app for learning Japanese vocabulary through spaced repetition
|
||||||
|
|
||||||
|
[Attached: avatar.png - anime mascot image]
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2. Technical Documentation
|
||||||
|
|
||||||
|
This section is for those interested in running their own instance of the Application.
|
||||||
|
|
||||||
|
### 2.1 System Requirements
|
||||||
|
|
||||||
|
- **Node.js**: Version 18.x or higher
|
||||||
|
- **Package Manager**: pnpm 10.19.0 (recommended) or npm
|
||||||
|
- **Operating System**: Linux, macOS, or Windows
|
||||||
|
- **API Keys**: Valid API keys for required services (see Environment Variables)
|
||||||
|
|
||||||
|
### 2.2 Technology Stack
|
||||||
|
|
||||||
|
The Application is built with the following technologies:
|
||||||
|
|
||||||
|
**Core Technologies:**
|
||||||
|
- **TypeScript** (v5.9.3): Type-safe JavaScript development
|
||||||
|
- **Discord.js** (v14.24.2): Discord bot framework and API wrapper
|
||||||
|
- **Anthropic SDK** (@anthropic-ai/sdk v0.68.0): Claude AI integration for text generation
|
||||||
|
- **Google GenAI** (@google/genai v1.28.0): Gemini AI integration for image generation
|
||||||
|
|
||||||
|
**Development Tools:**
|
||||||
|
- **ESLint** (v9.38.0): Code quality and linting
|
||||||
|
- **Custom Configurations**: @nhcarrigan/eslint-config, @nhcarrigan/typescript-config
|
||||||
|
|
||||||
|
**Utilities:**
|
||||||
|
- **@nhcarrigan/logger** (v1.1.1): Custom logging utility with Discord Analytics integration
|
||||||
|
- **@nhcarrigan/discord-analytics** (v0.0.6): Analytics tracking
|
||||||
|
|
||||||
|
### 2.3 Installation
|
||||||
|
|
||||||
|
1. **Clone the repository:**
|
||||||
|
```bash
|
||||||
|
git clone <repository-url>
|
||||||
|
cd nomena
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Install dependencies:**
|
||||||
|
```bash
|
||||||
|
pnpm install
|
||||||
|
```
|
||||||
|
|
||||||
|
Or using npm:
|
||||||
|
```bash
|
||||||
|
npm install
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Configure environment variables** (see section 2.4)
|
||||||
|
|
||||||
|
4. **Build the application:**
|
||||||
|
```bash
|
||||||
|
pnpm run build
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.4 Environment Variables
|
||||||
|
|
||||||
|
Create a `prod.env` file in the project root with the following variables:
|
||||||
|
|
||||||
|
```env
|
||||||
|
# Required: Anthropic API key for Claude AI
|
||||||
|
ANTHROPIC_API_KEY=your_anthropic_api_key_here
|
||||||
|
|
||||||
|
# Required: Google Gemini API key for image generation
|
||||||
|
GEMINI_API_KEY=your_gemini_api_key_here
|
||||||
|
|
||||||
|
# Required: Discord bot token
|
||||||
|
DISCORD_TOKEN=your_discord_bot_token_here
|
||||||
|
|
||||||
|
# Required: Logger token for Discord Analytics
|
||||||
|
LOG_TOKEN=your_log_token_here
|
||||||
|
```
|
||||||
|
|
||||||
|
**API Key Acquisition:**
|
||||||
|
- **Anthropic API Key**: Sign up at https://console.anthropic.com/
|
||||||
|
- **Gemini API Key**: Obtain from Google AI Studio at https://makersuite.google.com/
|
||||||
|
- **Discord Bot Token**: Create a bot at https://discord.com/developers/applications
|
||||||
|
- **Log Token**: Refer to @nhcarrigan/logger documentation
|
||||||
|
|
||||||
|
**All environment variables are required.** The Application will throw an error and refuse to start if any are missing.
|
||||||
|
|
||||||
|
### 2.5 Discord Bot Configuration
|
||||||
|
|
||||||
|
When creating your Discord bot at https://discord.com/developers/applications, ensure the following:
|
||||||
|
|
||||||
|
**Required Intents:**
|
||||||
|
- `Guilds`: Access to guild information
|
||||||
|
- `Guild Messages`: Receive messages in guilds
|
||||||
|
- `Message Content`: Read full message content (privileged intent)
|
||||||
|
|
||||||
|
**Bot Permissions:**
|
||||||
|
- Read Messages/View Channels
|
||||||
|
- Send Messages
|
||||||
|
- Attach Files
|
||||||
|
- Read Message History
|
||||||
|
|
||||||
|
**Note:** The Message Content intent is a privileged intent and must be enabled in the Discord Developer Portal.
|
||||||
|
|
||||||
|
### 2.6 Customization
|
||||||
|
|
||||||
|
#### User Access Control
|
||||||
|
|
||||||
|
By default, the bot is configured for single-user access. To modify access control, edit `src/index.ts:44`:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
if (message.author.id !== "YOUR_DISCORD_USER_ID") {
|
||||||
|
await message.reply("Sorry, I can only generate project ideas for authorized users.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
You can:
|
||||||
|
- Replace the user ID with your own Discord user ID
|
||||||
|
- Remove the check entirely to allow all users
|
||||||
|
- Implement role-based access control
|
||||||
|
- Add a whitelist of authorized user IDs
|
||||||
|
|
||||||
|
#### Bot Mention ID
|
||||||
|
|
||||||
|
Update the bot mention ID in `src/index.ts:37` to match your bot's ID:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
if (!message.mentions.has("YOUR_BOT_ID", {
|
||||||
|
ignoreEveryone: true,
|
||||||
|
ignoreRepliedUser: true,
|
||||||
|
ignoreRoles: true,
|
||||||
|
})) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Find your bot's ID in the Discord Developer Portal.
|
||||||
|
|
||||||
|
#### AI Model Configuration
|
||||||
|
|
||||||
|
To modify the AI models used:
|
||||||
|
|
||||||
|
**Text Generation (src/classes/ai.ts:78):**
|
||||||
|
```typescript
|
||||||
|
model: "claude-sonnet-4-5-20250929" // Change to any available Claude model
|
||||||
|
```
|
||||||
|
|
||||||
|
**Image Generation (src/classes/ai.ts:100):**
|
||||||
|
```typescript
|
||||||
|
model: "gemini-2.5-flash-image" // Change to any available Gemini model
|
||||||
|
```
|
||||||
|
|
||||||
|
**Token Limits (src/classes/ai.ts:71):**
|
||||||
|
```typescript
|
||||||
|
max_tokens: 1000 // Adjust based on your needs
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Project Name Uniqueness Check
|
||||||
|
|
||||||
|
The bot fetches existing project names from `https://data.nhcarrigan.com/projects.json`. To use your own data source, modify `src/classes/ai.ts:42-43`:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const projectRequest = await fetch("YOUR_DATA_SOURCE_URL");
|
||||||
|
```
|
||||||
|
|
||||||
|
Ensure your data source returns an array of objects with a `name` property:
|
||||||
|
```json
|
||||||
|
[
|
||||||
|
{ "name": "ProjectOne" },
|
||||||
|
{ "name": "ProjectTwo" }
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
To disable the uniqueness check entirely, comment out or remove the fetch logic and modify the system prompt accordingly.
|
||||||
|
|
||||||
|
#### Image Generation Prompts
|
||||||
|
|
||||||
|
Customize the mascot generation prompt in `src/classes/ai.ts:95-97`:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
systemInstruction: `Your custom prompt here`
|
||||||
|
```
|
||||||
|
|
||||||
|
Adjust the aspect ratio in `src/classes/ai.ts:95`:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
imageConfig: { aspectRatio: "3:4" } // Options: "1:1", "16:9", "9:16", etc.
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.7 Running the Application
|
||||||
|
|
||||||
|
**Development Mode:**
|
||||||
|
```bash
|
||||||
|
pnpm run build && node prod/index.js
|
||||||
|
```
|
||||||
|
|
||||||
|
**Production Mode (with 1Password integration):**
|
||||||
|
```bash
|
||||||
|
pnpm start
|
||||||
|
```
|
||||||
|
|
||||||
|
This runs: `op run --env-file=prod.env -- node prod/index.js`
|
||||||
|
|
||||||
|
**Without 1Password:**
|
||||||
|
|
||||||
|
If you're not using 1Password for secret management, modify the `start` script in `package.json`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"start": "node prod/index.js"
|
||||||
|
```
|
||||||
|
|
||||||
|
Then ensure your environment variables are loaded through your preferred method (e.g., dotenv, system environment variables).
|
||||||
|
|
||||||
|
### 2.8 Development Workflow
|
||||||
|
|
||||||
|
**Code Linting:**
|
||||||
|
```bash
|
||||||
|
pnpm run lint
|
||||||
|
```
|
||||||
|
|
||||||
|
Note: The lint script uses `--max-warnings 0`, meaning any warnings will cause the command to fail.
|
||||||
|
|
||||||
|
**Build Process:**
|
||||||
|
```bash
|
||||||
|
pnpm run build
|
||||||
|
```
|
||||||
|
|
||||||
|
This compiles TypeScript files from `src/` to `prod/`.
|
||||||
|
|
||||||
|
**Testing:**
|
||||||
|
```bash
|
||||||
|
pnpm test
|
||||||
|
```
|
||||||
|
|
||||||
|
Note: Currently no tests are implemented. This command returns successfully with exit code 0.
|
||||||
|
|
||||||
|
### 2.9 Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
nomena/
|
||||||
|
├── src/
|
||||||
|
│ ├── index.ts # Main entry point, Discord bot initialization
|
||||||
|
│ └── classes/
|
||||||
|
│ └── ai.ts # AI utility class for text and image generation
|
||||||
|
├── prod/ # Compiled JavaScript output (gitignored)
|
||||||
|
├── node_modules/ # Dependencies
|
||||||
|
├── prod.env # Environment variables (gitignored)
|
||||||
|
├── package.json # Project metadata and dependencies
|
||||||
|
├── tsconfig.json # TypeScript configuration
|
||||||
|
├── eslint.config.js # ESLint configuration
|
||||||
|
├── .gitignore # Git ignore rules
|
||||||
|
├── LICENSE.md # License information
|
||||||
|
├── CONTRIBUTING.md # Contributing guidelines
|
||||||
|
├── CODE_OF_CONDUCT.md # Code of conduct
|
||||||
|
├── PRIVACY.md # Privacy policy
|
||||||
|
├── SECURITY.md # Security policy
|
||||||
|
└── TERMS.md # Terms of service
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.10 Logging and Monitoring
|
||||||
|
|
||||||
|
The Application uses @nhcarrigan/logger for logging with Discord Analytics integration. Logs include:
|
||||||
|
|
||||||
|
- Bot startup events (debug level)
|
||||||
|
- Error tracking (when enabled in the logger)
|
||||||
|
- Custom analytics events (when configured)
|
||||||
|
|
||||||
|
To modify logging behavior, update the logger initialization in `src/index.ts:21`:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const logger = new Logger("Nomena", process.env.LOG_TOKEN);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.11 Troubleshooting
|
||||||
|
|
||||||
|
**Bot doesn't respond:**
|
||||||
|
- Verify the bot has the Message Content intent enabled
|
||||||
|
- Check that the bot mention ID in the code matches your bot's ID
|
||||||
|
- Ensure the bot has permission to read and send messages in the channel
|
||||||
|
|
||||||
|
**Environment variable errors:**
|
||||||
|
- Verify all four required environment variables are set
|
||||||
|
- Check for typos in variable names
|
||||||
|
- Ensure the `prod.env` file is in the project root
|
||||||
|
|
||||||
|
**Image generation failures:**
|
||||||
|
- Verify your Gemini API key is valid and has sufficient quota
|
||||||
|
- Check if the prompt violates content policies
|
||||||
|
- Review Google AI Studio logs for detailed error messages
|
||||||
|
|
||||||
|
**TypeScript compilation errors:**
|
||||||
|
- Ensure you're using Node.js 18.x or higher
|
||||||
|
- Delete `node_modules` and `prod` directories, then reinstall: `pnpm install && pnpm run build`
|
||||||
|
|
||||||
|
**Permission errors:**
|
||||||
|
- Verify bot permissions in Discord server settings
|
||||||
|
- Ensure the bot role has sufficient privileges
|
||||||
|
|
||||||
|
### 2.12 Production Deployment
|
||||||
|
|
||||||
|
**Recommended Setup:**
|
||||||
|
- Use a process manager (PM2, systemd, Docker)
|
||||||
|
- Implement log rotation
|
||||||
|
- Set up monitoring and alerting
|
||||||
|
- Use environment-specific configuration files
|
||||||
|
- Implement graceful shutdown handling
|
||||||
|
- Consider using a reverse proxy if exposing any HTTP endpoints
|
||||||
|
|
||||||
|
**Example PM2 Configuration:**
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"apps": [{
|
||||||
|
"name": "nomena",
|
||||||
|
"script": "prod/index.js",
|
||||||
|
"env_file": "prod.env",
|
||||||
|
"instances": 1,
|
||||||
|
"autorestart": true,
|
||||||
|
"watch": false,
|
||||||
|
"max_memory_restart": "1G"
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Docker Deployment:**
|
||||||
|
|
||||||
|
While no Dockerfile is provided, a basic setup would include:
|
||||||
|
```dockerfile
|
||||||
|
FROM node:18-alpine
|
||||||
|
WORKDIR /app
|
||||||
|
COPY package.json pnpm-lock.yaml ./
|
||||||
|
RUN npm install -g pnpm && pnpm install --frozen-lockfile
|
||||||
|
COPY . .
|
||||||
|
RUN pnpm run build
|
||||||
|
CMD ["node", "prod/index.js"]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.13 API Rate Limits and Costs
|
||||||
|
|
||||||
|
**Anthropic (Claude AI):**
|
||||||
|
- Free tier: Limited requests per month
|
||||||
|
- Paid tier: Pay-per-token pricing
|
||||||
|
- Token usage: ~100-500 tokens per request (depending on response length)
|
||||||
|
- Rate limits: Vary by tier
|
||||||
|
|
||||||
|
**Google Gemini:**
|
||||||
|
- Free tier: Limited image generations per day
|
||||||
|
- Paid tier: Pay-per-generation pricing
|
||||||
|
- Rate limits: Vary by model and tier
|
||||||
|
|
||||||
|
**Discord:**
|
||||||
|
- 50 requests per second per bot
|
||||||
|
- Additional limits on message size and file attachments
|
||||||
|
|
||||||
|
Monitor your usage to avoid unexpected costs or service interruptions.
|
||||||
|
|
||||||
|
## 3. Legal Documentation
|
||||||
|
|
||||||
|
This section is for expansions to our legal policies specific to the Application.
|
||||||
|
|
||||||
|
### 3.1 License
|
||||||
|
|
||||||
|
This Application is licensed under Naomi's Public License. The full license text is available at:
|
||||||
|
https://docs.nhcarrigan.com/#/license
|
||||||
|
|
||||||
|
Copyright is held by Naomi Carrigan.
|
||||||
|
|
||||||
|
### 3.2 Privacy Policy
|
||||||
|
|
||||||
|
The Application's privacy practices are governed by the global privacy policy available at:
|
||||||
|
https://docs.nhcarrigan.com/#/privacy
|
||||||
|
|
||||||
|
**Application-Specific Privacy Considerations:**
|
||||||
|
|
||||||
|
- **User Messages**: Discord messages mentioning the bot are processed to extract project descriptions
|
||||||
|
- **API Data**: Project descriptions are sent to third-party APIs (Anthropic and Google) for processing
|
||||||
|
- **Generated Content**: Project names and images are generated and stored temporarily in Discord
|
||||||
|
- **Logging**: Basic debug information is logged via Discord Analytics
|
||||||
|
- **User Identification**: Discord user IDs are checked for access control purposes
|
||||||
|
|
||||||
|
**Data Retention:**
|
||||||
|
- Messages are not stored beyond the request lifecycle
|
||||||
|
- Generated images are stored on Discord's servers as message attachments
|
||||||
|
- Logs may retain basic event information per the logger configuration
|
||||||
|
|
||||||
|
**Third-Party Services:**
|
||||||
|
- Anthropic (Claude AI): Processes project descriptions to generate names
|
||||||
|
- Google (Gemini AI): Processes project descriptions to generate images
|
||||||
|
- Discord Analytics: Receives basic logging information
|
||||||
|
|
||||||
|
Users should review the privacy policies of these third-party services.
|
||||||
|
|
||||||
|
### 3.3 Security Policy
|
||||||
|
|
||||||
|
The Application's security practices follow the global security policy available at:
|
||||||
|
https://docs.nhcarrigan.com/#/security
|
||||||
|
|
||||||
|
**Application-Specific Security Measures:**
|
||||||
|
|
||||||
|
- **Access Control**: User ID verification limits bot usage to authorized individuals
|
||||||
|
- **Environment Variables**: Sensitive API keys are stored in environment variables, not in code
|
||||||
|
- **Input Validation**: User input is sanitized before being sent to AI APIs
|
||||||
|
- **Error Handling**: Errors are caught and handled gracefully without exposing sensitive information
|
||||||
|
|
||||||
|
**Security Considerations for Self-Hosted Instances:**
|
||||||
|
|
||||||
|
- Keep dependencies updated to patch security vulnerabilities
|
||||||
|
- Protect your `prod.env` file and never commit it to version control
|
||||||
|
- Use strong, unique API keys
|
||||||
|
- Monitor API usage for unusual activity
|
||||||
|
- Implement rate limiting if exposing the bot publicly
|
||||||
|
- Review Discord permissions to follow the principle of least privilege
|
||||||
|
|
||||||
|
**Reporting Security Vulnerabilities:**
|
||||||
|
|
||||||
|
If you discover a security vulnerability, please report it following the guidelines at:
|
||||||
|
https://docs.nhcarrigan.com/#/security
|
||||||
|
|
||||||
|
### 3.4 Terms of Service
|
||||||
|
|
||||||
|
General terms of service can be found in the `TERMS.md` file in the repository.
|
||||||
|
|
||||||
|
**Application-Specific Terms:**
|
||||||
|
|
||||||
|
- The Application generates content using AI, which may occasionally produce unexpected or inappropriate results
|
||||||
|
- Generated project names are suggestions only and may not be suitable for all use cases
|
||||||
|
- Users are responsible for verifying that generated names are available for use (trademark searches, domain availability, etc.)
|
||||||
|
- The Application makes best efforts to generate unique names but does not guarantee uniqueness outside of the internal project database
|
||||||
|
- Generated images are created by AI and may not always match expectations or requirements
|
||||||
|
|
||||||
|
### 3.5 Content Policy
|
||||||
|
|
||||||
|
**Generated Content:**
|
||||||
|
- AI-generated names and images are provided "as-is"
|
||||||
|
- The Application attempts to filter inappropriate content through AI safety measures
|
||||||
|
- Users should review generated content before using it in production
|
||||||
|
|
||||||
|
**Image Generation:**
|
||||||
|
- Images are generated in an anime art style
|
||||||
|
- The system prompt instructs the AI to create appropriate content
|
||||||
|
- Image generation may fail if content policies are violated
|
||||||
|
|
||||||
|
### 3.6 Code of Conduct
|
||||||
|
|
||||||
|
Contributors and users are expected to follow the code of conduct available at:
|
||||||
|
https://docs.nhcarrigan.com/#/coc
|
||||||
|
|
||||||
|
## 4. Contributing Documentation
|
||||||
|
|
||||||
|
This section is for documentation related to contributing to the Application's codebase.
|
||||||
|
|
||||||
|
### 4.1 General Contributing Guidelines
|
||||||
|
|
||||||
|
General contributing guidelines can be found at:
|
||||||
|
https://docs.nhcarrigan.com/#/contributing
|
||||||
|
|
||||||
|
Please review these guidelines before contributing to the Application.
|
||||||
|
|
||||||
|
### 4.2 Development Setup
|
||||||
|
|
||||||
|
Follow the installation instructions in Section 2.3 (Installation) to set up your development environment.
|
||||||
|
|
||||||
|
**Additional Development Tools:**
|
||||||
|
|
||||||
|
- **ESLint**: Code is linted using @nhcarrigan/eslint-config
|
||||||
|
- **TypeScript**: Strong typing is enforced throughout the codebase
|
||||||
|
- **Code Style**: Follow the existing code style and conventions
|
||||||
|
|
||||||
|
### 4.3 Code Quality Standards
|
||||||
|
|
||||||
|
**Linting:**
|
||||||
|
- All code must pass `pnpm run lint` without warnings
|
||||||
|
- The project uses ESLint with maximum warnings set to 0
|
||||||
|
|
||||||
|
**TypeScript:**
|
||||||
|
- Use explicit types where appropriate
|
||||||
|
- Avoid `any` types unless absolutely necessary
|
||||||
|
- Document complex type definitions
|
||||||
|
|
||||||
|
**Comments and Documentation:**
|
||||||
|
- Use JSDoc comments for public methods and classes
|
||||||
|
- Include file headers with copyright, license, and author information
|
||||||
|
- Comment complex logic or non-obvious implementations
|
||||||
|
|
||||||
|
**Example File Header:**
|
||||||
|
```typescript
|
||||||
|
/**
|
||||||
|
* @copyright NHCarrigan
|
||||||
|
* @license Naomi's Public License
|
||||||
|
* @author Naomi Carrigan
|
||||||
|
*/
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.4 Contribution Workflow
|
||||||
|
|
||||||
|
1. **Fork the repository**
|
||||||
|
2. **Create a feature branch**: `git checkout -b feature/your-feature-name`
|
||||||
|
3. **Make your changes**
|
||||||
|
4. **Test your changes**: Build and run the application locally
|
||||||
|
5. **Lint your code**: `pnpm run lint`
|
||||||
|
6. **Commit your changes**: Follow conventional commit format
|
||||||
|
7. **Push to your fork**: `git push origin feature/your-feature-name`
|
||||||
|
8. **Open a pull request**
|
||||||
|
|
||||||
|
### 4.5 Pull Request Guidelines
|
||||||
|
|
||||||
|
**Before Submitting:**
|
||||||
|
- Ensure all code passes linting
|
||||||
|
- Test the application thoroughly
|
||||||
|
- Update documentation if necessary
|
||||||
|
- Keep commits atomic and well-described
|
||||||
|
|
||||||
|
**PR Description Should Include:**
|
||||||
|
- Summary of changes
|
||||||
|
- Motivation for the changes
|
||||||
|
- Any breaking changes
|
||||||
|
- Testing performed
|
||||||
|
- Screenshots (for UI changes, if applicable)
|
||||||
|
|
||||||
|
### 4.6 Areas for Contribution
|
||||||
|
|
||||||
|
**Feature Enhancements:**
|
||||||
|
- Multi-user support improvements
|
||||||
|
- Additional AI model options
|
||||||
|
- Customizable response formats
|
||||||
|
- Project name filtering options
|
||||||
|
- Image style customization
|
||||||
|
|
||||||
|
**Code Quality:**
|
||||||
|
- Adding unit tests
|
||||||
|
- Improving error handling
|
||||||
|
- Performance optimizations
|
||||||
|
- Code documentation
|
||||||
|
|
||||||
|
**Bug Fixes:**
|
||||||
|
- Reporting and fixing bugs
|
||||||
|
- Improving edge case handling
|
||||||
|
- Fixing typos or documentation errors
|
||||||
|
|
||||||
|
**Documentation:**
|
||||||
|
- Expanding user guides
|
||||||
|
- Adding code examples
|
||||||
|
- Improving setup instructions
|
||||||
|
- Creating video tutorials
|
||||||
|
|
||||||
|
### 4.7 Testing Guidelines
|
||||||
|
|
||||||
|
**Currently, the Application has no automated tests.** Contributors are encouraged to:
|
||||||
|
|
||||||
|
- Add unit tests for the `Ai` class (src/classes/ai.ts)
|
||||||
|
- Add integration tests for Discord bot functionality
|
||||||
|
- Test with various project descriptions and edge cases
|
||||||
|
- Verify image generation works with different prompts
|
||||||
|
|
||||||
|
**Recommended Testing Frameworks:**
|
||||||
|
- Jest for unit testing
|
||||||
|
- Discord.js test utilities for bot testing
|
||||||
|
|
||||||
|
### 4.8 Commit Message Format
|
||||||
|
|
||||||
|
Follow conventional commit format:
|
||||||
|
|
||||||
|
```
|
||||||
|
<type>(<scope>): <subject>
|
||||||
|
|
||||||
|
<body>
|
||||||
|
|
||||||
|
<footer>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Types:**
|
||||||
|
- `feat`: New feature
|
||||||
|
- `fix`: Bug fix
|
||||||
|
- `docs`: Documentation changes
|
||||||
|
- `style`: Code style changes (formatting, etc.)
|
||||||
|
- `refactor`: Code refactoring
|
||||||
|
- `test`: Adding tests
|
||||||
|
- `chore`: Maintenance tasks
|
||||||
|
|
||||||
|
**Examples:**
|
||||||
|
```
|
||||||
|
feat(ai): add support for custom image aspect ratios
|
||||||
|
|
||||||
|
docs(readme): update installation instructions
|
||||||
|
|
||||||
|
fix(bot): handle missing image generation gracefully
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.9 Review Process
|
||||||
|
|
||||||
|
- Pull requests will be reviewed by project maintainers
|
||||||
|
- Feedback may be provided for changes or improvements
|
||||||
|
- Once approved, PRs will be merged by maintainers
|
||||||
|
- Contributors will be credited in release notes
|
||||||
|
|
||||||
|
### 4.10 Getting Help
|
||||||
|
|
||||||
|
If you need help contributing:
|
||||||
|
|
||||||
|
- Review existing issues and pull requests
|
||||||
|
- Check the documentation at https://docs.nhcarrigan.com/
|
||||||
|
- Open a discussion in the repository
|
||||||
|
- Reach out to maintainers
|
||||||
|
|
||||||
|
### 4.11 Code Organization
|
||||||
|
|
||||||
|
When adding new features, follow the existing structure:
|
||||||
|
|
||||||
|
```
|
||||||
|
src/
|
||||||
|
├── index.ts # Main entry point, keep minimal
|
||||||
|
├── classes/ # Utility classes
|
||||||
|
│ └── ai.ts # AI-related functionality
|
||||||
|
└── (future additions) # Commands, utilities, etc.
|
||||||
|
```
|
||||||
|
|
||||||
|
**Best Practices:**
|
||||||
|
- Keep `index.ts` focused on bot initialization and event handling
|
||||||
|
- Create separate classes for distinct functionality
|
||||||
|
- Use dependency injection where appropriate
|
||||||
|
- Avoid tight coupling between components
|
||||||
|
|
||||||
|
### 4.12 Dependencies
|
||||||
|
|
||||||
|
**Adding Dependencies:**
|
||||||
|
- Prefer well-maintained packages with active communities
|
||||||
|
- Check licenses for compatibility
|
||||||
|
- Keep dependencies minimal
|
||||||
|
- Document why dependencies are needed
|
||||||
|
|
||||||
|
**Updating Dependencies:**
|
||||||
|
- Test thoroughly after updates
|
||||||
|
- Check for breaking changes
|
||||||
|
- Update documentation if APIs change
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Appendix
|
||||||
|
|
||||||
|
### A. Frequently Asked Questions
|
||||||
|
|
||||||
|
**Q: Can I use this bot in my Discord server?**
|
||||||
|
A: The bot is currently configured for single-user access. You can fork the repository and run your own instance with modified access controls.
|
||||||
|
|
||||||
|
**Q: Why does the bot only respond to one user?**
|
||||||
|
A: This is a deliberate design choice for the official instance. Self-hosted instances can modify access controls in the code.
|
||||||
|
|
||||||
|
**Q: Can I change the art style of generated images?**
|
||||||
|
A: Yes, modify the system instruction in `src/classes/ai.ts:95-97` to specify different art styles.
|
||||||
|
|
||||||
|
**Q: How much does it cost to run?**
|
||||||
|
A: Costs depend on API usage. Both Anthropic and Google offer free tiers, but high-volume usage may incur costs.
|
||||||
|
|
||||||
|
**Q: Can I use a different AI model?**
|
||||||
|
A: Yes, modify the model names in the code. Ensure compatibility with the SDK methods used.
|
||||||
|
|
||||||
|
**Q: Why did image generation fail?**
|
||||||
|
A: Common reasons include API quota limits, content policy violations, or temporary service unavailability.
|
||||||
|
|
||||||
|
**Q: How do I get my Discord user ID?**
|
||||||
|
A: Enable Developer Mode in Discord settings, then right-click your username and select "Copy User ID".
|
||||||
|
|
||||||
|
**Q: Can I deploy this on a free tier service?**
|
||||||
|
A: Yes, the bot can run on services like Heroku (legacy), Railway, Render, or Fly.io, subject to their free tier limitations.
|
||||||
|
|
||||||
|
### B. Glossary
|
||||||
|
|
||||||
|
- **AI**: Artificial Intelligence
|
||||||
|
- **API**: Application Programming Interface
|
||||||
|
- **Bot**: Automated Discord application
|
||||||
|
- **Claude**: Anthropic's AI language model
|
||||||
|
- **Discord**: Communication platform
|
||||||
|
- **ESLint**: JavaScript linting tool
|
||||||
|
- **Gemini**: Google's AI model family
|
||||||
|
- **Intent**: Discord API permission flag
|
||||||
|
- **Mascot**: Representative character image
|
||||||
|
- **pnpm**: Fast, disk space efficient package manager
|
||||||
|
- **TypeScript**: Typed superset of JavaScript
|
||||||
|
|
||||||
|
### C. Version History
|
||||||
|
|
||||||
|
- **v1.0.0**: Initial release
|
||||||
|
- Claude AI integration for project name generation
|
||||||
|
- Google Gemini integration for mascot image generation
|
||||||
|
- Discord bot functionality
|
||||||
|
- Single-user access control
|
||||||
|
- Uniqueness checking against existing projects
|
||||||
|
|
||||||
|
### D. Credits
|
||||||
|
|
||||||
|
- **Author**: Naomi Carrigan
|
||||||
|
- **License**: Naomi's Public License
|
||||||
|
- **Dependencies**: See package.json for full list
|
||||||
|
- **Special Thanks**: Anthropic, Google, and Discord.js contributors
|
||||||
|
|
||||||
|
### E. Additional Resources
|
||||||
|
|
||||||
|
- **Anthropic Documentation**: https://docs.anthropic.com/
|
||||||
|
- **Google AI Documentation**: https://ai.google.dev/
|
||||||
|
- **Discord.js Guide**: https://discordjs.guide/
|
||||||
|
- **TypeScript Documentation**: https://www.typescriptlang.org/docs/
|
||||||
|
- **Author's Documentation Hub**: https://docs.nhcarrigan.com/
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Last Updated**: 2025-10-31
|
||||||
|
**Documentation Version**: 1.0.0
|
||||||
@@ -0,0 +1,662 @@
|
|||||||
|
---
|
||||||
|
title: Scripts
|
||||||
|
---
|
||||||
|
|
||||||
|
Scripts Collection (hereinafter the "Application") is a collection of TypeScript-based automation utilities for managing translations, auditing repositories, automating Discord tasks, and processing media files.
|
||||||
|
|
||||||
|
## 1. User Documentation
|
||||||
|
|
||||||
|
This section is for those interacting with a live instance of the Application.
|
||||||
|
|
||||||
|
### Overview
|
||||||
|
|
||||||
|
The Scripts Collection provides standalone automation tools that can be executed individually as needed. Each script serves a specific purpose and operates independently.
|
||||||
|
|
||||||
|
### Available Scripts
|
||||||
|
|
||||||
|
#### 1.1 Crowdin Translation Management
|
||||||
|
|
||||||
|
**Purpose:** Manage translations and translation memory in Crowdin projects.
|
||||||
|
|
||||||
|
##### Write Data (`src/crowdin/writeData.ts`)
|
||||||
|
|
||||||
|
Fetches and persists Crowdin project data locally for use by other scripts.
|
||||||
|
|
||||||
|
**Usage:**
|
||||||
|
```bash
|
||||||
|
op run --env-file=.env -- tsx src/crowdin/writeData.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
**What it does:**
|
||||||
|
- Retrieves all files from your Crowdin project
|
||||||
|
- Retrieves all translatable strings
|
||||||
|
- Saves data to `data/crowdin-files.json` and `data/crowdin-strings.json`
|
||||||
|
|
||||||
|
**Required environment variables:**
|
||||||
|
- `CROWDIN_PROJECT_ID` - Your Crowdin project identifier
|
||||||
|
- `CROWDIN_API_URL` - Crowdin API base URL
|
||||||
|
- `CROWDIN_TOKEN` - Your Crowdin API token
|
||||||
|
|
||||||
|
##### Clear Hidden Translations (`src/crowdin/clearHiddenTranslations.ts`)
|
||||||
|
|
||||||
|
Removes translations for strings that have been marked as hidden in Crowdin.
|
||||||
|
|
||||||
|
**Usage:**
|
||||||
|
```bash
|
||||||
|
op run --env-file=.env -- tsx src/crowdin/clearHiddenTranslations.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
**What it does:**
|
||||||
|
- Reads cached string data from `data/crowdin-strings.json`
|
||||||
|
- Identifies strings marked as hidden
|
||||||
|
- Deletes existing translations for hidden strings across all languages
|
||||||
|
- Tracks processed strings in `data/crowdin-strings-hidden.txt` to avoid reprocessing
|
||||||
|
|
||||||
|
**Prerequisites:**
|
||||||
|
- Run `writeData.ts` first to cache the latest string data
|
||||||
|
- Same environment variables as above
|
||||||
|
|
||||||
|
##### Reapply Translations (`src/crowdin/reapplyTranslations.ts`)
|
||||||
|
|
||||||
|
Applies translation memory to project files using Crowdin's pre-translation feature.
|
||||||
|
|
||||||
|
**Usage:**
|
||||||
|
```bash
|
||||||
|
op run --env-file=.env -- tsx src/crowdin/reapplyTranslations.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
**What it does:**
|
||||||
|
- Initiates pre-translation jobs for all files using translation memory
|
||||||
|
- Uses "perfect match only" strategy with auto-approval
|
||||||
|
- Skips already approved translations
|
||||||
|
- Polls every 5 seconds for completion status
|
||||||
|
- Displays progress for each file
|
||||||
|
|
||||||
|
**Prerequisites:**
|
||||||
|
- Run `writeData.ts` first to cache the latest file data
|
||||||
|
- Same environment variables as above
|
||||||
|
|
||||||
|
#### 1.2 GitHub Repository Auditing
|
||||||
|
|
||||||
|
##### Audit NPM Packages (`src/github/auditNpmPackages.ts`)
|
||||||
|
|
||||||
|
Scans GitHub organization repositories for known vulnerable npm packages.
|
||||||
|
|
||||||
|
**Usage:**
|
||||||
|
```bash
|
||||||
|
op run --env-file=.env -- tsx src/github/auditNpmPackages.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
**What it does:**
|
||||||
|
- Fetches all repositories from the FreeCodeCamp GitHub organization
|
||||||
|
- Searches for 18 known vulnerable package versions in `package.json` files
|
||||||
|
- Generates a report of vulnerable dependencies
|
||||||
|
- Saves results to `data/npm-vulnerabilities.txt`
|
||||||
|
|
||||||
|
**Monitored vulnerable packages include:**
|
||||||
|
- ansi-regex (< 5.0.1)
|
||||||
|
- chalk (< 4.1.2)
|
||||||
|
- debug (< 4.3.1)
|
||||||
|
- color/color-string libraries (various versions)
|
||||||
|
- json-schema (< 0.4.0)
|
||||||
|
- And 13 others
|
||||||
|
|
||||||
|
**Required environment variables:**
|
||||||
|
- `GITHUB_TOKEN` - GitHub personal access token with repo read permissions
|
||||||
|
|
||||||
|
**Output format:**
|
||||||
|
```
|
||||||
|
Repository: <org>/<repo>
|
||||||
|
File: package.json
|
||||||
|
Vulnerable package: <package-name>@<version>
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 1.3 Discord Automation
|
||||||
|
|
||||||
|
##### Create Conference Threads (`src/discord/cycThreads.ts`)
|
||||||
|
|
||||||
|
Automatically creates forum threads for conference presentations.
|
||||||
|
|
||||||
|
**Usage:**
|
||||||
|
```bash
|
||||||
|
op run --env-file=.env -- tsx src/discord/cycThreads.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
**What it does:**
|
||||||
|
- Creates threads for 100+ conference talk entries
|
||||||
|
- Each thread is titled with speaker name and talk title
|
||||||
|
- Adds a welcome message to each thread
|
||||||
|
- Implements rate limiting with automatic retry on Discord API limits
|
||||||
|
|
||||||
|
**Required environment variables:**
|
||||||
|
- `DISCORD_TOKEN` - Discord bot token with thread creation permissions
|
||||||
|
|
||||||
|
**Target channel:** 1417221194536714393
|
||||||
|
|
||||||
|
**Note:** The conference data is currently hardcoded in the script and includes talks on web development, cloud technologies, AI/ML, architecture patterns, and more.
|
||||||
|
|
||||||
|
#### 1.4 Media File Processing
|
||||||
|
|
||||||
|
##### Add ID3 Tags (`src/music/id3v2.ts`)
|
||||||
|
|
||||||
|
Batch processes MP3 files to add metadata tags and album artwork.
|
||||||
|
|
||||||
|
**Usage:**
|
||||||
|
```bash
|
||||||
|
tsx src/music/id3v2.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
**What it does:**
|
||||||
|
- Scans `/home/naomi/down` for MP3 files
|
||||||
|
- Extracts title from filename (removing special characters)
|
||||||
|
- Sets artist to "Neuro-sama"
|
||||||
|
- Adds album artwork from `/home/naomi/neuro.png`
|
||||||
|
- Displays progress bar during processing
|
||||||
|
|
||||||
|
**Required system dependencies:**
|
||||||
|
- `eyeD3` - Python-based MP3 tag editor
|
||||||
|
- `id3v2` - Command-line ID3 tag tool
|
||||||
|
|
||||||
|
**Note:** File paths are currently hardcoded and should be modified for your use case.
|
||||||
|
|
||||||
|
### General Usage Tips
|
||||||
|
|
||||||
|
1. **Environment Variables:** Most scripts require environment variables. Use 1Password CLI (`op run`) for secure credential management.
|
||||||
|
|
||||||
|
2. **Execution Pattern:**
|
||||||
|
```bash
|
||||||
|
# With environment variables
|
||||||
|
op run --env-file=.env --no-masking -- tsx src/path/to/script.ts
|
||||||
|
|
||||||
|
# Without environment variables
|
||||||
|
tsx src/path/to/script.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Data Directory:** Scripts store cached data and results in the `data/` directory. Ensure this directory exists and is writable.
|
||||||
|
|
||||||
|
4. **Error Handling:** Scripts implement automatic retry logic for rate-limited API calls (429 errors) with a 5-second backoff.
|
||||||
|
|
||||||
|
5. **Progress Tracking:** Most scripts provide console output to track progress. Some use progress bars for visual feedback.
|
||||||
|
|
||||||
|
## 2. Technical Documentation
|
||||||
|
|
||||||
|
This section is for those interested in running their own instance of the Application.
|
||||||
|
|
||||||
|
### 2.1 System Requirements
|
||||||
|
|
||||||
|
**Runtime:**
|
||||||
|
- Node.js (compatible with v24.3.0 type definitions)
|
||||||
|
- pnpm 10.15.0 (package manager)
|
||||||
|
- TypeScript 5.9.2
|
||||||
|
|
||||||
|
**External Tools (for media processing):**
|
||||||
|
- eyeD3 (Python package)
|
||||||
|
- id3v2 (system package)
|
||||||
|
|
||||||
|
**Optional:**
|
||||||
|
- 1Password CLI (`op`) for secure environment variable management
|
||||||
|
|
||||||
|
### 2.2 Installation
|
||||||
|
|
||||||
|
1. **Clone the repository:**
|
||||||
|
```bash
|
||||||
|
git clone <repository-url>
|
||||||
|
cd scripts
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Install dependencies:**
|
||||||
|
```bash
|
||||||
|
pnpm install
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Create data directory:**
|
||||||
|
```bash
|
||||||
|
mkdir -p data
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Set up environment variables:**
|
||||||
|
|
||||||
|
Create a `.env` file with required credentials:
|
||||||
|
```env
|
||||||
|
# Crowdin (for translation scripts)
|
||||||
|
CROWDIN_PROJECT_ID=your_project_id
|
||||||
|
CROWDIN_API_URL=https://api.crowdin.com/api/v2
|
||||||
|
CROWDIN_TOKEN=your_api_token
|
||||||
|
|
||||||
|
# GitHub (for audit script)
|
||||||
|
GITHUB_TOKEN=your_github_token
|
||||||
|
|
||||||
|
# Discord (for thread creation script)
|
||||||
|
DISCORD_TOKEN=your_discord_bot_token
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Install external tools (for media processing):**
|
||||||
|
```bash
|
||||||
|
# On Ubuntu/Debian
|
||||||
|
sudo apt-get install id3v2
|
||||||
|
pip install eyeD3
|
||||||
|
|
||||||
|
# On macOS
|
||||||
|
brew install id3v2
|
||||||
|
pip install eyeD3
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.3 Configuration
|
||||||
|
|
||||||
|
**TypeScript Configuration (`tsconfig.json`):**
|
||||||
|
- Extends `@nhcarrigan/typescript-config`
|
||||||
|
- Source directory: `./src`
|
||||||
|
- Output directory: `./prod`
|
||||||
|
- ES Modules enabled
|
||||||
|
|
||||||
|
**ESLint Configuration (`eslint.config.js`):**
|
||||||
|
- Extends `@nhcarrigan/eslint-config`
|
||||||
|
- Console statements allowed
|
||||||
|
- Await in loops permitted (needed for sequential API calls)
|
||||||
|
- Relaxed unsafe assignment rules
|
||||||
|
|
||||||
|
**Modifying Target Locations:**
|
||||||
|
|
||||||
|
To adapt scripts for your use:
|
||||||
|
|
||||||
|
1. **Crowdin scripts:** Update environment variables in `.env`
|
||||||
|
2. **GitHub audit:** Modify organization name in `src/github/auditNpmPackages.ts:48`
|
||||||
|
3. **Discord threads:** Update channel ID and talk data in `src/discord/cycThreads.ts`
|
||||||
|
4. **Media processing:** Update file paths in `src/music/id3v2.ts:7-8`
|
||||||
|
|
||||||
|
### 2.4 Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
scripts/
|
||||||
|
├── src/ # Source code
|
||||||
|
│ ├── crowdin/ # Crowdin automation
|
||||||
|
│ │ ├── interfaces/ # TypeScript type definitions
|
||||||
|
│ │ ├── utils/ # Helper functions
|
||||||
|
│ │ ├── writeData.ts
|
||||||
|
│ │ ├── clearHiddenTranslations.ts
|
||||||
|
│ │ └── reapplyTranslations.ts
|
||||||
|
│ ├── discord/ # Discord automation
|
||||||
|
│ │ └── cycThreads.ts
|
||||||
|
│ ├── github/ # GitHub utilities
|
||||||
|
│ │ └── auditNpmPackages.ts
|
||||||
|
│ ├── music/ # Media processing
|
||||||
|
│ │ └── id3v2.ts
|
||||||
|
│ ├── utils/ # Shared utilities
|
||||||
|
│ │ ├── sleep.ts
|
||||||
|
│ │ ├── backoffAndRetry.ts
|
||||||
|
│ │ └── serialiseJsonOrError.ts
|
||||||
|
│ └── index.ts # Usage instructions
|
||||||
|
├── data/ # Cached data and output
|
||||||
|
├── prod/ # Compiled JavaScript
|
||||||
|
├── package.json # Dependencies and scripts
|
||||||
|
├── tsconfig.json # TypeScript configuration
|
||||||
|
└── eslint.config.js # Linting rules
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.5 Development
|
||||||
|
|
||||||
|
**Build the project:**
|
||||||
|
```bash
|
||||||
|
pnpm build
|
||||||
|
```
|
||||||
|
|
||||||
|
**Run linting:**
|
||||||
|
```bash
|
||||||
|
pnpm lint
|
||||||
|
```
|
||||||
|
|
||||||
|
**Execute scripts:**
|
||||||
|
```bash
|
||||||
|
# With tsx (no compilation needed)
|
||||||
|
tsx src/path/to/script.ts
|
||||||
|
|
||||||
|
# Or compile first, then run
|
||||||
|
pnpm build
|
||||||
|
node prod/path/to/script.js
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.6 API Integration Details
|
||||||
|
|
||||||
|
**Crowdin API:**
|
||||||
|
- Base URL: `https://api.crowdin.com/api/v2`
|
||||||
|
- Authentication: Bearer token in Authorization header
|
||||||
|
- Pagination: 500 items per page (limit parameter)
|
||||||
|
- Rate limiting: Handled with automatic retry (5-second backoff)
|
||||||
|
|
||||||
|
**GitHub API:**
|
||||||
|
- Client: `@octokit/rest`
|
||||||
|
- Authentication: Personal access token
|
||||||
|
- Pagination: 100 repositories per page
|
||||||
|
- Permissions needed: `repo` (read access)
|
||||||
|
|
||||||
|
**Discord API:**
|
||||||
|
- REST endpoint: `https://discord.com/api/v10`
|
||||||
|
- Authentication: Bot token
|
||||||
|
- Rate limiting: Automatic retry with backoff on 429 responses
|
||||||
|
- Required intents: Message content, guild messages
|
||||||
|
|
||||||
|
### 2.7 Data Storage
|
||||||
|
|
||||||
|
**Generated files:**
|
||||||
|
- `data/crowdin-files.json` (~12MB) - Cached Crowdin file metadata
|
||||||
|
- `data/crowdin-strings.json` (~275MB) - Cached translatable strings
|
||||||
|
- `data/crowdin-strings-hidden.txt` (~1.3MB) - Processed hidden string IDs
|
||||||
|
- `data/npm-vulnerabilities.txt` (~2KB) - Audit results
|
||||||
|
|
||||||
|
**Data freshness:**
|
||||||
|
- Crowdin data should be refreshed before running dependent scripts
|
||||||
|
- GitHub audit data is generated per run
|
||||||
|
- Discord thread creation is one-time execution
|
||||||
|
|
||||||
|
### 2.8 Security Considerations
|
||||||
|
|
||||||
|
1. **Secrets Management:**
|
||||||
|
- Use 1Password CLI or similar tools for environment variable injection
|
||||||
|
- Never commit `.env` files to version control
|
||||||
|
- Rotate API tokens regularly
|
||||||
|
|
||||||
|
2. **API Token Permissions:**
|
||||||
|
- Crowdin: Project member with translation management access
|
||||||
|
- GitHub: Read-only repository access (minimal scope)
|
||||||
|
- Discord: Bot token with thread creation and message posting permissions
|
||||||
|
|
||||||
|
3. **Rate Limiting:**
|
||||||
|
- All scripts implement retry logic for rate limit responses
|
||||||
|
- 5-second backoff before retrying
|
||||||
|
- Recursive retry until success
|
||||||
|
|
||||||
|
4. **Input Validation:**
|
||||||
|
- API responses are typed with TypeScript interfaces
|
||||||
|
- Safe JSON parsing with error handling
|
||||||
|
- File existence checks before processing
|
||||||
|
|
||||||
|
### 2.9 Troubleshooting
|
||||||
|
|
||||||
|
**Common Issues:**
|
||||||
|
|
||||||
|
1. **"Cannot find module" errors:**
|
||||||
|
- Ensure `pnpm install` completed successfully
|
||||||
|
- Check Node.js version compatibility
|
||||||
|
|
||||||
|
2. **API authentication failures:**
|
||||||
|
- Verify environment variables are set correctly
|
||||||
|
- Check token permissions and expiration
|
||||||
|
- Ensure `op run` is used for secure variable injection
|
||||||
|
|
||||||
|
3. **Rate limit errors persisting:**
|
||||||
|
- Check API quota limits on your account
|
||||||
|
- Increase backoff delay in `src/utils/backoffAndRetry.ts`
|
||||||
|
|
||||||
|
4. **Large JSON files causing memory issues:**
|
||||||
|
- Increase Node.js memory limit: `NODE_OPTIONS=--max-old-space-size=4096`
|
||||||
|
- Process data in smaller batches
|
||||||
|
|
||||||
|
5. **Media processing failures:**
|
||||||
|
- Verify `eyeD3` and `id3v2` are installed and in PATH
|
||||||
|
- Check file permissions for input/output directories
|
||||||
|
- Ensure cover image exists at specified path
|
||||||
|
|
||||||
|
## 3. Legal Documentation
|
||||||
|
|
||||||
|
This section is for expansions to our legal policies specific to the Application.
|
||||||
|
|
||||||
|
### 3.1 License
|
||||||
|
|
||||||
|
The Application is distributed under **Naomi's Public License**. See `LICENSE.md` for full terms.
|
||||||
|
|
||||||
|
### 3.2 Privacy Policy
|
||||||
|
|
||||||
|
Usage of these scripts may involve processing data from third-party services. Refer to `PRIVACY.md` for our privacy practices regarding:
|
||||||
|
- API credentials and tokens
|
||||||
|
- Cached translation data
|
||||||
|
- Repository metadata
|
||||||
|
- Discord message content
|
||||||
|
|
||||||
|
### 3.3 Terms of Service
|
||||||
|
|
||||||
|
By using these scripts, you agree to:
|
||||||
|
- Comply with third-party API terms of service (Crowdin, GitHub, Discord)
|
||||||
|
- Use automation responsibly and within rate limits
|
||||||
|
- Not use the scripts for malicious purposes
|
||||||
|
- Maintain security of API credentials
|
||||||
|
|
||||||
|
See `TERMS.md` for complete terms.
|
||||||
|
|
||||||
|
### 3.4 Security Policy
|
||||||
|
|
||||||
|
To report security vulnerabilities in the Application itself, please refer to `SECURITY.md`.
|
||||||
|
|
||||||
|
**Third-Party Security:**
|
||||||
|
- The GitHub audit script identifies known vulnerable npm packages
|
||||||
|
- Keep dependencies updated regularly
|
||||||
|
- Review `pnpm audit` output periodically
|
||||||
|
|
||||||
|
### 3.5 Data Retention
|
||||||
|
|
||||||
|
**Local Data Storage:**
|
||||||
|
- Cached data in `data/` directory is stored indefinitely until manually deleted
|
||||||
|
- No data is transmitted to external services except through official APIs
|
||||||
|
- API tokens are loaded at runtime and not persisted to disk
|
||||||
|
|
||||||
|
**Third-Party Data:**
|
||||||
|
- Crowdin: Translation data is fetched and cached locally
|
||||||
|
- GitHub: Repository metadata is accessed read-only
|
||||||
|
- Discord: Messages are posted to threads as configured
|
||||||
|
|
||||||
|
## 4. Contributing Documentation
|
||||||
|
|
||||||
|
This section is for documentation related to contributing to the Application's codebase.
|
||||||
|
|
||||||
|
### 4.1 Contributing Guidelines
|
||||||
|
|
||||||
|
Please refer to `CONTRIBUTING.md` for detailed contribution guidelines. Key points:
|
||||||
|
|
||||||
|
**Code Style:**
|
||||||
|
- Follow ESLint configuration (`@nhcarrigan/eslint-config`)
|
||||||
|
- Use TypeScript with strict type checking
|
||||||
|
- Include JSDoc comments for exported functions
|
||||||
|
- Add copyright headers to new files
|
||||||
|
|
||||||
|
**Commit Messages:**
|
||||||
|
- Use conventional commit format
|
||||||
|
- Examples: `feat:`, `fix:`, `docs:`, `refactor:`
|
||||||
|
|
||||||
|
**Testing:**
|
||||||
|
- While formal tests are not currently required, manually test all changes
|
||||||
|
- Verify scripts work with environment variables
|
||||||
|
- Test error handling and edge cases
|
||||||
|
|
||||||
|
### 4.2 Adding New Scripts
|
||||||
|
|
||||||
|
To add a new automation script:
|
||||||
|
|
||||||
|
1. **Create script file:**
|
||||||
|
```bash
|
||||||
|
# Choose appropriate directory or create new one
|
||||||
|
touch src/<category>/<script-name>.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Add copyright header:**
|
||||||
|
```typescript
|
||||||
|
/*
|
||||||
|
* Copyright (C) 2025 Naomi Carrigan
|
||||||
|
* Licensed under Naomi's Public License
|
||||||
|
*/
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Import shared utilities:**
|
||||||
|
```typescript
|
||||||
|
import { sleep } from "../utils/sleep.js";
|
||||||
|
import { backoffAndRetry } from "../utils/backoffAndRetry.js";
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Implement type safety:**
|
||||||
|
- Create interfaces in `<category>/interfaces/` if needed
|
||||||
|
- Use TypeScript types for all API responses
|
||||||
|
- Handle errors with try-catch blocks
|
||||||
|
|
||||||
|
5. **Add environment variable validation:**
|
||||||
|
```typescript
|
||||||
|
const token = process.env.SERVICE_TOKEN;
|
||||||
|
if (!token) {
|
||||||
|
console.error("Missing SERVICE_TOKEN environment variable");
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
6. **Document usage:**
|
||||||
|
- Add JSDoc comments to main functions
|
||||||
|
- Update this documentation file
|
||||||
|
- Include example usage in comments
|
||||||
|
|
||||||
|
7. **Test thoroughly:**
|
||||||
|
- Test with valid and invalid inputs
|
||||||
|
- Verify error handling
|
||||||
|
- Check rate limit handling
|
||||||
|
|
||||||
|
### 4.3 Code Organization
|
||||||
|
|
||||||
|
**Shared Utilities (`src/utils/`):**
|
||||||
|
Place reusable functions here:
|
||||||
|
- Async utilities (sleep, retry logic)
|
||||||
|
- Data transformation functions
|
||||||
|
- Error handling helpers
|
||||||
|
- Common type definitions
|
||||||
|
|
||||||
|
**Category-Specific Utilities:**
|
||||||
|
Place service-specific helpers in `<category>/utils/`:
|
||||||
|
- API client wrappers
|
||||||
|
- Data fetching functions
|
||||||
|
- Service-specific transformations
|
||||||
|
|
||||||
|
**Interfaces (`<category>/interfaces/`):**
|
||||||
|
Define TypeScript interfaces for:
|
||||||
|
- API request/response types
|
||||||
|
- Configuration objects
|
||||||
|
- Data models
|
||||||
|
|
||||||
|
### 4.4 Dependency Management
|
||||||
|
|
||||||
|
**Adding Dependencies:**
|
||||||
|
```bash
|
||||||
|
pnpm add <package-name>
|
||||||
|
pnpm add -D <dev-package-name>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Updating Dependencies:**
|
||||||
|
```bash
|
||||||
|
pnpm update
|
||||||
|
pnpm audit
|
||||||
|
```
|
||||||
|
|
||||||
|
**Preferred Libraries:**
|
||||||
|
- API clients: Official SDKs when available (`@octokit/rest`, etc.)
|
||||||
|
- CLI interactions: `@inquirer/prompts`
|
||||||
|
- Progress visualization: `cli-progress`
|
||||||
|
- HTTP requests: Native `fetch` with `backoffAndRetry` wrapper
|
||||||
|
|
||||||
|
### 4.5 Documentation Standards
|
||||||
|
|
||||||
|
**Code Comments:**
|
||||||
|
- Use JSDoc for all exported functions
|
||||||
|
- Include `@param`, `@returns`, and `@throws` tags
|
||||||
|
- Add inline comments for complex logic
|
||||||
|
|
||||||
|
**Example:**
|
||||||
|
```typescript
|
||||||
|
/**
|
||||||
|
* Fetches all files from a Crowdin project with pagination.
|
||||||
|
* @param projectId - The Crowdin project identifier
|
||||||
|
* @returns Promise resolving to array of file objects
|
||||||
|
* @throws Error if API request fails after retries
|
||||||
|
*/
|
||||||
|
export async function getFiles(projectId: string): Promise<File[]> {
|
||||||
|
// Implementation
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**README Updates:**
|
||||||
|
- Document new scripts in README.md
|
||||||
|
- Include usage examples
|
||||||
|
- List required environment variables
|
||||||
|
- Note any system dependencies
|
||||||
|
|
||||||
|
**Changelog:**
|
||||||
|
- Maintain a changelog for significant changes
|
||||||
|
- Use semantic versioning principles
|
||||||
|
- Document breaking changes prominently
|
||||||
|
|
||||||
|
### 4.6 Architecture Principles
|
||||||
|
|
||||||
|
**Script Independence:**
|
||||||
|
- Each script should be runnable standalone
|
||||||
|
- Avoid tight coupling between scripts
|
||||||
|
- Use data files for inter-script communication when needed
|
||||||
|
|
||||||
|
**Error Handling:**
|
||||||
|
- Always validate environment variables at startup
|
||||||
|
- Use try-catch blocks for external API calls
|
||||||
|
- Provide descriptive error messages
|
||||||
|
- Exit with non-zero code on failure
|
||||||
|
|
||||||
|
**API Best Practices:**
|
||||||
|
- Implement rate limit handling with exponential backoff
|
||||||
|
- Use pagination for large data sets
|
||||||
|
- Cache data locally when appropriate
|
||||||
|
- Respect API quotas and limits
|
||||||
|
|
||||||
|
**Type Safety:**
|
||||||
|
- Define interfaces for all external data
|
||||||
|
- Avoid `any` types when possible
|
||||||
|
- Use strict TypeScript configuration
|
||||||
|
- Validate data shapes at runtime when needed
|
||||||
|
|
||||||
|
### 4.7 Testing Approach
|
||||||
|
|
||||||
|
While formal unit tests are not currently implemented, contributors should:
|
||||||
|
|
||||||
|
1. **Manual Testing Checklist:**
|
||||||
|
- Run script with valid environment variables
|
||||||
|
- Test with missing environment variables
|
||||||
|
- Verify output files are created correctly
|
||||||
|
- Check error handling with invalid inputs
|
||||||
|
- Test rate limit retry logic (if applicable)
|
||||||
|
|
||||||
|
2. **Integration Testing:**
|
||||||
|
- Test against real APIs in development environments
|
||||||
|
- Verify data format compatibility
|
||||||
|
- Check for breaking changes in API responses
|
||||||
|
|
||||||
|
3. **Future Testing Goals:**
|
||||||
|
- Add unit tests for utility functions
|
||||||
|
- Mock API responses for integration tests
|
||||||
|
- Set up CI/CD pipeline for automated testing
|
||||||
|
- Implement test coverage reporting
|
||||||
|
|
||||||
|
### 4.8 Code of Conduct
|
||||||
|
|
||||||
|
All contributors must adhere to the Code of Conduct outlined in `CODE_OF_CONDUCT.md`. Key principles:
|
||||||
|
- Be respectful and inclusive
|
||||||
|
- Welcome newcomers and help them learn
|
||||||
|
- Focus on constructive feedback
|
||||||
|
- Respect differing opinions and experiences
|
||||||
|
|
||||||
|
### 4.9 Getting Help
|
||||||
|
|
||||||
|
**Resources:**
|
||||||
|
- Review existing scripts for examples
|
||||||
|
- Check JSDoc comments for function documentation
|
||||||
|
- Read official API documentation for services
|
||||||
|
- Consult TypeScript and ESLint documentation for style questions
|
||||||
|
|
||||||
|
**Contact:**
|
||||||
|
- Open an issue for bug reports or feature requests
|
||||||
|
- Submit pull requests for proposed changes
|
||||||
|
- Follow contribution guidelines for all submissions
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Document Version:** 1.0
|
||||||
|
**Last Updated:** 2025-10-31
|
||||||
|
**Maintained By:** Naomi Carrigan
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
---
|
||||||
|
title: Sylvara
|
||||||
|
---
|
||||||
|
|
||||||
|
Sylvara (hereinafter the "Application") is a micro-blogging social media platform.
|
||||||
|
|
||||||
|
## 1. User Documentation
|
||||||
|
|
||||||
|
:::note
|
||||||
|
This section is coming soon!
|
||||||
|
:::
|
||||||
|
|
||||||
|
This section is for those interacting with a live instance of the Application.
|
||||||
|
|
||||||
|
## 2. Technical Documentation
|
||||||
|
|
||||||
|
:::note
|
||||||
|
This section is coming soon!
|
||||||
|
:::
|
||||||
|
|
||||||
|
This section is for those interested in running their own instance of the Application.
|
||||||
|
|
||||||
|
## 3. Legal Documentation
|
||||||
|
|
||||||
|
:::note
|
||||||
|
This section is coming soon!
|
||||||
|
:::
|
||||||
|
|
||||||
|
This section is for expansions to our legal policies specific to the Application.
|
||||||
|
|
||||||
|
## 4. Contributing Documentation
|
||||||
|
|
||||||
|
:::note
|
||||||
|
This section is coming soon!
|
||||||
|
:::
|
||||||
|
|
||||||
|
This section is for documentation related to contributing to the Application's codebase.
|
||||||
|
|
||||||
Reference in New Issue
Block a user