docs/src/content/docs/projects/data-api.md

title

title
Data API

Data API (hereinafter the "Application") is a lightweight web API server designed to host and serve personal and professional data for the NHCarrigan ecosystem. The Application acts as a centralised headless CMS, providing structured data in both YAML and JSON formats to power multiple websites, applications, and Discord bots.

1. User Documentation

This section is for those interacting with a live instance of the Application.

1.1. API Overview

The Data Server provides a RESTful API that serves various categories of data through simple HTTP GET requests. All endpoints support both YAML and JSON formats.

Base URL: The server runs on `https://data.nhcarrigan.com'.

Root Endpoint:

• GET / - Returns a welcome message: "Welcome to the NHCarrigan Data API!"

1.2. Available Endpoints

All data endpoints are available in two formats by changing the file extension:

Projects Data

• GET /projects.json - Returns project portfolio data in JSON format
• GET /projects.yml - Returns project portfolio data in YAML format

Contains information about development projects including:

• Project names and descriptions
• Project URLs and repository links
• Project categories (websites, community tools, Discord bots, etc.)
• Avatar images
• Status indicators (premium, work-in-progress)

Resume Data

• GET /resume.json - Returns resume data in JSON format
• GET /resume.yml - Returns resume data in YAML format

Contains comprehensive professional information:

• Employment history (current and prior positions)
• Volunteer experience
• Education credentials
• Professional certifications
• Notable projects
• Publications and articles

Testimonials

• GET /testimonials.json - Returns testimonials in JSON format
• GET /testimonials.yml - Returns testimonials in YAML format

Contains professional testimonials and recommendations with:

• Testimonial author names
• Testimonial content
• Date information

Donation Information

• GET /donate.json - Returns donation platform data in JSON format
• GET /donate.yml - Returns donation platform data in YAML format

Contains details about donation platforms:

• Platform names and URLs
• Icon references
• Colour schemes
• Platform descriptions

Funding Information

• GET /funding.json - Returns comprehensive funding data in JSON format
• GET /funding.yml - Returns comprehensive funding data in YAML format

Contains detailed funding information:

• Entity details
• Project listings with unique identifiers
• Funding channel information
• Subscription plan details
• Financial history and records

1.3. CORS Support

The Application has CORS enabled for all origins, allowing consumption from any domain without cross-origin restrictions.

1.4. Content Types

The Application automatically sets appropriate Content-Type headers:

• JSON endpoints: application/json
• YAML endpoints: text/yaml or application/x-yaml

1.5. Rate Limiting

Currently, the Application does not implement rate limiting. Users should be respectful of the server resources and avoid excessive requests.

1.6. Data Freshness

Data is compiled from YAML source files during the build process. To receive updated data, the server must be rebuilt and restarted when source data changes.

2. Technical Documentation

This section is for those interested in running their own instance of the Application.

2.1. Prerequisites

Before running the Application, ensure you have the following installed:

• Node.js: Latest LTS version recommended
• pnpm: Version 10.17.0 (specified in package.json as packageManager)
• 1Password CLI (optional): For secure environment variable management in production

2.2. Technology Stack

• Runtime: Node.js with ES modules
• Language: TypeScript 5.9.2
• Web Framework: Fastify 5.6.1
• Data Format: YAML (source) → JSON (runtime)
• Testing: Vitest 3.2.4
• Linting: ESLint 9.36.0
• Package Manager: pnpm 10.17.0

2.3. Installation

1. Clone the repository:

git clone <repository-url>
cd data

2. Install dependencies using pnpm:

pnpm install

2.4. Project Structure

/data/
├── src/                           # TypeScript source code
│   ├── index.ts                   # Main entry point - Fastify server
│   ├── modules/
│   │   └── buildRoutes.ts         # Dynamic route builder
│   ├── interfaces/                # TypeScript type definitions
│   │   ├── projects.ts
│   │   ├── resume.ts
│   │   ├── testimonials.ts
│   │   ├── donate.ts
│   │   └── funding.ts
│   └── utils/
│       └── logger.ts              # Logger configuration
├── data/                          # Data directory (YAML sources and JSON builds)
│   ├── projects.yml / .json
│   ├── resume.yml / .json
│   ├── testimonials.yml / .json
│   ├── donate.yml / .json
│   └── funding.yml / .json
├── test/
│   └── yaml.spec.ts               # Comprehensive validation tests
├── prod/                          # Compiled JavaScript output
├── build.js                       # YAML to JSON build script
├── tsconfig.json                  # TypeScript configuration
└── package.json                   # Project metadata and scripts

2.5. Configuration

The Application uses the following configuration points:

Server Port

• Default: Port 9999
• Location: src/index.ts:34
• Can be modified by editing the port number in the source code

Data Directory

• Location: ./data
• Contains both source YAML files and built JSON files
• Referenced in: src/modules/buildRoutes.ts

CORS Settings

• Currently configured to allow all origins
• Location: src/index.ts:30
• Can be restricted by modifying the Fastify CORS configuration

2.6. Data Management

Adding New Data Endpoints

1. Create a new YAML file in the /data directory:

touch data/newdata.yml

2. Create a corresponding TypeScript interface in src/interfaces/:

// src/interfaces/newdata.ts
export interface NewData {
  // Define your data structure
}

3. The Application will automatically:
   • Generate routes for /newdata.yml and /newdata.json
   • Serve the content when the server restarts

Editing Existing Data

1. Edit the YAML files in the /data directory
2. Run the build command to convert YAML to JSON
3. Restart the server to serve updated data

The YAML files are the source of truth. Always edit YAML files, not JSON files, as JSON files are auto-generated.

2.7. Build Process

The build process consists of two steps:

1. YAML to JSON Conversion: node build.js

   • Reads all YAML files from the data directory
   • Generates corresponding JSON files
   • Preserves the human-readable YAML as the source of truth

2. TypeScript Compilation: tsc

   • Compiles TypeScript source code to JavaScript
   • Outputs to the /prod directory
   • Uses configuration from tsconfig.json (extends @nhcarrigan/typescript-config)

To run the complete build:

pnpm run build

2.8. Available Scripts

# Build (YAML → JSON + TypeScript compilation)
pnpm run build

# Start production server (with 1Password environment variables)
pnpm run start

# Run test suite
pnpm test

# Linting
pnpm run lint          # Run all linters
pnpm run lint:ts       # ESLint only
pnpm run lint:spelling # Spell check YAML files
pnpm run lint:yaml     # YAML validation

2.9. Testing

The Application includes comprehensive test coverage using Vitest.

Test File: test/yaml.spec.ts

What is tested:

• Data structure validation against TypeScript interfaces
• URL reachability for all external links (with retry logic)
• Date format validation
• Required field presence
• Data type correctness

Running tests:

pnpm test

Tests have a 120-second timeout to accommodate URL validation checks, which may be rate-limited.

2.10. Development Workflow

1. Make changes to data:

   • Edit YAML files in /data directory
   • Use your preferred text editor

2. Validate your changes:

   pnpm run lint:yaml     # Validate YAML syntax
   pnpm run lint:spelling # Check spelling
   pnpm test              # Run full test suite
   

3. Build the application:

   pnpm run build
   

4. Start the server locally:

   node ./prod/index.js
   

   Or with 1Password environment variables:

   pnpm run start
   

5. Test endpoints:

   curl http://localhost:9999/projects.json
   

2.11. Deployment

Environment Variables

The production start script uses 1Password CLI for secure environment variable management:

op run --env-file ./prod.env -- node ./prod/index.js

Create a prod.env file with 1Password references for any required environment variables.

Production Considerations

1. Reverse Proxy: Consider running the Application behind a reverse proxy (nginx, Apache) for:

   • SSL/TLS termination
   • Rate limiting
   • Caching
   • Load balancing

2. Process Management: Use a process manager to ensure uptime:

   • PM2
   • systemd
   • Docker with restart policies

3. Monitoring: Implement monitoring for:

   • Server health
   • Response times
   • Error rates

4. Backups: Regularly backup the /data directory, as it contains the source YAML files.

2.12. Troubleshooting

Server won't start

• Verify port 9999 is not in use
• Check that dependencies are installed (pnpm install)
• Ensure the build completed successfully (pnpm run build)

Data not updating

• Verify you edited the YAML files, not JSON files
• Run pnpm run build to regenerate JSON
• Restart the server

Test failures

• URL validation failures: May be due to rate limiting; tests include retry logic
• Data structure failures: Ensure YAML structure matches TypeScript interfaces
• Spelling failures: Add custom words to cspell configuration if needed

3. Legal Documentation

This section is for expansions to our legal policies specific to the Application.

3.1. Data Usage and Privacy

Data Served by the Application

The Application serves public, non-sensitive data intended for public consumption. The data categories include:

• Projects: Public information about open-source projects and portfolio work
• Resume: Professional experience, education, and certifications
• Testimonials: Public testimonials and recommendations
• Donation Information: Public donation platform links
• Funding Information: Public funding channel and subscription information

No Personal User Data Collection

The Application itself does not collect, store, or process any personal data from users accessing the API. The server:

• Does not require authentication
• Does not track user requests
• Does not use cookies or similar tracking technologies
• Does not log IP addresses or personally identifiable information

CORS and Third-Party Access

By enabling CORS for all origins, the Application explicitly permits third-party websites and applications to access the served data. This is intentional and by design, as the data is meant for public consumption.

3.2. Data Accuracy and Updates

The data served by the Application is maintained on a best-effort basis. While efforts are made to ensure accuracy:

• Data may become outdated between updates
• URLs and external links may become invalid over time
• No guarantee of real-time data freshness is provided

Users consuming this data should:

• Implement appropriate error handling for unavailable resources
• Cache responses responsibly
• Not rely on the Application for time-sensitive or critical decisions

3.3. Service Availability

The Application is provided "as-is" without guarantees of:

• Continuous availability
• Specific uptime percentages
• Response time performance
• Data freshness

Users should not build critical systems that depend on the Application's availability without implementing appropriate fallbacks and error handling.

3.4. Intellectual Property

Data Content

The data served by the Application contains information about projects, professional experience, and other content that may be subject to intellectual property rights:

• Project names, descriptions, and logos may be trademarks
• Code and software referenced may be under various open-source licences
• Testimonial content remains the intellectual property of the original authors

Users consuming this data should:

• Respect intellectual property rights
• Provide proper attribution when displaying data
• Review specific licences for any referenced projects or software

Application Code

The Application's source code is licensed under the terms specified in the LICENSE.md file. Copyright is held by Naomi Carrigan.

3.5. Terms of Service

The general Terms of Service can be found at: https://docs.nhcarrigan.com/#/terms

Application-Specific Terms

Users of the Application agree to:

• Use the API in good faith and not for malicious purposes
• Not attempt to overwhelm the server with excessive requests
• Not attempt to exploit vulnerabilities or security weaknesses
• Respect the intended public use of the data

3.6. Privacy Policy

The general Privacy Policy can be found at: https://docs.nhcarrigan.com/#/privacy

Application-Specific Privacy Notes

As noted in section 3.1, the Application does not collect user data. The privacy policy primarily applies to:

• Contributors to the codebase (governed by GitHub's privacy policy)
• Data maintainers with access to the data repository

4. Contributing Documentation

This section is for documentation related to contributing to the Application's codebase.

4.1. General Contributing Guidelines

General contributing guidelines can be found at: https://docs.nhcarrigan.com/#/contributing

Please review these guidelines before contributing to the Application.

4.2. Code of Conduct

Before interacting with the project or community, please read the Code of Conduct.

4.3. Ways to Contribute

4.3.1. Data Contributions

If you have corrections or updates to the data served by the Application:

1. Data corrections: If you notice incorrect information, please open an issue
2. Testimonials: If you'd like to add a testimonial, please contact through the official channels
3. Project updates: For updates to project information, verify you have permission to modify the data

4.3.2. Code Contributions

Contributors can help improve the Application in several ways:

Features:

• Enhanced API endpoints
• New data validation rules
• Performance optimisations
• Additional data format support
• API documentation improvements

Bug Fixes:

• Server stability issues
• Route handling bugs
• Build process errors
• Test failures

Quality Improvements:

• Code refactoring
• Test coverage expansion
• Documentation enhancements
• Type safety improvements

4.4. Development Setup

1. Fork and clone the repository
2. Install dependencies: pnpm install
3. Create a branch for your changes: git checkout -b feature/your-feature-name
4. Make your changes following the guidelines below
5. Test your changes: pnpm test
6. Lint your code: pnpm run lint
7. Build the project: pnpm run build
8. Commit and push your changes
9. Open a Pull Request

4.5. Code Style Guidelines

TypeScript

• Follow the ESLint configuration (@nhcarrigan/eslint-config)
• Use explicit type annotations for function parameters and return values
• Prefer interfaces over type aliases for object shapes
• Use ES modules (import/export) syntax
• Maintain strict type safety (no any types unless absolutely necessary)

Example:

// Good
export const processData = (input: string): ProcessedData => {
  // implementation
};

// Avoid
export const processData = (input) => {
  // implementation
};

File Organization

• Place interfaces in src/interfaces/
• Place utility functions in src/utils/
• Place core logic in src/modules/
• Keep the main entry point (src/index.ts) minimal

Error Handling

• Use the custom logger from @nhcarrigan/logger
• Provide meaningful error messages
• Handle errors gracefully without crashing the server

4.6. Testing Guidelines

All contributions should maintain or improve test coverage.

Writing Tests

• Use Vitest test framework
• Place tests in the /test directory
• Use descriptive test names

Example test structure:

describe("Feature Name", () => {
  it("should do something specific", () => {
    // Arrange
    const input = setupTestData();

    // Act
    const result = performAction(input);

    // Assert
    expect(result).toBe(expectedValue);
  });
});

Running Tests

Before submitting a PR:

pnpm test          # Run all tests

4.7. Documentation Guidelines

When contributing documentation:

• Use clear, concise language
• Include code examples where helpful
• Update relevant sections when changing functionality
• Check spelling with pnpm run lint:spelling

4.8. Pull Request Process

1. Ensure all checks pass:

   • All tests pass (pnpm test)
   • Code lints without warnings (pnpm run lint)
   • Build succeeds (pnpm run build)

2. Write a clear PR description:

   • Describe what changes were made
   • Explain why the changes were necessary
   • Reference any related issues

3. Respond to feedback:

   • Address review comments promptly
   • Be open to suggestions and improvements
   • Update the PR as requested

4. Squash commits if requested to maintain a clean git history

4.9. Adding New Data Categories

To add a new data category to the Application:

1. Create the TypeScript interface:

   // src/interfaces/newcategory.ts
   export interface NewCategory {
     id: string;
     name: string;
     // other fields
   }
   

2. Create the YAML data file:

   # data/newcategory.yml
   - id: example
     name: Example Item
   

3. Add tests:

   // test/yaml.spec.ts
   describe("New Category Data", () => {
     it("should have valid structure", () => {
       // validation tests
     });
   });
   

4. Update documentation:

   • Add endpoint documentation to section 1.2
   • Update project structure in section 2.4

5. Test the changes:

   pnpm run build
   node ./prod/index.js
   curl http://localhost:9999/newcategory.json
   

4.10. Security Considerations

When contributing, keep security in mind:

• Never commit sensitive data (API keys, passwords, personal information)
• Validate all input data in tests
• Follow secure coding practices
• Report security vulnerabilities privately before opening public issues

4.11. Getting Help

If you need help contributing:

• Post in the Bug Reports or Feature Requests categories on our support forum
• Join the Chat Server
• Email: contact@nhcarrigan.com

4.12. Recognition

All contributors will be recognised for their contributions. Significant contributions may be highlighted in release notes and project documentation.

---

Application Version: 1.0.0 Maintained by: NHCarrigan