IngestIQ
Getting StartedCore ConceptsData ConnectorsAI & ProcessingIntegrationsAPI ReferenceSelf-Hosting

Development

Local development setup and contribution guidelines

Development Setup#

Prerequisites#

  • Node.js 18 or later (check with node --version)
  • Docker & Docker Compose
  • PostgreSQL client (optional, for direct DB access)

Clone and Install#

git clone https://github.com/avesta-hq/ingestiq-backend.git
cd ingestiq-backend
npm install

Environment Configuration#

cp .env.example .env

Configure your local environment variables. See Configuration Reference for all options.

Start Development Services#

# Start infrastructure (DB, Redis, NATS, MinIO)
docker compose up -d

# Run migrations
npm run db:migrate

# Seed data
npm run db:seed

# Start dev server with hot reload
npm run dev

Project Structure#

ingestiq-backend/
├── src/
│   ├── application/      # Use Cases (business logic)
│   ├── domain/           # Entities & interfaces
│   ├── infrastructure/   # Database & external services
│   ├── presentation/     # Controllers & routes
│   ├── common/           # Shared utilities
│   ├── mcp/              # MCP Server
│   ├── nats-events/      # Event handlers
│   └── scheduler/        # Job scheduling
├── tests/                # Test suites
├── docs/                 # Documentation
└── docker-compose.yml    # Local services

Available Scripts#

ScriptDescription
npm run devStart development server with hot reload
npm run buildBuild for production
npm startStart production server
npm testRun test suite
npm run test:watchRun tests in watch mode
npm run test:coverageRun tests with coverage report
npm run lintRun ESLint
npm run lint:fixFix linting issues
npm run db:migrateRun database migrations
npm run db:migrate:undoRollback last migration
npm run db:seedRun database seeders

Database Migrations#

Create a New Migration#

npx sequelize-cli migration:generate --name your-migration-name

Run Migrations#

# Run all pending migrations
npm run db:migrate

# Undo last migration
npm run db:migrate:undo

# Undo all migrations
npm run db:migrate:undo:all

Testing#

Run All Tests#

npm test

Run Specific Tests#

# Run tests matching a pattern
npm test -- --testPathPattern="knowledgebase"

# Run a single test file
npm test -- tests/unit/useCases/knowledgebase.test.ts

Test Coverage#

npm run test:coverage

Coverage reports are generated in the coverage/ directory.

Code Style#

The project uses ESLint and Prettier for consistent code style.

# Check for issues
npm run lint

# Auto-fix issues
npm run lint:fix

Commit Hooks#

Husky is configured to run linting before commits. Commits that don't pass linting will be rejected.

Contributing#

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Make your changes
  4. Run tests (npm test)
  5. Commit your changes (git commit -m 'Add amazing feature')
  6. Push to the branch (git push origin feature/amazing-feature)
  7. Open a Pull Request

See CONTRIBUTING.md for detailed guidelines.

Debugging#

VS Code Launch Configuration#

Add to .vscode/launch.json:

{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "node",
      "request": "launch",
      "name": "Debug Server",
      "runtimeExecutable": "npm",
      "runtimeArgs": ["run", "dev"],
      "console": "integratedTerminal",
      "skipFiles": ["<node_internals>/**"]
    }
  ]
}

Debugging Database Queries#

Enable query logging in development:

DB_LOGGING=true

NATS Event Debugging#

View NATS messages:

docker exec -it nats nats sub ">"
Documentation