rachpradhan

Hey HN,

I’ve been using Zod for TypeScript validation, but I kept running into performance bottlenecks in high-throughput applications. So I built DHI – a WebAssembly-powered validation library that offers a familiar Zod-like API while being significantly faster.

Why I Built This:

• Zod is great, but it slows down at scale, especially with large datasets.

• DHI uses WebAssembly to execute validations at near-native speeds.

• It’s TypeScript-first and async-friendly, with built-in support for batch validation.

• I love to build things and this was just a passion project that I did in a day

Benchmarks (for 1,000,000 validations):

Library | Time Taken (ms) | Validations per sec

DHI | 3010.11 | 332,214 Zod | 5679.42 | 176,074

Features:

• Familiar API inspired by Zod.

• WebAssembly-powered for ultra-fast validation.

• Supports batch validation for high-throughput processing.

• Minimal memory overhead.

• Supports all major TypeScript types.

Example Usage:

import { dhi } from ‘dhi’;

const UserSchema = await dhi.object({ name: dhi.string(), age: dhi.number(), email: dhi.string() });

const result = UserSchema.validate({ name: “John”, age: 30, email: “john@example.com” });

console.log(result.success); // true or false

Would love your thoughts! Is this something you’d use in your projects? What’s been your biggest frustration with TypeScript validation?

There is also a live website to see the speed difference for yourself: https://dhi.trilok.ai

I built Kew after two years of building AI applications with FastAPI. Kept running into the same problems: LLM calls hogging threads, Celery workers being hard to configure properly, and RQ requiring complex process management. Traditional task queues weren't designed for the async world of AI applications where you need precise control over concurrent API calls.

Kew (https://github.com/justrach/kew) solves this by running directly in your FastAPI process. I'm using it in production for my AI apps where it handles thousands of LLM API calls daily with proper concurrency control.

Technical details: - Runs directly in your FastAPI process using asyncio - Uses Redis for persistence - True concurrency control with semaphores (crucial for managing expensive AI API calls) - Circuit breakers for handling API timeouts and failures - Millisecond-precision task scheduling

Simple example:

  # Create a queue with strict concurrency control
  await manager.create_queue(QueueConfig(
    name="llm_tasks",
    max_workers=4  # Strictly enforced for API rate limits
  ))

  # Use it in your FastAPI endpoint
  @app.post("/generate")
  async def generate_text(prompt: str):
    await manager.submit_task(
      queue_name="llm_tasks",
      task_func=call_llm_api,
      prompt=prompt
    )

Why: Current solutions (Celery/RQ/Huey) require separate worker processes which add complexity to FastAPI deployments. They weren't designed for async frameworks and require sync/async context switching. This becomes especially painful when dealing with AI API calls where you need precise control over concurrency.

This is running in production on several of my AI applications. It's particularly good at handling concurrent LLM API calls where you need to respect rate limits while maintaining high throughput.

The code is MIT licensed and available at https://github.com/justrach/kew.

Would appreciate feedback especially on the concurrency control implementation. Also curious to hear from others building AI apps about their task queue patterns.