Backend Engineer - AI Gateway

Client: Korean IT Services and IT Consulting

Working Location: NJ (initial), then Plano, TX

Employment Type: 1-year contract / W-2

About the Role

We are building an AI Gateway — a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale.

Qualifications & Skills

• Async Python & Network Programming (3+ years) – Strong hands-on experience with asyncio, aiohttp/anyio, low-level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection-pooling, keep-alive, and zero-copy buffering
• Streaming & Large-Payload Handling – Ability to manage chunked transfer, bidirectional streaming, back-pressure and high-throughput data paths for real-time LLM inference
• Protocol Knowledge – Working familiarity with HTTP/2 & HTTP/3 (multiplexing, HPACK), gRPC, and occasional custom protocol parsers
• FastAPI-Based Backend (3+ years) – Design, develop and test production-grade REST/Streaming APIs; pragmatic use of Pydantic, dependency injection and OpenAPI docs
• Containerization & Kubernetes – Docker, AKS/EKS/GKE deployment experience; HPA/VPA scaling, rolling updates, and service-mesh (e.g., Istio/Envoy) basics
• Async Messaging & Event-Driven Design – Production use of Redis Streams, RabbitMQ or Kafka for decoupled task queues, retries and back-off logic
• PostgreSQL & Vault – Advanced query tuning, partitioning, PgBouncer pooling, plus HashiCorp Vault (dynamic secrets, transit encryption) for key management
• Cloud Networking & Security – L4/L7 load balancing, TLS termination, reverse-proxy (NGINX/Envoy), mTLS between services, and automated cert lifecycle (ACME/Let’s Encrypt)
• IaC & CI/CD – Terraform modules for multi-env infra, plus GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue-green deployments

Preferred Qualifications

• Enterprise Security Gateways – Integration with CASB/DLP/WSS or similar edge-security platforms; centralized logging & audit trails
• IAM & SSO – Configuring Azure AD/Entra ID, Okta, SAML 2.0 or OIDC SSO flows; managing Conditional Access & SCIM provisioning
• AI/LLM Pipelines – Building prompt-engineering workflows, PII anonymization, and monitoring Azure OpenAI (or comparable) usage, rate limits & cost
• API-Gateway Operations – Custom webhook/REST integrations, circuit-breaker patterns, rate-limiting, auto-retry and health-checking
• Observability & FinOps – End-to-end monitoring with Datadog, Prometheus/Grafana or Azure Monitor; proactive cost-optimization and 24/7 incident response (SLA/SLO)

Candidate Profile

A proactive, detail-oriented professional with strong communication, ownership, and problem-solving skills who can collaborate effectively across business and technical teams.

Pay

$100,000 – $130,000/year (DOE)

Benefits

• Insurance
• 401K

Schedule

9 AM to 6 PM, onsite

Language: English

Visa sponsorship: Not provided