Avik Mukherjee
I build systems that have to survive reality. I like the problems where the easy implementation works at 10 users and falls apart at 10,000.
Currently building AI and security infrastructure at SuperAlign, in Hyderabad: ingestion for large endpoint fleets, the pipelines behind it, and the data layer underneath.
On my own time I ship things I wanted to exist: mindstate pgxray markdown-to-video. And I rebuild systems to understand them: job queues, a Kubernetes control plane, erasure coding. Then I write down what broke.
If that is a lot of words, there is a thirty-second version.
Focus
- Distributed systems
- Queues, workers, transactional boundaries, idempotency, partitioning, backpressure, failure recovery. Lately that has meant moving report generation onto async jobs, instrumenting it end to end, and fanning results out over WebSockets instead of holding a request open.
- AI infrastructure
- Asset discovery and inventory, risk assessment, telemetry pipelines, processing large volumes of endpoint data, and the extension framework that delivers AI prompt transcripts into a customer's own warehouse or SIEM.
- Security
- Cross-platform scanners, a browser extension that blocks sensitive data before it reaches ChatGPT, Claude, Gemini or Perplexity, and the MDM/GPO flow that puts it on 1,000+ devices. Also in-house auth (RS256 JWTs, session revocation, OIDC login), so self-hosted customers need no external identity provider.
- Data systems
- PostgreSQL, ClickHouse, DuckDB, Parquet on object storage, and a CEF stream that found 5x the throughput on a fraction of the memory once PyFlink came out of it. The interesting part is deciding where data lives and how much work happens synchronously.
Experience
Résumé (PDF, opens a download)I own the Go services behind a multi-tenant AI governance platform: asset inventory, auth, dashboard APIs, scanner policy distribution, plus the Surface backend end to end and the scanner feeding it. I also lead the cloud extension framework: per-org extension processes and prompt sinks that deliver AI transcripts to a customer's own S3 or Splunk, which took listing from 11s to 1.5s. Most of the work is taking expensive, failure-prone computation off the critical path: idempotent ingestion at 10K assets/second as the fleet grew from 200 to 1,000+ devices, partition-aware workers, Postgres locking semantics, and a memory-bounded DuckDB pipeline that turns ~70GB of firewall logs into a report in under 4 minutes.
Azure Data Factory pipelines with parameterised queries and Blob Storage, CSV-to-RDF conversion, and the frontend half of role-based access control.
Built most of a React Native app on Expo and cut page load times by 40% through API integration work.
Redux state management and Cypress automation, which took manual testing down by half.
Projects
202615
202514
20241
20231
Writing
All 37 posts →Approach
Generating code got cheap. Deciding whether it is correct did not. So I spend my attention on the parts a model cannot hand me.
Start with constraints. Latency, throughput, consistency, durability, cost, and what happens when it breaks. A model will build the wrong thing very quickly if nobody has said what right means.
Prefer the simplest architecture that survives them. Complexity is not sophistication. If Postgres is enough, use Postgres. If a queue is necessary, be able to say why out loud.
Measure before optimizing. A benchmark beats an opinion, and it beats a confident paragraph from a model.
Design for failure. Networks fail, workers die, requests retry, payloads duplicate, deploys go wrong. Generated code almost always assumes the happy path.
Understand the abstraction. I use high-level tools constantly, but I want to know enough of what sits underneath to debug it when it leaks. That is the skill that survives the tooling changing.
Connect
If you're working on hard problems in infrastructure, AI systems, security or developer tooling, I'd like to hear about it.
Find me on GitHub, X, LinkedIn or Peerlist, or just email me.