VidEngine
Someone uploads a 200MB video. Handle that inside the request that received it and the request times out, the connection drops, and the user loses the file and their patience. So the upload finishes as fast as the bytes arrive, the request returns an identifier immediately, and the work happens on a queue. What actually took the time was failure: a worker dies mid-transcode and you need to know whether to retry, how many times, and how to stop two workers picking up the same job and both writing output. None of that shows in a demo, and all of it is what separates a pipeline from a script.
Uploads go straight to storage and processing runs on a queue, so the request that accepts a large file returns in milliseconds instead of holding a connection open. Transcoding to several qualities happens afterwards, and the client polls rather than waits.
- Started
- 2025
- Stack
- Node.js, Express.js, JWT, OAuth, Cloudinary, MongoDB
- Kind
- Product
- Source
- Public
- Async
- Upload pipeline
- OAuth
- Social login
- Multi-format
- Transcoding
How it evolved
- 2025Scoped
Worked out what actually needed building before writing any code.
- 2025Built
Built and shipped the core of it.
- 2025Live
Shipped and still improving it.
Built by Syed Tasavour. More at syedtasavour.com.