Opportunity brief
About Marvedge:
Marvedge is a SaaS video product for creating interactive, trackable product demos - screen recordings with branding, watermarking, Al voiceover, subtitles, CTAs, and shareable QR codes, used by SaaS teams to convert prospects. We're now building V3: an engine that takes a customer's long-form video (webinars, product tours, podcasts) and automatically repurposes it into short, vertical, Al-voiced clips with embedded lead capture - turning passive content into a distribution and pipeline channel.
You'd be joining pre-team - there is no existing dev org yet. You will be one of the first engineers building this from scratch, working directly with the CTO.
What you'll actually build:
This is not a research role. The job is to get real AI/ML capability shipped into a production Next.js/TypeScript app in weeks, not months - which means integrating and orchestrating existing models and hosted-inference APIs, not training models from scratch or vendoring unmaintained academic repos.
Concretely, you will own:
- Transcription & virality scoring - run long-form source video through Whisper (word-level timestamps) and build an LLM-based scoring chain (OpenAl) that ranks candidate 15—60s segments by "viral potential," producing hook titles and start/end timestamps. (You'll be extending a pattern that already exists in our codebase for Al script tone-rewriting - you'll study and reuse it, not invent it from zero.) - Vertical reframing pipeline - integrate hosted inference for face/saliency detection and active-speaker detection (via vendor APIs such as Sieve/Replicate-style hosted models, not by trying to build/train MediaPipe AutoFlip or TalkNet-ASD in-house - both are unmaintained/academic and out of scope for our timeline) to drive 16:9 9:16 cropping and dual-speaker split-screen logic. - Kinetic subtitle generation - generate Advanced SubStation Alpha ( . ass ) caption files with word-level karaoke highlighting, burned in via ffmpeg/libass. We already burn stabilized captions for another feature; you'll extend that approach for short-form. - Al dubbing pipeline - integrate the Elevenl_abs Dubbing API (translation + voice cloning) for multi-lingual shorts, and reuse/adapt our existing audio-visual pacing-alignment logic (time- stretch vs. freeze-frame padding) that keeps dubbed audio in sync with video. - Pipeline orchestration - wire all of the above into background jobs (queue-based, on our existing Cloud Run/Node worker), with clear stage-by-stage state so failures at any step degrade gracefully instead of breaking the whole export.
What we need from you:
Must-have
- Strong Python fundamentals - comfortable writing production-quality (not notebook-only) code. - Working knowledge of how modern ML APIs are consumed in practice: REST/API integration, handling async jobs, webhooks, polling, rate limits, retries. - Basic understanding of audio/video concepts: frames, timestamps, sample rate, codecs - you don't need to be a video engineer, but you shouldn't be starting from zero. - Comfortable reading model/APl docs and being the person who evaluates "does this vendor's output quality actually work for our use case," not just "does the API call succeed. - Currently in 3rd or 4th year of a CS/related degree.
Strongly preferred
- Exposure to ffmpeg (even basic CLI usage - trimming, cropping, concatenation).
- Familiarity with any of: Whisper/speech-to-text APIs, TTS APIs (ElevenLabs, Deepgram, etc.),
computer vision basics (face detection, OpenCV). - Some exposure to LLM prompting/chains (OpenA API, LangChain or similar) - you'll be designing prompts that produce structured, reliable output (e.g., scored clip segments as JSON), not freeform chat. - Any past project (hackathon, coursework, personal) involving video/audio processing or applied ML - even a rough one. We care more about a working demo than a polished GitHub README.
Good fit if you
- Prefer "ship a working VI with a hosted APP' over "spend three weeks trying to get an academic repo to run." - Are comfortable with ambiguity - specs will be directional, not exhaustive, and you'll be expected to flag when something in a plan isn't actually feasible in the timeline rather than silently struggling with it. - Want ownership: as an early intern here, code you write in your first month can plausibly still be running in production in six. - Can communicate technical trade-offs clearly (e.g., "this vendor is cheaper but adds 2s of latency per clip") - you'll be making some of these calls, not just executing a spec.
What this role is explicitly NOT
- Not a "build a novel active-speaker-detection model" research role. We buy or wrap that capability; we don't build it in-house at this stage.
- Not a data science/analytics role - no dashboards-and-Jupyter work.
- Not purely backend - you'll write integration code, not deep infra, though you'll collaborate
closely with whoever owns the backend/queue layer