Software Engineer

Job Details:Job Title: Software Engineer - LLM Personalization BenchmarkingJob Duration: 9-12 month contract (Possibility for extension)Job Location: Austin TX – RemoteJob Summary

Create a framework for managing jobs on a combination of on-premises and external cloud compute, where a job is allocated a compute node to load an LLM, retrieve queries from remote storage, and generate responses from the LLM given the queries. The framework must be robust and include fault tolerance so that jobs can be restarted/recovered upon failure and child nodes are brought down cleanly upon success.

Required skills include:

* Experience working with Python

* Experience working with LLM inference libraries (e.g. vLLM, transformers, or nemotron)

* Experience with Kubernetes, including managing tasks given available infrastructure, and coordinating containers

* Experience building robust distributed applications that allow graceful failure and recovery

* Experience working with AWS/GCP cloud platforms

* Experience using basic git functionality, including branching, pull requests, reviews, and merging

* Experience writing unit tests

* Excellent collaboration skills are required

Bonus skills include:

* Experience with PyTorch

* Experience with API design

EEO:“Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of – Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans.”