Get up and running quickly with Truss’ developer workflow
We built Truss because we were frustrated with the long feedback loops in ML model deployment. When you have to wait for your server to rebuild every time you make a change, iteration is painful.Meanwhile, web developers have enjoyed live reload workflows for years, where changes are patched onto a running server and available almost instantly.With Truss, you get the same live reload workflow for serving ML models.
A Truss is an abstraction for a model server. But more literally, it’s a set of files. Running truss init creates those files in a target directory.To package a model with Truss, follow the quickstart, a step-by-step example, or an example from GitHub.See the CLI reference for more information on truss init.
A model server takes requests, passes them through an ML model, and returns the model’s output. When you’re ready to start testing your Truss, use truss push to spin up a model server with your model and config.See the CLI reference for more information on truss init.
Run the truss watch command in a new terminal tab in the same working directory, as you’ll need to leave it running while you work.
truss watch
When you make a change with truss watch running, it will automatically attempt to patch that change onto the model server. Most changes to model.py and config.yaml can be patched.
The following changes should not be made in a live reload workflow:
Updates to resources in config.yaml, which must be set before the first truss push
Changes to the model_name in config.yaml. Changing the model name requires a new truss push to create a new model server.
See the CLI reference for more information on truss watch.
