Modules
- Autodep configuration – can eventually be lazy_static parsed from a config file
- The manager (worker manager, resource allocator, allocator) is responsible for interfacing with a set of workers. The manager starts and stops workers, and forwards inference requests
- The worker’s RPC server
- Code for loading and running (trained) PyTorch models
- Network utility functions
- An inference worker listens for requests from the
Manager
and computes model inference in an isolated environment