Crate autodep

source ·

Modules

  • Autodep configuration – can eventually be lazy_static parsed from a config file
  • The manager (worker manager, resource allocator, allocator) is responsible for interfacing with a set of workers. The manager starts and stops workers, and forwards inference requests
  • The worker’s RPC server
  • Code for loading and running (trained) PyTorch models
  • Network utility functions
  • An inference worker listens for requests from the Manager and computes model inference in an isolated environment