pub struct Worker { /* private fields */ }
Expand description
A worker runs as a separate process, spawned by the resource manager. A worker runs an RPC server listening for requests to compute inference on its own local copy of the model
Implementations§
Trait Implementations§
source§impl Worker for Worker
impl Worker for Worker
source§fn compute_inference<'life0, 'async_trait>(
&'life0 self,
request: Request<InferenceTask>
) -> Pin<Box<dyn Future<Output = Result<Response<Inference>, Status>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
fn compute_inference<'life0, 'async_trait>( &'life0 self, request: Request<InferenceTask> ) -> Pin<Box<dyn Future<Output = Result<Response<Inference>, Status>> + Send + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait,
Handle requests for inference, and compute inference on this worker
fn get_stats<'life0, 'async_trait>( &'life0 self, _req: Request<Empty> ) -> Pin<Box<dyn Future<Output = Result<Response<Stats>, Status>> + Send + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait,
Auto Trait Implementations§
impl RefUnwindSafe for Worker
impl Send for Worker
impl Sync for Worker
impl Unpin for Worker
impl UnwindSafe for Worker
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
source§impl<T> Instrument for T
impl<T> Instrument for T
source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
Wrap the input message
T
in a tonic::Request