Fast Gradient Clipping Utils¶

class opacus.utils.fast_gradient_clipping_utils.DPLossFastGradientClipping(module, optimizer, criterion, loss_reduction='mean')[source]¶

Wrapper on the loss function to be used with Fast Gradient and Ghost Clipping. It computes the per-sample loss, and wraps it in DPTensorFastGradientClipping.

class opacus.utils.fast_gradient_clipping_utils.DPTensorFastGradientClipping(module, optimizer, loss_per_sample, loss_reduction='mean')[source]¶

Packages the training loop for Fast Gradient and Ghost Clipping into loss.backward().

Parameters:

• module (GradSampleModuleFastGradientClipping) – the module to train

• optimizer (DPOptimizerFastGradientClipping) – the optimizer used to train the module

• loss_per_sample (Tensor) – loss on each sample in the mini-batch of size [batch_size, 1]

backward()[source]¶

Repurposes loss.backward() to perform two backward passes, as well as the loss rescaling and hook operations in between