Uniform Sampler¶
- class opacus.utils.uniform_sampler.DistributedUniformWithReplacementSampler(*, total_size, sample_rate, shuffle=True, shuffle_seed=0, steps=None, generator=None)[source]¶
Distributed batch sampler.
- Each batch is sampled as follows:
Shuffle the dataset (enabled by default)
Split the dataset among the replicas into chunks of equal size (plus or minus one sample)
Each replica selects each sample of its chunk independently with probability sample_rate
Each replica outputs the selected samples, which form a local batch
The sum of the lengths of the local batches follows a Poisson distribution. In particular, the expected length of each local batch is: sample_rate * total_size / num_replicas
- Parameters:
total_size (
int
) – total number of samples to sample fromsample_rate (
float
) – number of samples to draw.shuffle (
bool
) – Flag indicating whether apply shuffle when dividing elements between workersshuffle_seed (
int
) – Random seed used to shuffle when dividing elements across workersgenerator – torch.Generator() object used as a source of randomness when selecting items for the next round on a given worker
- class opacus.utils.uniform_sampler.UniformWithReplacementSampler(*, num_samples, sample_rate, generator=None, steps=None)[source]¶
This sampler samples elements according to the Sampled Gaussian Mechanism. Each sample is selected with a probability equal to
sample_rate
. The sampler generatessteps
number of batches, that defaults to 1/sample_rate
.