How to fit a population receptive field model to simulated data¶
Author: Malte Lüken (m.luken@esciencecenter.nl)
Difficulty: Beginner
This tutorial explains how to fit a population receptive field (pRF) model to simulated data.
A pRF model maps neural activity in a region of interest in the brain (e.g., V1 in the human visual cortex) to an experimental stimulus (e.g., a bar moving through the visual field). Here, we use the visual domain as an example, where the part of the visual field that stimulates activity in the region of interest is the pRF.
Because prfmodel uses Keras for model fitting, we need to make sure that a backend is installed before we begin. In this tutorial, we use the TensorFlow backend.
import os
from importlib.util import find_spec
# Set keras backend to 'tensorflow' (this is normally the default)
os.environ["KERAS_BACKEND"] = "tensorflow"
# Hide tensorflow info messages
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "1"
if find_spec("tensorflow") is None:
msg = "Could not find the tensorflow package. Please install tensorflow with 'pip install .[tensorflow]'"
raise ImportError(msg)
Defining the stimulus¶
Let’s start with the first step: Defining the stimulus. In practice, we recommend that users save the stimulus they use in an experiment to a file and load it to avoid mismatches between experiment and analysis. Because we use simulated data in this tutorial, we load an example stimulus that is included in the package. The stimulus simulates a bar moving in different directions through a two-dimensional visual field.
from prfmodel.examples import load_2d_bar_stimulus
stimulus = load_2d_bar_stimulus()
print(stimulus)
Stimulus(design=array[200, 101, 101], grid=array[101, 101, 2], dimension_labels=['y', 'x'])
When printing the stimulus object, we can see that it has three attributes. The design attribute defines how
the visual field changes over time. It has shape (num_frames, width, height), where width and hight define the number of pixels at which the visual field is recorded. The grid attribute maps each pixel to its xy-coordinate in the visual field (i.e., the degree of visual angle).
We can visualize the stimulus using animate_2d_stimulus.
from IPython.display import HTML
from prfmodel.stimulus import animate_2d_stimulus
ani = animate_2d_stimulus(stimulus, interval=25) # Pause 25 ms between time frames
HTML(ani.to_html5_video())
Defining the pRF model¶
Now that we defined our stimulus, we can create a pRF model to predict a neural response to this stimulus in our (hypothetical) region of interest (e.g., V1). We use the most popular pRF model that is based on the seminal paper by Dumoulin and Wandell (2008): It assumes that the stimulus (our moving bar) elicits a response that follows a Gaussian shape in two-dimensional visual space. This response is then summed and convolved with an impulse response that follows the shape of the hemodynamic response in the brain. Finally, a baseline and amplitude parameter shift and scale our predicted response to the simulated (or observed) neural response.
The Gaussian2DPRFModel class performs all these steps to make a combined prediction.
from prfmodel.models.gaussian import Gaussian2DPRFModel
prf_model = Gaussian2DPRFModel()
To simulate a neural response to our stimulus with our Gaussian 2D pRF model, we need to define a set of parameters.
The list of parameters that need to be set to make model predictions can be obtained from the parameter_names property.
prf_model.parameter_names
['shift', 'rate', 'shape', 'baseline', 'sigma', 'amplitude', 'mu_x', 'mu_y']
The parameters mu_x, mu_y, and sigma define the location and size of the predicted Gaussian pRF and are of primary interest. We simulate a pRF with its center at (-2.1, 1.45) and a size of 1.35. The parameters shape, rate, and shift determine the impulse response that is convolved with our pRF response. The parameters baseline and amplitude shift and scale our convolved response, respectively. We store the parameter values in a pandas.DataFrame object.
import pandas as pd
true_params = pd.DataFrame(
{
"mu_x": [-2.1],
"mu_y": [1.45],
"sigma": [1.35],
"shape": [6.0],
"rate": [0.9],
"shift": [2.5],
"baseline": [10.0],
"amplitude": [1.2],
},
)
Using the “true” parameters, we simulate a response to our stimulus by making a prediction with our pRF model.
import matplotlib.pyplot as plt
simulated_response = prf_model(stimulus, true_params)
_ = plt.plot(simulated_response[0])
2026-01-21 16:42:33.947093: E external/local_xla/xla/stream_executor/cuda/cuda_platform.cc:51] failed call to cuInit: INTERNAL: CUDA error: Failed call to cuInit: UNKNOWN ERROR (303)
The predicted response contains increased activation followed by decreased activation compared to the baseline activity for each moving bar in our stimulus.
Fitting the pRF model¶
We will fit the pRF model to our simulated data using multiple stages. We begin with a grid search to find good
starting values for our parameters of interest (mu_x, mu_y, and sigma). Then, we use least squares to estimate the baseline and amplitude of
our model. Finally, we use stochastic gradient descent (SGD) to finetune our model fits.
Let’s start with the grid search by defining ranges of mu_x, mu_y, and sigma that we want to construct a grid
of parameter values from. For all other parameters, we only provide a single value so that they will stay constant
across the entire grid.
import numpy as np
param_ranges = {
"mu_x": np.linspace(-3.0, 3.0, 10),
"mu_y": np.linspace(-3.0, 3.0, 10),
"sigma": np.linspace(0.5, 3.0, 10),
"shape": [6.0],
"rate": [0.9],
"shift": [2.5],
"baseline": [0.0],
"amplitude": [1.0],
}
For all three parameters, we defined ranges of 10 values that will be used to construct the grid. That is, the grid search will evaluate all possible combinations of these values and return the combination that fits the simulated data best. This will result in a grid containing \(10 \times 10 \times 10 = 1000\) parameter combinations.
Let’s construct the GridFitter and perform the grid search. Note that we set chunk_size=20 to let the GridFitter
evaluate 20 parameter combinations at the same time (which saves us some memory).
from prfmodel.fitters.grid import GridFitter
grid_fitter = GridFitter(
model=prf_model,
stimulus=stimulus,
)
grid_history, grid_params = grid_fitter.fit(
data=simulated_response,
parameter_values=param_ranges,
chunk_size=20,
)
Processing parameter grid chunks: 0%| | 0/50 [00:00<?, ?it/s]
Processing parameter grid chunks: 0%| | 0/50 [00:00<?, ?it/s, loss=3.47e+3]
Processing parameter grid chunks: 0%| | 0/50 [00:00<?, ?it/s, loss=2.02e+3]
Processing parameter grid chunks: 4%|▍ | 2/50 [00:00<00:02, 19.10it/s, loss=2.02e+3]
Processing parameter grid chunks: 4%|▍ | 2/50 [00:00<00:02, 19.10it/s, loss=763]
Processing parameter grid chunks: 4%|▍ | 2/50 [00:00<00:02, 19.10it/s, loss=397]
Processing parameter grid chunks: 4%|▍ | 2/50 [00:00<00:02, 19.10it/s, loss=397]
Processing parameter grid chunks: 10%|█ | 5/50 [00:00<00:02, 15.64it/s, loss=397]
Processing parameter grid chunks: 10%|█ | 5/50 [00:00<00:02, 15.64it/s, loss=397]
Processing parameter grid chunks: 10%|█ | 5/50 [00:00<00:02, 15.64it/s, loss=397]
Processing parameter grid chunks: 10%|█ | 5/50 [00:00<00:02, 15.64it/s, loss=397]
Processing parameter grid chunks: 10%|█ | 5/50 [00:00<00:02, 15.64it/s, loss=161]
Processing parameter grid chunks: 10%|█ | 5/50 [00:00<00:02, 15.64it/s, loss=161]
Processing parameter grid chunks: 20%|██ | 10/50 [00:00<00:01, 25.98it/s, loss=161]
Processing parameter grid chunks: 20%|██ | 10/50 [00:00<00:01, 25.98it/s, loss=161]
Processing parameter grid chunks: 20%|██ | 10/50 [00:00<00:01, 25.98it/s, loss=161]
Processing parameter grid chunks: 20%|██ | 10/50 [00:00<00:01, 25.98it/s, loss=161]
Processing parameter grid chunks: 20%|██ | 10/50 [00:00<00:01, 25.98it/s, loss=161]
Processing parameter grid chunks: 20%|██ | 10/50 [00:00<00:01, 25.98it/s, loss=161]
Processing parameter grid chunks: 30%|███ | 15/50 [00:00<00:01, 32.21it/s, loss=161]
Processing parameter grid chunks: 30%|███ | 15/50 [00:00<00:01, 32.21it/s, loss=161]
Processing parameter grid chunks: 30%|███ | 15/50 [00:00<00:01, 32.21it/s, loss=161]
Processing parameter grid chunks: 30%|███ | 15/50 [00:00<00:01, 32.21it/s, loss=161]
Processing parameter grid chunks: 30%|███ | 15/50 [00:00<00:01, 32.21it/s, loss=161]
Processing parameter grid chunks: 30%|███ | 15/50 [00:00<00:01, 32.21it/s, loss=161]
Processing parameter grid chunks: 40%|████ | 20/50 [00:00<00:00, 35.96it/s, loss=161]
Processing parameter grid chunks: 40%|████ | 20/50 [00:00<00:00, 35.96it/s, loss=161]
Processing parameter grid chunks: 40%|████ | 20/50 [00:00<00:00, 35.96it/s, loss=161]
Processing parameter grid chunks: 40%|████ | 20/50 [00:00<00:00, 35.96it/s, loss=161]
Processing parameter grid chunks: 40%|████ | 20/50 [00:00<00:00, 35.96it/s, loss=161]
Processing parameter grid chunks: 40%|████ | 20/50 [00:00<00:00, 35.96it/s, loss=161]
Processing parameter grid chunks: 50%|█████ | 25/50 [00:00<00:00, 38.51it/s, loss=161]
Processing parameter grid chunks: 50%|█████ | 25/50 [00:00<00:00, 38.51it/s, loss=161]
Processing parameter grid chunks: 50%|█████ | 25/50 [00:00<00:00, 38.51it/s, loss=161]
Processing parameter grid chunks: 50%|█████ | 25/50 [00:00<00:00, 38.51it/s, loss=161]
Processing parameter grid chunks: 50%|█████ | 25/50 [00:00<00:00, 38.51it/s, loss=161]
Processing parameter grid chunks: 50%|█████ | 25/50 [00:00<00:00, 38.51it/s, loss=161]
Processing parameter grid chunks: 60%|██████ | 30/50 [00:00<00:00, 40.36it/s, loss=161]
Processing parameter grid chunks: 60%|██████ | 30/50 [00:00<00:00, 40.36it/s, loss=161]
Processing parameter grid chunks: 60%|██████ | 30/50 [00:00<00:00, 40.36it/s, loss=161]
Processing parameter grid chunks: 60%|██████ | 30/50 [00:00<00:00, 40.36it/s, loss=161]
Processing parameter grid chunks: 60%|██████ | 30/50 [00:00<00:00, 40.36it/s, loss=161]
Processing parameter grid chunks: 60%|██████ | 30/50 [00:00<00:00, 40.36it/s, loss=161]
Processing parameter grid chunks: 70%|███████ | 35/50 [00:00<00:00, 41.72it/s, loss=161]
Processing parameter grid chunks: 70%|███████ | 35/50 [00:01<00:00, 41.72it/s, loss=161]
Processing parameter grid chunks: 70%|███████ | 35/50 [00:01<00:00, 41.72it/s, loss=161]
Processing parameter grid chunks: 70%|███████ | 35/50 [00:01<00:00, 41.72it/s, loss=161]
Processing parameter grid chunks: 70%|███████ | 35/50 [00:01<00:00, 41.72it/s, loss=161]
Processing parameter grid chunks: 70%|███████ | 35/50 [00:01<00:00, 41.72it/s, loss=161]
Processing parameter grid chunks: 80%|████████ | 40/50 [00:01<00:00, 42.49it/s, loss=161]
Processing parameter grid chunks: 80%|████████ | 40/50 [00:01<00:00, 42.49it/s, loss=161]
Processing parameter grid chunks: 80%|████████ | 40/50 [00:01<00:00, 42.49it/s, loss=161]
Processing parameter grid chunks: 80%|████████ | 40/50 [00:01<00:00, 42.49it/s, loss=161]
Processing parameter grid chunks: 80%|████████ | 40/50 [00:01<00:00, 42.49it/s, loss=161]
Processing parameter grid chunks: 80%|████████ | 40/50 [00:01<00:00, 42.49it/s, loss=161]
Processing parameter grid chunks: 90%|█████████ | 45/50 [00:01<00:00, 42.91it/s, loss=161]
Processing parameter grid chunks: 90%|█████████ | 45/50 [00:01<00:00, 42.91it/s, loss=161]
Processing parameter grid chunks: 90%|█████████ | 45/50 [00:01<00:00, 42.91it/s, loss=161]
Processing parameter grid chunks: 90%|█████████ | 45/50 [00:01<00:00, 42.91it/s, loss=161]
Processing parameter grid chunks: 90%|█████████ | 45/50 [00:01<00:00, 42.91it/s, loss=161]
Processing parameter grid chunks: 90%|█████████ | 45/50 [00:01<00:00, 42.91it/s, loss=161]
Processing parameter grid chunks: 100%|██████████| 50/50 [00:01<00:00, 43.38it/s, loss=161]
Processing parameter grid chunks: 100%|██████████| 50/50 [00:01<00:00, 37.52it/s, loss=161]
grid_params
| mu_x | mu_y | sigma | shape | rate | shift | baseline | amplitude | |
|---|---|---|---|---|---|---|---|---|
| 0 | -2.333333 | 1.666667 | 1.611111 | 6.0 | 0.9 | 2.5 | 0.0 | 1.0 |
We can see that the estimates for mu_x, mu_y, and sigma are one combination in our grid. However, because the
grid did not contain the “true” parameters we used to simulate the original response, the estimates differ from the
“true” parameters.
Using the parameter estimates resulting from the grid search we can make model predictions and compare them against the original simulated response.
grid_pred_response = prf_model(stimulus, grid_params)
fig, ax = plt.subplots()
ax.plot(simulated_response[0], label="True")
ax.plot(grid_pred_response[0], label="Predicted (grid)")
fig.legend();
We can see that the predicted response follows the shape of the original (true) response but still shows some deviation in the amplitude of the activation peaks and the baseline activation.
Using least squares, we can estimate the baseline and amplitude parameters of our model.
from prfmodel.fitters.linear import LeastSquaresFitter
ls_fitter = LeastSquaresFitter(
model=prf_model,
stimulus=stimulus,
)
ls_history, ls_params = ls_fitter.fit(
data=simulated_response,
parameters=grid_params,
slope_name="amplitude", # Names of parameters to be optimized with least squares
intercept_name="baseline",
)
ls_params
| mu_x | mu_y | sigma | shape | rate | shift | baseline | amplitude | |
|---|---|---|---|---|---|---|---|---|
| 0 | -2.333333 | 1.666667 | 1.611111 | 6.0 | 0.9 | 2.5 | 10.00065 | 1.060192 |
Looking at the parameters, we can see that the model compensates the deviation in the peaks by adjusting the
baseline and amplitude parameters. We can also plot the predicted response.
ls_pred_response = prf_model(stimulus, ls_params)
fig, ax = plt.subplots()
ax.plot(simulated_response[0], label="True")
ax.plot(ls_pred_response[0], label="Predicted (least-squares)")
fig.legend();
To finetune our model fits, we use SGD to iteratively optimize model parameters using the gradient of a loss function that is computed between data and model predictions. The default loss function in prfmodel is the means squared error. As initial parameters, we use the result from the grid search and least squares fit. We fix the parameters related to the impulse response to their initial values (which are the “true” values).
from prfmodel.fitters.sgd import SGDFitter
sgd_fitter = SGDFitter(
model=prf_model,
stimulus=stimulus,
)
sgd_history, sgd_params = sgd_fitter.fit(
data=simulated_response,
init_parameters=ls_params,
fixed_parameters=["shape", "rate", "shift"],
)
0%| | 0/1000 [00:00<?, ?it/s]
2026-01-21 16:42:36.018112: E tensorflow/core/util/util.cc:131] oneDNN supports DT_INT32 only on platforms with AVX-512. Falling back to the default Eigen-based implementation if present.
0%| | 0/1000 [00:00<?, ?it/s, loss=46]
0%| | 0/1000 [00:00<?, ?it/s, loss=45.8]
0%| | 2/1000 [00:00<01:02, 16.00it/s, loss=45.8]
0%| | 2/1000 [00:00<01:02, 16.00it/s, loss=45.6]
0%| | 2/1000 [00:00<01:02, 16.00it/s, loss=45.4]
0%| | 2/1000 [00:00<01:02, 16.00it/s, loss=45.2]
0%| | 5/1000 [00:00<00:46, 21.30it/s, loss=45.2]
0%| | 5/1000 [00:00<00:46, 21.30it/s, loss=45.1]
0%| | 5/1000 [00:00<00:46, 21.30it/s, loss=44.9]
0%| | 5/1000 [00:00<00:46, 21.30it/s, loss=44.7]
1%| | 8/1000 [00:00<00:42, 23.29it/s, loss=44.7]
1%| | 8/1000 [00:00<00:42, 23.29it/s, loss=44.5]
1%| | 8/1000 [00:00<00:42, 23.29it/s, loss=44.3]
1%| | 8/1000 [00:00<00:42, 23.29it/s, loss=44.1]
1%| | 11/1000 [00:00<00:40, 24.34it/s, loss=44.1]
1%| | 11/1000 [00:00<00:40, 24.34it/s, loss=43.9]
1%| | 11/1000 [00:00<00:40, 24.34it/s, loss=43.7]
1%| | 11/1000 [00:00<00:40, 24.34it/s, loss=43.5]
1%|▏ | 14/1000 [00:00<00:39, 24.73it/s, loss=43.5]
1%|▏ | 14/1000 [00:00<00:39, 24.73it/s, loss=43.4]
1%|▏ | 14/1000 [00:00<00:39, 24.73it/s, loss=43.2]
1%|▏ | 14/1000 [00:00<00:39, 24.73it/s, loss=43]
2%|▏ | 17/1000 [00:00<00:39, 25.04it/s, loss=43]
2%|▏ | 17/1000 [00:00<00:39, 25.04it/s, loss=42.8]
2%|▏ | 17/1000 [00:00<00:39, 25.04it/s, loss=42.6]
2%|▏ | 17/1000 [00:00<00:39, 25.04it/s, loss=42.4]
2%|▏ | 20/1000 [00:00<00:38, 25.31it/s, loss=42.4]
2%|▏ | 20/1000 [00:00<00:38, 25.31it/s, loss=42.3]
2%|▏ | 20/1000 [00:00<00:38, 25.31it/s, loss=42.1]
2%|▏ | 20/1000 [00:00<00:38, 25.31it/s, loss=41.9]
2%|▏ | 23/1000 [00:00<00:38, 25.37it/s, loss=41.9]
2%|▏ | 23/1000 [00:00<00:38, 25.37it/s, loss=41.7]
2%|▏ | 23/1000 [00:01<00:38, 25.37it/s, loss=41.5]
2%|▏ | 23/1000 [00:01<00:38, 25.37it/s, loss=41.3]
3%|▎ | 26/1000 [00:01<00:38, 25.59it/s, loss=41.3]
3%|▎ | 26/1000 [00:01<00:38, 25.59it/s, loss=41.2]
3%|▎ | 26/1000 [00:01<00:38, 25.59it/s, loss=41]
3%|▎ | 26/1000 [00:01<00:38, 25.59it/s, loss=40.8]
3%|▎ | 29/1000 [00:01<00:37, 25.59it/s, loss=40.8]
3%|▎ | 29/1000 [00:01<00:37, 25.59it/s, loss=40.6]
3%|▎ | 29/1000 [00:01<00:37, 25.59it/s, loss=40.5]
3%|▎ | 29/1000 [00:01<00:37, 25.59it/s, loss=40.3]
3%|▎ | 32/1000 [00:01<00:37, 25.72it/s, loss=40.3]
3%|▎ | 32/1000 [00:01<00:37, 25.72it/s, loss=40.1]
3%|▎ | 32/1000 [00:01<00:37, 25.72it/s, loss=39.9]
3%|▎ | 32/1000 [00:01<00:37, 25.72it/s, loss=39.8]
4%|▎ | 35/1000 [00:01<00:37, 25.72it/s, loss=39.8]
4%|▎ | 35/1000 [00:01<00:37, 25.72it/s, loss=39.6]
4%|▎ | 35/1000 [00:01<00:37, 25.72it/s, loss=39.4]
4%|▎ | 35/1000 [00:01<00:37, 25.72it/s, loss=39.3]
4%|▍ | 38/1000 [00:01<00:37, 25.86it/s, loss=39.3]
4%|▍ | 38/1000 [00:01<00:37, 25.86it/s, loss=39.1]
4%|▍ | 38/1000 [00:01<00:37, 25.86it/s, loss=38.9]
4%|▍ | 38/1000 [00:01<00:37, 25.86it/s, loss=38.7]
4%|▍ | 41/1000 [00:01<00:37, 25.81it/s, loss=38.7]
4%|▍ | 41/1000 [00:01<00:37, 25.81it/s, loss=38.6]
4%|▍ | 41/1000 [00:01<00:37, 25.81it/s, loss=38.4]
4%|▍ | 41/1000 [00:01<00:37, 25.81it/s, loss=38.2]
4%|▍ | 44/1000 [00:01<00:36, 25.84it/s, loss=38.2]
4%|▍ | 44/1000 [00:01<00:36, 25.84it/s, loss=38.1]
4%|▍ | 44/1000 [00:01<00:36, 25.84it/s, loss=37.9]
4%|▍ | 44/1000 [00:01<00:36, 25.84it/s, loss=37.8]
5%|▍ | 47/1000 [00:01<00:36, 25.82it/s, loss=37.8]
5%|▍ | 47/1000 [00:01<00:36, 25.82it/s, loss=37.6]
5%|▍ | 47/1000 [00:01<00:36, 25.82it/s, loss=37.4]
5%|▍ | 47/1000 [00:01<00:36, 25.82it/s, loss=37.3]
5%|▌ | 50/1000 [00:01<00:37, 25.61it/s, loss=37.3]
5%|▌ | 50/1000 [00:02<00:37, 25.61it/s, loss=37.1]
5%|▌ | 50/1000 [00:02<00:37, 25.61it/s, loss=36.9]
5%|▌ | 50/1000 [00:02<00:37, 25.61it/s, loss=36.8]
5%|▌ | 53/1000 [00:02<00:36, 25.68it/s, loss=36.8]
5%|▌ | 53/1000 [00:02<00:36, 25.68it/s, loss=36.6]
5%|▌ | 53/1000 [00:02<00:36, 25.68it/s, loss=36.5]
5%|▌ | 53/1000 [00:02<00:36, 25.68it/s, loss=36.3]
6%|▌ | 56/1000 [00:02<00:37, 25.51it/s, loss=36.3]
6%|▌ | 56/1000 [00:02<00:37, 25.51it/s, loss=36.1]
6%|▌ | 56/1000 [00:02<00:37, 25.51it/s, loss=36]
6%|▌ | 56/1000 [00:02<00:37, 25.51it/s, loss=35.8]
6%|▌ | 59/1000 [00:02<00:38, 24.72it/s, loss=35.8]
6%|▌ | 59/1000 [00:02<00:38, 24.72it/s, loss=35.7]
6%|▌ | 59/1000 [00:02<00:38, 24.72it/s, loss=35.5]
6%|▌ | 59/1000 [00:02<00:38, 24.72it/s, loss=35.4]
6%|▌ | 62/1000 [00:02<00:38, 24.44it/s, loss=35.4]
6%|▌ | 62/1000 [00:02<00:38, 24.44it/s, loss=35.2]
6%|▌ | 62/1000 [00:02<00:38, 24.44it/s, loss=35.1]
6%|▌ | 62/1000 [00:02<00:38, 24.44it/s, loss=34.9]
6%|▋ | 65/1000 [00:02<00:37, 24.75it/s, loss=34.9]
6%|▋ | 65/1000 [00:02<00:37, 24.75it/s, loss=34.8]
6%|▋ | 65/1000 [00:02<00:37, 24.75it/s, loss=34.6]
6%|▋ | 65/1000 [00:02<00:37, 24.75it/s, loss=34.5]
7%|▋ | 68/1000 [00:02<00:37, 25.17it/s, loss=34.5]
7%|▋ | 68/1000 [00:02<00:37, 25.17it/s, loss=34.3]
7%|▋ | 68/1000 [00:02<00:37, 25.17it/s, loss=34.2]
7%|▋ | 68/1000 [00:02<00:37, 25.17it/s, loss=34]
7%|▋ | 71/1000 [00:02<00:36, 25.54it/s, loss=34]
7%|▋ | 71/1000 [00:02<00:36, 25.54it/s, loss=33.9]
7%|▋ | 71/1000 [00:02<00:36, 25.54it/s, loss=33.7]
7%|▋ | 71/1000 [00:02<00:36, 25.54it/s, loss=33.6]
7%|▋ | 74/1000 [00:02<00:35, 25.78it/s, loss=33.6]
7%|▋ | 74/1000 [00:02<00:35, 25.78it/s, loss=33.4]
7%|▋ | 74/1000 [00:03<00:35, 25.78it/s, loss=33.3]
7%|▋ | 74/1000 [00:03<00:35, 25.78it/s, loss=33.1]
8%|▊ | 77/1000 [00:03<00:35, 25.94it/s, loss=33.1]
8%|▊ | 77/1000 [00:03<00:35, 25.94it/s, loss=33]
8%|▊ | 77/1000 [00:03<00:35, 25.94it/s, loss=32.9]
8%|▊ | 77/1000 [00:03<00:35, 25.94it/s, loss=32.7]
8%|▊ | 80/1000 [00:03<00:35, 26.02it/s, loss=32.7]
8%|▊ | 80/1000 [00:03<00:35, 26.02it/s, loss=32.6]
8%|▊ | 80/1000 [00:03<00:35, 26.02it/s, loss=32.4]
8%|▊ | 80/1000 [00:03<00:35, 26.02it/s, loss=32.3]
8%|▊ | 83/1000 [00:03<00:35, 26.03it/s, loss=32.3]
8%|▊ | 83/1000 [00:03<00:35, 26.03it/s, loss=32.2]
8%|▊ | 83/1000 [00:03<00:35, 26.03it/s, loss=32]
8%|▊ | 83/1000 [00:03<00:35, 26.03it/s, loss=31.9]
9%|▊ | 86/1000 [00:03<00:35, 26.03it/s, loss=31.9]
9%|▊ | 86/1000 [00:03<00:35, 26.03it/s, loss=31.7]
9%|▊ | 86/1000 [00:03<00:35, 26.03it/s, loss=31.6]
9%|▊ | 86/1000 [00:03<00:35, 26.03it/s, loss=31.5]
9%|▉ | 89/1000 [00:03<00:35, 25.98it/s, loss=31.5]
9%|▉ | 89/1000 [00:03<00:35, 25.98it/s, loss=31.3]
9%|▉ | 89/1000 [00:03<00:35, 25.98it/s, loss=31.2]
9%|▉ | 89/1000 [00:03<00:35, 25.98it/s, loss=31.1]
9%|▉ | 92/1000 [00:03<00:34, 26.05it/s, loss=31.1]
9%|▉ | 92/1000 [00:03<00:34, 26.05it/s, loss=30.9]
9%|▉ | 92/1000 [00:03<00:34, 26.05it/s, loss=30.8]
9%|▉ | 92/1000 [00:03<00:34, 26.05it/s, loss=30.7]
10%|▉ | 95/1000 [00:03<00:34, 26.15it/s, loss=30.7]
10%|▉ | 95/1000 [00:03<00:34, 26.15it/s, loss=30.5]
10%|▉ | 95/1000 [00:03<00:34, 26.15it/s, loss=30.4]
10%|▉ | 95/1000 [00:03<00:34, 26.15it/s, loss=30.3]
10%|▉ | 98/1000 [00:03<00:34, 26.07it/s, loss=30.3]
10%|▉ | 98/1000 [00:03<00:34, 26.07it/s, loss=30.1]
10%|▉ | 98/1000 [00:03<00:34, 26.07it/s, loss=30]
10%|▉ | 98/1000 [00:03<00:34, 26.07it/s, loss=29.9]
10%|█ | 101/1000 [00:03<00:34, 26.13it/s, loss=29.9]
10%|█ | 101/1000 [00:04<00:34, 26.13it/s, loss=29.8]
10%|█ | 101/1000 [00:04<00:34, 26.13it/s, loss=29.6]
10%|█ | 101/1000 [00:04<00:34, 26.13it/s, loss=29.5]
10%|█ | 104/1000 [00:04<00:34, 26.20it/s, loss=29.5]
10%|█ | 104/1000 [00:04<00:34, 26.20it/s, loss=29.4]
10%|█ | 104/1000 [00:04<00:34, 26.20it/s, loss=29.3]
10%|█ | 104/1000 [00:04<00:34, 26.20it/s, loss=29.1]
11%|█ | 107/1000 [00:04<00:34, 26.09it/s, loss=29.1]
11%|█ | 107/1000 [00:04<00:34, 26.09it/s, loss=29]
11%|█ | 107/1000 [00:04<00:34, 26.09it/s, loss=28.9]
11%|█ | 107/1000 [00:04<00:34, 26.09it/s, loss=28.8]
11%|█ | 110/1000 [00:04<00:33, 26.19it/s, loss=28.8]
11%|█ | 110/1000 [00:04<00:33, 26.19it/s, loss=28.6]
11%|█ | 110/1000 [00:04<00:33, 26.19it/s, loss=28.5]
11%|█ | 110/1000 [00:04<00:33, 26.19it/s, loss=28.4]
11%|█▏ | 113/1000 [00:04<00:33, 26.21it/s, loss=28.4]
11%|█▏ | 113/1000 [00:04<00:33, 26.21it/s, loss=28.3]
11%|█▏ | 113/1000 [00:04<00:33, 26.21it/s, loss=28.2]
11%|█▏ | 113/1000 [00:04<00:33, 26.21it/s, loss=28]
12%|█▏ | 116/1000 [00:04<00:33, 26.42it/s, loss=28]
12%|█▏ | 116/1000 [00:04<00:33, 26.42it/s, loss=27.9]
12%|█▏ | 116/1000 [00:04<00:33, 26.42it/s, loss=27.8]
12%|█▏ | 116/1000 [00:04<00:33, 26.42it/s, loss=27.7]
12%|█▏ | 119/1000 [00:04<00:33, 26.58it/s, loss=27.7]
12%|█▏ | 119/1000 [00:04<00:33, 26.58it/s, loss=27.6]
12%|█▏ | 119/1000 [00:04<00:33, 26.58it/s, loss=27.5]
12%|█▏ | 119/1000 [00:04<00:33, 26.58it/s, loss=27.3]
12%|█▏ | 122/1000 [00:04<00:33, 26.58it/s, loss=27.3]
12%|█▏ | 122/1000 [00:04<00:33, 26.58it/s, loss=27.2]
12%|█▏ | 122/1000 [00:04<00:33, 26.58it/s, loss=27.1]
12%|█▏ | 122/1000 [00:04<00:33, 26.58it/s, loss=27]
12%|█▎ | 125/1000 [00:04<00:33, 26.41it/s, loss=27]
12%|█▎ | 125/1000 [00:04<00:33, 26.41it/s, loss=26.9]
12%|█▎ | 125/1000 [00:04<00:33, 26.41it/s, loss=26.8]
12%|█▎ | 125/1000 [00:04<00:33, 26.41it/s, loss=26.7]
13%|█▎ | 128/1000 [00:04<00:33, 26.36it/s, loss=26.7]
13%|█▎ | 128/1000 [00:05<00:33, 26.36it/s, loss=26.6]
13%|█▎ | 128/1000 [00:05<00:33, 26.36it/s, loss=26.4]
13%|█▎ | 128/1000 [00:05<00:33, 26.36it/s, loss=26.3]
13%|█▎ | 131/1000 [00:05<00:32, 26.45it/s, loss=26.3]
13%|█▎ | 131/1000 [00:05<00:32, 26.45it/s, loss=26.2]
13%|█▎ | 131/1000 [00:05<00:32, 26.45it/s, loss=26.1]
13%|█▎ | 131/1000 [00:05<00:32, 26.45it/s, loss=26]
13%|█▎ | 134/1000 [00:05<00:32, 26.72it/s, loss=26]
13%|█▎ | 134/1000 [00:05<00:32, 26.72it/s, loss=25.9]
13%|█▎ | 134/1000 [00:05<00:32, 26.72it/s, loss=25.8]
13%|█▎ | 134/1000 [00:05<00:32, 26.72it/s, loss=25.7]
14%|█▎ | 137/1000 [00:05<00:32, 26.85it/s, loss=25.7]
14%|█▎ | 137/1000 [00:05<00:32, 26.85it/s, loss=25.6]
14%|█▎ | 137/1000 [00:05<00:32, 26.85it/s, loss=25.5]
14%|█▎ | 137/1000 [00:05<00:32, 26.85it/s, loss=25.4]
14%|█▍ | 140/1000 [00:05<00:32, 26.80it/s, loss=25.4]
14%|█▍ | 140/1000 [00:05<00:32, 26.80it/s, loss=25.3]
14%|█▍ | 140/1000 [00:05<00:32, 26.80it/s, loss=25.2]
14%|█▍ | 140/1000 [00:05<00:32, 26.80it/s, loss=25.1]
14%|█▍ | 143/1000 [00:05<00:32, 26.73it/s, loss=25.1]
14%|█▍ | 143/1000 [00:05<00:32, 26.73it/s, loss=25]
14%|█▍ | 143/1000 [00:05<00:32, 26.73it/s, loss=24.9]
14%|█▍ | 143/1000 [00:05<00:32, 26.73it/s, loss=24.8]
15%|█▍ | 146/1000 [00:05<00:32, 26.62it/s, loss=24.8]
15%|█▍ | 146/1000 [00:05<00:32, 26.62it/s, loss=24.7]
15%|█▍ | 146/1000 [00:05<00:32, 26.62it/s, loss=24.6]
15%|█▍ | 146/1000 [00:05<00:32, 26.62it/s, loss=24.5]
15%|█▍ | 149/1000 [00:05<00:32, 26.59it/s, loss=24.5]
15%|█▍ | 149/1000 [00:05<00:32, 26.59it/s, loss=24.4]
15%|█▍ | 149/1000 [00:05<00:32, 26.59it/s, loss=24.3]
15%|█▍ | 149/1000 [00:05<00:32, 26.59it/s, loss=24.2]
15%|█▌ | 152/1000 [00:05<00:31, 26.54it/s, loss=24.2]
15%|█▌ | 152/1000 [00:05<00:31, 26.54it/s, loss=24.1]
15%|█▌ | 152/1000 [00:05<00:31, 26.54it/s, loss=24]
15%|█▌ | 152/1000 [00:06<00:31, 26.54it/s, loss=23.9]
16%|█▌ | 155/1000 [00:06<00:31, 26.57it/s, loss=23.9]
16%|█▌ | 155/1000 [00:06<00:31, 26.57it/s, loss=23.8]
16%|█▌ | 155/1000 [00:06<00:31, 26.57it/s, loss=23.7]
16%|█▌ | 155/1000 [00:06<00:31, 26.57it/s, loss=23.6]
16%|█▌ | 158/1000 [00:06<00:31, 26.60it/s, loss=23.6]
16%|█▌ | 158/1000 [00:06<00:31, 26.60it/s, loss=23.5]
16%|█▌ | 158/1000 [00:06<00:31, 26.60it/s, loss=23.4]
16%|█▌ | 158/1000 [00:06<00:31, 26.60it/s, loss=23.3]
16%|█▌ | 161/1000 [00:06<00:31, 26.60it/s, loss=23.3]
16%|█▌ | 161/1000 [00:06<00:31, 26.60it/s, loss=23.2]
16%|█▌ | 161/1000 [00:06<00:31, 26.60it/s, loss=23.1]
16%|█▌ | 161/1000 [00:06<00:31, 26.60it/s, loss=23]
16%|█▋ | 164/1000 [00:06<00:31, 26.37it/s, loss=23]
16%|█▋ | 164/1000 [00:06<00:31, 26.37it/s, loss=22.9]
16%|█▋ | 164/1000 [00:06<00:31, 26.37it/s, loss=22.8]
16%|█▋ | 164/1000 [00:06<00:31, 26.37it/s, loss=22.7]
17%|█▋ | 167/1000 [00:06<00:31, 26.16it/s, loss=22.7]
17%|█▋ | 167/1000 [00:06<00:31, 26.16it/s, loss=22.6]
17%|█▋ | 167/1000 [00:06<00:31, 26.16it/s, loss=22.6]
17%|█▋ | 167/1000 [00:06<00:31, 26.16it/s, loss=22.5]
17%|█▋ | 170/1000 [00:06<00:32, 25.50it/s, loss=22.5]
17%|█▋ | 170/1000 [00:06<00:32, 25.50it/s, loss=22.4]
17%|█▋ | 170/1000 [00:06<00:32, 25.50it/s, loss=22.3]
17%|█▋ | 170/1000 [00:06<00:32, 25.50it/s, loss=22.2]
17%|█▋ | 173/1000 [00:06<00:32, 25.61it/s, loss=22.2]
17%|█▋ | 173/1000 [00:06<00:32, 25.61it/s, loss=22.1]
17%|█▋ | 173/1000 [00:06<00:32, 25.61it/s, loss=22]
17%|█▋ | 173/1000 [00:06<00:32, 25.61it/s, loss=21.9]
18%|█▊ | 176/1000 [00:06<00:31, 25.95it/s, loss=21.9]
18%|█▊ | 176/1000 [00:06<00:31, 25.95it/s, loss=21.9]
18%|█▊ | 176/1000 [00:06<00:31, 25.95it/s, loss=21.8]
18%|█▊ | 176/1000 [00:06<00:31, 25.95it/s, loss=21.7]
18%|█▊ | 179/1000 [00:06<00:31, 26.19it/s, loss=21.7]
18%|█▊ | 179/1000 [00:06<00:31, 26.19it/s, loss=21.6]
18%|█▊ | 179/1000 [00:07<00:31, 26.19it/s, loss=21.5]
18%|█▊ | 179/1000 [00:07<00:31, 26.19it/s, loss=21.4]
18%|█▊ | 182/1000 [00:07<00:30, 26.51it/s, loss=21.4]
18%|█▊ | 182/1000 [00:07<00:30, 26.51it/s, loss=21.3]
18%|█▊ | 182/1000 [00:07<00:30, 26.51it/s, loss=21.3]
18%|█▊ | 182/1000 [00:07<00:30, 26.51it/s, loss=21.2]
18%|█▊ | 185/1000 [00:07<00:30, 26.56it/s, loss=21.2]
18%|█▊ | 185/1000 [00:07<00:30, 26.56it/s, loss=21.1]
18%|█▊ | 185/1000 [00:07<00:30, 26.56it/s, loss=21]
18%|█▊ | 185/1000 [00:07<00:30, 26.56it/s, loss=20.9]
19%|█▉ | 188/1000 [00:07<00:30, 26.71it/s, loss=20.9]
19%|█▉ | 188/1000 [00:07<00:30, 26.71it/s, loss=20.9]
19%|█▉ | 188/1000 [00:07<00:30, 26.71it/s, loss=20.8]
19%|█▉ | 188/1000 [00:07<00:30, 26.71it/s, loss=20.7]
19%|█▉ | 191/1000 [00:07<00:30, 26.77it/s, loss=20.7]
19%|█▉ | 191/1000 [00:07<00:30, 26.77it/s, loss=20.6]
19%|█▉ | 191/1000 [00:07<00:30, 26.77it/s, loss=20.5]
19%|█▉ | 191/1000 [00:07<00:30, 26.77it/s, loss=20.5]
19%|█▉ | 194/1000 [00:07<00:29, 26.88it/s, loss=20.5]
19%|█▉ | 194/1000 [00:07<00:29, 26.88it/s, loss=20.4]
19%|█▉ | 194/1000 [00:07<00:29, 26.88it/s, loss=20.3]
19%|█▉ | 194/1000 [00:07<00:29, 26.88it/s, loss=20.2]
20%|█▉ | 197/1000 [00:07<00:29, 27.06it/s, loss=20.2]
20%|█▉ | 197/1000 [00:07<00:29, 27.06it/s, loss=20.1]
20%|█▉ | 197/1000 [00:07<00:29, 27.06it/s, loss=20.1]
20%|█▉ | 197/1000 [00:07<00:29, 27.06it/s, loss=20]
20%|██ | 200/1000 [00:07<00:29, 27.08it/s, loss=20]
20%|██ | 200/1000 [00:07<00:29, 27.08it/s, loss=19.9]
20%|██ | 200/1000 [00:07<00:29, 27.08it/s, loss=19.8]
20%|██ | 200/1000 [00:07<00:29, 27.08it/s, loss=19.8]
20%|██ | 203/1000 [00:07<00:29, 27.03it/s, loss=19.8]
20%|██ | 203/1000 [00:07<00:29, 27.03it/s, loss=19.7]
20%|██ | 203/1000 [00:07<00:29, 27.03it/s, loss=19.6]
20%|██ | 203/1000 [00:07<00:29, 27.03it/s, loss=19.5]
21%|██ | 206/1000 [00:07<00:29, 27.05it/s, loss=19.5]
21%|██ | 206/1000 [00:07<00:29, 27.05it/s, loss=19.5]
21%|██ | 206/1000 [00:08<00:29, 27.05it/s, loss=19.4]
21%|██ | 206/1000 [00:08<00:29, 27.05it/s, loss=19.3]
21%|██ | 209/1000 [00:08<00:29, 27.22it/s, loss=19.3]
21%|██ | 209/1000 [00:08<00:29, 27.22it/s, loss=19.3]
21%|██ | 209/1000 [00:08<00:29, 27.22it/s, loss=19.2]
21%|██ | 209/1000 [00:08<00:29, 27.22it/s, loss=19.1]
21%|██ | 212/1000 [00:08<00:28, 27.34it/s, loss=19.1]
21%|██ | 212/1000 [00:08<00:28, 27.34it/s, loss=19]
21%|██ | 212/1000 [00:08<00:28, 27.34it/s, loss=19]
21%|██ | 212/1000 [00:08<00:28, 27.34it/s, loss=18.9]
22%|██▏ | 215/1000 [00:08<00:28, 27.25it/s, loss=18.9]
22%|██▏ | 215/1000 [00:08<00:28, 27.25it/s, loss=18.8]
22%|██▏ | 215/1000 [00:08<00:28, 27.25it/s, loss=18.8]
22%|██▏ | 215/1000 [00:08<00:28, 27.25it/s, loss=18.7]
22%|██▏ | 218/1000 [00:08<00:28, 27.34it/s, loss=18.7]
22%|██▏ | 218/1000 [00:08<00:28, 27.34it/s, loss=18.6]
22%|██▏ | 218/1000 [00:08<00:28, 27.34it/s, loss=18.6]
22%|██▏ | 218/1000 [00:08<00:28, 27.34it/s, loss=18.5]
22%|██▏ | 221/1000 [00:08<00:28, 26.97it/s, loss=18.5]
22%|██▏ | 221/1000 [00:08<00:28, 26.97it/s, loss=18.4]
22%|██▏ | 221/1000 [00:08<00:28, 26.97it/s, loss=18.4]
22%|██▏ | 221/1000 [00:08<00:28, 26.97it/s, loss=18.3]
22%|██▏ | 224/1000 [00:08<00:28, 26.79it/s, loss=18.3]
22%|██▏ | 224/1000 [00:08<00:28, 26.79it/s, loss=18.2]
22%|██▏ | 224/1000 [00:08<00:28, 26.79it/s, loss=18.2]
22%|██▏ | 224/1000 [00:08<00:28, 26.79it/s, loss=18.1]
23%|██▎ | 227/1000 [00:08<00:28, 26.67it/s, loss=18.1]
23%|██▎ | 227/1000 [00:08<00:28, 26.67it/s, loss=18]
23%|██▎ | 227/1000 [00:08<00:28, 26.67it/s, loss=18]
23%|██▎ | 227/1000 [00:08<00:28, 26.67it/s, loss=17.9]
23%|██▎ | 230/1000 [00:08<00:28, 26.71it/s, loss=17.9]
23%|██▎ | 230/1000 [00:08<00:28, 26.71it/s, loss=17.8]
23%|██▎ | 230/1000 [00:08<00:28, 26.71it/s, loss=17.8]
23%|██▎ | 230/1000 [00:08<00:28, 26.71it/s, loss=17.7]
23%|██▎ | 233/1000 [00:08<00:28, 26.72it/s, loss=17.7]
23%|██▎ | 233/1000 [00:08<00:28, 26.72it/s, loss=17.6]
23%|██▎ | 233/1000 [00:09<00:28, 26.72it/s, loss=17.6]
23%|██▎ | 233/1000 [00:09<00:28, 26.72it/s, loss=17.5]
24%|██▎ | 236/1000 [00:09<00:28, 26.78it/s, loss=17.5]
24%|██▎ | 236/1000 [00:09<00:28, 26.78it/s, loss=17.5]
24%|██▎ | 236/1000 [00:09<00:28, 26.78it/s, loss=17.4]
24%|██▎ | 236/1000 [00:09<00:28, 26.78it/s, loss=17.3]
24%|██▍ | 239/1000 [00:09<00:28, 26.75it/s, loss=17.3]
24%|██▍ | 239/1000 [00:09<00:28, 26.75it/s, loss=17.3]
24%|██▍ | 239/1000 [00:09<00:28, 26.75it/s, loss=17.2]
24%|██▍ | 239/1000 [00:09<00:28, 26.75it/s, loss=17.2]
24%|██▍ | 242/1000 [00:09<00:28, 26.70it/s, loss=17.2]
24%|██▍ | 242/1000 [00:09<00:28, 26.70it/s, loss=17.1]
24%|██▍ | 242/1000 [00:09<00:28, 26.70it/s, loss=17]
24%|██▍ | 242/1000 [00:09<00:28, 26.70it/s, loss=17]
24%|██▍ | 245/1000 [00:09<00:28, 26.81it/s, loss=17]
24%|██▍ | 245/1000 [00:09<00:28, 26.81it/s, loss=16.9]
24%|██▍ | 245/1000 [00:09<00:28, 26.81it/s, loss=16.9]
24%|██▍ | 245/1000 [00:09<00:28, 26.81it/s, loss=16.8]
25%|██▍ | 248/1000 [00:09<00:28, 26.80it/s, loss=16.8]
25%|██▍ | 248/1000 [00:09<00:28, 26.80it/s, loss=16.7]
25%|██▍ | 248/1000 [00:09<00:28, 26.80it/s, loss=16.7]
25%|██▍ | 248/1000 [00:09<00:28, 26.80it/s, loss=16.6]
25%|██▌ | 251/1000 [00:09<00:27, 26.79it/s, loss=16.6]
25%|██▌ | 251/1000 [00:09<00:27, 26.79it/s, loss=16.6]
25%|██▌ | 251/1000 [00:09<00:27, 26.79it/s, loss=16.5]
25%|██▌ | 251/1000 [00:09<00:27, 26.79it/s, loss=16.5]
25%|██▌ | 254/1000 [00:09<00:27, 26.71it/s, loss=16.5]
25%|██▌ | 254/1000 [00:09<00:27, 26.71it/s, loss=16.4]
25%|██▌ | 254/1000 [00:09<00:27, 26.71it/s, loss=16.3]
25%|██▌ | 254/1000 [00:09<00:27, 26.71it/s, loss=16.3]
26%|██▌ | 257/1000 [00:09<00:27, 26.72it/s, loss=16.3]
26%|██▌ | 257/1000 [00:09<00:27, 26.72it/s, loss=16.2]
26%|██▌ | 257/1000 [00:09<00:27, 26.72it/s, loss=16.2]
26%|██▌ | 257/1000 [00:09<00:27, 26.72it/s, loss=16.1]
26%|██▌ | 260/1000 [00:09<00:27, 26.76it/s, loss=16.1]
26%|██▌ | 260/1000 [00:09<00:27, 26.76it/s, loss=16.1]
26%|██▌ | 260/1000 [00:10<00:27, 26.76it/s, loss=16]
26%|██▌ | 260/1000 [00:10<00:27, 26.76it/s, loss=16]
26%|██▋ | 263/1000 [00:10<00:27, 26.87it/s, loss=16]
26%|██▋ | 263/1000 [00:10<00:27, 26.87it/s, loss=15.9]
26%|██▋ | 263/1000 [00:10<00:27, 26.87it/s, loss=15.9]
26%|██▋ | 263/1000 [00:10<00:27, 26.87it/s, loss=15.8]
27%|██▋ | 266/1000 [00:10<00:27, 26.95it/s, loss=15.8]
27%|██▋ | 266/1000 [00:10<00:27, 26.95it/s, loss=15.7]
27%|██▋ | 266/1000 [00:10<00:27, 26.95it/s, loss=15.7]
27%|██▋ | 266/1000 [00:10<00:27, 26.95it/s, loss=15.6]
27%|██▋ | 269/1000 [00:10<00:27, 26.92it/s, loss=15.6]
27%|██▋ | 269/1000 [00:10<00:27, 26.92it/s, loss=15.6]
27%|██▋ | 269/1000 [00:10<00:27, 26.92it/s, loss=15.5]
27%|██▋ | 269/1000 [00:10<00:27, 26.92it/s, loss=15.5]
27%|██▋ | 272/1000 [00:10<00:26, 27.02it/s, loss=15.5]
27%|██▋ | 272/1000 [00:10<00:26, 27.02it/s, loss=15.4]
27%|██▋ | 272/1000 [00:10<00:26, 27.02it/s, loss=15.4]
27%|██▋ | 272/1000 [00:10<00:26, 27.02it/s, loss=15.3]
28%|██▊ | 275/1000 [00:10<00:26, 27.12it/s, loss=15.3]
28%|██▊ | 275/1000 [00:10<00:26, 27.12it/s, loss=15.3]
28%|██▊ | 275/1000 [00:10<00:26, 27.12it/s, loss=15.2]
28%|██▊ | 275/1000 [00:10<00:26, 27.12it/s, loss=15.2]
28%|██▊ | 278/1000 [00:10<00:26, 26.96it/s, loss=15.2]
28%|██▊ | 278/1000 [00:10<00:26, 26.96it/s, loss=15.1]
28%|██▊ | 278/1000 [00:10<00:26, 26.96it/s, loss=15.1]
28%|██▊ | 278/1000 [00:10<00:26, 26.96it/s, loss=15]
28%|██▊ | 281/1000 [00:10<00:26, 26.75it/s, loss=15]
28%|██▊ | 281/1000 [00:10<00:26, 26.75it/s, loss=15]
28%|██▊ | 281/1000 [00:10<00:26, 26.75it/s, loss=14.9]
28%|██▊ | 281/1000 [00:10<00:26, 26.75it/s, loss=14.9]
28%|██▊ | 284/1000 [00:10<00:27, 25.87it/s, loss=14.9]
28%|██▊ | 284/1000 [00:10<00:27, 25.87it/s, loss=14.8]
28%|██▊ | 284/1000 [00:10<00:27, 25.87it/s, loss=14.8]
28%|██▊ | 284/1000 [00:10<00:27, 25.87it/s, loss=14.8]
29%|██▊ | 287/1000 [00:10<00:27, 26.37it/s, loss=14.8]
29%|██▊ | 287/1000 [00:10<00:27, 26.37it/s, loss=14.7]
29%|██▊ | 287/1000 [00:11<00:27, 26.37it/s, loss=14.7]
29%|██▊ | 287/1000 [00:11<00:27, 26.37it/s, loss=14.6]
29%|██▉ | 290/1000 [00:11<00:26, 26.42it/s, loss=14.6]
29%|██▉ | 290/1000 [00:11<00:26, 26.42it/s, loss=14.6]
29%|██▉ | 290/1000 [00:11<00:26, 26.42it/s, loss=14.5]
29%|██▉ | 290/1000 [00:11<00:26, 26.42it/s, loss=14.5]
29%|██▉ | 293/1000 [00:11<00:26, 26.46it/s, loss=14.5]
29%|██▉ | 293/1000 [00:11<00:26, 26.46it/s, loss=14.4]
29%|██▉ | 293/1000 [00:11<00:26, 26.46it/s, loss=14.4]
29%|██▉ | 293/1000 [00:11<00:26, 26.46it/s, loss=14.3]
30%|██▉ | 296/1000 [00:11<00:26, 26.58it/s, loss=14.3]
30%|██▉ | 296/1000 [00:11<00:26, 26.58it/s, loss=14.3]
30%|██▉ | 296/1000 [00:11<00:26, 26.58it/s, loss=14.2]
30%|██▉ | 296/1000 [00:11<00:26, 26.58it/s, loss=14.2]
30%|██▉ | 299/1000 [00:11<00:26, 26.66it/s, loss=14.2]
30%|██▉ | 299/1000 [00:11<00:26, 26.66it/s, loss=14.2]
30%|██▉ | 299/1000 [00:11<00:26, 26.66it/s, loss=14.1]
30%|██▉ | 299/1000 [00:11<00:26, 26.66it/s, loss=14.1]
30%|███ | 302/1000 [00:11<00:26, 26.62it/s, loss=14.1]
30%|███ | 302/1000 [00:11<00:26, 26.62it/s, loss=14]
30%|███ | 302/1000 [00:11<00:26, 26.62it/s, loss=14]
30%|███ | 302/1000 [00:11<00:26, 26.62it/s, loss=13.9]
30%|███ | 305/1000 [00:11<00:25, 26.80it/s, loss=13.9]
30%|███ | 305/1000 [00:11<00:25, 26.80it/s, loss=13.9]
30%|███ | 305/1000 [00:11<00:25, 26.80it/s, loss=13.9]
30%|███ | 305/1000 [00:11<00:25, 26.80it/s, loss=13.8]
31%|███ | 308/1000 [00:11<00:25, 27.06it/s, loss=13.8]
31%|███ | 308/1000 [00:11<00:25, 27.06it/s, loss=13.8]
31%|███ | 308/1000 [00:11<00:25, 27.06it/s, loss=13.7]
31%|███ | 308/1000 [00:11<00:25, 27.06it/s, loss=13.7]
31%|███ | 311/1000 [00:11<00:25, 27.20it/s, loss=13.7]
31%|███ | 311/1000 [00:11<00:25, 27.20it/s, loss=13.7]
31%|███ | 311/1000 [00:11<00:25, 27.20it/s, loss=13.6]
31%|███ | 311/1000 [00:11<00:25, 27.20it/s, loss=13.6]
31%|███▏ | 314/1000 [00:11<00:25, 27.11it/s, loss=13.6]
31%|███▏ | 314/1000 [00:11<00:25, 27.11it/s, loss=13.5]
31%|███▏ | 314/1000 [00:12<00:25, 27.11it/s, loss=13.5]
31%|███▏ | 314/1000 [00:12<00:25, 27.11it/s, loss=13.4]
32%|███▏ | 317/1000 [00:12<00:25, 26.91it/s, loss=13.4]
32%|███▏ | 317/1000 [00:12<00:25, 26.91it/s, loss=13.4]
32%|███▏ | 317/1000 [00:12<00:25, 26.91it/s, loss=13.4]
32%|███▏ | 317/1000 [00:12<00:25, 26.91it/s, loss=13.3]
32%|███▏ | 320/1000 [00:12<00:25, 26.86it/s, loss=13.3]
32%|███▏ | 320/1000 [00:12<00:25, 26.86it/s, loss=13.3]
32%|███▏ | 320/1000 [00:12<00:25, 26.86it/s, loss=13.2]
32%|███▏ | 320/1000 [00:12<00:25, 26.86it/s, loss=13.2]
32%|███▏ | 323/1000 [00:12<00:25, 26.62it/s, loss=13.2]
32%|███▏ | 323/1000 [00:12<00:25, 26.62it/s, loss=13.2]
32%|███▏ | 323/1000 [00:12<00:25, 26.62it/s, loss=13.1]
32%|███▏ | 323/1000 [00:12<00:25, 26.62it/s, loss=13.1]
33%|███▎ | 326/1000 [00:12<00:25, 26.56it/s, loss=13.1]
33%|███▎ | 326/1000 [00:12<00:25, 26.56it/s, loss=13.1]
33%|███▎ | 326/1000 [00:12<00:25, 26.56it/s, loss=13]
33%|███▎ | 326/1000 [00:12<00:25, 26.56it/s, loss=13]
33%|███▎ | 329/1000 [00:12<00:25, 26.59it/s, loss=13]
33%|███▎ | 329/1000 [00:12<00:25, 26.59it/s, loss=12.9]
33%|███▎ | 329/1000 [00:12<00:25, 26.59it/s, loss=12.9]
33%|███▎ | 329/1000 [00:12<00:25, 26.59it/s, loss=12.9]
33%|███▎ | 332/1000 [00:12<00:24, 27.05it/s, loss=12.9]
33%|███▎ | 332/1000 [00:12<00:24, 27.05it/s, loss=12.8]
33%|███▎ | 332/1000 [00:12<00:24, 27.05it/s, loss=12.8]
33%|███▎ | 332/1000 [00:12<00:24, 27.05it/s, loss=12.8]
34%|███▎ | 335/1000 [00:12<00:24, 27.08it/s, loss=12.8]
34%|███▎ | 335/1000 [00:12<00:24, 27.08it/s, loss=12.7]
34%|███▎ | 335/1000 [00:12<00:24, 27.08it/s, loss=12.7]
34%|███▎ | 335/1000 [00:12<00:24, 27.08it/s, loss=12.7]
34%|███▍ | 338/1000 [00:12<00:24, 26.96it/s, loss=12.7]
34%|███▍ | 338/1000 [00:12<00:24, 26.96it/s, loss=12.6]
34%|███▍ | 338/1000 [00:12<00:24, 26.96it/s, loss=12.6]
34%|███▍ | 338/1000 [00:12<00:24, 26.96it/s, loss=12.5]
34%|███▍ | 341/1000 [00:12<00:24, 26.64it/s, loss=12.5]
34%|███▍ | 341/1000 [00:13<00:24, 26.64it/s, loss=12.5]
34%|███▍ | 341/1000 [00:13<00:24, 26.64it/s, loss=12.5]
34%|███▍ | 341/1000 [00:13<00:24, 26.64it/s, loss=12.4]
34%|███▍ | 344/1000 [00:13<00:25, 25.80it/s, loss=12.4]
34%|███▍ | 344/1000 [00:13<00:25, 25.80it/s, loss=12.4]
34%|███▍ | 344/1000 [00:13<00:25, 25.80it/s, loss=12.4]
34%|███▍ | 344/1000 [00:13<00:25, 25.80it/s, loss=12.3]
35%|███▍ | 347/1000 [00:13<00:25, 25.94it/s, loss=12.3]
35%|███▍ | 347/1000 [00:13<00:25, 25.94it/s, loss=12.3]
35%|███▍ | 347/1000 [00:13<00:25, 25.94it/s, loss=12.3]
35%|███▍ | 347/1000 [00:13<00:25, 25.94it/s, loss=12.2]
35%|███▌ | 350/1000 [00:13<00:25, 25.91it/s, loss=12.2]
35%|███▌ | 350/1000 [00:13<00:25, 25.91it/s, loss=12.2]
35%|███▌ | 350/1000 [00:13<00:25, 25.91it/s, loss=12.2]
35%|███▌ | 350/1000 [00:13<00:25, 25.91it/s, loss=12.1]
35%|███▌ | 353/1000 [00:13<00:24, 25.97it/s, loss=12.1]
35%|███▌ | 353/1000 [00:13<00:24, 25.97it/s, loss=12.1]
35%|███▌ | 353/1000 [00:13<00:24, 25.97it/s, loss=12.1]
35%|███▌ | 353/1000 [00:13<00:24, 25.97it/s, loss=12]
36%|███▌ | 356/1000 [00:13<00:24, 26.11it/s, loss=12]
36%|███▌ | 356/1000 [00:13<00:24, 26.11it/s, loss=12]
36%|███▌ | 356/1000 [00:13<00:24, 26.11it/s, loss=12]
36%|███▌ | 356/1000 [00:13<00:24, 26.11it/s, loss=11.9]
36%|███▌ | 359/1000 [00:13<00:24, 26.41it/s, loss=11.9]
36%|███▌ | 359/1000 [00:13<00:24, 26.41it/s, loss=11.9]
36%|███▌ | 359/1000 [00:13<00:24, 26.41it/s, loss=11.9]
36%|███▌ | 359/1000 [00:13<00:24, 26.41it/s, loss=11.8]
36%|███▌ | 362/1000 [00:13<00:23, 26.66it/s, loss=11.8]
36%|███▌ | 362/1000 [00:13<00:23, 26.66it/s, loss=11.8]
36%|███▌ | 362/1000 [00:13<00:23, 26.66it/s, loss=11.8]
36%|███▌ | 362/1000 [00:13<00:23, 26.66it/s, loss=11.7]
36%|███▋ | 365/1000 [00:13<00:23, 26.65it/s, loss=11.7]
36%|███▋ | 365/1000 [00:13<00:23, 26.65it/s, loss=11.7]
36%|███▋ | 365/1000 [00:13<00:23, 26.65it/s, loss=11.7]
36%|███▋ | 365/1000 [00:13<00:23, 26.65it/s, loss=11.7]
37%|███▋ | 368/1000 [00:13<00:23, 26.96it/s, loss=11.7]
37%|███▋ | 368/1000 [00:14<00:23, 26.96it/s, loss=11.6]
37%|███▋ | 368/1000 [00:14<00:23, 26.96it/s, loss=11.6]
37%|███▋ | 368/1000 [00:14<00:23, 26.96it/s, loss=11.6]
37%|███▋ | 371/1000 [00:14<00:23, 26.83it/s, loss=11.6]
37%|███▋ | 371/1000 [00:14<00:23, 26.83it/s, loss=11.5]
37%|███▋ | 371/1000 [00:14<00:23, 26.83it/s, loss=11.5]
37%|███▋ | 371/1000 [00:14<00:23, 26.83it/s, loss=11.5]
37%|███▋ | 374/1000 [00:14<00:23, 26.68it/s, loss=11.5]
37%|███▋ | 374/1000 [00:14<00:23, 26.68it/s, loss=11.4]
37%|███▋ | 374/1000 [00:14<00:23, 26.68it/s, loss=11.4]
37%|███▋ | 374/1000 [00:14<00:23, 26.68it/s, loss=11.4]
38%|███▊ | 377/1000 [00:14<00:23, 26.83it/s, loss=11.4]
38%|███▊ | 377/1000 [00:14<00:23, 26.83it/s, loss=11.3]
38%|███▊ | 377/1000 [00:14<00:23, 26.83it/s, loss=11.3]
38%|███▊ | 377/1000 [00:14<00:23, 26.83it/s, loss=11.3]
38%|███▊ | 380/1000 [00:14<00:23, 26.84it/s, loss=11.3]
38%|███▊ | 380/1000 [00:14<00:23, 26.84it/s, loss=11.3]
38%|███▊ | 380/1000 [00:14<00:23, 26.84it/s, loss=11.2]
38%|███▊ | 380/1000 [00:14<00:23, 26.84it/s, loss=11.2]
38%|███▊ | 383/1000 [00:14<00:22, 26.95it/s, loss=11.2]
38%|███▊ | 383/1000 [00:14<00:22, 26.95it/s, loss=11.2]
38%|███▊ | 383/1000 [00:14<00:22, 26.95it/s, loss=11.1]
38%|███▊ | 383/1000 [00:14<00:22, 26.95it/s, loss=11.1]
39%|███▊ | 386/1000 [00:14<00:22, 27.29it/s, loss=11.1]
39%|███▊ | 386/1000 [00:14<00:22, 27.29it/s, loss=11.1]
39%|███▊ | 386/1000 [00:14<00:22, 27.29it/s, loss=11.1]
39%|███▊ | 386/1000 [00:14<00:22, 27.29it/s, loss=11]
39%|███▉ | 389/1000 [00:14<00:22, 27.37it/s, loss=11]
39%|███▉ | 389/1000 [00:14<00:22, 27.37it/s, loss=11]
39%|███▉ | 389/1000 [00:14<00:22, 27.37it/s, loss=11]
39%|███▉ | 389/1000 [00:14<00:22, 27.37it/s, loss=11]
39%|███▉ | 392/1000 [00:14<00:22, 27.21it/s, loss=11]
39%|███▉ | 392/1000 [00:14<00:22, 27.21it/s, loss=10.9]
39%|███▉ | 392/1000 [00:14<00:22, 27.21it/s, loss=10.9]
39%|███▉ | 392/1000 [00:14<00:22, 27.21it/s, loss=10.9]
40%|███▉ | 395/1000 [00:14<00:22, 27.03it/s, loss=10.9]
40%|███▉ | 395/1000 [00:15<00:22, 27.03it/s, loss=10.8]
40%|███▉ | 395/1000 [00:15<00:22, 27.03it/s, loss=10.8]
40%|███▉ | 395/1000 [00:15<00:22, 27.03it/s, loss=10.8]
40%|███▉ | 398/1000 [00:15<00:22, 26.90it/s, loss=10.8]
40%|███▉ | 398/1000 [00:15<00:22, 26.90it/s, loss=10.8]
40%|███▉ | 398/1000 [00:15<00:22, 26.90it/s, loss=10.7]
40%|███▉ | 398/1000 [00:15<00:22, 26.90it/s, loss=10.7]
40%|████ | 401/1000 [00:15<00:22, 27.04it/s, loss=10.7]
40%|████ | 401/1000 [00:15<00:22, 27.04it/s, loss=10.7]
40%|████ | 401/1000 [00:15<00:22, 27.04it/s, loss=10.7]
40%|████ | 401/1000 [00:15<00:22, 27.04it/s, loss=10.6]
40%|████ | 404/1000 [00:15<00:22, 27.00it/s, loss=10.6]
40%|████ | 404/1000 [00:15<00:22, 27.00it/s, loss=10.6]
40%|████ | 404/1000 [00:15<00:22, 27.00it/s, loss=10.6]
40%|████ | 404/1000 [00:15<00:22, 27.00it/s, loss=10.6]
41%|████ | 407/1000 [00:15<00:21, 27.08it/s, loss=10.6]
41%|████ | 407/1000 [00:15<00:21, 27.08it/s, loss=10.5]
41%|████ | 407/1000 [00:15<00:21, 27.08it/s, loss=10.5]
41%|████ | 407/1000 [00:15<00:21, 27.08it/s, loss=10.5]
41%|████ | 410/1000 [00:15<00:21, 27.16it/s, loss=10.5]
41%|████ | 410/1000 [00:15<00:21, 27.16it/s, loss=10.5]
41%|████ | 410/1000 [00:15<00:21, 27.16it/s, loss=10.4]
41%|████ | 410/1000 [00:15<00:21, 27.16it/s, loss=10.4]
41%|████▏ | 413/1000 [00:15<00:21, 27.30it/s, loss=10.4]
41%|████▏ | 413/1000 [00:15<00:21, 27.30it/s, loss=10.4]
41%|████▏ | 413/1000 [00:15<00:21, 27.30it/s, loss=10.4]
41%|████▏ | 413/1000 [00:15<00:21, 27.30it/s, loss=10.3]
42%|████▏ | 416/1000 [00:15<00:21, 27.28it/s, loss=10.3]
42%|████▏ | 416/1000 [00:15<00:21, 27.28it/s, loss=10.3]
42%|████▏ | 416/1000 [00:15<00:21, 27.28it/s, loss=10.3]
42%|████▏ | 416/1000 [00:15<00:21, 27.28it/s, loss=10.3]
42%|████▏ | 419/1000 [00:15<00:21, 27.10it/s, loss=10.3]
42%|████▏ | 419/1000 [00:15<00:21, 27.10it/s, loss=10.2]
42%|████▏ | 419/1000 [00:15<00:21, 27.10it/s, loss=10.2]
42%|████▏ | 419/1000 [00:15<00:21, 27.10it/s, loss=10.2]
42%|████▏ | 422/1000 [00:15<00:21, 27.05it/s, loss=10.2]
42%|████▏ | 422/1000 [00:16<00:21, 27.05it/s, loss=10.2]
42%|████▏ | 422/1000 [00:16<00:21, 27.05it/s, loss=10.1]
42%|████▏ | 422/1000 [00:16<00:21, 27.05it/s, loss=10.1]
42%|████▎ | 425/1000 [00:16<00:21, 27.21it/s, loss=10.1]
42%|████▎ | 425/1000 [00:16<00:21, 27.21it/s, loss=10.1]
42%|████▎ | 425/1000 [00:16<00:21, 27.21it/s, loss=10.1]
42%|████▎ | 425/1000 [00:16<00:21, 27.21it/s, loss=10]
43%|████▎ | 428/1000 [00:16<00:21, 27.18it/s, loss=10]
43%|████▎ | 428/1000 [00:16<00:21, 27.18it/s, loss=10]
43%|████▎ | 428/1000 [00:16<00:21, 27.18it/s, loss=10]
43%|████▎ | 428/1000 [00:16<00:21, 27.18it/s, loss=9.97]
43%|████▎ | 431/1000 [00:16<00:21, 27.09it/s, loss=9.97]
43%|████▎ | 431/1000 [00:16<00:21, 27.09it/s, loss=9.95]
43%|████▎ | 431/1000 [00:16<00:21, 27.09it/s, loss=9.93]
43%|████▎ | 431/1000 [00:16<00:21, 27.09it/s, loss=9.9]
43%|████▎ | 434/1000 [00:16<00:20, 27.34it/s, loss=9.9]
43%|████▎ | 434/1000 [00:16<00:20, 27.34it/s, loss=9.88]
43%|████▎ | 434/1000 [00:16<00:20, 27.34it/s, loss=9.86]
43%|████▎ | 434/1000 [00:16<00:20, 27.34it/s, loss=9.84]
44%|████▎ | 437/1000 [00:16<00:20, 27.40it/s, loss=9.84]
44%|████▎ | 437/1000 [00:16<00:20, 27.40it/s, loss=9.81]
44%|████▎ | 437/1000 [00:16<00:20, 27.40it/s, loss=9.79]
44%|████▎ | 437/1000 [00:16<00:20, 27.40it/s, loss=9.77]
44%|████▍ | 440/1000 [00:16<00:20, 27.59it/s, loss=9.77]
44%|████▍ | 440/1000 [00:16<00:20, 27.59it/s, loss=9.75]
44%|████▍ | 440/1000 [00:16<00:20, 27.59it/s, loss=9.73]
44%|████▍ | 440/1000 [00:16<00:20, 27.59it/s, loss=9.7]
44%|████▍ | 443/1000 [00:16<00:20, 27.40it/s, loss=9.7]
44%|████▍ | 443/1000 [00:16<00:20, 27.40it/s, loss=9.68]
44%|████▍ | 443/1000 [00:16<00:20, 27.40it/s, loss=9.66]
44%|████▍ | 443/1000 [00:16<00:20, 27.40it/s, loss=9.64]
45%|████▍ | 446/1000 [00:16<00:20, 27.31it/s, loss=9.64]
45%|████▍ | 446/1000 [00:16<00:20, 27.31it/s, loss=9.62]
45%|████▍ | 446/1000 [00:16<00:20, 27.31it/s, loss=9.6]
45%|████▍ | 446/1000 [00:16<00:20, 27.31it/s, loss=9.58]
45%|████▍ | 449/1000 [00:16<00:20, 27.09it/s, loss=9.58]
45%|████▍ | 449/1000 [00:17<00:20, 27.09it/s, loss=9.55]
45%|████▍ | 449/1000 [00:17<00:20, 27.09it/s, loss=9.53]
45%|████▍ | 449/1000 [00:17<00:20, 27.09it/s, loss=9.51]
45%|████▌ | 452/1000 [00:17<00:20, 26.90it/s, loss=9.51]
45%|████▌ | 452/1000 [00:17<00:20, 26.90it/s, loss=9.49]
45%|████▌ | 452/1000 [00:17<00:20, 26.90it/s, loss=9.47]
45%|████▌ | 452/1000 [00:17<00:20, 26.90it/s, loss=9.45]
46%|████▌ | 455/1000 [00:17<00:20, 26.72it/s, loss=9.45]
46%|████▌ | 455/1000 [00:17<00:20, 26.72it/s, loss=9.43]
46%|████▌ | 455/1000 [00:17<00:20, 26.72it/s, loss=9.41]
46%|████▌ | 455/1000 [00:17<00:20, 26.72it/s, loss=9.39]
46%|████▌ | 458/1000 [00:17<00:20, 26.52it/s, loss=9.39]
46%|████▌ | 458/1000 [00:17<00:20, 26.52it/s, loss=9.37]
46%|████▌ | 458/1000 [00:17<00:20, 26.52it/s, loss=9.35]
46%|████▌ | 458/1000 [00:17<00:20, 26.52it/s, loss=9.33]
46%|████▌ | 461/1000 [00:17<00:20, 26.32it/s, loss=9.33]
46%|████▌ | 461/1000 [00:17<00:20, 26.32it/s, loss=9.31]
46%|████▌ | 461/1000 [00:17<00:20, 26.32it/s, loss=9.29]
46%|████▌ | 461/1000 [00:17<00:20, 26.32it/s, loss=9.27]
46%|████▋ | 464/1000 [00:17<00:20, 26.18it/s, loss=9.27]
46%|████▋ | 464/1000 [00:17<00:20, 26.18it/s, loss=9.25]
46%|████▋ | 464/1000 [00:17<00:20, 26.18it/s, loss=9.23]
46%|████▋ | 464/1000 [00:17<00:20, 26.18it/s, loss=9.21]
47%|████▋ | 467/1000 [00:17<00:20, 26.06it/s, loss=9.21]
47%|████▋ | 467/1000 [00:17<00:20, 26.06it/s, loss=9.19]
47%|████▋ | 467/1000 [00:17<00:20, 26.06it/s, loss=9.17]
47%|████▋ | 467/1000 [00:17<00:20, 26.06it/s, loss=9.15]
47%|████▋ | 470/1000 [00:17<00:20, 26.08it/s, loss=9.15]
47%|████▋ | 470/1000 [00:17<00:20, 26.08it/s, loss=9.13]
47%|████▋ | 470/1000 [00:17<00:20, 26.08it/s, loss=9.11]
47%|████▋ | 470/1000 [00:17<00:20, 26.08it/s, loss=9.09]
47%|████▋ | 473/1000 [00:17<00:20, 26.19it/s, loss=9.09]
47%|████▋ | 473/1000 [00:17<00:20, 26.19it/s, loss=9.07]
47%|████▋ | 473/1000 [00:17<00:20, 26.19it/s, loss=9.05]
47%|████▋ | 473/1000 [00:18<00:20, 26.19it/s, loss=9.03]
48%|████▊ | 476/1000 [00:18<00:19, 26.30it/s, loss=9.03]
48%|████▊ | 476/1000 [00:18<00:19, 26.30it/s, loss=9.02]
48%|████▊ | 476/1000 [00:18<00:19, 26.30it/s, loss=9]
48%|████▊ | 476/1000 [00:18<00:19, 26.30it/s, loss=8.98]
48%|████▊ | 479/1000 [00:18<00:19, 26.29it/s, loss=8.98]
48%|████▊ | 479/1000 [00:18<00:19, 26.29it/s, loss=8.96]
48%|████▊ | 479/1000 [00:18<00:19, 26.29it/s, loss=8.94]
48%|████▊ | 479/1000 [00:18<00:19, 26.29it/s, loss=8.92]
48%|████▊ | 482/1000 [00:18<00:19, 26.36it/s, loss=8.92]
48%|████▊ | 482/1000 [00:18<00:19, 26.36it/s, loss=8.9]
48%|████▊ | 482/1000 [00:18<00:19, 26.36it/s, loss=8.89]
48%|████▊ | 482/1000 [00:18<00:19, 26.36it/s, loss=8.87]
48%|████▊ | 485/1000 [00:18<00:19, 26.48it/s, loss=8.87]
48%|████▊ | 485/1000 [00:18<00:19, 26.48it/s, loss=8.85]
48%|████▊ | 485/1000 [00:18<00:19, 26.48it/s, loss=8.83]
48%|████▊ | 485/1000 [00:18<00:19, 26.48it/s, loss=8.81]
49%|████▉ | 488/1000 [00:18<00:19, 26.50it/s, loss=8.81]
49%|████▉ | 488/1000 [00:18<00:19, 26.50it/s, loss=8.8]
49%|████▉ | 488/1000 [00:18<00:19, 26.50it/s, loss=8.78]
49%|████▉ | 488/1000 [00:18<00:19, 26.50it/s, loss=8.76]
49%|████▉ | 491/1000 [00:18<00:19, 26.47it/s, loss=8.76]
49%|████▉ | 491/1000 [00:18<00:19, 26.47it/s, loss=8.74]
49%|████▉ | 491/1000 [00:18<00:19, 26.47it/s, loss=8.72]
49%|████▉ | 491/1000 [00:18<00:19, 26.47it/s, loss=8.71]
49%|████▉ | 494/1000 [00:18<00:19, 26.22it/s, loss=8.71]
49%|████▉ | 494/1000 [00:18<00:19, 26.22it/s, loss=8.69]
49%|████▉ | 494/1000 [00:18<00:19, 26.22it/s, loss=8.67]
49%|████▉ | 494/1000 [00:18<00:19, 26.22it/s, loss=8.65]
50%|████▉ | 497/1000 [00:18<00:19, 26.09it/s, loss=8.65]
50%|████▉ | 497/1000 [00:18<00:19, 26.09it/s, loss=8.64]
50%|████▉ | 497/1000 [00:18<00:19, 26.09it/s, loss=8.62]
50%|████▉ | 497/1000 [00:18<00:19, 26.09it/s, loss=8.6]
50%|█████ | 500/1000 [00:18<00:19, 26.01it/s, loss=8.6]
50%|█████ | 500/1000 [00:18<00:19, 26.01it/s, loss=8.59]
50%|█████ | 500/1000 [00:19<00:19, 26.01it/s, loss=8.57]
50%|█████ | 500/1000 [00:19<00:19, 26.01it/s, loss=8.55]
50%|█████ | 503/1000 [00:19<00:19, 26.02it/s, loss=8.55]
50%|█████ | 503/1000 [00:19<00:19, 26.02it/s, loss=8.53]
50%|█████ | 503/1000 [00:19<00:19, 26.02it/s, loss=8.52]
50%|█████ | 503/1000 [00:19<00:19, 26.02it/s, loss=8.5]
51%|█████ | 506/1000 [00:19<00:19, 25.67it/s, loss=8.5]
51%|█████ | 506/1000 [00:19<00:19, 25.67it/s, loss=8.48]
51%|█████ | 506/1000 [00:19<00:19, 25.67it/s, loss=8.47]
51%|█████ | 506/1000 [00:19<00:19, 25.67it/s, loss=8.45]
51%|█████ | 509/1000 [00:19<00:19, 25.40it/s, loss=8.45]
51%|█████ | 509/1000 [00:19<00:19, 25.40it/s, loss=8.43]
51%|█████ | 509/1000 [00:19<00:19, 25.40it/s, loss=8.42]
51%|█████ | 509/1000 [00:19<00:19, 25.40it/s, loss=8.4]
51%|█████ | 512/1000 [00:19<00:19, 25.39it/s, loss=8.4]
51%|█████ | 512/1000 [00:19<00:19, 25.39it/s, loss=8.38]
51%|█████ | 512/1000 [00:19<00:19, 25.39it/s, loss=8.37]
51%|█████ | 512/1000 [00:19<00:19, 25.39it/s, loss=8.35]
52%|█████▏ | 515/1000 [00:19<00:19, 25.49it/s, loss=8.35]
52%|█████▏ | 515/1000 [00:19<00:19, 25.49it/s, loss=8.34]
52%|█████▏ | 515/1000 [00:19<00:19, 25.49it/s, loss=8.32]
52%|█████▏ | 515/1000 [00:19<00:19, 25.49it/s, loss=8.3]
52%|█████▏ | 518/1000 [00:19<00:18, 25.96it/s, loss=8.3]
52%|█████▏ | 518/1000 [00:19<00:18, 25.96it/s, loss=8.29]
52%|█████▏ | 518/1000 [00:19<00:18, 25.96it/s, loss=8.27]
52%|█████▏ | 518/1000 [00:19<00:18, 25.96it/s, loss=8.26]
52%|█████▏ | 521/1000 [00:19<00:18, 26.22it/s, loss=8.26]
52%|█████▏ | 521/1000 [00:19<00:18, 26.22it/s, loss=8.24]
52%|█████▏ | 521/1000 [00:19<00:18, 26.22it/s, loss=8.22]
52%|█████▏ | 521/1000 [00:19<00:18, 26.22it/s, loss=8.21]
52%|█████▏ | 524/1000 [00:19<00:17, 26.57it/s, loss=8.21]
52%|█████▏ | 524/1000 [00:19<00:17, 26.57it/s, loss=8.19]
52%|█████▏ | 524/1000 [00:19<00:17, 26.57it/s, loss=8.18]
52%|█████▏ | 524/1000 [00:19<00:17, 26.57it/s, loss=8.16]
53%|█████▎ | 527/1000 [00:19<00:17, 26.57it/s, loss=8.16]
53%|█████▎ | 527/1000 [00:19<00:17, 26.57it/s, loss=8.15]
53%|█████▎ | 527/1000 [00:20<00:17, 26.57it/s, loss=8.13]
53%|█████▎ | 527/1000 [00:20<00:17, 26.57it/s, loss=8.12]
53%|█████▎ | 530/1000 [00:20<00:17, 26.34it/s, loss=8.12]
53%|█████▎ | 530/1000 [00:20<00:17, 26.34it/s, loss=8.1]
53%|█████▎ | 530/1000 [00:20<00:17, 26.34it/s, loss=8.09]
53%|█████▎ | 530/1000 [00:20<00:17, 26.34it/s, loss=8.07]
53%|█████▎ | 533/1000 [00:20<00:17, 26.27it/s, loss=8.07]
53%|█████▎ | 533/1000 [00:20<00:17, 26.27it/s, loss=8.06]
53%|█████▎ | 533/1000 [00:20<00:17, 26.27it/s, loss=8.04]
53%|█████▎ | 533/1000 [00:20<00:17, 26.27it/s, loss=8.03]
54%|█████▎ | 536/1000 [00:20<00:17, 26.31it/s, loss=8.03]
54%|█████▎ | 536/1000 [00:20<00:17, 26.31it/s, loss=8.01]
54%|█████▎ | 536/1000 [00:20<00:17, 26.31it/s, loss=8]
54%|█████▎ | 536/1000 [00:20<00:17, 26.31it/s, loss=7.98]
54%|█████▍ | 539/1000 [00:20<00:17, 26.45it/s, loss=7.98]
54%|█████▍ | 539/1000 [00:20<00:17, 26.45it/s, loss=7.97]
54%|█████▍ | 539/1000 [00:20<00:17, 26.45it/s, loss=7.95]
54%|█████▍ | 539/1000 [00:20<00:17, 26.45it/s, loss=7.94]
54%|█████▍ | 542/1000 [00:20<00:17, 26.51it/s, loss=7.94]
54%|█████▍ | 542/1000 [00:20<00:17, 26.51it/s, loss=7.92]
54%|█████▍ | 542/1000 [00:20<00:17, 26.51it/s, loss=7.91]
54%|█████▍ | 542/1000 [00:20<00:17, 26.51it/s, loss=7.89]
55%|█████▍ | 545/1000 [00:20<00:17, 26.56it/s, loss=7.89]
55%|█████▍ | 545/1000 [00:20<00:17, 26.56it/s, loss=7.88]
55%|█████▍ | 545/1000 [00:20<00:17, 26.56it/s, loss=7.86]
55%|█████▍ | 545/1000 [00:20<00:17, 26.56it/s, loss=7.85]
55%|█████▍ | 548/1000 [00:20<00:17, 26.56it/s, loss=7.85]
55%|█████▍ | 548/1000 [00:20<00:17, 26.56it/s, loss=7.84]
55%|█████▍ | 548/1000 [00:20<00:17, 26.56it/s, loss=7.82]
55%|█████▍ | 548/1000 [00:20<00:17, 26.56it/s, loss=7.81]
55%|█████▌ | 551/1000 [00:20<00:16, 26.50it/s, loss=7.81]
55%|█████▌ | 551/1000 [00:20<00:16, 26.50it/s, loss=7.79]
55%|█████▌ | 551/1000 [00:20<00:16, 26.50it/s, loss=7.78]
55%|█████▌ | 551/1000 [00:20<00:16, 26.50it/s, loss=7.76]
55%|█████▌ | 554/1000 [00:20<00:16, 26.52it/s, loss=7.76]
55%|█████▌ | 554/1000 [00:21<00:16, 26.52it/s, loss=7.75]
55%|█████▌ | 554/1000 [00:21<00:16, 26.52it/s, loss=7.74]
55%|█████▌ | 554/1000 [00:21<00:16, 26.52it/s, loss=7.72]
56%|█████▌ | 557/1000 [00:21<00:16, 26.54it/s, loss=7.72]
56%|█████▌ | 557/1000 [00:21<00:16, 26.54it/s, loss=7.71]
56%|█████▌ | 557/1000 [00:21<00:16, 26.54it/s, loss=7.7]
56%|█████▌ | 557/1000 [00:21<00:16, 26.54it/s, loss=7.68]
56%|█████▌ | 560/1000 [00:21<00:16, 26.42it/s, loss=7.68]
56%|█████▌ | 560/1000 [00:21<00:16, 26.42it/s, loss=7.67]
56%|█████▌ | 560/1000 [00:21<00:16, 26.42it/s, loss=7.65]
56%|█████▌ | 560/1000 [00:21<00:16, 26.42it/s, loss=7.64]
56%|█████▋ | 563/1000 [00:21<00:16, 26.07it/s, loss=7.64]
56%|█████▋ | 563/1000 [00:21<00:16, 26.07it/s, loss=7.63]
56%|█████▋ | 563/1000 [00:21<00:16, 26.07it/s, loss=7.61]
56%|█████▋ | 563/1000 [00:21<00:16, 26.07it/s, loss=7.6]
57%|█████▋ | 566/1000 [00:21<00:16, 26.09it/s, loss=7.6]
57%|█████▋ | 566/1000 [00:21<00:16, 26.09it/s, loss=7.59]
57%|█████▋ | 566/1000 [00:21<00:16, 26.09it/s, loss=7.57]
57%|█████▋ | 566/1000 [00:21<00:16, 26.09it/s, loss=7.56]
57%|█████▋ | 569/1000 [00:21<00:16, 26.22it/s, loss=7.56]
57%|█████▋ | 569/1000 [00:21<00:16, 26.22it/s, loss=7.55]
57%|█████▋ | 569/1000 [00:21<00:16, 26.22it/s, loss=7.53]
57%|█████▋ | 569/1000 [00:21<00:16, 26.22it/s, loss=7.52]
57%|█████▋ | 572/1000 [00:21<00:16, 26.23it/s, loss=7.52]
57%|█████▋ | 572/1000 [00:21<00:16, 26.23it/s, loss=7.51]
57%|█████▋ | 572/1000 [00:21<00:16, 26.23it/s, loss=7.49]
57%|█████▋ | 572/1000 [00:21<00:16, 26.23it/s, loss=7.48]
57%|█████▊ | 575/1000 [00:21<00:16, 26.18it/s, loss=7.48]
57%|█████▊ | 575/1000 [00:21<00:16, 26.18it/s, loss=7.47]
57%|█████▊ | 575/1000 [00:21<00:16, 26.18it/s, loss=7.46]
57%|█████▊ | 575/1000 [00:21<00:16, 26.18it/s, loss=7.44]
58%|█████▊ | 578/1000 [00:21<00:16, 26.04it/s, loss=7.44]
58%|█████▊ | 578/1000 [00:21<00:16, 26.04it/s, loss=7.43]
58%|█████▊ | 578/1000 [00:21<00:16, 26.04it/s, loss=7.42]
58%|█████▊ | 578/1000 [00:22<00:16, 26.04it/s, loss=7.4]
58%|█████▊ | 581/1000 [00:22<00:16, 26.04it/s, loss=7.4]
58%|█████▊ | 581/1000 [00:22<00:16, 26.04it/s, loss=7.39]
58%|█████▊ | 581/1000 [00:22<00:16, 26.04it/s, loss=7.38]
58%|█████▊ | 581/1000 [00:22<00:16, 26.04it/s, loss=7.37]
58%|█████▊ | 584/1000 [00:22<00:16, 25.93it/s, loss=7.37]
58%|█████▊ | 584/1000 [00:22<00:16, 25.93it/s, loss=7.35]
58%|█████▊ | 584/1000 [00:22<00:16, 25.93it/s, loss=7.34]
58%|█████▊ | 584/1000 [00:22<00:16, 25.93it/s, loss=7.33]
59%|█████▊ | 587/1000 [00:22<00:15, 25.96it/s, loss=7.33]
59%|█████▊ | 587/1000 [00:22<00:15, 25.96it/s, loss=7.32]
59%|█████▊ | 587/1000 [00:22<00:15, 25.96it/s, loss=7.3]
59%|█████▊ | 587/1000 [00:22<00:15, 25.96it/s, loss=7.29]
59%|█████▉ | 590/1000 [00:22<00:15, 25.99it/s, loss=7.29]
59%|█████▉ | 590/1000 [00:22<00:15, 25.99it/s, loss=7.28]
59%|█████▉ | 590/1000 [00:22<00:15, 25.99it/s, loss=7.27]
59%|█████▉ | 590/1000 [00:22<00:15, 25.99it/s, loss=7.25]
59%|█████▉ | 593/1000 [00:22<00:15, 26.02it/s, loss=7.25]
59%|█████▉ | 593/1000 [00:22<00:15, 26.02it/s, loss=7.24]
59%|█████▉ | 593/1000 [00:22<00:15, 26.02it/s, loss=7.23]
59%|█████▉ | 593/1000 [00:22<00:15, 26.02it/s, loss=7.22]
60%|█████▉ | 596/1000 [00:22<00:15, 26.13it/s, loss=7.22]
60%|█████▉ | 596/1000 [00:22<00:15, 26.13it/s, loss=7.21]
60%|█████▉ | 596/1000 [00:22<00:15, 26.13it/s, loss=7.19]
60%|█████▉ | 596/1000 [00:22<00:15, 26.13it/s, loss=7.18]
60%|█████▉ | 599/1000 [00:22<00:15, 26.16it/s, loss=7.18]
60%|█████▉ | 599/1000 [00:22<00:15, 26.16it/s, loss=7.17]
60%|█████▉ | 599/1000 [00:22<00:15, 26.16it/s, loss=7.16]
60%|█████▉ | 599/1000 [00:22<00:15, 26.16it/s, loss=7.15]
60%|██████ | 602/1000 [00:22<00:15, 25.89it/s, loss=7.15]
60%|██████ | 602/1000 [00:22<00:15, 25.89it/s, loss=7.13]
60%|██████ | 602/1000 [00:22<00:15, 25.89it/s, loss=7.12]
60%|██████ | 602/1000 [00:22<00:15, 25.89it/s, loss=7.11]
60%|██████ | 605/1000 [00:22<00:15, 26.16it/s, loss=7.11]
60%|██████ | 605/1000 [00:22<00:15, 26.16it/s, loss=7.1]
60%|██████ | 605/1000 [00:23<00:15, 26.16it/s, loss=7.09]
60%|██████ | 605/1000 [00:23<00:15, 26.16it/s, loss=7.08]
61%|██████ | 608/1000 [00:23<00:14, 26.45it/s, loss=7.08]
61%|██████ | 608/1000 [00:23<00:14, 26.45it/s, loss=7.06]
61%|██████ | 608/1000 [00:23<00:14, 26.45it/s, loss=7.05]
61%|██████ | 608/1000 [00:23<00:14, 26.45it/s, loss=7.04]
61%|██████ | 611/1000 [00:23<00:14, 26.60it/s, loss=7.04]
61%|██████ | 611/1000 [00:23<00:14, 26.60it/s, loss=7.03]
61%|██████ | 611/1000 [00:23<00:14, 26.60it/s, loss=7.02]
61%|██████ | 611/1000 [00:23<00:14, 26.60it/s, loss=7.01]
61%|██████▏ | 614/1000 [00:23<00:14, 26.45it/s, loss=7.01]
61%|██████▏ | 614/1000 [00:23<00:14, 26.45it/s, loss=7]
61%|██████▏ | 614/1000 [00:23<00:14, 26.45it/s, loss=6.98]
61%|██████▏ | 614/1000 [00:23<00:14, 26.45it/s, loss=6.97]
62%|██████▏ | 617/1000 [00:23<00:14, 26.35it/s, loss=6.97]
62%|██████▏ | 617/1000 [00:23<00:14, 26.35it/s, loss=6.96]
62%|██████▏ | 617/1000 [00:23<00:14, 26.35it/s, loss=6.95]
62%|██████▏ | 617/1000 [00:23<00:14, 26.35it/s, loss=6.94]
62%|██████▏ | 620/1000 [00:23<00:14, 26.27it/s, loss=6.94]
62%|██████▏ | 620/1000 [00:23<00:14, 26.27it/s, loss=6.93]
62%|██████▏ | 620/1000 [00:23<00:14, 26.27it/s, loss=6.92]
62%|██████▏ | 620/1000 [00:23<00:14, 26.27it/s, loss=6.91]
62%|██████▏ | 623/1000 [00:23<00:14, 26.11it/s, loss=6.91]
62%|██████▏ | 623/1000 [00:23<00:14, 26.11it/s, loss=6.89]
62%|██████▏ | 623/1000 [00:23<00:14, 26.11it/s, loss=6.88]
62%|██████▏ | 623/1000 [00:23<00:14, 26.11it/s, loss=6.87]
63%|██████▎ | 626/1000 [00:23<00:14, 26.31it/s, loss=6.87]
63%|██████▎ | 626/1000 [00:23<00:14, 26.31it/s, loss=6.86]
63%|██████▎ | 626/1000 [00:23<00:14, 26.31it/s, loss=6.85]
63%|██████▎ | 626/1000 [00:23<00:14, 26.31it/s, loss=6.84]
63%|██████▎ | 629/1000 [00:23<00:14, 26.43it/s, loss=6.84]
63%|██████▎ | 629/1000 [00:23<00:14, 26.43it/s, loss=6.83]
63%|██████▎ | 629/1000 [00:23<00:14, 26.43it/s, loss=6.82]
63%|██████▎ | 629/1000 [00:23<00:14, 26.43it/s, loss=6.81]
63%|██████▎ | 632/1000 [00:23<00:14, 26.25it/s, loss=6.81]
63%|██████▎ | 632/1000 [00:23<00:14, 26.25it/s, loss=6.8]
63%|██████▎ | 632/1000 [00:24<00:14, 26.25it/s, loss=6.79]
63%|██████▎ | 632/1000 [00:24<00:14, 26.25it/s, loss=6.77]
64%|██████▎ | 635/1000 [00:24<00:13, 26.44it/s, loss=6.77]
64%|██████▎ | 635/1000 [00:24<00:13, 26.44it/s, loss=6.76]
64%|██████▎ | 635/1000 [00:24<00:13, 26.44it/s, loss=6.75]
64%|██████▎ | 635/1000 [00:24<00:13, 26.44it/s, loss=6.74]
64%|██████▍ | 638/1000 [00:24<00:13, 26.32it/s, loss=6.74]
64%|██████▍ | 638/1000 [00:24<00:13, 26.32it/s, loss=6.73]
64%|██████▍ | 638/1000 [00:24<00:13, 26.32it/s, loss=6.72]
64%|██████▍ | 638/1000 [00:24<00:13, 26.32it/s, loss=6.71]
64%|██████▍ | 641/1000 [00:24<00:13, 26.38it/s, loss=6.71]
64%|██████▍ | 641/1000 [00:24<00:13, 26.38it/s, loss=6.7]
64%|██████▍ | 641/1000 [00:24<00:13, 26.38it/s, loss=6.69]
64%|██████▍ | 641/1000 [00:24<00:13, 26.38it/s, loss=6.68]
64%|██████▍ | 644/1000 [00:24<00:13, 26.14it/s, loss=6.68]
64%|██████▍ | 644/1000 [00:24<00:13, 26.14it/s, loss=6.67]
64%|██████▍ | 644/1000 [00:24<00:13, 26.14it/s, loss=6.66]
64%|██████▍ | 644/1000 [00:24<00:13, 26.14it/s, loss=6.65]
65%|██████▍ | 647/1000 [00:24<00:13, 26.04it/s, loss=6.65]
65%|██████▍ | 647/1000 [00:24<00:13, 26.04it/s, loss=6.64]
65%|██████▍ | 647/1000 [00:24<00:13, 26.04it/s, loss=6.63]
65%|██████▍ | 647/1000 [00:24<00:13, 26.04it/s, loss=6.62]
65%|██████▌ | 650/1000 [00:24<00:13, 26.18it/s, loss=6.62]
65%|██████▌ | 650/1000 [00:24<00:13, 26.18it/s, loss=6.61]
65%|██████▌ | 650/1000 [00:24<00:13, 26.18it/s, loss=6.6]
65%|██████▌ | 650/1000 [00:24<00:13, 26.18it/s, loss=6.59]
65%|██████▌ | 653/1000 [00:24<00:13, 26.42it/s, loss=6.59]
65%|██████▌ | 653/1000 [00:24<00:13, 26.42it/s, loss=6.58]
65%|██████▌ | 653/1000 [00:24<00:13, 26.42it/s, loss=6.57]
65%|██████▌ | 653/1000 [00:24<00:13, 26.42it/s, loss=6.56]
66%|██████▌ | 656/1000 [00:24<00:12, 26.59it/s, loss=6.56]
66%|██████▌ | 656/1000 [00:24<00:12, 26.59it/s, loss=6.55]
66%|██████▌ | 656/1000 [00:24<00:12, 26.59it/s, loss=6.54]
66%|██████▌ | 656/1000 [00:24<00:12, 26.59it/s, loss=6.53]
66%|██████▌ | 659/1000 [00:24<00:12, 26.54it/s, loss=6.53]
66%|██████▌ | 659/1000 [00:25<00:12, 26.54it/s, loss=6.52]
66%|██████▌ | 659/1000 [00:25<00:12, 26.54it/s, loss=6.51]
66%|██████▌ | 659/1000 [00:25<00:12, 26.54it/s, loss=6.5]
66%|██████▌ | 662/1000 [00:25<00:12, 26.42it/s, loss=6.5]
66%|██████▌ | 662/1000 [00:25<00:12, 26.42it/s, loss=6.49]
66%|██████▌ | 662/1000 [00:25<00:12, 26.42it/s, loss=6.48]
66%|██████▌ | 662/1000 [00:25<00:12, 26.42it/s, loss=6.47]
66%|██████▋ | 665/1000 [00:25<00:12, 26.40it/s, loss=6.47]
66%|██████▋ | 665/1000 [00:25<00:12, 26.40it/s, loss=6.46]
66%|██████▋ | 665/1000 [00:25<00:12, 26.40it/s, loss=6.45]
66%|██████▋ | 665/1000 [00:25<00:12, 26.40it/s, loss=6.44]
67%|██████▋ | 668/1000 [00:25<00:12, 26.40it/s, loss=6.44]
67%|██████▋ | 668/1000 [00:25<00:12, 26.40it/s, loss=6.43]
67%|██████▋ | 668/1000 [00:25<00:12, 26.40it/s, loss=6.42]
67%|██████▋ | 668/1000 [00:25<00:12, 26.40it/s, loss=6.41]
67%|██████▋ | 671/1000 [00:25<00:12, 26.37it/s, loss=6.41]
67%|██████▋ | 671/1000 [00:25<00:12, 26.37it/s, loss=6.4]
67%|██████▋ | 671/1000 [00:25<00:12, 26.37it/s, loss=6.39]
67%|██████▋ | 671/1000 [00:25<00:12, 26.37it/s, loss=6.38]
67%|██████▋ | 674/1000 [00:25<00:12, 26.28it/s, loss=6.38]
67%|██████▋ | 674/1000 [00:25<00:12, 26.28it/s, loss=6.37]
67%|██████▋ | 674/1000 [00:25<00:12, 26.28it/s, loss=6.36]
67%|██████▋ | 674/1000 [00:25<00:12, 26.28it/s, loss=6.35]
68%|██████▊ | 677/1000 [00:25<00:12, 26.17it/s, loss=6.35]
68%|██████▊ | 677/1000 [00:25<00:12, 26.17it/s, loss=6.35]
68%|██████▊ | 677/1000 [00:25<00:12, 26.17it/s, loss=6.34]
68%|██████▊ | 677/1000 [00:25<00:12, 26.17it/s, loss=6.33]
68%|██████▊ | 680/1000 [00:25<00:12, 26.19it/s, loss=6.33]
68%|██████▊ | 680/1000 [00:25<00:12, 26.19it/s, loss=6.32]
68%|██████▊ | 680/1000 [00:25<00:12, 26.19it/s, loss=6.31]
68%|██████▊ | 680/1000 [00:25<00:12, 26.19it/s, loss=6.3]
68%|██████▊ | 683/1000 [00:25<00:11, 26.44it/s, loss=6.3]
68%|██████▊ | 683/1000 [00:25<00:11, 26.44it/s, loss=6.29]
68%|██████▊ | 683/1000 [00:25<00:11, 26.44it/s, loss=6.28]
68%|██████▊ | 683/1000 [00:26<00:11, 26.44it/s, loss=6.27]
69%|██████▊ | 686/1000 [00:26<00:11, 26.76it/s, loss=6.27]
69%|██████▊ | 686/1000 [00:26<00:11, 26.76it/s, loss=6.26]
69%|██████▊ | 686/1000 [00:26<00:11, 26.76it/s, loss=6.25]
69%|██████▊ | 686/1000 [00:26<00:11, 26.76it/s, loss=6.24]
69%|██████▉ | 689/1000 [00:26<00:11, 26.89it/s, loss=6.24]
69%|██████▉ | 689/1000 [00:26<00:11, 26.89it/s, loss=6.23]
69%|██████▉ | 689/1000 [00:26<00:11, 26.89it/s, loss=6.23]
69%|██████▉ | 689/1000 [00:26<00:11, 26.89it/s, loss=6.22]
69%|██████▉ | 692/1000 [00:26<00:11, 27.03it/s, loss=6.22]
69%|██████▉ | 692/1000 [00:26<00:11, 27.03it/s, loss=6.21]
69%|██████▉ | 692/1000 [00:26<00:11, 27.03it/s, loss=6.2]
69%|██████▉ | 692/1000 [00:26<00:11, 27.03it/s, loss=6.19]
70%|██████▉ | 695/1000 [00:26<00:11, 26.98it/s, loss=6.19]
70%|██████▉ | 695/1000 [00:26<00:11, 26.98it/s, loss=6.18]
70%|██████▉ | 695/1000 [00:26<00:11, 26.98it/s, loss=6.17]
70%|██████▉ | 695/1000 [00:26<00:11, 26.98it/s, loss=6.16]
70%|██████▉ | 698/1000 [00:26<00:11, 26.92it/s, loss=6.16]
70%|██████▉ | 698/1000 [00:26<00:11, 26.92it/s, loss=6.15]
70%|██████▉ | 698/1000 [00:26<00:11, 26.92it/s, loss=6.15]
70%|██████▉ | 698/1000 [00:26<00:11, 26.92it/s, loss=6.14]
70%|███████ | 701/1000 [00:26<00:11, 26.88it/s, loss=6.14]
70%|███████ | 701/1000 [00:26<00:11, 26.88it/s, loss=6.13]
70%|███████ | 701/1000 [00:26<00:11, 26.88it/s, loss=6.12]
70%|███████ | 701/1000 [00:26<00:11, 26.88it/s, loss=6.11]
70%|███████ | 704/1000 [00:26<00:11, 26.86it/s, loss=6.11]
70%|███████ | 704/1000 [00:26<00:11, 26.86it/s, loss=6.1]
70%|███████ | 704/1000 [00:26<00:11, 26.86it/s, loss=6.09]
70%|███████ | 704/1000 [00:26<00:11, 26.86it/s, loss=6.08]
71%|███████ | 707/1000 [00:26<00:10, 26.76it/s, loss=6.08]
71%|███████ | 707/1000 [00:26<00:10, 26.76it/s, loss=6.08]
71%|███████ | 707/1000 [00:26<00:10, 26.76it/s, loss=6.07]
71%|███████ | 707/1000 [00:26<00:10, 26.76it/s, loss=6.06]
71%|███████ | 710/1000 [00:26<00:10, 26.62it/s, loss=6.06]
71%|███████ | 710/1000 [00:26<00:10, 26.62it/s, loss=6.05]
71%|███████ | 710/1000 [00:26<00:10, 26.62it/s, loss=6.04]
71%|███████ | 710/1000 [00:27<00:10, 26.62it/s, loss=6.03]
71%|███████▏ | 713/1000 [00:27<00:10, 26.55it/s, loss=6.03]
71%|███████▏ | 713/1000 [00:27<00:10, 26.55it/s, loss=6.03]
71%|███████▏ | 713/1000 [00:27<00:10, 26.55it/s, loss=6.02]
71%|███████▏ | 713/1000 [00:27<00:10, 26.55it/s, loss=6.01]
72%|███████▏ | 716/1000 [00:27<00:10, 26.52it/s, loss=6.01]
72%|███████▏ | 716/1000 [00:27<00:10, 26.52it/s, loss=6]
72%|███████▏ | 716/1000 [00:27<00:10, 26.52it/s, loss=5.99]
72%|███████▏ | 716/1000 [00:27<00:10, 26.52it/s, loss=5.98]
72%|███████▏ | 719/1000 [00:27<00:10, 26.37it/s, loss=5.98]
72%|███████▏ | 719/1000 [00:27<00:10, 26.37it/s, loss=5.97]
72%|███████▏ | 719/1000 [00:27<00:10, 26.37it/s, loss=5.97]
72%|███████▏ | 719/1000 [00:27<00:10, 26.37it/s, loss=5.96]
72%|███████▏ | 722/1000 [00:27<00:10, 26.37it/s, loss=5.96]
72%|███████▏ | 722/1000 [00:27<00:10, 26.37it/s, loss=5.95]
72%|███████▏ | 722/1000 [00:27<00:10, 26.37it/s, loss=5.94]
72%|███████▏ | 722/1000 [00:27<00:10, 26.37it/s, loss=5.93]
72%|███████▎ | 725/1000 [00:27<00:10, 26.47it/s, loss=5.93]
72%|███████▎ | 725/1000 [00:27<00:10, 26.47it/s, loss=5.93]
72%|███████▎ | 725/1000 [00:27<00:10, 26.47it/s, loss=5.92]
72%|███████▎ | 725/1000 [00:27<00:10, 26.47it/s, loss=5.91]
73%|███████▎ | 728/1000 [00:27<00:10, 26.21it/s, loss=5.91]
73%|███████▎ | 728/1000 [00:27<00:10, 26.21it/s, loss=5.9]
73%|███████▎ | 728/1000 [00:27<00:10, 26.21it/s, loss=5.89]
73%|███████▎ | 728/1000 [00:27<00:10, 26.21it/s, loss=5.89]
73%|███████▎ | 731/1000 [00:27<00:10, 25.99it/s, loss=5.89]
73%|███████▎ | 731/1000 [00:27<00:10, 25.99it/s, loss=5.88]
73%|███████▎ | 731/1000 [00:27<00:10, 25.99it/s, loss=5.87]
73%|███████▎ | 731/1000 [00:27<00:10, 25.99it/s, loss=5.86]
73%|███████▎ | 734/1000 [00:27<00:10, 25.89it/s, loss=5.86]
73%|███████▎ | 734/1000 [00:27<00:10, 25.89it/s, loss=5.85]
73%|███████▎ | 734/1000 [00:27<00:10, 25.89it/s, loss=5.85]
73%|███████▎ | 734/1000 [00:27<00:10, 25.89it/s, loss=5.84]
74%|███████▎ | 737/1000 [00:27<00:10, 25.66it/s, loss=5.84]
74%|███████▎ | 737/1000 [00:27<00:10, 25.66it/s, loss=5.83]
74%|███████▎ | 737/1000 [00:28<00:10, 25.66it/s, loss=5.82]
74%|███████▎ | 737/1000 [00:28<00:10, 25.66it/s, loss=5.81]
74%|███████▍ | 740/1000 [00:28<00:10, 25.85it/s, loss=5.81]
74%|███████▍ | 740/1000 [00:28<00:10, 25.85it/s, loss=5.81]
74%|███████▍ | 740/1000 [00:28<00:10, 25.85it/s, loss=5.8]
74%|███████▍ | 740/1000 [00:28<00:10, 25.85it/s, loss=5.79]
74%|███████▍ | 743/1000 [00:28<00:09, 25.94it/s, loss=5.79]
74%|███████▍ | 743/1000 [00:28<00:09, 25.94it/s, loss=5.78]
74%|███████▍ | 743/1000 [00:28<00:09, 25.94it/s, loss=5.77]
74%|███████▍ | 743/1000 [00:28<00:09, 25.94it/s, loss=5.77]
75%|███████▍ | 746/1000 [00:28<00:09, 26.09it/s, loss=5.77]
75%|███████▍ | 746/1000 [00:28<00:09, 26.09it/s, loss=5.76]
75%|███████▍ | 746/1000 [00:28<00:09, 26.09it/s, loss=5.75]
75%|███████▍ | 746/1000 [00:28<00:09, 26.09it/s, loss=5.74]
75%|███████▍ | 749/1000 [00:28<00:09, 26.20it/s, loss=5.74]
75%|███████▍ | 749/1000 [00:28<00:09, 26.20it/s, loss=5.74]
75%|███████▍ | 749/1000 [00:28<00:09, 26.20it/s, loss=5.73]
75%|███████▍ | 749/1000 [00:28<00:09, 26.20it/s, loss=5.72]
75%|███████▌ | 752/1000 [00:28<00:09, 26.27it/s, loss=5.72]
75%|███████▌ | 752/1000 [00:28<00:09, 26.27it/s, loss=5.71]
75%|███████▌ | 752/1000 [00:28<00:09, 26.27it/s, loss=5.71]
75%|███████▌ | 752/1000 [00:28<00:09, 26.27it/s, loss=5.7]
76%|███████▌ | 755/1000 [00:28<00:09, 26.53it/s, loss=5.7]
76%|███████▌ | 755/1000 [00:28<00:09, 26.53it/s, loss=5.69]
76%|███████▌ | 755/1000 [00:28<00:09, 26.53it/s, loss=5.68]
76%|███████▌ | 755/1000 [00:28<00:09, 26.53it/s, loss=5.68]
76%|███████▌ | 758/1000 [00:28<00:09, 26.65it/s, loss=5.68]
76%|███████▌ | 758/1000 [00:28<00:09, 26.65it/s, loss=5.67]
76%|███████▌ | 758/1000 [00:28<00:09, 26.65it/s, loss=5.66]
76%|███████▌ | 758/1000 [00:28<00:09, 26.65it/s, loss=5.65]
76%|███████▌ | 761/1000 [00:28<00:08, 26.80it/s, loss=5.65]
76%|███████▌ | 761/1000 [00:28<00:08, 26.80it/s, loss=5.65]
76%|███████▌ | 761/1000 [00:28<00:08, 26.80it/s, loss=5.64]
76%|███████▌ | 761/1000 [00:28<00:08, 26.80it/s, loss=5.63]
76%|███████▋ | 764/1000 [00:28<00:08, 26.90it/s, loss=5.63]
76%|███████▋ | 764/1000 [00:28<00:08, 26.90it/s, loss=5.62]
76%|███████▋ | 764/1000 [00:29<00:08, 26.90it/s, loss=5.62]
76%|███████▋ | 764/1000 [00:29<00:08, 26.90it/s, loss=5.61]
77%|███████▋ | 767/1000 [00:29<00:08, 26.93it/s, loss=5.61]
77%|███████▋ | 767/1000 [00:29<00:08, 26.93it/s, loss=5.6]
77%|███████▋ | 767/1000 [00:29<00:08, 26.93it/s, loss=5.59]
77%|███████▋ | 767/1000 [00:29<00:08, 26.93it/s, loss=5.59]
77%|███████▋ | 770/1000 [00:29<00:08, 27.07it/s, loss=5.59]
77%|███████▋ | 770/1000 [00:29<00:08, 27.07it/s, loss=5.58]
77%|███████▋ | 770/1000 [00:29<00:08, 27.07it/s, loss=5.57]
77%|███████▋ | 770/1000 [00:29<00:08, 27.07it/s, loss=5.57]
77%|███████▋ | 773/1000 [00:29<00:08, 27.24it/s, loss=5.57]
77%|███████▋ | 773/1000 [00:29<00:08, 27.24it/s, loss=5.56]
77%|███████▋ | 773/1000 [00:29<00:08, 27.24it/s, loss=5.55]
77%|███████▋ | 773/1000 [00:29<00:08, 27.24it/s, loss=5.54]
78%|███████▊ | 776/1000 [00:29<00:08, 27.22it/s, loss=5.54]
78%|███████▊ | 776/1000 [00:29<00:08, 27.22it/s, loss=5.54]
78%|███████▊ | 776/1000 [00:29<00:08, 27.22it/s, loss=5.53]
78%|███████▊ | 776/1000 [00:29<00:08, 27.22it/s, loss=5.52]
78%|███████▊ | 779/1000 [00:29<00:08, 27.20it/s, loss=5.52]
78%|███████▊ | 779/1000 [00:29<00:08, 27.20it/s, loss=5.52]
78%|███████▊ | 779/1000 [00:29<00:08, 27.20it/s, loss=5.51]
78%|███████▊ | 779/1000 [00:29<00:08, 27.20it/s, loss=5.5]
78%|███████▊ | 782/1000 [00:29<00:07, 27.30it/s, loss=5.5]
78%|███████▊ | 782/1000 [00:29<00:07, 27.30it/s, loss=5.49]
78%|███████▊ | 782/1000 [00:29<00:07, 27.30it/s, loss=5.49]
78%|███████▊ | 782/1000 [00:29<00:07, 27.30it/s, loss=5.48]
78%|███████▊ | 785/1000 [00:29<00:07, 26.94it/s, loss=5.48]
78%|███████▊ | 785/1000 [00:29<00:07, 26.94it/s, loss=5.47]
78%|███████▊ | 785/1000 [00:29<00:07, 26.94it/s, loss=5.47]
78%|███████▊ | 785/1000 [00:29<00:07, 26.94it/s, loss=5.46]
79%|███████▉ | 788/1000 [00:29<00:07, 26.59it/s, loss=5.46]
79%|███████▉ | 788/1000 [00:29<00:07, 26.59it/s, loss=5.45]
79%|███████▉ | 788/1000 [00:29<00:07, 26.59it/s, loss=5.45]
79%|███████▉ | 788/1000 [00:29<00:07, 26.59it/s, loss=5.44]
79%|███████▉ | 791/1000 [00:29<00:07, 26.51it/s, loss=5.44]
79%|███████▉ | 791/1000 [00:29<00:07, 26.51it/s, loss=5.43]
79%|███████▉ | 791/1000 [00:30<00:07, 26.51it/s, loss=5.42]
79%|███████▉ | 791/1000 [00:30<00:07, 26.51it/s, loss=5.42]
79%|███████▉ | 794/1000 [00:30<00:07, 26.08it/s, loss=5.42]
79%|███████▉ | 794/1000 [00:30<00:07, 26.08it/s, loss=5.41]
79%|███████▉ | 794/1000 [00:30<00:07, 26.08it/s, loss=5.4]
79%|███████▉ | 794/1000 [00:30<00:07, 26.08it/s, loss=5.4]
80%|███████▉ | 797/1000 [00:30<00:07, 26.17it/s, loss=5.4]
80%|███████▉ | 797/1000 [00:30<00:07, 26.17it/s, loss=5.39]
80%|███████▉ | 797/1000 [00:30<00:07, 26.17it/s, loss=5.38]
80%|███████▉ | 797/1000 [00:30<00:07, 26.17it/s, loss=5.38]
80%|████████ | 800/1000 [00:30<00:07, 26.26it/s, loss=5.38]
80%|████████ | 800/1000 [00:30<00:07, 26.26it/s, loss=5.37]
80%|████████ | 800/1000 [00:30<00:07, 26.26it/s, loss=5.36]
80%|████████ | 800/1000 [00:30<00:07, 26.26it/s, loss=5.36]
80%|████████ | 803/1000 [00:30<00:07, 26.25it/s, loss=5.36]
80%|████████ | 803/1000 [00:30<00:07, 26.25it/s, loss=5.35]
80%|████████ | 803/1000 [00:30<00:07, 26.25it/s, loss=5.34]
80%|████████ | 803/1000 [00:30<00:07, 26.25it/s, loss=5.34]
81%|████████ | 806/1000 [00:30<00:07, 26.16it/s, loss=5.34]
81%|████████ | 806/1000 [00:30<00:07, 26.16it/s, loss=5.33]
81%|████████ | 806/1000 [00:30<00:07, 26.16it/s, loss=5.32]
81%|████████ | 806/1000 [00:30<00:07, 26.16it/s, loss=5.32]
81%|████████ | 809/1000 [00:30<00:07, 26.26it/s, loss=5.32]
81%|████████ | 809/1000 [00:30<00:07, 26.26it/s, loss=5.31]
81%|████████ | 809/1000 [00:30<00:07, 26.26it/s, loss=5.3]
81%|████████ | 809/1000 [00:30<00:07, 26.26it/s, loss=5.3]
81%|████████ | 812/1000 [00:30<00:07, 26.21it/s, loss=5.3]
81%|████████ | 812/1000 [00:30<00:07, 26.21it/s, loss=5.29]
81%|████████ | 812/1000 [00:30<00:07, 26.21it/s, loss=5.29]
81%|████████ | 812/1000 [00:30<00:07, 26.21it/s, loss=5.28]
82%|████████▏ | 815/1000 [00:30<00:07, 26.37it/s, loss=5.28]
82%|████████▏ | 815/1000 [00:30<00:07, 26.37it/s, loss=5.27]
82%|████████▏ | 815/1000 [00:30<00:07, 26.37it/s, loss=5.27]
82%|████████▏ | 815/1000 [00:30<00:07, 26.37it/s, loss=5.26]
82%|████████▏ | 818/1000 [00:30<00:06, 26.50it/s, loss=5.26]
82%|████████▏ | 818/1000 [00:31<00:06, 26.50it/s, loss=5.25]
82%|████████▏ | 818/1000 [00:31<00:06, 26.50it/s, loss=5.25]
82%|████████▏ | 818/1000 [00:31<00:06, 26.50it/s, loss=5.24]
82%|████████▏ | 821/1000 [00:31<00:06, 26.26it/s, loss=5.24]
82%|████████▏ | 821/1000 [00:31<00:06, 26.26it/s, loss=5.23]
82%|████████▏ | 821/1000 [00:31<00:06, 26.26it/s, loss=5.23]
82%|████████▏ | 821/1000 [00:31<00:06, 26.26it/s, loss=5.22]
82%|████████▏ | 824/1000 [00:31<00:06, 26.09it/s, loss=5.22]
82%|████████▏ | 824/1000 [00:31<00:06, 26.09it/s, loss=5.21]
82%|████████▏ | 824/1000 [00:31<00:06, 26.09it/s, loss=5.21]
82%|████████▏ | 824/1000 [00:31<00:06, 26.09it/s, loss=5.2]
83%|████████▎ | 827/1000 [00:31<00:06, 25.92it/s, loss=5.2]
83%|████████▎ | 827/1000 [00:31<00:06, 25.92it/s, loss=5.2]
83%|████████▎ | 827/1000 [00:31<00:06, 25.92it/s, loss=5.19]
83%|████████▎ | 827/1000 [00:31<00:06, 25.92it/s, loss=5.18]
83%|████████▎ | 830/1000 [00:31<00:06, 25.65it/s, loss=5.18]
83%|████████▎ | 830/1000 [00:31<00:06, 25.65it/s, loss=5.18]
83%|████████▎ | 830/1000 [00:31<00:06, 25.65it/s, loss=5.17]
83%|████████▎ | 830/1000 [00:31<00:06, 25.65it/s, loss=5.16]
83%|████████▎ | 833/1000 [00:31<00:06, 25.54it/s, loss=5.16]
83%|████████▎ | 833/1000 [00:31<00:06, 25.54it/s, loss=5.16]
83%|████████▎ | 833/1000 [00:31<00:06, 25.54it/s, loss=5.15]
83%|████████▎ | 833/1000 [00:31<00:06, 25.54it/s, loss=5.15]
84%|████████▎ | 836/1000 [00:31<00:06, 25.61it/s, loss=5.15]
84%|████████▎ | 836/1000 [00:31<00:06, 25.61it/s, loss=5.14]
84%|████████▎ | 836/1000 [00:31<00:06, 25.61it/s, loss=5.13]
84%|████████▎ | 836/1000 [00:31<00:06, 25.61it/s, loss=5.13]
84%|████████▍ | 839/1000 [00:31<00:06, 25.83it/s, loss=5.13]
84%|████████▍ | 839/1000 [00:31<00:06, 25.83it/s, loss=5.12]
84%|████████▍ | 839/1000 [00:31<00:06, 25.83it/s, loss=5.12]
84%|████████▍ | 839/1000 [00:31<00:06, 25.83it/s, loss=5.11]
84%|████████▍ | 842/1000 [00:31<00:06, 26.12it/s, loss=5.11]
84%|████████▍ | 842/1000 [00:31<00:06, 26.12it/s, loss=5.1]
84%|████████▍ | 842/1000 [00:31<00:06, 26.12it/s, loss=5.1]
84%|████████▍ | 842/1000 [00:32<00:06, 26.12it/s, loss=5.09]
84%|████████▍ | 845/1000 [00:32<00:05, 26.07it/s, loss=5.09]
84%|████████▍ | 845/1000 [00:32<00:05, 26.07it/s, loss=5.09]
84%|████████▍ | 845/1000 [00:32<00:05, 26.07it/s, loss=5.08]
84%|████████▍ | 845/1000 [00:32<00:05, 26.07it/s, loss=5.07]
85%|████████▍ | 848/1000 [00:32<00:05, 26.02it/s, loss=5.07]
85%|████████▍ | 848/1000 [00:32<00:05, 26.02it/s, loss=5.07]
85%|████████▍ | 848/1000 [00:32<00:05, 26.02it/s, loss=5.06]
85%|████████▍ | 848/1000 [00:32<00:05, 26.02it/s, loss=5.06]
85%|████████▌ | 851/1000 [00:32<00:05, 26.19it/s, loss=5.06]
85%|████████▌ | 851/1000 [00:32<00:05, 26.19it/s, loss=5.05]
85%|████████▌ | 851/1000 [00:32<00:05, 26.19it/s, loss=5.04]
85%|████████▌ | 851/1000 [00:32<00:05, 26.19it/s, loss=5.04]
85%|████████▌ | 854/1000 [00:32<00:05, 26.15it/s, loss=5.04]
85%|████████▌ | 854/1000 [00:32<00:05, 26.15it/s, loss=5.03]
85%|████████▌ | 854/1000 [00:32<00:05, 26.15it/s, loss=5.03]
85%|████████▌ | 854/1000 [00:32<00:05, 26.15it/s, loss=5.02]
86%|████████▌ | 857/1000 [00:32<00:05, 25.70it/s, loss=5.02]
86%|████████▌ | 857/1000 [00:32<00:05, 25.70it/s, loss=5.01]
86%|████████▌ | 857/1000 [00:32<00:05, 25.70it/s, loss=5.01]
86%|████████▌ | 857/1000 [00:32<00:05, 25.70it/s, loss=5]
86%|████████▌ | 860/1000 [00:32<00:05, 25.49it/s, loss=5]
86%|████████▌ | 860/1000 [00:32<00:05, 25.49it/s, loss=5]
86%|████████▌ | 860/1000 [00:32<00:05, 25.49it/s, loss=4.99]
86%|████████▌ | 860/1000 [00:32<00:05, 25.49it/s, loss=4.98]
86%|████████▋ | 863/1000 [00:32<00:05, 25.13it/s, loss=4.98]
86%|████████▋ | 863/1000 [00:32<00:05, 25.13it/s, loss=4.98]
86%|████████▋ | 863/1000 [00:32<00:05, 25.13it/s, loss=4.97]
86%|████████▋ | 863/1000 [00:32<00:05, 25.13it/s, loss=4.97]
87%|████████▋ | 866/1000 [00:32<00:05, 25.04it/s, loss=4.97]
87%|████████▋ | 866/1000 [00:32<00:05, 25.04it/s, loss=4.96]
87%|████████▋ | 866/1000 [00:32<00:05, 25.04it/s, loss=4.96]
87%|████████▋ | 866/1000 [00:32<00:05, 25.04it/s, loss=4.95]
87%|████████▋ | 869/1000 [00:32<00:05, 25.01it/s, loss=4.95]
87%|████████▋ | 869/1000 [00:33<00:05, 25.01it/s, loss=4.94]
87%|████████▋ | 869/1000 [00:33<00:05, 25.01it/s, loss=4.94]
87%|████████▋ | 869/1000 [00:33<00:05, 25.01it/s, loss=4.93]
87%|████████▋ | 872/1000 [00:33<00:05, 23.86it/s, loss=4.93]
87%|████████▋ | 872/1000 [00:33<00:05, 23.86it/s, loss=4.93]
87%|████████▋ | 872/1000 [00:33<00:05, 23.86it/s, loss=4.92]
87%|████████▋ | 872/1000 [00:33<00:05, 23.86it/s, loss=4.92]
88%|████████▊ | 875/1000 [00:33<00:05, 24.29it/s, loss=4.92]
88%|████████▊ | 875/1000 [00:33<00:05, 24.29it/s, loss=4.91]
88%|████████▊ | 875/1000 [00:33<00:05, 24.29it/s, loss=4.91]
88%|████████▊ | 875/1000 [00:33<00:05, 24.29it/s, loss=4.9]
88%|████████▊ | 878/1000 [00:33<00:04, 24.67it/s, loss=4.9]
88%|████████▊ | 878/1000 [00:33<00:04, 24.67it/s, loss=4.89]
88%|████████▊ | 878/1000 [00:33<00:04, 24.67it/s, loss=4.89]
88%|████████▊ | 878/1000 [00:33<00:04, 24.67it/s, loss=4.88]
88%|████████▊ | 881/1000 [00:33<00:04, 24.93it/s, loss=4.88]
88%|████████▊ | 881/1000 [00:33<00:04, 24.93it/s, loss=4.88]
88%|████████▊ | 881/1000 [00:33<00:04, 24.93it/s, loss=4.87]
88%|████████▊ | 881/1000 [00:33<00:04, 24.93it/s, loss=4.87]
88%|████████▊ | 884/1000 [00:33<00:04, 25.02it/s, loss=4.87]
88%|████████▊ | 884/1000 [00:33<00:04, 25.02it/s, loss=4.86]
88%|████████▊ | 884/1000 [00:33<00:04, 25.02it/s, loss=4.86]
88%|████████▊ | 884/1000 [00:33<00:04, 25.02it/s, loss=4.85]
89%|████████▊ | 887/1000 [00:33<00:04, 25.22it/s, loss=4.85]
89%|████████▊ | 887/1000 [00:33<00:04, 25.22it/s, loss=4.84]
89%|████████▊ | 887/1000 [00:33<00:04, 25.22it/s, loss=4.84]
89%|████████▊ | 887/1000 [00:33<00:04, 25.22it/s, loss=4.83]
89%|████████▉ | 890/1000 [00:33<00:04, 25.19it/s, loss=4.83]
89%|████████▉ | 890/1000 [00:33<00:04, 25.19it/s, loss=4.83]
89%|████████▉ | 890/1000 [00:33<00:04, 25.19it/s, loss=4.82]
89%|████████▉ | 890/1000 [00:33<00:04, 25.19it/s, loss=4.82]
89%|████████▉ | 893/1000 [00:33<00:04, 25.15it/s, loss=4.82]
89%|████████▉ | 893/1000 [00:33<00:04, 25.15it/s, loss=4.81]
89%|████████▉ | 893/1000 [00:34<00:04, 25.15it/s, loss=4.81]
89%|████████▉ | 893/1000 [00:34<00:04, 25.15it/s, loss=4.8]
90%|████████▉ | 896/1000 [00:34<00:04, 25.13it/s, loss=4.8]
90%|████████▉ | 896/1000 [00:34<00:04, 25.13it/s, loss=4.8]
90%|████████▉ | 896/1000 [00:34<00:04, 25.13it/s, loss=4.79]
90%|████████▉ | 896/1000 [00:34<00:04, 25.13it/s, loss=4.79]
90%|████████▉ | 899/1000 [00:34<00:04, 25.15it/s, loss=4.79]
90%|████████▉ | 899/1000 [00:34<00:04, 25.15it/s, loss=4.78]
90%|████████▉ | 899/1000 [00:34<00:04, 25.15it/s, loss=4.77]
90%|████████▉ | 899/1000 [00:34<00:04, 25.15it/s, loss=4.77]
90%|█████████ | 902/1000 [00:34<00:03, 25.33it/s, loss=4.77]
90%|█████████ | 902/1000 [00:34<00:03, 25.33it/s, loss=4.76]
90%|█████████ | 902/1000 [00:34<00:03, 25.33it/s, loss=4.76]
90%|█████████ | 902/1000 [00:34<00:03, 25.33it/s, loss=4.75]
90%|█████████ | 905/1000 [00:34<00:03, 25.32it/s, loss=4.75]
90%|█████████ | 905/1000 [00:34<00:03, 25.32it/s, loss=4.75]
90%|█████████ | 905/1000 [00:34<00:03, 25.32it/s, loss=4.74]
90%|█████████ | 905/1000 [00:34<00:03, 25.32it/s, loss=4.74]
91%|█████████ | 908/1000 [00:34<00:03, 25.56it/s, loss=4.74]
91%|█████████ | 908/1000 [00:34<00:03, 25.56it/s, loss=4.73]
91%|█████████ | 908/1000 [00:34<00:03, 25.56it/s, loss=4.73]
91%|█████████ | 908/1000 [00:34<00:03, 25.56it/s, loss=4.72]
91%|█████████ | 911/1000 [00:34<00:03, 25.62it/s, loss=4.72]
91%|█████████ | 911/1000 [00:34<00:03, 25.62it/s, loss=4.72]
91%|█████████ | 911/1000 [00:34<00:03, 25.62it/s, loss=4.71]
91%|█████████ | 911/1000 [00:34<00:03, 25.62it/s, loss=4.71]
91%|█████████▏| 914/1000 [00:34<00:03, 25.90it/s, loss=4.71]
91%|█████████▏| 914/1000 [00:34<00:03, 25.90it/s, loss=4.7]
91%|█████████▏| 914/1000 [00:34<00:03, 25.90it/s, loss=4.7]
91%|█████████▏| 914/1000 [00:34<00:03, 25.90it/s, loss=4.69]
92%|█████████▏| 917/1000 [00:34<00:03, 25.84it/s, loss=4.69]
92%|█████████▏| 917/1000 [00:34<00:03, 25.84it/s, loss=4.69]
92%|█████████▏| 917/1000 [00:34<00:03, 25.84it/s, loss=4.68]
92%|█████████▏| 917/1000 [00:35<00:03, 25.84it/s, loss=4.68]
92%|█████████▏| 920/1000 [00:35<00:03, 25.58it/s, loss=4.68]
92%|█████████▏| 920/1000 [00:35<00:03, 25.58it/s, loss=4.67]
92%|█████████▏| 920/1000 [00:35<00:03, 25.58it/s, loss=4.67]
92%|█████████▏| 920/1000 [00:35<00:03, 25.58it/s, loss=4.66]
92%|█████████▏| 923/1000 [00:35<00:03, 25.51it/s, loss=4.66]
92%|█████████▏| 923/1000 [00:35<00:03, 25.51it/s, loss=4.66]
92%|█████████▏| 923/1000 [00:35<00:03, 25.51it/s, loss=4.65]
92%|█████████▏| 923/1000 [00:35<00:03, 25.51it/s, loss=4.65]
93%|█████████▎| 926/1000 [00:35<00:02, 25.51it/s, loss=4.65]
93%|█████████▎| 926/1000 [00:35<00:02, 25.51it/s, loss=4.64]
93%|█████████▎| 926/1000 [00:35<00:02, 25.51it/s, loss=4.64]
93%|█████████▎| 926/1000 [00:35<00:02, 25.51it/s, loss=4.63]
93%|█████████▎| 929/1000 [00:35<00:02, 25.62it/s, loss=4.63]
93%|█████████▎| 929/1000 [00:35<00:02, 25.62it/s, loss=4.63]
93%|█████████▎| 929/1000 [00:35<00:02, 25.62it/s, loss=4.62]
93%|█████████▎| 929/1000 [00:35<00:02, 25.62it/s, loss=4.62]
93%|█████████▎| 932/1000 [00:35<00:02, 25.67it/s, loss=4.62]
93%|█████████▎| 932/1000 [00:35<00:02, 25.67it/s, loss=4.61]
93%|█████████▎| 932/1000 [00:35<00:02, 25.67it/s, loss=4.61]
93%|█████████▎| 932/1000 [00:35<00:02, 25.67it/s, loss=4.6]
94%|█████████▎| 935/1000 [00:35<00:02, 25.77it/s, loss=4.6]
94%|█████████▎| 935/1000 [00:35<00:02, 25.77it/s, loss=4.6]
94%|█████████▎| 935/1000 [00:35<00:02, 25.77it/s, loss=4.59]
94%|█████████▎| 935/1000 [00:35<00:02, 25.77it/s, loss=4.59]
94%|█████████▍| 938/1000 [00:35<00:02, 25.77it/s, loss=4.59]
94%|█████████▍| 938/1000 [00:35<00:02, 25.77it/s, loss=4.58]
94%|█████████▍| 938/1000 [00:35<00:02, 25.77it/s, loss=4.58]
94%|█████████▍| 938/1000 [00:35<00:02, 25.77it/s, loss=4.57]
94%|█████████▍| 941/1000 [00:35<00:02, 25.84it/s, loss=4.57]
94%|█████████▍| 941/1000 [00:35<00:02, 25.84it/s, loss=4.57]
94%|█████████▍| 941/1000 [00:35<00:02, 25.84it/s, loss=4.56]
94%|█████████▍| 941/1000 [00:35<00:02, 25.84it/s, loss=4.56]
94%|█████████▍| 944/1000 [00:35<00:02, 25.73it/s, loss=4.56]
94%|█████████▍| 944/1000 [00:35<00:02, 25.73it/s, loss=4.55]
94%|█████████▍| 944/1000 [00:36<00:02, 25.73it/s, loss=4.55]
94%|█████████▍| 944/1000 [00:36<00:02, 25.73it/s, loss=4.54]
95%|█████████▍| 947/1000 [00:36<00:02, 25.89it/s, loss=4.54]
95%|█████████▍| 947/1000 [00:36<00:02, 25.89it/s, loss=4.54]
95%|█████████▍| 947/1000 [00:36<00:02, 25.89it/s, loss=4.53]
95%|█████████▍| 947/1000 [00:36<00:02, 25.89it/s, loss=4.53]
95%|█████████▌| 950/1000 [00:36<00:01, 26.01it/s, loss=4.53]
95%|█████████▌| 950/1000 [00:36<00:01, 26.01it/s, loss=4.52]
95%|█████████▌| 950/1000 [00:36<00:01, 26.01it/s, loss=4.52]
95%|█████████▌| 950/1000 [00:36<00:01, 26.01it/s, loss=4.51]
95%|█████████▌| 953/1000 [00:36<00:01, 26.18it/s, loss=4.51]
95%|█████████▌| 953/1000 [00:36<00:01, 26.18it/s, loss=4.51]
95%|█████████▌| 953/1000 [00:36<00:01, 26.18it/s, loss=4.5]
95%|█████████▌| 953/1000 [00:36<00:01, 26.18it/s, loss=4.5]
96%|█████████▌| 956/1000 [00:36<00:01, 26.21it/s, loss=4.5]
96%|█████████▌| 956/1000 [00:36<00:01, 26.21it/s, loss=4.5]
96%|█████████▌| 956/1000 [00:36<00:01, 26.21it/s, loss=4.49]
96%|█████████▌| 956/1000 [00:36<00:01, 26.21it/s, loss=4.49]
96%|█████████▌| 959/1000 [00:36<00:01, 25.87it/s, loss=4.49]
96%|█████████▌| 959/1000 [00:36<00:01, 25.87it/s, loss=4.48]
96%|█████████▌| 959/1000 [00:36<00:01, 25.87it/s, loss=4.48]
96%|█████████▌| 959/1000 [00:36<00:01, 25.87it/s, loss=4.47]
96%|█████████▌| 962/1000 [00:36<00:01, 25.78it/s, loss=4.47]
96%|█████████▌| 962/1000 [00:36<00:01, 25.78it/s, loss=4.47]
96%|█████████▌| 962/1000 [00:36<00:01, 25.78it/s, loss=4.46]
96%|█████████▌| 962/1000 [00:36<00:01, 25.78it/s, loss=4.46]
96%|█████████▋| 965/1000 [00:36<00:01, 25.46it/s, loss=4.46]
96%|█████████▋| 965/1000 [00:36<00:01, 25.46it/s, loss=4.45]
96%|█████████▋| 965/1000 [00:36<00:01, 25.46it/s, loss=4.45]
96%|█████████▋| 965/1000 [00:36<00:01, 25.46it/s, loss=4.44]
97%|█████████▋| 968/1000 [00:36<00:01, 25.43it/s, loss=4.44]
97%|█████████▋| 968/1000 [00:36<00:01, 25.43it/s, loss=4.44]
97%|█████████▋| 968/1000 [00:36<00:01, 25.43it/s, loss=4.44]
97%|█████████▋| 968/1000 [00:36<00:01, 25.43it/s, loss=4.43]
97%|█████████▋| 971/1000 [00:36<00:01, 25.38it/s, loss=4.43]
97%|█████████▋| 971/1000 [00:37<00:01, 25.38it/s, loss=4.43]
97%|█████████▋| 971/1000 [00:37<00:01, 25.38it/s, loss=4.42]
97%|█████████▋| 971/1000 [00:37<00:01, 25.38it/s, loss=4.42]
97%|█████████▋| 974/1000 [00:37<00:01, 25.40it/s, loss=4.42]
97%|█████████▋| 974/1000 [00:37<00:01, 25.40it/s, loss=4.41]
97%|█████████▋| 974/1000 [00:37<00:01, 25.40it/s, loss=4.41]
97%|█████████▋| 974/1000 [00:37<00:01, 25.40it/s, loss=4.4]
98%|█████████▊| 977/1000 [00:37<00:00, 25.65it/s, loss=4.4]
98%|█████████▊| 977/1000 [00:37<00:00, 25.65it/s, loss=4.4]
98%|█████████▊| 977/1000 [00:37<00:00, 25.65it/s, loss=4.39]
98%|█████████▊| 977/1000 [00:37<00:00, 25.65it/s, loss=4.39]
98%|█████████▊| 980/1000 [00:37<00:00, 25.82it/s, loss=4.39]
98%|█████████▊| 980/1000 [00:37<00:00, 25.82it/s, loss=4.39]
98%|█████████▊| 980/1000 [00:37<00:00, 25.82it/s, loss=4.38]
98%|█████████▊| 980/1000 [00:37<00:00, 25.82it/s, loss=4.38]
98%|█████████▊| 983/1000 [00:37<00:00, 25.80it/s, loss=4.38]
98%|█████████▊| 983/1000 [00:37<00:00, 25.80it/s, loss=4.37]
98%|█████████▊| 983/1000 [00:37<00:00, 25.80it/s, loss=4.37]
98%|█████████▊| 983/1000 [00:37<00:00, 25.80it/s, loss=4.36]
99%|█████████▊| 986/1000 [00:37<00:00, 25.76it/s, loss=4.36]
99%|█████████▊| 986/1000 [00:37<00:00, 25.76it/s, loss=4.36]
99%|█████████▊| 986/1000 [00:37<00:00, 25.76it/s, loss=4.35]
99%|█████████▊| 986/1000 [00:37<00:00, 25.76it/s, loss=4.35]
99%|█████████▉| 989/1000 [00:37<00:00, 25.97it/s, loss=4.35]
99%|█████████▉| 989/1000 [00:37<00:00, 25.97it/s, loss=4.35]
99%|█████████▉| 989/1000 [00:37<00:00, 25.97it/s, loss=4.34]
99%|█████████▉| 989/1000 [00:37<00:00, 25.97it/s, loss=4.34]
99%|█████████▉| 992/1000 [00:37<00:00, 26.09it/s, loss=4.34]
99%|█████████▉| 992/1000 [00:37<00:00, 26.09it/s, loss=4.33]
99%|█████████▉| 992/1000 [00:37<00:00, 26.09it/s, loss=4.33]
99%|█████████▉| 992/1000 [00:37<00:00, 26.09it/s, loss=4.32]
100%|█████████▉| 995/1000 [00:37<00:00, 26.13it/s, loss=4.32]
100%|█████████▉| 995/1000 [00:37<00:00, 26.13it/s, loss=4.32]
100%|█████████▉| 995/1000 [00:37<00:00, 26.13it/s, loss=4.31]
100%|█████████▉| 995/1000 [00:38<00:00, 26.13it/s, loss=4.31]
100%|█████████▉| 998/1000 [00:38<00:00, 26.27it/s, loss=4.31]
100%|█████████▉| 998/1000 [00:38<00:00, 26.27it/s, loss=4.31]
100%|█████████▉| 998/1000 [00:38<00:00, 26.27it/s, loss=4.3]
100%|██████████| 1000/1000 [00:38<00:00, 26.25it/s, loss=4.3]
sgd_params
| mu_x | mu_y | sigma | shape | rate | shift | baseline | amplitude | |
|---|---|---|---|---|---|---|---|---|
| 0 | -2.100004 | 1.450003 | 1.350009 | 6.0 | 0.9 | 2.5 | 9.999946 | 1.199994 |
We can plot the predicted model response and see that it matches the original simulated response almost perfectly.
sgd_pred_response = prf_model(stimulus, sgd_params)
fig, ax = plt.subplots()
ax.plot(simulated_response[0], label="True")
ax.plot(sgd_pred_response[0], "--", label="Predicted (SGD)")
fig.legend();
Conclusion¶
In this tutorial, we showed how to setup a standard Gaussian pRF model for a two-dimensional stimulus. We demonstrated how to fit the model to simulated data (without noise) using a multi-stage workflow: First, we used a grid search to find good starting values, then, we estimated baseline and amplitude using least squares, and finally we finetuned the model fit using stochastic gradient descent. At each stage, we compared the predicted model response against the original simulated response to check how well the model fit the data.
Stay Tuned¶
More tutorials on fitting models to empirical data and creating custom models are in the making.
For questions and issues, please make an issue on GitHub or contact Malte Lüken (m.luken@esciencecenter.nl).
References¶
Dumoulin, S. O., & Wandell, B. A. (2008). Population receptive field estimates in human visual cortex. NeuroImage, 39(2), 647–660. https://doi.org/10.1016/j.neuroimage.2007.09.034