How to fit a population receptive field model to simulated data¶
Author: Malte Lüken (m.luken@esciencecenter.nl)
Difficulty: Beginner
This tutorial explains how to fit a population receptive field (pRF) model to simulated data.
A pRF model maps neural activity in a region of interest in the brain (e.g., V1 in the human visual cortex) to an experimental stimulus (e.g., a bar moving through the visual field). Here, we use the visual domain as an example, where the part of the visual field that stimulates activity in the region of interest is the pRF.
Because prfmodel uses Keras for model fitting, we need to make sure that a backend is installed before we begin. In this tutorial, we use the TensorFlow backend.
import os
from importlib.util import find_spec
# Set keras backend to 'tensorflow' (this is normally the default)
os.environ["KERAS_BACKEND"] = "tensorflow"
# Hide tensorflow info messages
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "1"
if find_spec("tensorflow") is None:
msg = "Could not find the tensorflow package. Please install tensorflow with 'pip install .[tensorflow]'"
raise ImportError(msg)
Defining the stimulus¶
Let’s start with the first step: Defining the stimulus. In practice, we recommend that users save the stimulus they use in an experiment to a file and load it to avoid mismatches between experiment and analysis. Because we use simulated data in this tutorial, we load an example stimulus that is included in the package. The stimulus simulates a bar moving in different directions through a two-dimensional visual field.
from prfmodel.examples import load_2d_bar_stimulus
num_frames = 200 # Simulate 200 time frames
stimulus = load_2d_bar_stimulus()
print(stimulus)
Stimulus(design=array[200, 101, 101], grid=array[101, 101, 2], dimension_labels=['y', 'x'])
When printing the stimulus object, we can see that it has three attributes. The design attribute defines how
the visual field changes over time. It has shape (num_frames, width, height), where width and hight define the number of pixels at which the visual field is recorded. The grid attribute maps each pixel to its xy-coordinate in the visual field (i.e., the degree of visual angle).
We can visualize the stimulus using animate_2d_stimulus.
from IPython.display import HTML
from prfmodel.stimulus import animate_2d_stimulus
ani = animate_2d_stimulus(stimulus, interval=25) # Pause 25 ms between time frames
HTML(ani.to_html5_video())
Defining the pRF model¶
Now that we defined our stimulus, we can create a pRF model to predict a neural response to this stimulus in our (hypothetical) region of interest (e.g., V1). We use the most popular pRF model that is based on the seminal paper by Dumoulin and Wandell (2008): It assumes that the stimulus (our moving bar) elicits a response that follows a Gaussian shape in two-dimensional visual space. This response is then summed and convolved with an impulse response that follows the shape of the hemodynamic response in the brain. Finally, a baseline and amplitude parameter shift and scale our predicted response to the simulated (or observed) neural response.
The Gaussian2DPRFModel class performs all these steps to make a combined prediction.
from prfmodel.models.gaussian import Gaussian2DPRFModel
prf_model = Gaussian2DPRFModel()
To simulate a neural response to our stimulus with our Gaussian 2D pRF model, we need to define a set of parameters.
The list of parameters that need to be set to make model predictions can be obtained from the parameter_names property.
prf_model.parameter_names
['shape_2',
'baseline',
'weight',
'shape_1',
'mu_y',
'rate_2',
'amplitude',
'mu_x',
'sigma',
'rate_1']
The parameters mu_x, mu_y, and sigma define the location and size of the predicted Gaussian pRF and are of primary interest. We simulate a pRF with its center at (-2.1, 1.45) and a size of 1.35. We will ignore the other parameters for now (however, we still need to set values for them). We store the parameter values in a pandas.DataFrame object.
import pandas as pd
true_params = pd.DataFrame(
{
"mu_x": [-2.1],
"mu_y": [1.45],
"sigma": [1.35],
"shape_1": [6.0],
"rate_1": [0.9],
"shape_2": [12.0],
"rate_2": [0.9],
"weight": [0.35],
"baseline": [10.0],
"amplitude": [1.2],
},
)
Using the “true” parameters, we simulate a response to our stimulus by making a prediction with our pRF model.
import matplotlib.pyplot as plt
simulated_response = prf_model(stimulus, true_params)
_ = plt.plot(simulated_response[0])
2025-11-27 14:29:23.806543: E external/local_xla/xla/stream_executor/cuda/cuda_platform.cc:51] failed call to cuInit: INTERNAL: CUDA error: Failed call to cuInit: UNKNOWN ERROR (303)
The predicted response contains increased activation followed by decreased activation compared to the baseline activity for each moving bar in our stimulus.
Fitting the pRF model¶
We will fit the pRF model to our simulated data using multiple stages. We begin with a grid search to find good
starting values for our parameters of interest (mu_x, mu_y, and sigma). Then, we use least squares to estimate the baseline and amplitude of
our model. Finally, we use stochastic gradient descent (SGD) to finetune our model fits.
Let’s start with the grid search by defining ranges of mu_x, mu_y, and sigma that we want to construct a grid
of parameter values from. For all other parameters, we only provide a single value so that they will stay constant
across the entire grid.
import numpy as np
param_ranges = {
"mu_x": np.linspace(-3.0, 3.0, 10),
"mu_y": np.linspace(-3.0, 3.0, 10),
"sigma": np.linspace(0.5, 3.0, 10),
"shape_1": [6.0],
"rate_1": [0.9],
"shape_2": [12.0],
"rate_2": [0.9],
"weight": [0.35],
"baseline": [0.0],
"amplitude": [1.0],
}
For all three parameters, we defined ranges of 10 values that will be used to construct the grid. That is, the grid search will evaluate all possible combinations of these values and return the combination that fits the simulated data best. This will result in a grid containing \(10 \times 10 \times 10 = 1000\) parameter combinations.
Let’s construct the GridFitter and perform the grid search. Note that we set chunk_size=20 to let the GridFitter
evaluate 20 parameter combinations at the same time (which saves us some memory).
from prfmodel.fitters.grid import GridFitter
grid_fitter = GridFitter(
model=prf_model,
stimulus=stimulus,
)
grid_history, grid_params = grid_fitter.fit(
data=simulated_response,
parameter_values=param_ranges,
chunk_size=20,
)
Processing parameter grid chunks: 0%| | 0/50 [00:00<?, ?it/s]
Processing parameter grid chunks: 0%| | 0/50 [00:00<?, ?it/s, loss=3.1e+3]
Processing parameter grid chunks: 0%| | 0/50 [00:00<?, ?it/s, loss=1.6e+3]
Processing parameter grid chunks: 4%|▍ | 2/50 [00:00<00:02, 18.15it/s, loss=1.6e+3]
Processing parameter grid chunks: 4%|▍ | 2/50 [00:00<00:02, 18.15it/s, loss=587]
Processing parameter grid chunks: 4%|▍ | 2/50 [00:00<00:02, 18.15it/s, loss=297]
Processing parameter grid chunks: 4%|▍ | 2/50 [00:00<00:02, 18.15it/s, loss=297]
Processing parameter grid chunks: 10%|█ | 5/50 [00:00<00:02, 20.04it/s, loss=297]
Processing parameter grid chunks: 10%|█ | 5/50 [00:00<00:02, 20.04it/s, loss=297]
Processing parameter grid chunks: 10%|█ | 5/50 [00:00<00:02, 20.04it/s, loss=297]
Processing parameter grid chunks: 10%|█ | 5/50 [00:00<00:02, 20.04it/s, loss=297]
Processing parameter grid chunks: 16%|█▌ | 8/50 [00:00<00:02, 20.35it/s, loss=297]
Processing parameter grid chunks: 16%|█▌ | 8/50 [00:00<00:02, 20.35it/s, loss=116]
Processing parameter grid chunks: 16%|█▌ | 8/50 [00:00<00:02, 20.35it/s, loss=116]
Processing parameter grid chunks: 16%|█▌ | 8/50 [00:00<00:02, 20.35it/s, loss=116]
Processing parameter grid chunks: 22%|██▏ | 11/50 [00:00<00:01, 20.64it/s, loss=116]
Processing parameter grid chunks: 22%|██▏ | 11/50 [00:00<00:01, 20.64it/s, loss=116]
Processing parameter grid chunks: 22%|██▏ | 11/50 [00:00<00:01, 20.64it/s, loss=116]
Processing parameter grid chunks: 22%|██▏ | 11/50 [00:00<00:01, 20.64it/s, loss=116]
Processing parameter grid chunks: 28%|██▊ | 14/50 [00:00<00:01, 20.87it/s, loss=116]
Processing parameter grid chunks: 28%|██▊ | 14/50 [00:00<00:01, 20.87it/s, loss=116]
Processing parameter grid chunks: 28%|██▊ | 14/50 [00:00<00:01, 20.87it/s, loss=116]
Processing parameter grid chunks: 28%|██▊ | 14/50 [00:00<00:01, 20.87it/s, loss=116]
Processing parameter grid chunks: 34%|███▍ | 17/50 [00:00<00:01, 20.97it/s, loss=116]
Processing parameter grid chunks: 34%|███▍ | 17/50 [00:00<00:01, 20.97it/s, loss=116]
Processing parameter grid chunks: 34%|███▍ | 17/50 [00:00<00:01, 20.97it/s, loss=116]
Processing parameter grid chunks: 34%|███▍ | 17/50 [00:00<00:01, 20.97it/s, loss=116]
Processing parameter grid chunks: 40%|████ | 20/50 [00:00<00:01, 20.78it/s, loss=116]
Processing parameter grid chunks: 40%|████ | 20/50 [00:01<00:01, 20.78it/s, loss=116]
Processing parameter grid chunks: 40%|████ | 20/50 [00:01<00:01, 20.78it/s, loss=116]
Processing parameter grid chunks: 40%|████ | 20/50 [00:01<00:01, 20.78it/s, loss=116]
Processing parameter grid chunks: 46%|████▌ | 23/50 [00:01<00:01, 20.91it/s, loss=116]
Processing parameter grid chunks: 46%|████▌ | 23/50 [00:01<00:01, 20.91it/s, loss=116]
Processing parameter grid chunks: 46%|████▌ | 23/50 [00:01<00:01, 20.91it/s, loss=116]
Processing parameter grid chunks: 46%|████▌ | 23/50 [00:01<00:01, 20.91it/s, loss=116]
Processing parameter grid chunks: 52%|█████▏ | 26/50 [00:01<00:01, 20.95it/s, loss=116]
Processing parameter grid chunks: 52%|█████▏ | 26/50 [00:01<00:01, 20.95it/s, loss=116]
Processing parameter grid chunks: 52%|█████▏ | 26/50 [00:01<00:01, 20.95it/s, loss=116]
Processing parameter grid chunks: 52%|█████▏ | 26/50 [00:01<00:01, 20.95it/s, loss=116]
Processing parameter grid chunks: 58%|█████▊ | 29/50 [00:01<00:00, 21.09it/s, loss=116]
Processing parameter grid chunks: 58%|█████▊ | 29/50 [00:01<00:00, 21.09it/s, loss=116]
Processing parameter grid chunks: 58%|█████▊ | 29/50 [00:01<00:00, 21.09it/s, loss=116]
Processing parameter grid chunks: 58%|█████▊ | 29/50 [00:01<00:00, 21.09it/s, loss=116]
Processing parameter grid chunks: 64%|██████▍ | 32/50 [00:01<00:00, 21.07it/s, loss=116]
Processing parameter grid chunks: 64%|██████▍ | 32/50 [00:01<00:00, 21.07it/s, loss=116]
Processing parameter grid chunks: 64%|██████▍ | 32/50 [00:01<00:00, 21.07it/s, loss=116]
Processing parameter grid chunks: 64%|██████▍ | 32/50 [00:01<00:00, 21.07it/s, loss=116]
Processing parameter grid chunks: 70%|███████ | 35/50 [00:01<00:00, 21.05it/s, loss=116]
Processing parameter grid chunks: 70%|███████ | 35/50 [00:01<00:00, 21.05it/s, loss=116]
Processing parameter grid chunks: 70%|███████ | 35/50 [00:01<00:00, 21.05it/s, loss=116]
Processing parameter grid chunks: 70%|███████ | 35/50 [00:01<00:00, 21.05it/s, loss=116]
Processing parameter grid chunks: 76%|███████▌ | 38/50 [00:01<00:00, 21.14it/s, loss=116]
Processing parameter grid chunks: 76%|███████▌ | 38/50 [00:01<00:00, 21.14it/s, loss=116]
Processing parameter grid chunks: 76%|███████▌ | 38/50 [00:01<00:00, 21.14it/s, loss=116]
Processing parameter grid chunks: 76%|███████▌ | 38/50 [00:01<00:00, 21.14it/s, loss=116]
Processing parameter grid chunks: 82%|████████▏ | 41/50 [00:01<00:00, 21.29it/s, loss=116]
Processing parameter grid chunks: 82%|████████▏ | 41/50 [00:02<00:00, 21.29it/s, loss=116]
Processing parameter grid chunks: 82%|████████▏ | 41/50 [00:02<00:00, 21.29it/s, loss=116]
Processing parameter grid chunks: 82%|████████▏ | 41/50 [00:02<00:00, 21.29it/s, loss=116]
Processing parameter grid chunks: 88%|████████▊ | 44/50 [00:02<00:00, 21.35it/s, loss=116]
Processing parameter grid chunks: 88%|████████▊ | 44/50 [00:02<00:00, 21.35it/s, loss=116]
Processing parameter grid chunks: 88%|████████▊ | 44/50 [00:02<00:00, 21.35it/s, loss=116]
Processing parameter grid chunks: 88%|████████▊ | 44/50 [00:02<00:00, 21.35it/s, loss=116]
Processing parameter grid chunks: 94%|█████████▍| 47/50 [00:02<00:00, 21.36it/s, loss=116]
Processing parameter grid chunks: 94%|█████████▍| 47/50 [00:02<00:00, 21.36it/s, loss=116]
Processing parameter grid chunks: 94%|█████████▍| 47/50 [00:02<00:00, 21.36it/s, loss=116]
Processing parameter grid chunks: 94%|█████████▍| 47/50 [00:02<00:00, 21.36it/s, loss=116]
Processing parameter grid chunks: 100%|██████████| 50/50 [00:02<00:00, 21.40it/s, loss=116]
Processing parameter grid chunks: 100%|██████████| 50/50 [00:02<00:00, 21.01it/s, loss=116]
grid_params
| mu_x | mu_y | sigma | shape_1 | rate_1 | shape_2 | rate_2 | weight | baseline | amplitude | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | -2.333333 | 1.666667 | 1.888889 | 6.0 | 0.9 | 12.0 | 0.9 | 0.35 | 0.0 | 1.0 |
We can see that the estimates for mu_x, mu_y, and sigma are one combination in our grid. However, because the
grid did not contain the “true” parameters we used to simulate the original response, the estimates differ from the
“true” parameters.
Using the parameter estimates resulting from the grid search we can make model predictions and compare them against the original simulated response.
grid_pred_response = prf_model(stimulus, grid_params)
fig, ax = plt.subplots()
ax.plot(simulated_response[0], label="True")
ax.plot(grid_pred_response[0], label="Predicted (grid)")
fig.legend();
We can see that the predicted response follows the shape of the original (true) response but still shows some deviation in the amplitude of the activation peaks and the baseline activation.
Using least squares, we can estimate the baseline and amplitude parameters of our model.
from prfmodel.fitters.linear import LeastSquaresFitter
ls_fitter = LeastSquaresFitter(
model=prf_model,
stimulus=stimulus,
)
ls_history, ls_params = ls_fitter.fit(
data=simulated_response,
parameters=grid_params,
target_parameters=["baseline", "amplitude"],
)
ls_params
| mu_x | mu_y | sigma | shape_1 | rate_1 | shape_2 | rate_2 | weight | baseline | amplitude | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | -2.333333 | 1.666667 | 1.888889 | 6.0 | 0.9 | 12.0 | 0.9 | 0.35 | 7.995422 | 0.919782 |
Looking at the parameters, we can see that the model compensates the deviation in the peaks by adjusting the
baseline and amplitude parameters. We can also plot the predicted response.
ls_pred_response = prf_model(stimulus, ls_params)
fig, ax = plt.subplots()
ax.plot(simulated_response[0], label="True")
ax.plot(ls_pred_response[0], label="Predicted (least-squares)")
fig.legend();
To finetune our model fits, we use SGD to iteratively optimize model parameters using the gradient of a loss function that is computed between data and model predictions. The default loss function in prfmodel is the means squared error. As initial parameters, we use the result from the grid search and least squares fit. We fix the parameters related to the impulse response to their initial values (which are the “true” values).
from prfmodel.fitters.sgd import SGDFitter
sgd_fitter = SGDFitter(
model=prf_model,
stimulus=stimulus,
)
sgd_history, sgd_params = sgd_fitter.fit(
data=simulated_response,
init_parameters=ls_params,
fixed_parameters=["shape_1", "shape_2", "rate_1", "rate_2", "weight"],
)
0%| | 0/1000 [00:00<?, ?it/s]
0%| | 0/1000 [00:00<?, ?it/s, loss=41.5]
0%| | 0/1000 [00:00<?, ?it/s, loss=41.4]
0%| | 2/1000 [00:00<00:58, 16.97it/s, loss=41.4]
0%| | 2/1000 [00:00<00:58, 16.97it/s, loss=41.2]
0%| | 2/1000 [00:00<00:58, 16.97it/s, loss=41.1]
0%| | 4/1000 [00:00<00:54, 18.21it/s, loss=41.1]
0%| | 4/1000 [00:00<00:54, 18.21it/s, loss=41]
0%| | 4/1000 [00:00<00:54, 18.21it/s, loss=40.8]
1%| | 6/1000 [00:00<00:53, 18.59it/s, loss=40.8]
1%| | 6/1000 [00:00<00:53, 18.59it/s, loss=40.7]
1%| | 6/1000 [00:00<00:53, 18.59it/s, loss=40.6]
1%| | 8/1000 [00:00<00:52, 18.76it/s, loss=40.6]
1%| | 8/1000 [00:00<00:52, 18.76it/s, loss=40.4]
1%| | 8/1000 [00:00<00:52, 18.76it/s, loss=40.3]
1%| | 10/1000 [00:00<00:52, 18.93it/s, loss=40.3]
1%| | 10/1000 [00:00<00:52, 18.93it/s, loss=40.2]
1%| | 10/1000 [00:00<00:52, 18.93it/s, loss=40]
1%| | 12/1000 [00:00<00:52, 18.96it/s, loss=40]
1%| | 12/1000 [00:00<00:52, 18.96it/s, loss=39.9]
1%| | 12/1000 [00:00<00:52, 18.96it/s, loss=39.8]
1%|▏ | 14/1000 [00:00<00:51, 18.99it/s, loss=39.8]
1%|▏ | 14/1000 [00:00<00:51, 18.99it/s, loss=39.7]
1%|▏ | 14/1000 [00:00<00:51, 18.99it/s, loss=39.5]
2%|▏ | 16/1000 [00:00<00:51, 18.99it/s, loss=39.5]
2%|▏ | 16/1000 [00:00<00:51, 18.99it/s, loss=39.4]
2%|▏ | 16/1000 [00:00<00:51, 18.99it/s, loss=39.3]
2%|▏ | 18/1000 [00:00<00:51, 19.11it/s, loss=39.3]
2%|▏ | 18/1000 [00:01<00:51, 19.11it/s, loss=39.1]
2%|▏ | 18/1000 [00:01<00:51, 19.11it/s, loss=39]
2%|▏ | 20/1000 [00:01<00:51, 19.12it/s, loss=39]
2%|▏ | 20/1000 [00:01<00:51, 19.12it/s, loss=38.9]
2%|▏ | 20/1000 [00:01<00:51, 19.12it/s, loss=38.8]
2%|▏ | 22/1000 [00:01<00:51, 19.11it/s, loss=38.8]
2%|▏ | 22/1000 [00:01<00:51, 19.11it/s, loss=38.6]
2%|▏ | 22/1000 [00:01<00:51, 19.11it/s, loss=38.5]
2%|▏ | 24/1000 [00:01<00:50, 19.14it/s, loss=38.5]
2%|▏ | 24/1000 [00:01<00:50, 19.14it/s, loss=38.4]
2%|▏ | 24/1000 [00:01<00:50, 19.14it/s, loss=38.2]
3%|▎ | 26/1000 [00:01<00:50, 19.19it/s, loss=38.2]
3%|▎ | 26/1000 [00:01<00:50, 19.19it/s, loss=38.1]
3%|▎ | 26/1000 [00:01<00:50, 19.19it/s, loss=38]
3%|▎ | 28/1000 [00:01<00:50, 19.16it/s, loss=38]
3%|▎ | 28/1000 [00:01<00:50, 19.16it/s, loss=37.9]
3%|▎ | 28/1000 [00:01<00:50, 19.16it/s, loss=37.8]
3%|▎ | 30/1000 [00:01<00:50, 19.24it/s, loss=37.8]
3%|▎ | 30/1000 [00:01<00:50, 19.24it/s, loss=37.6]
3%|▎ | 30/1000 [00:01<00:50, 19.24it/s, loss=37.5]
3%|▎ | 32/1000 [00:01<00:50, 19.26it/s, loss=37.5]
3%|▎ | 32/1000 [00:01<00:50, 19.26it/s, loss=37.4]
3%|▎ | 32/1000 [00:01<00:50, 19.26it/s, loss=37.3]
3%|▎ | 34/1000 [00:01<00:50, 19.27it/s, loss=37.3]
3%|▎ | 34/1000 [00:01<00:50, 19.27it/s, loss=37.1]
3%|▎ | 34/1000 [00:01<00:50, 19.27it/s, loss=37]
4%|▎ | 36/1000 [00:01<00:50, 19.25it/s, loss=37]
4%|▎ | 36/1000 [00:01<00:50, 19.25it/s, loss=36.9]
4%|▎ | 36/1000 [00:01<00:50, 19.25it/s, loss=36.8]
4%|▍ | 38/1000 [00:01<00:50, 19.22it/s, loss=36.8]
4%|▍ | 38/1000 [00:02<00:50, 19.22it/s, loss=36.7]
4%|▍ | 38/1000 [00:02<00:50, 19.22it/s, loss=36.5]
4%|▍ | 40/1000 [00:02<00:49, 19.24it/s, loss=36.5]
4%|▍ | 40/1000 [00:02<00:49, 19.24it/s, loss=36.4]
4%|▍ | 40/1000 [00:02<00:49, 19.24it/s, loss=36.3]
4%|▍ | 42/1000 [00:02<00:49, 19.23it/s, loss=36.3]
4%|▍ | 42/1000 [00:02<00:49, 19.23it/s, loss=36.2]
4%|▍ | 42/1000 [00:02<00:49, 19.23it/s, loss=36.1]
4%|▍ | 44/1000 [00:02<00:49, 19.25it/s, loss=36.1]
4%|▍ | 44/1000 [00:02<00:49, 19.25it/s, loss=35.9]
4%|▍ | 44/1000 [00:02<00:49, 19.25it/s, loss=35.8]
5%|▍ | 46/1000 [00:02<00:49, 19.22it/s, loss=35.8]
5%|▍ | 46/1000 [00:02<00:49, 19.22it/s, loss=35.7]
5%|▍ | 46/1000 [00:02<00:49, 19.22it/s, loss=35.6]
5%|▍ | 48/1000 [00:02<00:49, 19.19it/s, loss=35.6]
5%|▍ | 48/1000 [00:02<00:49, 19.19it/s, loss=35.5]
5%|▍ | 48/1000 [00:02<00:49, 19.19it/s, loss=35.4]
5%|▌ | 50/1000 [00:02<00:49, 19.21it/s, loss=35.4]
5%|▌ | 50/1000 [00:02<00:49, 19.21it/s, loss=35.3]
5%|▌ | 50/1000 [00:02<00:49, 19.21it/s, loss=35.1]
5%|▌ | 52/1000 [00:02<00:49, 19.31it/s, loss=35.1]
5%|▌ | 52/1000 [00:02<00:49, 19.31it/s, loss=35]
5%|▌ | 52/1000 [00:02<00:49, 19.31it/s, loss=34.9]
5%|▌ | 54/1000 [00:02<00:49, 19.28it/s, loss=34.9]
5%|▌ | 54/1000 [00:02<00:49, 19.28it/s, loss=34.8]
5%|▌ | 54/1000 [00:02<00:49, 19.28it/s, loss=34.7]
6%|▌ | 56/1000 [00:02<00:48, 19.28it/s, loss=34.7]
6%|▌ | 56/1000 [00:02<00:48, 19.28it/s, loss=34.6]
6%|▌ | 56/1000 [00:03<00:48, 19.28it/s, loss=34.5]
6%|▌ | 58/1000 [00:03<00:48, 19.27it/s, loss=34.5]
6%|▌ | 58/1000 [00:03<00:48, 19.27it/s, loss=34.4]
6%|▌ | 58/1000 [00:03<00:48, 19.27it/s, loss=34.2]
6%|▌ | 60/1000 [00:03<00:48, 19.28it/s, loss=34.2]
6%|▌ | 60/1000 [00:03<00:48, 19.28it/s, loss=34.1]
6%|▌ | 60/1000 [00:03<00:48, 19.28it/s, loss=34]
6%|▌ | 62/1000 [00:03<00:48, 19.29it/s, loss=34]
6%|▌ | 62/1000 [00:03<00:48, 19.29it/s, loss=33.9]
6%|▌ | 62/1000 [00:03<00:48, 19.29it/s, loss=33.8]
6%|▋ | 64/1000 [00:03<00:48, 19.25it/s, loss=33.8]
6%|▋ | 64/1000 [00:03<00:48, 19.25it/s, loss=33.7]
6%|▋ | 64/1000 [00:03<00:48, 19.25it/s, loss=33.6]
7%|▋ | 66/1000 [00:03<00:48, 19.17it/s, loss=33.6]
7%|▋ | 66/1000 [00:03<00:48, 19.17it/s, loss=33.5]
7%|▋ | 66/1000 [00:03<00:48, 19.17it/s, loss=33.4]
7%|▋ | 68/1000 [00:03<00:48, 19.14it/s, loss=33.4]
7%|▋ | 68/1000 [00:03<00:48, 19.14it/s, loss=33.3]
7%|▋ | 68/1000 [00:03<00:48, 19.14it/s, loss=33.2]
7%|▋ | 70/1000 [00:03<00:48, 19.22it/s, loss=33.2]
7%|▋ | 70/1000 [00:03<00:48, 19.22it/s, loss=33.1]
7%|▋ | 70/1000 [00:03<00:48, 19.22it/s, loss=32.9]
7%|▋ | 72/1000 [00:03<00:48, 19.28it/s, loss=32.9]
7%|▋ | 72/1000 [00:03<00:48, 19.28it/s, loss=32.8]
7%|▋ | 72/1000 [00:03<00:48, 19.28it/s, loss=32.7]
7%|▋ | 74/1000 [00:03<00:48, 19.23it/s, loss=32.7]
7%|▋ | 74/1000 [00:03<00:48, 19.23it/s, loss=32.6]
7%|▋ | 74/1000 [00:03<00:48, 19.23it/s, loss=32.5]
8%|▊ | 76/1000 [00:03<00:48, 19.25it/s, loss=32.5]
8%|▊ | 76/1000 [00:04<00:48, 19.25it/s, loss=32.4]
8%|▊ | 76/1000 [00:04<00:48, 19.25it/s, loss=32.3]
8%|▊ | 78/1000 [00:04<00:47, 19.26it/s, loss=32.3]
8%|▊ | 78/1000 [00:04<00:47, 19.26it/s, loss=32.2]
8%|▊ | 78/1000 [00:04<00:47, 19.26it/s, loss=32.1]
8%|▊ | 80/1000 [00:04<00:47, 19.20it/s, loss=32.1]
8%|▊ | 80/1000 [00:04<00:47, 19.20it/s, loss=32]
8%|▊ | 80/1000 [00:04<00:47, 19.20it/s, loss=31.9]
8%|▊ | 82/1000 [00:04<00:47, 19.23it/s, loss=31.9]
8%|▊ | 82/1000 [00:04<00:47, 19.23it/s, loss=31.8]
8%|▊ | 82/1000 [00:04<00:47, 19.23it/s, loss=31.7]
8%|▊ | 84/1000 [00:04<00:47, 19.26it/s, loss=31.7]
8%|▊ | 84/1000 [00:04<00:47, 19.26it/s, loss=31.6]
8%|▊ | 84/1000 [00:04<00:47, 19.26it/s, loss=31.5]
9%|▊ | 86/1000 [00:04<00:47, 19.26it/s, loss=31.5]
9%|▊ | 86/1000 [00:04<00:47, 19.26it/s, loss=31.4]
9%|▊ | 86/1000 [00:04<00:47, 19.26it/s, loss=31.3]
9%|▉ | 88/1000 [00:04<00:47, 19.20it/s, loss=31.3]
9%|▉ | 88/1000 [00:04<00:47, 19.20it/s, loss=31.2]
9%|▉ | 88/1000 [00:04<00:47, 19.20it/s, loss=31.1]
9%|▉ | 90/1000 [00:04<00:47, 19.17it/s, loss=31.1]
9%|▉ | 90/1000 [00:04<00:47, 19.17it/s, loss=31]
9%|▉ | 90/1000 [00:04<00:47, 19.17it/s, loss=30.9]
9%|▉ | 92/1000 [00:04<00:47, 19.25it/s, loss=30.9]
9%|▉ | 92/1000 [00:04<00:47, 19.25it/s, loss=30.8]
9%|▉ | 92/1000 [00:04<00:47, 19.25it/s, loss=30.7]
9%|▉ | 94/1000 [00:04<00:47, 19.24it/s, loss=30.7]
9%|▉ | 94/1000 [00:04<00:47, 19.24it/s, loss=30.6]
9%|▉ | 94/1000 [00:05<00:47, 19.24it/s, loss=30.5]
10%|▉ | 96/1000 [00:05<00:47, 19.18it/s, loss=30.5]
10%|▉ | 96/1000 [00:05<00:47, 19.18it/s, loss=30.4]
10%|▉ | 96/1000 [00:05<00:47, 19.18it/s, loss=30.4]
10%|▉ | 98/1000 [00:05<00:47, 19.16it/s, loss=30.4]
10%|▉ | 98/1000 [00:05<00:47, 19.16it/s, loss=30.3]
10%|▉ | 98/1000 [00:05<00:47, 19.16it/s, loss=30.2]
10%|█ | 100/1000 [00:05<00:46, 19.18it/s, loss=30.2]
10%|█ | 100/1000 [00:05<00:46, 19.18it/s, loss=30.1]
10%|█ | 100/1000 [00:05<00:46, 19.18it/s, loss=30]
10%|█ | 102/1000 [00:05<00:46, 19.19it/s, loss=30]
10%|█ | 102/1000 [00:05<00:46, 19.19it/s, loss=29.9]
10%|█ | 102/1000 [00:05<00:46, 19.19it/s, loss=29.8]
10%|█ | 104/1000 [00:05<00:46, 19.14it/s, loss=29.8]
10%|█ | 104/1000 [00:05<00:46, 19.14it/s, loss=29.7]
10%|█ | 104/1000 [00:05<00:46, 19.14it/s, loss=29.6]
11%|█ | 106/1000 [00:05<00:46, 19.22it/s, loss=29.6]
11%|█ | 106/1000 [00:05<00:46, 19.22it/s, loss=29.5]
11%|█ | 106/1000 [00:05<00:46, 19.22it/s, loss=29.4]
11%|█ | 108/1000 [00:05<00:46, 19.15it/s, loss=29.4]
11%|█ | 108/1000 [00:05<00:46, 19.15it/s, loss=29.3]
11%|█ | 108/1000 [00:05<00:46, 19.15it/s, loss=29.3]
11%|█ | 110/1000 [00:05<00:46, 19.12it/s, loss=29.3]
11%|█ | 110/1000 [00:05<00:46, 19.12it/s, loss=29.2]
11%|█ | 110/1000 [00:05<00:46, 19.12it/s, loss=29.1]
11%|█ | 112/1000 [00:05<00:46, 19.12it/s, loss=29.1]
11%|█ | 112/1000 [00:05<00:46, 19.12it/s, loss=29]
11%|█ | 112/1000 [00:05<00:46, 19.12it/s, loss=28.9]
11%|█▏ | 114/1000 [00:05<00:46, 19.14it/s, loss=28.9]
11%|█▏ | 114/1000 [00:06<00:46, 19.14it/s, loss=28.8]
11%|█▏ | 114/1000 [00:06<00:46, 19.14it/s, loss=28.7]
12%|█▏ | 116/1000 [00:06<00:46, 19.21it/s, loss=28.7]
12%|█▏ | 116/1000 [00:06<00:46, 19.21it/s, loss=28.6]
12%|█▏ | 116/1000 [00:06<00:46, 19.21it/s, loss=28.5]
12%|█▏ | 118/1000 [00:06<00:45, 19.29it/s, loss=28.5]
12%|█▏ | 118/1000 [00:06<00:45, 19.29it/s, loss=28.5]
12%|█▏ | 118/1000 [00:06<00:45, 19.29it/s, loss=28.4]
12%|█▏ | 120/1000 [00:06<00:45, 19.31it/s, loss=28.4]
12%|█▏ | 120/1000 [00:06<00:45, 19.31it/s, loss=28.3]
12%|█▏ | 120/1000 [00:06<00:45, 19.31it/s, loss=28.2]
12%|█▏ | 122/1000 [00:06<00:45, 19.29it/s, loss=28.2]
12%|█▏ | 122/1000 [00:06<00:45, 19.29it/s, loss=28.1]
12%|█▏ | 122/1000 [00:06<00:45, 19.29it/s, loss=28]
12%|█▏ | 124/1000 [00:06<00:45, 19.26it/s, loss=28]
12%|█▏ | 124/1000 [00:06<00:45, 19.26it/s, loss=27.9]
12%|█▏ | 124/1000 [00:06<00:45, 19.26it/s, loss=27.9]
13%|█▎ | 126/1000 [00:06<00:45, 19.28it/s, loss=27.9]
13%|█▎ | 126/1000 [00:06<00:45, 19.28it/s, loss=27.8]
13%|█▎ | 126/1000 [00:06<00:45, 19.28it/s, loss=27.7]
13%|█▎ | 128/1000 [00:06<00:45, 19.26it/s, loss=27.7]
13%|█▎ | 128/1000 [00:06<00:45, 19.26it/s, loss=27.6]
13%|█▎ | 128/1000 [00:06<00:45, 19.26it/s, loss=27.5]
13%|█▎ | 130/1000 [00:06<00:45, 19.17it/s, loss=27.5]
13%|█▎ | 130/1000 [00:06<00:45, 19.17it/s, loss=27.5]
13%|█▎ | 130/1000 [00:06<00:45, 19.17it/s, loss=27.4]
13%|█▎ | 132/1000 [00:06<00:45, 19.20it/s, loss=27.4]
13%|█▎ | 132/1000 [00:06<00:45, 19.20it/s, loss=27.3]
13%|█▎ | 132/1000 [00:06<00:45, 19.20it/s, loss=27.2]
13%|█▎ | 134/1000 [00:06<00:45, 19.19it/s, loss=27.2]
13%|█▎ | 134/1000 [00:07<00:45, 19.19it/s, loss=27.1]
13%|█▎ | 134/1000 [00:07<00:45, 19.19it/s, loss=27]
14%|█▎ | 136/1000 [00:07<00:44, 19.21it/s, loss=27]
14%|█▎ | 136/1000 [00:07<00:44, 19.21it/s, loss=27]
14%|█▎ | 136/1000 [00:07<00:44, 19.21it/s, loss=26.9]
14%|█▍ | 138/1000 [00:07<00:44, 19.25it/s, loss=26.9]
14%|█▍ | 138/1000 [00:07<00:44, 19.25it/s, loss=26.8]
14%|█▍ | 138/1000 [00:07<00:44, 19.25it/s, loss=26.7]
14%|█▍ | 140/1000 [00:07<00:44, 19.25it/s, loss=26.7]
14%|█▍ | 140/1000 [00:07<00:44, 19.25it/s, loss=26.7]
14%|█▍ | 140/1000 [00:07<00:44, 19.25it/s, loss=26.6]
14%|█▍ | 142/1000 [00:07<00:44, 19.24it/s, loss=26.6]
14%|█▍ | 142/1000 [00:07<00:44, 19.24it/s, loss=26.5]
14%|█▍ | 142/1000 [00:07<00:44, 19.24it/s, loss=26.4]
14%|█▍ | 144/1000 [00:07<00:44, 19.23it/s, loss=26.4]
14%|█▍ | 144/1000 [00:07<00:44, 19.23it/s, loss=26.3]
14%|█▍ | 144/1000 [00:07<00:44, 19.23it/s, loss=26.3]
15%|█▍ | 146/1000 [00:07<00:44, 19.21it/s, loss=26.3]
15%|█▍ | 146/1000 [00:07<00:44, 19.21it/s, loss=26.2]
15%|█▍ | 146/1000 [00:07<00:44, 19.21it/s, loss=26.1]
15%|█▍ | 148/1000 [00:07<00:44, 19.20it/s, loss=26.1]
15%|█▍ | 148/1000 [00:07<00:44, 19.20it/s, loss=26]
15%|█▍ | 148/1000 [00:07<00:44, 19.20it/s, loss=26]
15%|█▌ | 150/1000 [00:07<00:44, 19.12it/s, loss=26]
15%|█▌ | 150/1000 [00:07<00:44, 19.12it/s, loss=25.9]
15%|█▌ | 150/1000 [00:07<00:44, 19.12it/s, loss=25.8]
15%|█▌ | 152/1000 [00:07<00:44, 19.13it/s, loss=25.8]
15%|█▌ | 152/1000 [00:07<00:44, 19.13it/s, loss=25.7]
15%|█▌ | 152/1000 [00:08<00:44, 19.13it/s, loss=25.7]
15%|█▌ | 154/1000 [00:08<00:44, 19.08it/s, loss=25.7]
15%|█▌ | 154/1000 [00:08<00:44, 19.08it/s, loss=25.6]
15%|█▌ | 154/1000 [00:08<00:44, 19.08it/s, loss=25.5]
16%|█▌ | 156/1000 [00:08<00:43, 19.19it/s, loss=25.5]
16%|█▌ | 156/1000 [00:08<00:43, 19.19it/s, loss=25.4]
16%|█▌ | 156/1000 [00:08<00:43, 19.19it/s, loss=25.4]
16%|█▌ | 158/1000 [00:08<00:43, 19.22it/s, loss=25.4]
16%|█▌ | 158/1000 [00:08<00:43, 19.22it/s, loss=25.3]
16%|█▌ | 158/1000 [00:08<00:43, 19.22it/s, loss=25.2]
16%|█▌ | 160/1000 [00:08<00:43, 19.24it/s, loss=25.2]
16%|█▌ | 160/1000 [00:08<00:43, 19.24it/s, loss=25.2]
16%|█▌ | 160/1000 [00:08<00:43, 19.24it/s, loss=25.1]
16%|█▌ | 162/1000 [00:08<00:43, 19.25it/s, loss=25.1]
16%|█▌ | 162/1000 [00:08<00:43, 19.25it/s, loss=25]
16%|█▌ | 162/1000 [00:08<00:43, 19.25it/s, loss=24.9]
16%|█▋ | 164/1000 [00:08<00:43, 19.26it/s, loss=24.9]
16%|█▋ | 164/1000 [00:08<00:43, 19.26it/s, loss=24.9]
16%|█▋ | 164/1000 [00:08<00:43, 19.26it/s, loss=24.8]
17%|█▋ | 166/1000 [00:08<00:43, 19.27it/s, loss=24.8]
17%|█▋ | 166/1000 [00:08<00:43, 19.27it/s, loss=24.7]
17%|█▋ | 166/1000 [00:08<00:43, 19.27it/s, loss=24.7]
17%|█▋ | 168/1000 [00:08<00:43, 19.34it/s, loss=24.7]
17%|█▋ | 168/1000 [00:08<00:43, 19.34it/s, loss=24.6]
17%|█▋ | 168/1000 [00:08<00:43, 19.34it/s, loss=24.5]
17%|█▋ | 170/1000 [00:08<00:43, 19.27it/s, loss=24.5]
17%|█▋ | 170/1000 [00:08<00:43, 19.27it/s, loss=24.4]
17%|█▋ | 170/1000 [00:08<00:43, 19.27it/s, loss=24.4]
17%|█▋ | 172/1000 [00:08<00:43, 19.23it/s, loss=24.4]
17%|█▋ | 172/1000 [00:09<00:43, 19.23it/s, loss=24.3]
17%|█▋ | 172/1000 [00:09<00:43, 19.23it/s, loss=24.2]
17%|█▋ | 174/1000 [00:09<00:42, 19.22it/s, loss=24.2]
17%|█▋ | 174/1000 [00:09<00:42, 19.22it/s, loss=24.2]
17%|█▋ | 174/1000 [00:09<00:42, 19.22it/s, loss=24.1]
18%|█▊ | 176/1000 [00:09<00:42, 19.24it/s, loss=24.1]
18%|█▊ | 176/1000 [00:09<00:42, 19.24it/s, loss=24]
18%|█▊ | 176/1000 [00:09<00:42, 19.24it/s, loss=24]
18%|█▊ | 178/1000 [00:09<00:42, 19.12it/s, loss=24]
18%|█▊ | 178/1000 [00:09<00:42, 19.12it/s, loss=23.9]
18%|█▊ | 178/1000 [00:09<00:42, 19.12it/s, loss=23.8]
18%|█▊ | 180/1000 [00:09<00:42, 19.17it/s, loss=23.8]
18%|█▊ | 180/1000 [00:09<00:42, 19.17it/s, loss=23.8]
18%|█▊ | 180/1000 [00:09<00:42, 19.17it/s, loss=23.7]
18%|█▊ | 182/1000 [00:09<00:42, 19.17it/s, loss=23.7]
18%|█▊ | 182/1000 [00:09<00:42, 19.17it/s, loss=23.6]
18%|█▊ | 182/1000 [00:09<00:42, 19.17it/s, loss=23.6]
18%|█▊ | 184/1000 [00:09<00:42, 19.17it/s, loss=23.6]
18%|█▊ | 184/1000 [00:09<00:42, 19.17it/s, loss=23.5]
18%|█▊ | 184/1000 [00:09<00:42, 19.17it/s, loss=23.5]
19%|█▊ | 186/1000 [00:09<00:42, 19.21it/s, loss=23.5]
19%|█▊ | 186/1000 [00:09<00:42, 19.21it/s, loss=23.4]
19%|█▊ | 186/1000 [00:09<00:42, 19.21it/s, loss=23.3]
19%|█▉ | 188/1000 [00:09<00:42, 19.18it/s, loss=23.3]
19%|█▉ | 188/1000 [00:09<00:42, 19.18it/s, loss=23.3]
19%|█▉ | 188/1000 [00:09<00:42, 19.18it/s, loss=23.2]
19%|█▉ | 190/1000 [00:09<00:42, 19.20it/s, loss=23.2]
19%|█▉ | 190/1000 [00:09<00:42, 19.20it/s, loss=23.1]
19%|█▉ | 190/1000 [00:10<00:42, 19.20it/s, loss=23.1]
19%|█▉ | 192/1000 [00:10<00:42, 19.19it/s, loss=23.1]
19%|█▉ | 192/1000 [00:10<00:42, 19.19it/s, loss=23]
19%|█▉ | 192/1000 [00:10<00:42, 19.19it/s, loss=22.9]
19%|█▉ | 194/1000 [00:10<00:41, 19.21it/s, loss=22.9]
19%|█▉ | 194/1000 [00:10<00:41, 19.21it/s, loss=22.9]
19%|█▉ | 194/1000 [00:10<00:41, 19.21it/s, loss=22.8]
20%|█▉ | 196/1000 [00:10<00:41, 19.24it/s, loss=22.8]
20%|█▉ | 196/1000 [00:10<00:41, 19.24it/s, loss=22.8]
20%|█▉ | 196/1000 [00:10<00:41, 19.24it/s, loss=22.7]
20%|█▉ | 198/1000 [00:10<00:41, 19.25it/s, loss=22.7]
20%|█▉ | 198/1000 [00:10<00:41, 19.25it/s, loss=22.6]
20%|█▉ | 198/1000 [00:10<00:41, 19.25it/s, loss=22.6]
20%|██ | 200/1000 [00:10<00:41, 19.23it/s, loss=22.6]
20%|██ | 200/1000 [00:10<00:41, 19.23it/s, loss=22.5]
20%|██ | 200/1000 [00:10<00:41, 19.23it/s, loss=22.5]
20%|██ | 202/1000 [00:10<00:41, 19.22it/s, loss=22.5]
20%|██ | 202/1000 [00:10<00:41, 19.22it/s, loss=22.4]
20%|██ | 202/1000 [00:10<00:41, 19.22it/s, loss=22.3]
20%|██ | 204/1000 [00:10<00:41, 19.13it/s, loss=22.3]
20%|██ | 204/1000 [00:10<00:41, 19.13it/s, loss=22.3]
20%|██ | 204/1000 [00:10<00:41, 19.13it/s, loss=22.2]
21%|██ | 206/1000 [00:10<00:41, 19.14it/s, loss=22.2]
21%|██ | 206/1000 [00:10<00:41, 19.14it/s, loss=22.2]
21%|██ | 206/1000 [00:10<00:41, 19.14it/s, loss=22.1]
21%|██ | 208/1000 [00:10<00:41, 19.11it/s, loss=22.1]
21%|██ | 208/1000 [00:10<00:41, 19.11it/s, loss=22]
21%|██ | 208/1000 [00:10<00:41, 19.11it/s, loss=22]
21%|██ | 210/1000 [00:10<00:41, 19.18it/s, loss=22]
21%|██ | 210/1000 [00:10<00:41, 19.18it/s, loss=21.9]
21%|██ | 210/1000 [00:11<00:41, 19.18it/s, loss=21.9]
21%|██ | 212/1000 [00:11<00:41, 19.17it/s, loss=21.9]
21%|██ | 212/1000 [00:11<00:41, 19.17it/s, loss=21.8]
21%|██ | 212/1000 [00:11<00:41, 19.17it/s, loss=21.8]
21%|██▏ | 214/1000 [00:11<00:41, 19.15it/s, loss=21.8]
21%|██▏ | 214/1000 [00:11<00:41, 19.15it/s, loss=21.7]
21%|██▏ | 214/1000 [00:11<00:41, 19.15it/s, loss=21.6]
22%|██▏ | 216/1000 [00:11<00:40, 19.17it/s, loss=21.6]
22%|██▏ | 216/1000 [00:11<00:40, 19.17it/s, loss=21.6]
22%|██▏ | 216/1000 [00:11<00:40, 19.17it/s, loss=21.5]
22%|██▏ | 218/1000 [00:11<00:40, 19.15it/s, loss=21.5]
22%|██▏ | 218/1000 [00:11<00:40, 19.15it/s, loss=21.5]
22%|██▏ | 218/1000 [00:11<00:40, 19.15it/s, loss=21.4]
22%|██▏ | 220/1000 [00:11<00:40, 19.18it/s, loss=21.4]
22%|██▏ | 220/1000 [00:11<00:40, 19.18it/s, loss=21.4]
22%|██▏ | 220/1000 [00:11<00:40, 19.18it/s, loss=21.3]
22%|██▏ | 222/1000 [00:11<00:40, 19.23it/s, loss=21.3]
22%|██▏ | 222/1000 [00:11<00:40, 19.23it/s, loss=21.3]
22%|██▏ | 222/1000 [00:11<00:40, 19.23it/s, loss=21.2]
22%|██▏ | 224/1000 [00:11<00:40, 19.27it/s, loss=21.2]
22%|██▏ | 224/1000 [00:11<00:40, 19.27it/s, loss=21.2]
22%|██▏ | 224/1000 [00:11<00:40, 19.27it/s, loss=21.1]
23%|██▎ | 226/1000 [00:11<00:39, 19.42it/s, loss=21.1]
23%|██▎ | 226/1000 [00:11<00:39, 19.42it/s, loss=21]
23%|██▎ | 226/1000 [00:11<00:39, 19.42it/s, loss=21]
23%|██▎ | 228/1000 [00:11<00:39, 19.34it/s, loss=21]
23%|██▎ | 228/1000 [00:11<00:39, 19.34it/s, loss=20.9]
23%|██▎ | 228/1000 [00:11<00:39, 19.34it/s, loss=20.9]
23%|██▎ | 230/1000 [00:11<00:39, 19.30it/s, loss=20.9]
23%|██▎ | 230/1000 [00:12<00:39, 19.30it/s, loss=20.8]
23%|██▎ | 230/1000 [00:12<00:39, 19.30it/s, loss=20.8]
23%|██▎ | 232/1000 [00:12<00:40, 19.20it/s, loss=20.8]
23%|██▎ | 232/1000 [00:12<00:40, 19.20it/s, loss=20.7]
23%|██▎ | 232/1000 [00:12<00:40, 19.20it/s, loss=20.7]
23%|██▎ | 234/1000 [00:12<00:39, 19.22it/s, loss=20.7]
23%|██▎ | 234/1000 [00:12<00:39, 19.22it/s, loss=20.6]
23%|██▎ | 234/1000 [00:12<00:39, 19.22it/s, loss=20.6]
24%|██▎ | 236/1000 [00:12<00:39, 19.26it/s, loss=20.6]
24%|██▎ | 236/1000 [00:12<00:39, 19.26it/s, loss=20.5]
24%|██▎ | 236/1000 [00:12<00:39, 19.26it/s, loss=20.5]
24%|██▍ | 238/1000 [00:12<00:39, 19.29it/s, loss=20.5]
24%|██▍ | 238/1000 [00:12<00:39, 19.29it/s, loss=20.4]
24%|██▍ | 238/1000 [00:12<00:39, 19.29it/s, loss=20.4]
24%|██▍ | 240/1000 [00:12<00:39, 19.34it/s, loss=20.4]
24%|██▍ | 240/1000 [00:12<00:39, 19.34it/s, loss=20.3]
24%|██▍ | 240/1000 [00:12<00:39, 19.34it/s, loss=20.3]
24%|██▍ | 242/1000 [00:12<00:39, 19.34it/s, loss=20.3]
24%|██▍ | 242/1000 [00:12<00:39, 19.34it/s, loss=20.2]
24%|██▍ | 242/1000 [00:12<00:39, 19.34it/s, loss=20.2]
24%|██▍ | 244/1000 [00:12<00:39, 19.35it/s, loss=20.2]
24%|██▍ | 244/1000 [00:12<00:39, 19.35it/s, loss=20.1]
24%|██▍ | 244/1000 [00:12<00:39, 19.35it/s, loss=20.1]
25%|██▍ | 246/1000 [00:12<00:38, 19.40it/s, loss=20.1]
25%|██▍ | 246/1000 [00:12<00:38, 19.40it/s, loss=20]
25%|██▍ | 246/1000 [00:12<00:38, 19.40it/s, loss=20]
25%|██▍ | 248/1000 [00:12<00:38, 19.35it/s, loss=20]
25%|██▍ | 248/1000 [00:12<00:38, 19.35it/s, loss=19.9]
25%|██▍ | 248/1000 [00:13<00:38, 19.35it/s, loss=19.9]
25%|██▌ | 250/1000 [00:13<00:38, 19.33it/s, loss=19.9]
25%|██▌ | 250/1000 [00:13<00:38, 19.33it/s, loss=19.8]
25%|██▌ | 250/1000 [00:13<00:38, 19.33it/s, loss=19.8]
25%|██▌ | 252/1000 [00:13<00:38, 19.29it/s, loss=19.8]
25%|██▌ | 252/1000 [00:13<00:38, 19.29it/s, loss=19.7]
25%|██▌ | 252/1000 [00:13<00:38, 19.29it/s, loss=19.7]
25%|██▌ | 254/1000 [00:13<00:38, 19.31it/s, loss=19.7]
25%|██▌ | 254/1000 [00:13<00:38, 19.31it/s, loss=19.6]
25%|██▌ | 254/1000 [00:13<00:38, 19.31it/s, loss=19.6]
26%|██▌ | 256/1000 [00:13<00:38, 19.28it/s, loss=19.6]
26%|██▌ | 256/1000 [00:13<00:38, 19.28it/s, loss=19.5]
26%|██▌ | 256/1000 [00:13<00:38, 19.28it/s, loss=19.5]
26%|██▌ | 258/1000 [00:13<00:38, 19.28it/s, loss=19.5]
26%|██▌ | 258/1000 [00:13<00:38, 19.28it/s, loss=19.4]
26%|██▌ | 258/1000 [00:13<00:38, 19.28it/s, loss=19.4]
26%|██▌ | 260/1000 [00:13<00:38, 19.35it/s, loss=19.4]
26%|██▌ | 260/1000 [00:13<00:38, 19.35it/s, loss=19.3]
26%|██▌ | 260/1000 [00:13<00:38, 19.35it/s, loss=19.3]
26%|██▌ | 262/1000 [00:13<00:38, 19.34it/s, loss=19.3]
26%|██▌ | 262/1000 [00:13<00:38, 19.34it/s, loss=19.3]
26%|██▌ | 262/1000 [00:13<00:38, 19.34it/s, loss=19.2]
26%|██▋ | 264/1000 [00:13<00:38, 19.32it/s, loss=19.2]
26%|██▋ | 264/1000 [00:13<00:38, 19.32it/s, loss=19.2]
26%|██▋ | 264/1000 [00:13<00:38, 19.32it/s, loss=19.1]
27%|██▋ | 266/1000 [00:13<00:38, 19.31it/s, loss=19.1]
27%|██▋ | 266/1000 [00:13<00:38, 19.31it/s, loss=19.1]
27%|██▋ | 266/1000 [00:13<00:38, 19.31it/s, loss=19]
27%|██▋ | 268/1000 [00:13<00:37, 19.30it/s, loss=19]
27%|██▋ | 268/1000 [00:14<00:37, 19.30it/s, loss=19]
27%|██▋ | 268/1000 [00:14<00:37, 19.30it/s, loss=18.9]
27%|██▋ | 270/1000 [00:14<00:37, 19.27it/s, loss=18.9]
27%|██▋ | 270/1000 [00:14<00:37, 19.27it/s, loss=18.9]
27%|██▋ | 270/1000 [00:14<00:37, 19.27it/s, loss=18.8]
27%|██▋ | 272/1000 [00:14<00:37, 19.25it/s, loss=18.8]
27%|██▋ | 272/1000 [00:14<00:37, 19.25it/s, loss=18.8]
27%|██▋ | 272/1000 [00:14<00:37, 19.25it/s, loss=18.8]
27%|██▋ | 274/1000 [00:14<00:37, 19.26it/s, loss=18.8]
27%|██▋ | 274/1000 [00:14<00:37, 19.26it/s, loss=18.7]
27%|██▋ | 274/1000 [00:14<00:37, 19.26it/s, loss=18.7]
28%|██▊ | 276/1000 [00:14<00:37, 19.20it/s, loss=18.7]
28%|██▊ | 276/1000 [00:14<00:37, 19.20it/s, loss=18.6]
28%|██▊ | 276/1000 [00:14<00:37, 19.20it/s, loss=18.6]
28%|██▊ | 278/1000 [00:14<00:37, 19.19it/s, loss=18.6]
28%|██▊ | 278/1000 [00:14<00:37, 19.19it/s, loss=18.5]
28%|██▊ | 278/1000 [00:14<00:37, 19.19it/s, loss=18.5]
28%|██▊ | 280/1000 [00:14<00:37, 19.19it/s, loss=18.5]
28%|██▊ | 280/1000 [00:14<00:37, 19.19it/s, loss=18.5]
28%|██▊ | 280/1000 [00:14<00:37, 19.19it/s, loss=18.4]
28%|██▊ | 282/1000 [00:14<00:37, 19.25it/s, loss=18.4]
28%|██▊ | 282/1000 [00:14<00:37, 19.25it/s, loss=18.4]
28%|██▊ | 282/1000 [00:14<00:37, 19.25it/s, loss=18.3]
28%|██▊ | 284/1000 [00:14<00:37, 19.24it/s, loss=18.3]
28%|██▊ | 284/1000 [00:14<00:37, 19.24it/s, loss=18.3]
28%|██▊ | 284/1000 [00:14<00:37, 19.24it/s, loss=18.2]
29%|██▊ | 286/1000 [00:14<00:37, 19.22it/s, loss=18.2]
29%|██▊ | 286/1000 [00:14<00:37, 19.22it/s, loss=18.2]
29%|██▊ | 286/1000 [00:14<00:37, 19.22it/s, loss=18.2]
29%|██▉ | 288/1000 [00:14<00:37, 19.20it/s, loss=18.2]
29%|██▉ | 288/1000 [00:15<00:37, 19.20it/s, loss=18.1]
29%|██▉ | 288/1000 [00:15<00:37, 19.20it/s, loss=18.1]
29%|██▉ | 290/1000 [00:15<00:36, 19.21it/s, loss=18.1]
29%|██▉ | 290/1000 [00:15<00:36, 19.21it/s, loss=18]
29%|██▉ | 290/1000 [00:15<00:36, 19.21it/s, loss=18]
29%|██▉ | 292/1000 [00:15<00:36, 19.22it/s, loss=18]
29%|██▉ | 292/1000 [00:15<00:36, 19.22it/s, loss=18]
29%|██▉ | 292/1000 [00:15<00:36, 19.22it/s, loss=17.9]
29%|██▉ | 294/1000 [00:15<00:36, 19.26it/s, loss=17.9]
29%|██▉ | 294/1000 [00:15<00:36, 19.26it/s, loss=17.9]
29%|██▉ | 294/1000 [00:15<00:36, 19.26it/s, loss=17.8]
30%|██▉ | 296/1000 [00:15<00:36, 19.25it/s, loss=17.8]
30%|██▉ | 296/1000 [00:15<00:36, 19.25it/s, loss=17.8]
30%|██▉ | 296/1000 [00:15<00:36, 19.25it/s, loss=17.8]
30%|██▉ | 298/1000 [00:15<00:36, 19.24it/s, loss=17.8]
30%|██▉ | 298/1000 [00:15<00:36, 19.24it/s, loss=17.7]
30%|██▉ | 298/1000 [00:15<00:36, 19.24it/s, loss=17.7]
30%|███ | 300/1000 [00:15<00:36, 19.28it/s, loss=17.7]
30%|███ | 300/1000 [00:15<00:36, 19.28it/s, loss=17.6]
30%|███ | 300/1000 [00:15<00:36, 19.28it/s, loss=17.6]
30%|███ | 302/1000 [00:15<00:36, 19.34it/s, loss=17.6]
30%|███ | 302/1000 [00:15<00:36, 19.34it/s, loss=17.6]
30%|███ | 302/1000 [00:15<00:36, 19.34it/s, loss=17.5]
30%|███ | 304/1000 [00:15<00:36, 19.22it/s, loss=17.5]
30%|███ | 304/1000 [00:15<00:36, 19.22it/s, loss=17.5]
30%|███ | 304/1000 [00:15<00:36, 19.22it/s, loss=17.4]
31%|███ | 306/1000 [00:15<00:36, 19.24it/s, loss=17.4]
31%|███ | 306/1000 [00:15<00:36, 19.24it/s, loss=17.4]
31%|███ | 306/1000 [00:16<00:36, 19.24it/s, loss=17.4]
31%|███ | 308/1000 [00:16<00:35, 19.31it/s, loss=17.4]
31%|███ | 308/1000 [00:16<00:35, 19.31it/s, loss=17.3]
31%|███ | 308/1000 [00:16<00:35, 19.31it/s, loss=17.3]
31%|███ | 310/1000 [00:16<00:35, 19.25it/s, loss=17.3]
31%|███ | 310/1000 [00:16<00:35, 19.25it/s, loss=17.3]
31%|███ | 310/1000 [00:16<00:35, 19.25it/s, loss=17.2]
31%|███ | 312/1000 [00:16<00:35, 19.31it/s, loss=17.2]
31%|███ | 312/1000 [00:16<00:35, 19.31it/s, loss=17.2]
31%|███ | 312/1000 [00:16<00:35, 19.31it/s, loss=17.1]
31%|███▏ | 314/1000 [00:16<00:35, 19.24it/s, loss=17.1]
31%|███▏ | 314/1000 [00:16<00:35, 19.24it/s, loss=17.1]
31%|███▏ | 314/1000 [00:16<00:35, 19.24it/s, loss=17.1]
32%|███▏ | 316/1000 [00:16<00:35, 19.26it/s, loss=17.1]
32%|███▏ | 316/1000 [00:16<00:35, 19.26it/s, loss=17]
32%|███▏ | 316/1000 [00:16<00:35, 19.26it/s, loss=17]
32%|███▏ | 318/1000 [00:16<00:35, 19.21it/s, loss=17]
32%|███▏ | 318/1000 [00:16<00:35, 19.21it/s, loss=17]
32%|███▏ | 318/1000 [00:16<00:35, 19.21it/s, loss=16.9]
32%|███▏ | 320/1000 [00:16<00:35, 19.20it/s, loss=16.9]
32%|███▏ | 320/1000 [00:16<00:35, 19.20it/s, loss=16.9]
32%|███▏ | 320/1000 [00:16<00:35, 19.20it/s, loss=16.8]
32%|███▏ | 322/1000 [00:16<00:35, 19.22it/s, loss=16.8]
32%|███▏ | 322/1000 [00:16<00:35, 19.22it/s, loss=16.8]
32%|███▏ | 322/1000 [00:16<00:35, 19.22it/s, loss=16.8]
32%|███▏ | 324/1000 [00:16<00:35, 19.22it/s, loss=16.8]
32%|███▏ | 324/1000 [00:16<00:35, 19.22it/s, loss=16.7]
32%|███▏ | 324/1000 [00:16<00:35, 19.22it/s, loss=16.7]
33%|███▎ | 326/1000 [00:16<00:35, 19.23it/s, loss=16.7]
33%|███▎ | 326/1000 [00:17<00:35, 19.23it/s, loss=16.7]
33%|███▎ | 326/1000 [00:17<00:35, 19.23it/s, loss=16.6]
33%|███▎ | 328/1000 [00:17<00:34, 19.25it/s, loss=16.6]
33%|███▎ | 328/1000 [00:17<00:34, 19.25it/s, loss=16.6]
33%|███▎ | 328/1000 [00:17<00:34, 19.25it/s, loss=16.6]
33%|███▎ | 330/1000 [00:17<00:34, 19.21it/s, loss=16.6]
33%|███▎ | 330/1000 [00:17<00:34, 19.21it/s, loss=16.5]
33%|███▎ | 330/1000 [00:17<00:34, 19.21it/s, loss=16.5]
33%|███▎ | 332/1000 [00:17<00:34, 19.19it/s, loss=16.5]
33%|███▎ | 332/1000 [00:17<00:34, 19.19it/s, loss=16.5]
33%|███▎ | 332/1000 [00:17<00:34, 19.19it/s, loss=16.4]
33%|███▎ | 334/1000 [00:17<00:34, 19.15it/s, loss=16.4]
33%|███▎ | 334/1000 [00:17<00:34, 19.15it/s, loss=16.4]
33%|███▎ | 334/1000 [00:17<00:34, 19.15it/s, loss=16.4]
34%|███▎ | 336/1000 [00:17<00:34, 19.14it/s, loss=16.4]
34%|███▎ | 336/1000 [00:17<00:34, 19.14it/s, loss=16.3]
34%|███▎ | 336/1000 [00:17<00:34, 19.14it/s, loss=16.3]
34%|███▍ | 338/1000 [00:17<00:34, 19.17it/s, loss=16.3]
34%|███▍ | 338/1000 [00:17<00:34, 19.17it/s, loss=16.3]
34%|███▍ | 338/1000 [00:17<00:34, 19.17it/s, loss=16.2]
34%|███▍ | 340/1000 [00:17<00:34, 19.11it/s, loss=16.2]
34%|███▍ | 340/1000 [00:17<00:34, 19.11it/s, loss=16.2]
34%|███▍ | 340/1000 [00:17<00:34, 19.11it/s, loss=16.2]
34%|███▍ | 342/1000 [00:17<00:34, 19.13it/s, loss=16.2]
34%|███▍ | 342/1000 [00:17<00:34, 19.13it/s, loss=16.1]
34%|███▍ | 342/1000 [00:17<00:34, 19.13it/s, loss=16.1]
34%|███▍ | 344/1000 [00:17<00:34, 19.16it/s, loss=16.1]
34%|███▍ | 344/1000 [00:17<00:34, 19.16it/s, loss=16.1]
34%|███▍ | 344/1000 [00:18<00:34, 19.16it/s, loss=16]
35%|███▍ | 346/1000 [00:18<00:34, 19.14it/s, loss=16]
35%|███▍ | 346/1000 [00:18<00:34, 19.14it/s, loss=16]
35%|███▍ | 346/1000 [00:18<00:34, 19.14it/s, loss=16]
35%|███▍ | 348/1000 [00:18<00:34, 19.11it/s, loss=16]
35%|███▍ | 348/1000 [00:18<00:34, 19.11it/s, loss=15.9]
35%|███▍ | 348/1000 [00:18<00:34, 19.11it/s, loss=15.9]
35%|███▌ | 350/1000 [00:18<00:33, 19.14it/s, loss=15.9]
35%|███▌ | 350/1000 [00:18<00:33, 19.14it/s, loss=15.9]
35%|███▌ | 350/1000 [00:18<00:33, 19.14it/s, loss=15.8]
35%|███▌ | 352/1000 [00:18<00:33, 19.09it/s, loss=15.8]
35%|███▌ | 352/1000 [00:18<00:33, 19.09it/s, loss=15.8]
35%|███▌ | 352/1000 [00:18<00:33, 19.09it/s, loss=15.8]
35%|███▌ | 354/1000 [00:18<00:33, 19.11it/s, loss=15.8]
35%|███▌ | 354/1000 [00:18<00:33, 19.11it/s, loss=15.7]
35%|███▌ | 354/1000 [00:18<00:33, 19.11it/s, loss=15.7]
36%|███▌ | 356/1000 [00:18<00:33, 19.06it/s, loss=15.7]
36%|███▌ | 356/1000 [00:18<00:33, 19.06it/s, loss=15.7]
36%|███▌ | 356/1000 [00:18<00:33, 19.06it/s, loss=15.6]
36%|███▌ | 358/1000 [00:18<00:33, 19.07it/s, loss=15.6]
36%|███▌ | 358/1000 [00:18<00:33, 19.07it/s, loss=15.6]
36%|███▌ | 358/1000 [00:18<00:33, 19.07it/s, loss=15.6]
36%|███▌ | 360/1000 [00:18<00:33, 19.03it/s, loss=15.6]
36%|███▌ | 360/1000 [00:18<00:33, 19.03it/s, loss=15.5]
36%|███▌ | 360/1000 [00:18<00:33, 19.03it/s, loss=15.5]
36%|███▌ | 362/1000 [00:18<00:33, 19.08it/s, loss=15.5]
36%|███▌ | 362/1000 [00:18<00:33, 19.08it/s, loss=15.5]
36%|███▌ | 362/1000 [00:18<00:33, 19.08it/s, loss=15.4]
36%|███▋ | 364/1000 [00:18<00:33, 19.09it/s, loss=15.4]
36%|███▋ | 364/1000 [00:19<00:33, 19.09it/s, loss=15.4]
36%|███▋ | 364/1000 [00:19<00:33, 19.09it/s, loss=15.4]
37%|███▋ | 366/1000 [00:19<00:33, 19.15it/s, loss=15.4]
37%|███▋ | 366/1000 [00:19<00:33, 19.15it/s, loss=15.4]
37%|███▋ | 366/1000 [00:19<00:33, 19.15it/s, loss=15.3]
37%|███▋ | 368/1000 [00:19<00:32, 19.19it/s, loss=15.3]
37%|███▋ | 368/1000 [00:19<00:32, 19.19it/s, loss=15.3]
37%|███▋ | 368/1000 [00:19<00:32, 19.19it/s, loss=15.3]
37%|███▋ | 370/1000 [00:19<00:32, 19.22it/s, loss=15.3]
37%|███▋ | 370/1000 [00:19<00:32, 19.22it/s, loss=15.2]
37%|███▋ | 370/1000 [00:19<00:32, 19.22it/s, loss=15.2]
37%|███▋ | 372/1000 [00:19<00:32, 19.19it/s, loss=15.2]
37%|███▋ | 372/1000 [00:19<00:32, 19.19it/s, loss=15.2]
37%|███▋ | 372/1000 [00:19<00:32, 19.19it/s, loss=15.1]
37%|███▋ | 374/1000 [00:19<00:32, 19.16it/s, loss=15.1]
37%|███▋ | 374/1000 [00:19<00:32, 19.16it/s, loss=15.1]
37%|███▋ | 374/1000 [00:19<00:32, 19.16it/s, loss=15.1]
38%|███▊ | 376/1000 [00:19<00:32, 19.11it/s, loss=15.1]
38%|███▊ | 376/1000 [00:19<00:32, 19.11it/s, loss=15.1]
38%|███▊ | 376/1000 [00:19<00:32, 19.11it/s, loss=15]
38%|███▊ | 378/1000 [00:19<00:32, 19.07it/s, loss=15]
38%|███▊ | 378/1000 [00:19<00:32, 19.07it/s, loss=15]
38%|███▊ | 378/1000 [00:19<00:32, 19.07it/s, loss=15]
38%|███▊ | 380/1000 [00:19<00:32, 19.13it/s, loss=15]
38%|███▊ | 380/1000 [00:19<00:32, 19.13it/s, loss=14.9]
38%|███▊ | 380/1000 [00:19<00:32, 19.13it/s, loss=14.9]
38%|███▊ | 382/1000 [00:19<00:32, 19.08it/s, loss=14.9]
38%|███▊ | 382/1000 [00:19<00:32, 19.08it/s, loss=14.9]
38%|███▊ | 382/1000 [00:20<00:32, 19.08it/s, loss=14.9]
38%|███▊ | 384/1000 [00:20<00:32, 19.09it/s, loss=14.9]
38%|███▊ | 384/1000 [00:20<00:32, 19.09it/s, loss=14.8]
38%|███▊ | 384/1000 [00:20<00:32, 19.09it/s, loss=14.8]
39%|███▊ | 386/1000 [00:20<00:32, 19.12it/s, loss=14.8]
39%|███▊ | 386/1000 [00:20<00:32, 19.12it/s, loss=14.8]
39%|███▊ | 386/1000 [00:20<00:32, 19.12it/s, loss=14.7]
39%|███▉ | 388/1000 [00:20<00:32, 19.11it/s, loss=14.7]
39%|███▉ | 388/1000 [00:20<00:32, 19.11it/s, loss=14.7]
39%|███▉ | 388/1000 [00:20<00:32, 19.11it/s, loss=14.7]
39%|███▉ | 390/1000 [00:20<00:31, 19.11it/s, loss=14.7]
39%|███▉ | 390/1000 [00:20<00:31, 19.11it/s, loss=14.7]
39%|███▉ | 390/1000 [00:20<00:31, 19.11it/s, loss=14.6]
39%|███▉ | 392/1000 [00:20<00:31, 19.01it/s, loss=14.6]
39%|███▉ | 392/1000 [00:20<00:31, 19.01it/s, loss=14.6]
39%|███▉ | 392/1000 [00:20<00:31, 19.01it/s, loss=14.6]
39%|███▉ | 394/1000 [00:20<00:31, 19.06it/s, loss=14.6]
39%|███▉ | 394/1000 [00:20<00:31, 19.06it/s, loss=14.5]
39%|███▉ | 394/1000 [00:20<00:31, 19.06it/s, loss=14.5]
40%|███▉ | 396/1000 [00:20<00:31, 19.06it/s, loss=14.5]
40%|███▉ | 396/1000 [00:20<00:31, 19.06it/s, loss=14.5]
40%|███▉ | 396/1000 [00:20<00:31, 19.06it/s, loss=14.5]
40%|███▉ | 398/1000 [00:20<00:31, 19.02it/s, loss=14.5]
40%|███▉ | 398/1000 [00:20<00:31, 19.02it/s, loss=14.4]
40%|███▉ | 398/1000 [00:20<00:31, 19.02it/s, loss=14.4]
40%|████ | 400/1000 [00:20<00:31, 19.01it/s, loss=14.4]
40%|████ | 400/1000 [00:20<00:31, 19.01it/s, loss=14.4]
40%|████ | 400/1000 [00:20<00:31, 19.01it/s, loss=14.4]
40%|████ | 402/1000 [00:20<00:31, 19.08it/s, loss=14.4]
40%|████ | 402/1000 [00:20<00:31, 19.08it/s, loss=14.3]
40%|████ | 402/1000 [00:21<00:31, 19.08it/s, loss=14.3]
40%|████ | 404/1000 [00:21<00:31, 19.03it/s, loss=14.3]
40%|████ | 404/1000 [00:21<00:31, 19.03it/s, loss=14.3]
40%|████ | 404/1000 [00:21<00:31, 19.03it/s, loss=14.2]
41%|████ | 406/1000 [00:21<00:31, 19.09it/s, loss=14.2]
41%|████ | 406/1000 [00:21<00:31, 19.09it/s, loss=14.2]
41%|████ | 406/1000 [00:21<00:31, 19.09it/s, loss=14.2]
41%|████ | 408/1000 [00:21<00:31, 19.08it/s, loss=14.2]
41%|████ | 408/1000 [00:21<00:31, 19.08it/s, loss=14.2]
41%|████ | 408/1000 [00:21<00:31, 19.08it/s, loss=14.1]
41%|████ | 410/1000 [00:21<00:30, 19.14it/s, loss=14.1]
41%|████ | 410/1000 [00:21<00:30, 19.14it/s, loss=14.1]
41%|████ | 410/1000 [00:21<00:30, 19.14it/s, loss=14.1]
41%|████ | 412/1000 [00:21<00:30, 19.18it/s, loss=14.1]
41%|████ | 412/1000 [00:21<00:30, 19.18it/s, loss=14.1]
41%|████ | 412/1000 [00:21<00:30, 19.18it/s, loss=14]
41%|████▏ | 414/1000 [00:21<00:30, 19.18it/s, loss=14]
41%|████▏ | 414/1000 [00:21<00:30, 19.18it/s, loss=14]
41%|████▏ | 414/1000 [00:21<00:30, 19.18it/s, loss=14]
42%|████▏ | 416/1000 [00:21<00:30, 19.17it/s, loss=14]
42%|████▏ | 416/1000 [00:21<00:30, 19.17it/s, loss=14]
42%|████▏ | 416/1000 [00:21<00:30, 19.17it/s, loss=13.9]
42%|████▏ | 418/1000 [00:21<00:30, 19.20it/s, loss=13.9]
42%|████▏ | 418/1000 [00:21<00:30, 19.20it/s, loss=13.9]
42%|████▏ | 418/1000 [00:21<00:30, 19.20it/s, loss=13.9]
42%|████▏ | 420/1000 [00:21<00:30, 19.22it/s, loss=13.9]
42%|████▏ | 420/1000 [00:21<00:30, 19.22it/s, loss=13.9]
42%|████▏ | 420/1000 [00:21<00:30, 19.22it/s, loss=13.8]
42%|████▏ | 422/1000 [00:21<00:30, 19.16it/s, loss=13.8]
42%|████▏ | 422/1000 [00:22<00:30, 19.16it/s, loss=13.8]
42%|████▏ | 422/1000 [00:22<00:30, 19.16it/s, loss=13.8]
42%|████▏ | 424/1000 [00:22<00:30, 19.17it/s, loss=13.8]
42%|████▏ | 424/1000 [00:22<00:30, 19.17it/s, loss=13.8]
42%|████▏ | 424/1000 [00:22<00:30, 19.17it/s, loss=13.7]
43%|████▎ | 426/1000 [00:22<00:29, 19.21it/s, loss=13.7]
43%|████▎ | 426/1000 [00:22<00:29, 19.21it/s, loss=13.7]
43%|████▎ | 426/1000 [00:22<00:29, 19.21it/s, loss=13.7]
43%|████▎ | 428/1000 [00:22<00:29, 19.17it/s, loss=13.7]
43%|████▎ | 428/1000 [00:22<00:29, 19.17it/s, loss=13.7]
43%|████▎ | 428/1000 [00:22<00:29, 19.17it/s, loss=13.6]
43%|████▎ | 430/1000 [00:22<00:29, 19.19it/s, loss=13.6]
43%|████▎ | 430/1000 [00:22<00:29, 19.19it/s, loss=13.6]
43%|████▎ | 430/1000 [00:22<00:29, 19.19it/s, loss=13.6]
43%|████▎ | 432/1000 [00:22<00:29, 19.23it/s, loss=13.6]
43%|████▎ | 432/1000 [00:22<00:29, 19.23it/s, loss=13.6]
43%|████▎ | 432/1000 [00:22<00:29, 19.23it/s, loss=13.5]
43%|████▎ | 434/1000 [00:22<00:29, 19.17it/s, loss=13.5]
43%|████▎ | 434/1000 [00:22<00:29, 19.17it/s, loss=13.5]
43%|████▎ | 434/1000 [00:22<00:29, 19.17it/s, loss=13.5]
44%|████▎ | 436/1000 [00:22<00:29, 19.16it/s, loss=13.5]
44%|████▎ | 436/1000 [00:22<00:29, 19.16it/s, loss=13.5]
44%|████▎ | 436/1000 [00:22<00:29, 19.16it/s, loss=13.4]
44%|████▍ | 438/1000 [00:22<00:29, 19.22it/s, loss=13.4]
44%|████▍ | 438/1000 [00:22<00:29, 19.22it/s, loss=13.4]
44%|████▍ | 438/1000 [00:22<00:29, 19.22it/s, loss=13.4]
44%|████▍ | 440/1000 [00:22<00:29, 19.15it/s, loss=13.4]
44%|████▍ | 440/1000 [00:22<00:29, 19.15it/s, loss=13.4]
44%|████▍ | 440/1000 [00:23<00:29, 19.15it/s, loss=13.3]
44%|████▍ | 442/1000 [00:23<00:29, 19.19it/s, loss=13.3]
44%|████▍ | 442/1000 [00:23<00:29, 19.19it/s, loss=13.3]
44%|████▍ | 442/1000 [00:23<00:29, 19.19it/s, loss=13.3]
44%|████▍ | 444/1000 [00:23<00:28, 19.25it/s, loss=13.3]
44%|████▍ | 444/1000 [00:23<00:28, 19.25it/s, loss=13.3]
44%|████▍ | 444/1000 [00:23<00:28, 19.25it/s, loss=13.3]
45%|████▍ | 446/1000 [00:23<00:28, 19.26it/s, loss=13.3]
45%|████▍ | 446/1000 [00:23<00:28, 19.26it/s, loss=13.2]
45%|████▍ | 446/1000 [00:23<00:28, 19.26it/s, loss=13.2]
45%|████▍ | 448/1000 [00:23<00:28, 19.25it/s, loss=13.2]
45%|████▍ | 448/1000 [00:23<00:28, 19.25it/s, loss=13.2]
45%|████▍ | 448/1000 [00:23<00:28, 19.25it/s, loss=13.2]
45%|████▌ | 450/1000 [00:23<00:28, 19.25it/s, loss=13.2]
45%|████▌ | 450/1000 [00:23<00:28, 19.25it/s, loss=13.1]
45%|████▌ | 450/1000 [00:23<00:28, 19.25it/s, loss=13.1]
45%|████▌ | 452/1000 [00:23<00:28, 19.26it/s, loss=13.1]
45%|████▌ | 452/1000 [00:23<00:28, 19.26it/s, loss=13.1]
45%|████▌ | 452/1000 [00:23<00:28, 19.26it/s, loss=13.1]
45%|████▌ | 454/1000 [00:23<00:28, 19.25it/s, loss=13.1]
45%|████▌ | 454/1000 [00:23<00:28, 19.25it/s, loss=13]
45%|████▌ | 454/1000 [00:23<00:28, 19.25it/s, loss=13]
46%|████▌ | 456/1000 [00:23<00:28, 19.25it/s, loss=13]
46%|████▌ | 456/1000 [00:23<00:28, 19.25it/s, loss=13]
46%|████▌ | 456/1000 [00:23<00:28, 19.25it/s, loss=13]
46%|████▌ | 458/1000 [00:23<00:28, 19.20it/s, loss=13]
46%|████▌ | 458/1000 [00:23<00:28, 19.20it/s, loss=13]
46%|████▌ | 458/1000 [00:23<00:28, 19.20it/s, loss=12.9]
46%|████▌ | 460/1000 [00:23<00:28, 19.20it/s, loss=12.9]
46%|████▌ | 460/1000 [00:24<00:28, 19.20it/s, loss=12.9]
46%|████▌ | 460/1000 [00:24<00:28, 19.20it/s, loss=12.9]
46%|████▌ | 462/1000 [00:24<00:27, 19.24it/s, loss=12.9]
46%|████▌ | 462/1000 [00:24<00:27, 19.24it/s, loss=12.9]
46%|████▌ | 462/1000 [00:24<00:27, 19.24it/s, loss=12.8]
46%|████▋ | 464/1000 [00:24<00:27, 19.24it/s, loss=12.8]
46%|████▋ | 464/1000 [00:24<00:27, 19.24it/s, loss=12.8]
46%|████▋ | 464/1000 [00:24<00:27, 19.24it/s, loss=12.8]
47%|████▋ | 466/1000 [00:24<00:27, 19.23it/s, loss=12.8]
47%|████▋ | 466/1000 [00:24<00:27, 19.23it/s, loss=12.8]
47%|████▋ | 466/1000 [00:24<00:27, 19.23it/s, loss=12.8]
47%|████▋ | 468/1000 [00:24<00:27, 19.25it/s, loss=12.8]
47%|████▋ | 468/1000 [00:24<00:27, 19.25it/s, loss=12.7]
47%|████▋ | 468/1000 [00:24<00:27, 19.25it/s, loss=12.7]
47%|████▋ | 470/1000 [00:24<00:27, 19.28it/s, loss=12.7]
47%|████▋ | 470/1000 [00:24<00:27, 19.28it/s, loss=12.7]
47%|████▋ | 470/1000 [00:24<00:27, 19.28it/s, loss=12.7]
47%|████▋ | 472/1000 [00:24<00:27, 19.34it/s, loss=12.7]
47%|████▋ | 472/1000 [00:24<00:27, 19.34it/s, loss=12.7]
47%|████▋ | 472/1000 [00:24<00:27, 19.34it/s, loss=12.6]
47%|████▋ | 474/1000 [00:24<00:27, 19.27it/s, loss=12.6]
47%|████▋ | 474/1000 [00:24<00:27, 19.27it/s, loss=12.6]
47%|████▋ | 474/1000 [00:24<00:27, 19.27it/s, loss=12.6]
48%|████▊ | 476/1000 [00:24<00:27, 19.25it/s, loss=12.6]
48%|████▊ | 476/1000 [00:24<00:27, 19.25it/s, loss=12.6]
48%|████▊ | 476/1000 [00:24<00:27, 19.25it/s, loss=12.5]
48%|████▊ | 478/1000 [00:24<00:27, 19.17it/s, loss=12.5]
48%|████▊ | 478/1000 [00:24<00:27, 19.17it/s, loss=12.5]
48%|████▊ | 478/1000 [00:25<00:27, 19.17it/s, loss=12.5]
48%|████▊ | 480/1000 [00:25<00:27, 19.11it/s, loss=12.5]
48%|████▊ | 480/1000 [00:25<00:27, 19.11it/s, loss=12.5]
48%|████▊ | 480/1000 [00:25<00:27, 19.11it/s, loss=12.5]
48%|████▊ | 482/1000 [00:25<00:27, 19.08it/s, loss=12.5]
48%|████▊ | 482/1000 [00:25<00:27, 19.08it/s, loss=12.4]
48%|████▊ | 482/1000 [00:25<00:27, 19.08it/s, loss=12.4]
48%|████▊ | 484/1000 [00:25<00:26, 19.17it/s, loss=12.4]
48%|████▊ | 484/1000 [00:25<00:26, 19.17it/s, loss=12.4]
48%|████▊ | 484/1000 [00:25<00:26, 19.17it/s, loss=12.4]
49%|████▊ | 486/1000 [00:25<00:26, 19.13it/s, loss=12.4]
49%|████▊ | 486/1000 [00:25<00:26, 19.13it/s, loss=12.4]
49%|████▊ | 486/1000 [00:25<00:26, 19.13it/s, loss=12.3]
49%|████▉ | 488/1000 [00:25<00:26, 19.13it/s, loss=12.3]
49%|████▉ | 488/1000 [00:25<00:26, 19.13it/s, loss=12.3]
49%|████▉ | 488/1000 [00:25<00:26, 19.13it/s, loss=12.3]
49%|████▉ | 490/1000 [00:25<00:26, 19.19it/s, loss=12.3]
49%|████▉ | 490/1000 [00:25<00:26, 19.19it/s, loss=12.3]
49%|████▉ | 490/1000 [00:25<00:26, 19.19it/s, loss=12.3]
49%|████▉ | 492/1000 [00:25<00:26, 19.18it/s, loss=12.3]
49%|████▉ | 492/1000 [00:25<00:26, 19.18it/s, loss=12.2]
49%|████▉ | 492/1000 [00:25<00:26, 19.18it/s, loss=12.2]
49%|████▉ | 494/1000 [00:25<00:26, 19.18it/s, loss=12.2]
49%|████▉ | 494/1000 [00:25<00:26, 19.18it/s, loss=12.2]
49%|████▉ | 494/1000 [00:25<00:26, 19.18it/s, loss=12.2]
50%|████▉ | 496/1000 [00:25<00:26, 19.19it/s, loss=12.2]
50%|████▉ | 496/1000 [00:25<00:26, 19.19it/s, loss=12.2]
50%|████▉ | 496/1000 [00:25<00:26, 19.19it/s, loss=12.1]
50%|████▉ | 498/1000 [00:25<00:26, 19.22it/s, loss=12.1]
50%|████▉ | 498/1000 [00:25<00:26, 19.22it/s, loss=12.1]
50%|████▉ | 498/1000 [00:26<00:26, 19.22it/s, loss=12.1]
50%|█████ | 500/1000 [00:26<00:25, 19.24it/s, loss=12.1]
50%|█████ | 500/1000 [00:26<00:25, 19.24it/s, loss=12.1]
50%|█████ | 500/1000 [00:26<00:25, 19.24it/s, loss=12.1]
50%|█████ | 502/1000 [00:26<00:25, 19.25it/s, loss=12.1]
50%|█████ | 502/1000 [00:26<00:25, 19.25it/s, loss=12]
50%|█████ | 502/1000 [00:26<00:25, 19.25it/s, loss=12]
50%|█████ | 504/1000 [00:26<00:25, 19.19it/s, loss=12]
50%|█████ | 504/1000 [00:26<00:25, 19.19it/s, loss=12]
50%|█████ | 504/1000 [00:26<00:25, 19.19it/s, loss=12]
51%|█████ | 506/1000 [00:26<00:25, 19.19it/s, loss=12]
51%|█████ | 506/1000 [00:26<00:25, 19.19it/s, loss=12]
51%|█████ | 506/1000 [00:26<00:25, 19.19it/s, loss=11.9]
51%|█████ | 508/1000 [00:26<00:25, 19.26it/s, loss=11.9]
51%|█████ | 508/1000 [00:26<00:25, 19.26it/s, loss=11.9]
51%|█████ | 508/1000 [00:26<00:25, 19.26it/s, loss=11.9]
51%|█████ | 510/1000 [00:26<00:25, 19.17it/s, loss=11.9]
51%|█████ | 510/1000 [00:26<00:25, 19.17it/s, loss=11.9]
51%|█████ | 510/1000 [00:26<00:25, 19.17it/s, loss=11.9]
51%|█████ | 512/1000 [00:26<00:25, 19.15it/s, loss=11.9]
51%|█████ | 512/1000 [00:26<00:25, 19.15it/s, loss=11.9]
51%|█████ | 512/1000 [00:26<00:25, 19.15it/s, loss=11.8]
51%|█████▏ | 514/1000 [00:26<00:25, 19.19it/s, loss=11.8]
51%|█████▏ | 514/1000 [00:26<00:25, 19.19it/s, loss=11.8]
51%|█████▏ | 514/1000 [00:26<00:25, 19.19it/s, loss=11.8]
52%|█████▏ | 516/1000 [00:26<00:25, 19.14it/s, loss=11.8]
52%|█████▏ | 516/1000 [00:26<00:25, 19.14it/s, loss=11.8]
52%|█████▏ | 516/1000 [00:26<00:25, 19.14it/s, loss=11.8]
52%|█████▏ | 518/1000 [00:26<00:25, 19.16it/s, loss=11.8]
52%|█████▏ | 518/1000 [00:27<00:25, 19.16it/s, loss=11.7]
52%|█████▏ | 518/1000 [00:27<00:25, 19.16it/s, loss=11.7]
52%|█████▏ | 520/1000 [00:27<00:25, 19.19it/s, loss=11.7]
52%|█████▏ | 520/1000 [00:27<00:25, 19.19it/s, loss=11.7]
52%|█████▏ | 520/1000 [00:27<00:25, 19.19it/s, loss=11.7]
52%|█████▏ | 522/1000 [00:27<00:24, 19.24it/s, loss=11.7]
52%|█████▏ | 522/1000 [00:27<00:24, 19.24it/s, loss=11.7]
52%|█████▏ | 522/1000 [00:27<00:24, 19.24it/s, loss=11.7]
52%|█████▏ | 524/1000 [00:27<00:24, 19.24it/s, loss=11.7]
52%|█████▏ | 524/1000 [00:27<00:24, 19.24it/s, loss=11.6]
52%|█████▏ | 524/1000 [00:27<00:24, 19.24it/s, loss=11.6]
53%|█████▎ | 526/1000 [00:27<00:24, 19.25it/s, loss=11.6]
53%|█████▎ | 526/1000 [00:27<00:24, 19.25it/s, loss=11.6]
53%|█████▎ | 526/1000 [00:27<00:24, 19.25it/s, loss=11.6]
53%|█████▎ | 528/1000 [00:27<00:24, 19.22it/s, loss=11.6]
53%|█████▎ | 528/1000 [00:27<00:24, 19.22it/s, loss=11.6]
53%|█████▎ | 528/1000 [00:27<00:24, 19.22it/s, loss=11.5]
53%|█████▎ | 530/1000 [00:27<00:24, 19.09it/s, loss=11.5]
53%|█████▎ | 530/1000 [00:27<00:24, 19.09it/s, loss=11.5]
53%|█████▎ | 530/1000 [00:27<00:24, 19.09it/s, loss=11.5]
53%|█████▎ | 532/1000 [00:27<00:24, 19.15it/s, loss=11.5]
53%|█████▎ | 532/1000 [00:27<00:24, 19.15it/s, loss=11.5]
53%|█████▎ | 532/1000 [00:27<00:24, 19.15it/s, loss=11.5]
53%|█████▎ | 534/1000 [00:27<00:24, 19.19it/s, loss=11.5]
53%|█████▎ | 534/1000 [00:27<00:24, 19.19it/s, loss=11.5]
53%|█████▎ | 534/1000 [00:27<00:24, 19.19it/s, loss=11.4]
54%|█████▎ | 536/1000 [00:27<00:24, 19.18it/s, loss=11.4]
54%|█████▎ | 536/1000 [00:27<00:24, 19.18it/s, loss=11.4]
54%|█████▎ | 536/1000 [00:28<00:24, 19.18it/s, loss=11.4]
54%|█████▍ | 538/1000 [00:28<00:24, 19.13it/s, loss=11.4]
54%|█████▍ | 538/1000 [00:28<00:24, 19.13it/s, loss=11.4]
54%|█████▍ | 538/1000 [00:28<00:24, 19.13it/s, loss=11.4]
54%|█████▍ | 540/1000 [00:28<00:24, 19.14it/s, loss=11.4]
54%|█████▍ | 540/1000 [00:28<00:24, 19.14it/s, loss=11.3]
54%|█████▍ | 540/1000 [00:28<00:24, 19.14it/s, loss=11.3]
54%|█████▍ | 542/1000 [00:28<00:23, 19.23it/s, loss=11.3]
54%|█████▍ | 542/1000 [00:28<00:23, 19.23it/s, loss=11.3]
54%|█████▍ | 542/1000 [00:28<00:23, 19.23it/s, loss=11.3]
54%|█████▍ | 544/1000 [00:28<00:23, 19.22it/s, loss=11.3]
54%|█████▍ | 544/1000 [00:28<00:23, 19.22it/s, loss=11.3]
54%|█████▍ | 544/1000 [00:28<00:23, 19.22it/s, loss=11.3]
55%|█████▍ | 546/1000 [00:28<00:23, 19.22it/s, loss=11.3]
55%|█████▍ | 546/1000 [00:28<00:23, 19.22it/s, loss=11.2]
55%|█████▍ | 546/1000 [00:28<00:23, 19.22it/s, loss=11.2]
55%|█████▍ | 548/1000 [00:28<00:23, 19.19it/s, loss=11.2]
55%|█████▍ | 548/1000 [00:28<00:23, 19.19it/s, loss=11.2]
55%|█████▍ | 548/1000 [00:28<00:23, 19.19it/s, loss=11.2]
55%|█████▌ | 550/1000 [00:28<00:23, 19.18it/s, loss=11.2]
55%|█████▌ | 550/1000 [00:28<00:23, 19.18it/s, loss=11.2]
55%|█████▌ | 550/1000 [00:28<00:23, 19.18it/s, loss=11.2]
55%|█████▌ | 552/1000 [00:28<00:23, 19.17it/s, loss=11.2]
55%|█████▌ | 552/1000 [00:28<00:23, 19.17it/s, loss=11.1]
55%|█████▌ | 552/1000 [00:28<00:23, 19.17it/s, loss=11.1]
55%|█████▌ | 554/1000 [00:28<00:23, 19.16it/s, loss=11.1]
55%|█████▌ | 554/1000 [00:28<00:23, 19.16it/s, loss=11.1]
55%|█████▌ | 554/1000 [00:28<00:23, 19.16it/s, loss=11.1]
56%|█████▌ | 556/1000 [00:28<00:23, 19.15it/s, loss=11.1]
56%|█████▌ | 556/1000 [00:29<00:23, 19.15it/s, loss=11.1]
56%|█████▌ | 556/1000 [00:29<00:23, 19.15it/s, loss=11.1]
56%|█████▌ | 558/1000 [00:29<00:23, 19.13it/s, loss=11.1]
56%|█████▌ | 558/1000 [00:29<00:23, 19.13it/s, loss=11]
56%|█████▌ | 558/1000 [00:29<00:23, 19.13it/s, loss=11]
56%|█████▌ | 560/1000 [00:29<00:22, 19.14it/s, loss=11]
56%|█████▌ | 560/1000 [00:29<00:22, 19.14it/s, loss=11]
56%|█████▌ | 560/1000 [00:29<00:22, 19.14it/s, loss=11]
56%|█████▌ | 562/1000 [00:29<00:22, 19.18it/s, loss=11]
56%|█████▌ | 562/1000 [00:29<00:22, 19.18it/s, loss=11]
56%|█████▌ | 562/1000 [00:29<00:22, 19.18it/s, loss=11]
56%|█████▋ | 564/1000 [00:29<00:22, 19.12it/s, loss=11]
56%|█████▋ | 564/1000 [00:29<00:22, 19.12it/s, loss=10.9]
56%|█████▋ | 564/1000 [00:29<00:22, 19.12it/s, loss=10.9]
57%|█████▋ | 566/1000 [00:29<00:22, 19.14it/s, loss=10.9]
57%|█████▋ | 566/1000 [00:29<00:22, 19.14it/s, loss=10.9]
57%|█████▋ | 566/1000 [00:29<00:22, 19.14it/s, loss=10.9]
57%|█████▋ | 568/1000 [00:29<00:22, 19.14it/s, loss=10.9]
57%|█████▋ | 568/1000 [00:29<00:22, 19.14it/s, loss=10.9]
57%|█████▋ | 568/1000 [00:29<00:22, 19.14it/s, loss=10.9]
57%|█████▋ | 570/1000 [00:29<00:22, 19.17it/s, loss=10.9]
57%|█████▋ | 570/1000 [00:29<00:22, 19.17it/s, loss=10.8]
57%|█████▋ | 570/1000 [00:29<00:22, 19.17it/s, loss=10.8]
57%|█████▋ | 572/1000 [00:29<00:22, 19.17it/s, loss=10.8]
57%|█████▋ | 572/1000 [00:29<00:22, 19.17it/s, loss=10.8]
57%|█████▋ | 572/1000 [00:29<00:22, 19.17it/s, loss=10.8]
57%|█████▋ | 574/1000 [00:29<00:22, 19.18it/s, loss=10.8]
57%|█████▋ | 574/1000 [00:29<00:22, 19.18it/s, loss=10.8]
57%|█████▋ | 574/1000 [00:30<00:22, 19.18it/s, loss=10.8]
58%|█████▊ | 576/1000 [00:30<00:22, 19.27it/s, loss=10.8]
58%|█████▊ | 576/1000 [00:30<00:22, 19.27it/s, loss=10.8]
58%|█████▊ | 576/1000 [00:30<00:22, 19.27it/s, loss=10.7]
58%|█████▊ | 578/1000 [00:30<00:21, 19.27it/s, loss=10.7]
58%|█████▊ | 578/1000 [00:30<00:21, 19.27it/s, loss=10.7]
58%|█████▊ | 578/1000 [00:30<00:21, 19.27it/s, loss=10.7]
58%|█████▊ | 580/1000 [00:30<00:21, 19.29it/s, loss=10.7]
58%|█████▊ | 580/1000 [00:30<00:21, 19.29it/s, loss=10.7]
58%|█████▊ | 580/1000 [00:30<00:21, 19.29it/s, loss=10.7]
58%|█████▊ | 582/1000 [00:30<00:21, 19.28it/s, loss=10.7]
58%|█████▊ | 582/1000 [00:30<00:21, 19.28it/s, loss=10.7]
58%|█████▊ | 582/1000 [00:30<00:21, 19.28it/s, loss=10.6]
58%|█████▊ | 584/1000 [00:30<00:21, 19.35it/s, loss=10.6]
58%|█████▊ | 584/1000 [00:30<00:21, 19.35it/s, loss=10.6]
58%|█████▊ | 584/1000 [00:30<00:21, 19.35it/s, loss=10.6]
59%|█████▊ | 586/1000 [00:30<00:21, 19.27it/s, loss=10.6]
59%|█████▊ | 586/1000 [00:30<00:21, 19.27it/s, loss=10.6]
59%|█████▊ | 586/1000 [00:30<00:21, 19.27it/s, loss=10.6]
59%|█████▉ | 588/1000 [00:30<00:21, 19.29it/s, loss=10.6]
59%|█████▉ | 588/1000 [00:30<00:21, 19.29it/s, loss=10.6]
59%|█████▉ | 588/1000 [00:30<00:21, 19.29it/s, loss=10.6]
59%|█████▉ | 590/1000 [00:30<00:21, 19.20it/s, loss=10.6]
59%|█████▉ | 590/1000 [00:30<00:21, 19.20it/s, loss=10.5]
59%|█████▉ | 590/1000 [00:30<00:21, 19.20it/s, loss=10.5]
59%|█████▉ | 592/1000 [00:30<00:21, 19.26it/s, loss=10.5]
59%|█████▉ | 592/1000 [00:30<00:21, 19.26it/s, loss=10.5]
59%|█████▉ | 592/1000 [00:30<00:21, 19.26it/s, loss=10.5]
59%|█████▉ | 594/1000 [00:30<00:21, 19.18it/s, loss=10.5]
59%|█████▉ | 594/1000 [00:30<00:21, 19.18it/s, loss=10.5]
59%|█████▉ | 594/1000 [00:31<00:21, 19.18it/s, loss=10.5]
60%|█████▉ | 596/1000 [00:31<00:21, 19.12it/s, loss=10.5]
60%|█████▉ | 596/1000 [00:31<00:21, 19.12it/s, loss=10.4]
60%|█████▉ | 596/1000 [00:31<00:21, 19.12it/s, loss=10.4]
60%|█████▉ | 598/1000 [00:31<00:21, 19.14it/s, loss=10.4]
60%|█████▉ | 598/1000 [00:31<00:21, 19.14it/s, loss=10.4]
60%|█████▉ | 598/1000 [00:31<00:21, 19.14it/s, loss=10.4]
60%|██████ | 600/1000 [00:31<00:20, 19.13it/s, loss=10.4]
60%|██████ | 600/1000 [00:31<00:20, 19.13it/s, loss=10.4]
60%|██████ | 600/1000 [00:31<00:20, 19.13it/s, loss=10.4]
60%|██████ | 602/1000 [00:31<00:20, 19.15it/s, loss=10.4]
60%|██████ | 602/1000 [00:31<00:20, 19.15it/s, loss=10.4]
60%|██████ | 602/1000 [00:31<00:20, 19.15it/s, loss=10.3]
60%|██████ | 604/1000 [00:31<00:20, 19.20it/s, loss=10.3]
60%|██████ | 604/1000 [00:31<00:20, 19.20it/s, loss=10.3]
60%|██████ | 604/1000 [00:31<00:20, 19.20it/s, loss=10.3]
61%|██████ | 606/1000 [00:31<00:20, 19.22it/s, loss=10.3]
61%|██████ | 606/1000 [00:31<00:20, 19.22it/s, loss=10.3]
61%|██████ | 606/1000 [00:31<00:20, 19.22it/s, loss=10.3]
61%|██████ | 608/1000 [00:31<00:20, 19.30it/s, loss=10.3]
61%|██████ | 608/1000 [00:31<00:20, 19.30it/s, loss=10.3]
61%|██████ | 608/1000 [00:31<00:20, 19.30it/s, loss=10.3]
61%|██████ | 610/1000 [00:31<00:20, 19.29it/s, loss=10.3]
61%|██████ | 610/1000 [00:31<00:20, 19.29it/s, loss=10.2]
61%|██████ | 610/1000 [00:31<00:20, 19.29it/s, loss=10.2]
61%|██████ | 612/1000 [00:31<00:20, 19.23it/s, loss=10.2]
61%|██████ | 612/1000 [00:31<00:20, 19.23it/s, loss=10.2]
61%|██████ | 612/1000 [00:31<00:20, 19.23it/s, loss=10.2]
61%|██████▏ | 614/1000 [00:31<00:20, 19.23it/s, loss=10.2]
61%|██████▏ | 614/1000 [00:32<00:20, 19.23it/s, loss=10.2]
61%|██████▏ | 614/1000 [00:32<00:20, 19.23it/s, loss=10.2]
62%|██████▏ | 616/1000 [00:32<00:19, 19.25it/s, loss=10.2]
62%|██████▏ | 616/1000 [00:32<00:19, 19.25it/s, loss=10.2]
62%|██████▏ | 616/1000 [00:32<00:19, 19.25it/s, loss=10.1]
62%|██████▏ | 618/1000 [00:32<00:19, 19.25it/s, loss=10.1]
62%|██████▏ | 618/1000 [00:32<00:19, 19.25it/s, loss=10.1]
62%|██████▏ | 618/1000 [00:32<00:19, 19.25it/s, loss=10.1]
62%|██████▏ | 620/1000 [00:32<00:19, 19.25it/s, loss=10.1]
62%|██████▏ | 620/1000 [00:32<00:19, 19.25it/s, loss=10.1]
62%|██████▏ | 620/1000 [00:32<00:19, 19.25it/s, loss=10.1]
62%|██████▏ | 622/1000 [00:32<00:19, 19.27it/s, loss=10.1]
62%|██████▏ | 622/1000 [00:32<00:19, 19.27it/s, loss=10.1]
62%|██████▏ | 622/1000 [00:32<00:19, 19.27it/s, loss=10.1]
62%|██████▏ | 624/1000 [00:32<00:19, 19.21it/s, loss=10.1]
62%|██████▏ | 624/1000 [00:32<00:19, 19.21it/s, loss=10]
62%|██████▏ | 624/1000 [00:32<00:19, 19.21it/s, loss=10]
63%|██████▎ | 626/1000 [00:32<00:19, 19.24it/s, loss=10]
63%|██████▎ | 626/1000 [00:32<00:19, 19.24it/s, loss=10]
63%|██████▎ | 626/1000 [00:32<00:19, 19.24it/s, loss=10]
63%|██████▎ | 628/1000 [00:32<00:19, 19.31it/s, loss=10]
63%|██████▎ | 628/1000 [00:32<00:19, 19.31it/s, loss=9.99]
63%|██████▎ | 628/1000 [00:32<00:19, 19.31it/s, loss=9.98]
63%|██████▎ | 630/1000 [00:32<00:19, 19.32it/s, loss=9.98]
63%|██████▎ | 630/1000 [00:32<00:19, 19.32it/s, loss=9.97]
63%|██████▎ | 630/1000 [00:32<00:19, 19.32it/s, loss=9.95]
63%|██████▎ | 632/1000 [00:32<00:19, 19.29it/s, loss=9.95]
63%|██████▎ | 632/1000 [00:32<00:19, 19.29it/s, loss=9.94]
63%|██████▎ | 632/1000 [00:33<00:19, 19.29it/s, loss=9.93]
63%|██████▎ | 634/1000 [00:33<00:19, 19.24it/s, loss=9.93]
63%|██████▎ | 634/1000 [00:33<00:19, 19.24it/s, loss=9.91]
63%|██████▎ | 634/1000 [00:33<00:19, 19.24it/s, loss=9.9]
64%|██████▎ | 636/1000 [00:33<00:18, 19.30it/s, loss=9.9]
64%|██████▎ | 636/1000 [00:33<00:18, 19.30it/s, loss=9.89]
64%|██████▎ | 636/1000 [00:33<00:18, 19.30it/s, loss=9.87]
64%|██████▍ | 638/1000 [00:33<00:18, 19.35it/s, loss=9.87]
64%|██████▍ | 638/1000 [00:33<00:18, 19.35it/s, loss=9.86]
64%|██████▍ | 638/1000 [00:33<00:18, 19.35it/s, loss=9.85]
64%|██████▍ | 640/1000 [00:33<00:18, 19.33it/s, loss=9.85]
64%|██████▍ | 640/1000 [00:33<00:18, 19.33it/s, loss=9.83]
64%|██████▍ | 640/1000 [00:33<00:18, 19.33it/s, loss=9.82]
64%|██████▍ | 642/1000 [00:33<00:18, 19.30it/s, loss=9.82]
64%|██████▍ | 642/1000 [00:33<00:18, 19.30it/s, loss=9.81]
64%|██████▍ | 642/1000 [00:33<00:18, 19.30it/s, loss=9.79]
64%|██████▍ | 644/1000 [00:33<00:18, 19.31it/s, loss=9.79]
64%|██████▍ | 644/1000 [00:33<00:18, 19.31it/s, loss=9.78]
64%|██████▍ | 644/1000 [00:33<00:18, 19.31it/s, loss=9.77]
65%|██████▍ | 646/1000 [00:33<00:18, 19.29it/s, loss=9.77]
65%|██████▍ | 646/1000 [00:33<00:18, 19.29it/s, loss=9.75]
65%|██████▍ | 646/1000 [00:33<00:18, 19.29it/s, loss=9.74]
65%|██████▍ | 648/1000 [00:33<00:18, 19.27it/s, loss=9.74]
65%|██████▍ | 648/1000 [00:33<00:18, 19.27it/s, loss=9.73]
65%|██████▍ | 648/1000 [00:33<00:18, 19.27it/s, loss=9.72]
65%|██████▌ | 650/1000 [00:33<00:18, 19.23it/s, loss=9.72]
65%|██████▌ | 650/1000 [00:33<00:18, 19.23it/s, loss=9.7]
65%|██████▌ | 650/1000 [00:33<00:18, 19.23it/s, loss=9.69]
65%|██████▌ | 652/1000 [00:33<00:18, 19.27it/s, loss=9.69]
65%|██████▌ | 652/1000 [00:34<00:18, 19.27it/s, loss=9.68]
65%|██████▌ | 652/1000 [00:34<00:18, 19.27it/s, loss=9.66]
65%|██████▌ | 654/1000 [00:34<00:17, 19.23it/s, loss=9.66]
65%|██████▌ | 654/1000 [00:34<00:17, 19.23it/s, loss=9.65]
65%|██████▌ | 654/1000 [00:34<00:17, 19.23it/s, loss=9.64]
66%|██████▌ | 656/1000 [00:34<00:17, 19.29it/s, loss=9.64]
66%|██████▌ | 656/1000 [00:34<00:17, 19.29it/s, loss=9.63]
66%|██████▌ | 656/1000 [00:34<00:17, 19.29it/s, loss=9.61]
66%|██████▌ | 658/1000 [00:34<00:17, 19.34it/s, loss=9.61]
66%|██████▌ | 658/1000 [00:34<00:17, 19.34it/s, loss=9.6]
66%|██████▌ | 658/1000 [00:34<00:17, 19.34it/s, loss=9.59]
66%|██████▌ | 660/1000 [00:34<00:17, 19.34it/s, loss=9.59]
66%|██████▌ | 660/1000 [00:34<00:17, 19.34it/s, loss=9.58]
66%|██████▌ | 660/1000 [00:34<00:17, 19.34it/s, loss=9.56]
66%|██████▌ | 662/1000 [00:34<00:17, 19.29it/s, loss=9.56]
66%|██████▌ | 662/1000 [00:34<00:17, 19.29it/s, loss=9.55]
66%|██████▌ | 662/1000 [00:34<00:17, 19.29it/s, loss=9.54]
66%|██████▋ | 664/1000 [00:34<00:17, 19.21it/s, loss=9.54]
66%|██████▋ | 664/1000 [00:34<00:17, 19.21it/s, loss=9.53]
66%|██████▋ | 664/1000 [00:34<00:17, 19.21it/s, loss=9.51]
67%|██████▋ | 666/1000 [00:34<00:17, 19.17it/s, loss=9.51]
67%|██████▋ | 666/1000 [00:34<00:17, 19.17it/s, loss=9.5]
67%|██████▋ | 666/1000 [00:34<00:17, 19.17it/s, loss=9.49]
67%|██████▋ | 668/1000 [00:34<00:17, 19.20it/s, loss=9.49]
67%|██████▋ | 668/1000 [00:34<00:17, 19.20it/s, loss=9.48]
67%|██████▋ | 668/1000 [00:34<00:17, 19.20it/s, loss=9.46]
67%|██████▋ | 670/1000 [00:34<00:17, 19.18it/s, loss=9.46]
67%|██████▋ | 670/1000 [00:34<00:17, 19.18it/s, loss=9.45]
67%|██████▋ | 670/1000 [00:34<00:17, 19.18it/s, loss=9.44]
67%|██████▋ | 672/1000 [00:34<00:17, 19.20it/s, loss=9.44]
67%|██████▋ | 672/1000 [00:35<00:17, 19.20it/s, loss=9.43]
67%|██████▋ | 672/1000 [00:35<00:17, 19.20it/s, loss=9.41]
67%|██████▋ | 674/1000 [00:35<00:17, 19.16it/s, loss=9.41]
67%|██████▋ | 674/1000 [00:35<00:17, 19.16it/s, loss=9.4]
67%|██████▋ | 674/1000 [00:35<00:17, 19.16it/s, loss=9.39]
68%|██████▊ | 676/1000 [00:35<00:16, 19.17it/s, loss=9.39]
68%|██████▊ | 676/1000 [00:35<00:16, 19.17it/s, loss=9.38]
68%|██████▊ | 676/1000 [00:35<00:16, 19.17it/s, loss=9.37]
68%|██████▊ | 678/1000 [00:35<00:16, 19.21it/s, loss=9.37]
68%|██████▊ | 678/1000 [00:35<00:16, 19.21it/s, loss=9.35]
68%|██████▊ | 678/1000 [00:35<00:16, 19.21it/s, loss=9.34]
68%|██████▊ | 680/1000 [00:35<00:16, 19.22it/s, loss=9.34]
68%|██████▊ | 680/1000 [00:35<00:16, 19.22it/s, loss=9.33]
68%|██████▊ | 680/1000 [00:35<00:16, 19.22it/s, loss=9.32]
68%|██████▊ | 682/1000 [00:35<00:16, 19.18it/s, loss=9.32]
68%|██████▊ | 682/1000 [00:35<00:16, 19.18it/s, loss=9.31]
68%|██████▊ | 682/1000 [00:35<00:16, 19.18it/s, loss=9.29]
68%|██████▊ | 684/1000 [00:35<00:16, 19.22it/s, loss=9.29]
68%|██████▊ | 684/1000 [00:35<00:16, 19.22it/s, loss=9.28]
68%|██████▊ | 684/1000 [00:35<00:16, 19.22it/s, loss=9.27]
69%|██████▊ | 686/1000 [00:35<00:16, 19.23it/s, loss=9.27]
69%|██████▊ | 686/1000 [00:35<00:16, 19.23it/s, loss=9.26]
69%|██████▊ | 686/1000 [00:35<00:16, 19.23it/s, loss=9.25]
69%|██████▉ | 688/1000 [00:35<00:16, 19.17it/s, loss=9.25]
69%|██████▉ | 688/1000 [00:35<00:16, 19.17it/s, loss=9.23]
69%|██████▉ | 688/1000 [00:35<00:16, 19.17it/s, loss=9.22]
69%|██████▉ | 690/1000 [00:35<00:16, 18.87it/s, loss=9.22]
69%|██████▉ | 690/1000 [00:35<00:16, 18.87it/s, loss=9.21]
69%|██████▉ | 690/1000 [00:36<00:16, 18.87it/s, loss=9.2]
69%|██████▉ | 692/1000 [00:36<00:16, 18.83it/s, loss=9.2]
69%|██████▉ | 692/1000 [00:36<00:16, 18.83it/s, loss=9.19]
69%|██████▉ | 692/1000 [00:36<00:16, 18.83it/s, loss=9.18]
69%|██████▉ | 694/1000 [00:36<00:16, 18.94it/s, loss=9.18]
69%|██████▉ | 694/1000 [00:36<00:16, 18.94it/s, loss=9.16]
69%|██████▉ | 694/1000 [00:36<00:16, 18.94it/s, loss=9.15]
70%|██████▉ | 696/1000 [00:36<00:15, 19.05it/s, loss=9.15]
70%|██████▉ | 696/1000 [00:36<00:15, 19.05it/s, loss=9.14]
70%|██████▉ | 696/1000 [00:36<00:15, 19.05it/s, loss=9.13]
70%|██████▉ | 698/1000 [00:36<00:15, 19.09it/s, loss=9.13]
70%|██████▉ | 698/1000 [00:36<00:15, 19.09it/s, loss=9.12]
70%|██████▉ | 698/1000 [00:36<00:15, 19.09it/s, loss=9.11]
70%|███████ | 700/1000 [00:36<00:15, 19.20it/s, loss=9.11]
70%|███████ | 700/1000 [00:36<00:15, 19.20it/s, loss=9.1]
70%|███████ | 700/1000 [00:36<00:15, 19.20it/s, loss=9.08]
70%|███████ | 702/1000 [00:36<00:15, 19.24it/s, loss=9.08]
70%|███████ | 702/1000 [00:36<00:15, 19.24it/s, loss=9.07]
70%|███████ | 702/1000 [00:36<00:15, 19.24it/s, loss=9.06]
70%|███████ | 704/1000 [00:36<00:15, 19.15it/s, loss=9.06]
70%|███████ | 704/1000 [00:36<00:15, 19.15it/s, loss=9.05]
70%|███████ | 704/1000 [00:36<00:15, 19.15it/s, loss=9.04]
71%|███████ | 706/1000 [00:36<00:15, 19.11it/s, loss=9.04]
71%|███████ | 706/1000 [00:36<00:15, 19.11it/s, loss=9.03]
71%|███████ | 706/1000 [00:36<00:15, 19.11it/s, loss=9.02]
71%|███████ | 708/1000 [00:36<00:15, 19.09it/s, loss=9.02]
71%|███████ | 708/1000 [00:36<00:15, 19.09it/s, loss=9.01]
71%|███████ | 708/1000 [00:36<00:15, 19.09it/s, loss=8.99]
71%|███████ | 710/1000 [00:36<00:15, 19.16it/s, loss=8.99]
71%|███████ | 710/1000 [00:37<00:15, 19.16it/s, loss=8.98]
71%|███████ | 710/1000 [00:37<00:15, 19.16it/s, loss=8.97]
71%|███████ | 712/1000 [00:37<00:15, 19.17it/s, loss=8.97]
71%|███████ | 712/1000 [00:37<00:15, 19.17it/s, loss=8.96]
71%|███████ | 712/1000 [00:37<00:15, 19.17it/s, loss=8.95]
71%|███████▏ | 714/1000 [00:37<00:14, 19.23it/s, loss=8.95]
71%|███████▏ | 714/1000 [00:37<00:14, 19.23it/s, loss=8.94]
71%|███████▏ | 714/1000 [00:37<00:14, 19.23it/s, loss=8.93]
72%|███████▏ | 716/1000 [00:37<00:14, 19.26it/s, loss=8.93]
72%|███████▏ | 716/1000 [00:37<00:14, 19.26it/s, loss=8.92]
72%|███████▏ | 716/1000 [00:37<00:14, 19.26it/s, loss=8.91]
72%|███████▏ | 718/1000 [00:37<00:14, 19.23it/s, loss=8.91]
72%|███████▏ | 718/1000 [00:37<00:14, 19.23it/s, loss=8.89]
72%|███████▏ | 718/1000 [00:37<00:14, 19.23it/s, loss=8.88]
72%|███████▏ | 720/1000 [00:37<00:14, 19.30it/s, loss=8.88]
72%|███████▏ | 720/1000 [00:37<00:14, 19.30it/s, loss=8.87]
72%|███████▏ | 720/1000 [00:37<00:14, 19.30it/s, loss=8.86]
72%|███████▏ | 722/1000 [00:37<00:14, 19.30it/s, loss=8.86]
72%|███████▏ | 722/1000 [00:37<00:14, 19.30it/s, loss=8.85]
72%|███████▏ | 722/1000 [00:37<00:14, 19.30it/s, loss=8.84]
72%|███████▏ | 724/1000 [00:37<00:14, 19.23it/s, loss=8.84]
72%|███████▏ | 724/1000 [00:37<00:14, 19.23it/s, loss=8.83]
72%|███████▏ | 724/1000 [00:37<00:14, 19.23it/s, loss=8.82]
73%|███████▎ | 726/1000 [00:37<00:14, 19.25it/s, loss=8.82]
73%|███████▎ | 726/1000 [00:37<00:14, 19.25it/s, loss=8.81]
73%|███████▎ | 726/1000 [00:37<00:14, 19.25it/s, loss=8.8]
73%|███████▎ | 728/1000 [00:37<00:14, 19.31it/s, loss=8.8]
73%|███████▎ | 728/1000 [00:37<00:14, 19.31it/s, loss=8.79]
73%|███████▎ | 728/1000 [00:38<00:14, 19.31it/s, loss=8.78]
73%|███████▎ | 730/1000 [00:38<00:14, 19.27it/s, loss=8.78]
73%|███████▎ | 730/1000 [00:38<00:14, 19.27it/s, loss=8.77]
73%|███████▎ | 730/1000 [00:38<00:14, 19.27it/s, loss=8.75]
73%|███████▎ | 732/1000 [00:38<00:13, 19.32it/s, loss=8.75]
73%|███████▎ | 732/1000 [00:38<00:13, 19.32it/s, loss=8.74]
73%|███████▎ | 732/1000 [00:38<00:13, 19.32it/s, loss=8.73]
73%|███████▎ | 734/1000 [00:38<00:13, 19.35it/s, loss=8.73]
73%|███████▎ | 734/1000 [00:38<00:13, 19.35it/s, loss=8.72]
73%|███████▎ | 734/1000 [00:38<00:13, 19.35it/s, loss=8.71]
74%|███████▎ | 736/1000 [00:38<00:13, 19.34it/s, loss=8.71]
74%|███████▎ | 736/1000 [00:38<00:13, 19.34it/s, loss=8.7]
74%|███████▎ | 736/1000 [00:38<00:13, 19.34it/s, loss=8.69]
74%|███████▍ | 738/1000 [00:38<00:13, 19.32it/s, loss=8.69]
74%|███████▍ | 738/1000 [00:38<00:13, 19.32it/s, loss=8.68]
74%|███████▍ | 738/1000 [00:38<00:13, 19.32it/s, loss=8.67]
74%|███████▍ | 740/1000 [00:38<00:13, 19.20it/s, loss=8.67]
74%|███████▍ | 740/1000 [00:38<00:13, 19.20it/s, loss=8.66]
74%|███████▍ | 740/1000 [00:38<00:13, 19.20it/s, loss=8.65]
74%|███████▍ | 742/1000 [00:38<00:13, 19.16it/s, loss=8.65]
74%|███████▍ | 742/1000 [00:38<00:13, 19.16it/s, loss=8.64]
74%|███████▍ | 742/1000 [00:38<00:13, 19.16it/s, loss=8.63]
74%|███████▍ | 744/1000 [00:38<00:13, 19.19it/s, loss=8.63]
74%|███████▍ | 744/1000 [00:38<00:13, 19.19it/s, loss=8.62]
74%|███████▍ | 744/1000 [00:38<00:13, 19.19it/s, loss=8.61]
75%|███████▍ | 746/1000 [00:38<00:13, 19.23it/s, loss=8.61]
75%|███████▍ | 746/1000 [00:38<00:13, 19.23it/s, loss=8.6]
75%|███████▍ | 746/1000 [00:38<00:13, 19.23it/s, loss=8.59]
75%|███████▍ | 748/1000 [00:38<00:13, 19.18it/s, loss=8.59]
75%|███████▍ | 748/1000 [00:39<00:13, 19.18it/s, loss=8.58]
75%|███████▍ | 748/1000 [00:39<00:13, 19.18it/s, loss=8.57]
75%|███████▌ | 750/1000 [00:39<00:13, 19.12it/s, loss=8.57]
75%|███████▌ | 750/1000 [00:39<00:13, 19.12it/s, loss=8.56]
75%|███████▌ | 750/1000 [00:39<00:13, 19.12it/s, loss=8.55]
75%|███████▌ | 752/1000 [00:39<00:12, 19.13it/s, loss=8.55]
75%|███████▌ | 752/1000 [00:39<00:12, 19.13it/s, loss=8.54]
75%|███████▌ | 752/1000 [00:39<00:12, 19.13it/s, loss=8.53]
75%|███████▌ | 754/1000 [00:39<00:12, 19.16it/s, loss=8.53]
75%|███████▌ | 754/1000 [00:39<00:12, 19.16it/s, loss=8.52]
75%|███████▌ | 754/1000 [00:39<00:12, 19.16it/s, loss=8.51]
76%|███████▌ | 756/1000 [00:39<00:12, 19.23it/s, loss=8.51]
76%|███████▌ | 756/1000 [00:39<00:12, 19.23it/s, loss=8.5]
76%|███████▌ | 756/1000 [00:39<00:12, 19.23it/s, loss=8.49]
76%|███████▌ | 758/1000 [00:39<00:12, 19.24it/s, loss=8.49]
76%|███████▌ | 758/1000 [00:39<00:12, 19.24it/s, loss=8.48]
76%|███████▌ | 758/1000 [00:39<00:12, 19.24it/s, loss=8.47]
76%|███████▌ | 760/1000 [00:39<00:12, 19.27it/s, loss=8.47]
76%|███████▌ | 760/1000 [00:39<00:12, 19.27it/s, loss=8.46]
76%|███████▌ | 760/1000 [00:39<00:12, 19.27it/s, loss=8.45]
76%|███████▌ | 762/1000 [00:39<00:12, 19.23it/s, loss=8.45]
76%|███████▌ | 762/1000 [00:39<00:12, 19.23it/s, loss=8.44]
76%|███████▌ | 762/1000 [00:39<00:12, 19.23it/s, loss=8.43]
76%|███████▋ | 764/1000 [00:39<00:12, 19.23it/s, loss=8.43]
76%|███████▋ | 764/1000 [00:39<00:12, 19.23it/s, loss=8.42]
76%|███████▋ | 764/1000 [00:39<00:12, 19.23it/s, loss=8.41]
77%|███████▋ | 766/1000 [00:39<00:12, 19.25it/s, loss=8.41]
77%|███████▋ | 766/1000 [00:39<00:12, 19.25it/s, loss=8.4]
77%|███████▋ | 766/1000 [00:40<00:12, 19.25it/s, loss=8.39]
77%|███████▋ | 768/1000 [00:40<00:12, 19.24it/s, loss=8.39]
77%|███████▋ | 768/1000 [00:40<00:12, 19.24it/s, loss=8.38]
77%|███████▋ | 768/1000 [00:40<00:12, 19.24it/s, loss=8.37]
77%|███████▋ | 770/1000 [00:40<00:11, 19.24it/s, loss=8.37]
77%|███████▋ | 770/1000 [00:40<00:11, 19.24it/s, loss=8.36]
77%|███████▋ | 770/1000 [00:40<00:11, 19.24it/s, loss=8.35]
77%|███████▋ | 772/1000 [00:40<00:11, 19.21it/s, loss=8.35]
77%|███████▋ | 772/1000 [00:40<00:11, 19.21it/s, loss=8.34]
77%|███████▋ | 772/1000 [00:40<00:11, 19.21it/s, loss=8.33]
77%|███████▋ | 774/1000 [00:40<00:11, 19.22it/s, loss=8.33]
77%|███████▋ | 774/1000 [00:40<00:11, 19.22it/s, loss=8.32]
77%|███████▋ | 774/1000 [00:40<00:11, 19.22it/s, loss=8.31]
78%|███████▊ | 776/1000 [00:40<00:11, 19.18it/s, loss=8.31]
78%|███████▊ | 776/1000 [00:40<00:11, 19.18it/s, loss=8.3]
78%|███████▊ | 776/1000 [00:40<00:11, 19.18it/s, loss=8.29]
78%|███████▊ | 778/1000 [00:40<00:11, 19.13it/s, loss=8.29]
78%|███████▊ | 778/1000 [00:40<00:11, 19.13it/s, loss=8.28]
78%|███████▊ | 778/1000 [00:40<00:11, 19.13it/s, loss=8.27]
78%|███████▊ | 780/1000 [00:40<00:11, 19.12it/s, loss=8.27]
78%|███████▊ | 780/1000 [00:40<00:11, 19.12it/s, loss=8.26]
78%|███████▊ | 780/1000 [00:40<00:11, 19.12it/s, loss=8.25]
78%|███████▊ | 782/1000 [00:40<00:11, 19.21it/s, loss=8.25]
78%|███████▊ | 782/1000 [00:40<00:11, 19.21it/s, loss=8.24]
78%|███████▊ | 782/1000 [00:40<00:11, 19.21it/s, loss=8.23]
78%|███████▊ | 784/1000 [00:40<00:11, 19.24it/s, loss=8.23]
78%|███████▊ | 784/1000 [00:40<00:11, 19.24it/s, loss=8.22]
78%|███████▊ | 784/1000 [00:40<00:11, 19.24it/s, loss=8.22]
79%|███████▊ | 786/1000 [00:40<00:11, 19.26it/s, loss=8.22]
79%|███████▊ | 786/1000 [00:40<00:11, 19.26it/s, loss=8.21]
79%|███████▊ | 786/1000 [00:41<00:11, 19.26it/s, loss=8.2]
79%|███████▉ | 788/1000 [00:41<00:11, 19.25it/s, loss=8.2]
79%|███████▉ | 788/1000 [00:41<00:11, 19.25it/s, loss=8.19]
79%|███████▉ | 788/1000 [00:41<00:11, 19.25it/s, loss=8.18]
79%|███████▉ | 790/1000 [00:41<00:10, 19.28it/s, loss=8.18]
79%|███████▉ | 790/1000 [00:41<00:10, 19.28it/s, loss=8.17]
79%|███████▉ | 790/1000 [00:41<00:10, 19.28it/s, loss=8.16]
79%|███████▉ | 792/1000 [00:41<00:10, 19.26it/s, loss=8.16]
79%|███████▉ | 792/1000 [00:41<00:10, 19.26it/s, loss=8.15]
79%|███████▉ | 792/1000 [00:41<00:10, 19.26it/s, loss=8.14]
79%|███████▉ | 794/1000 [00:41<00:10, 19.26it/s, loss=8.14]
79%|███████▉ | 794/1000 [00:41<00:10, 19.26it/s, loss=8.13]
79%|███████▉ | 794/1000 [00:41<00:10, 19.26it/s, loss=8.12]
80%|███████▉ | 796/1000 [00:41<00:10, 19.28it/s, loss=8.12]
80%|███████▉ | 796/1000 [00:41<00:10, 19.28it/s, loss=8.11]
80%|███████▉ | 796/1000 [00:41<00:10, 19.28it/s, loss=8.1]
80%|███████▉ | 798/1000 [00:41<00:10, 19.30it/s, loss=8.1]
80%|███████▉ | 798/1000 [00:41<00:10, 19.30it/s, loss=8.1]
80%|███████▉ | 798/1000 [00:41<00:10, 19.30it/s, loss=8.09]
80%|████████ | 800/1000 [00:41<00:10, 19.36it/s, loss=8.09]
80%|████████ | 800/1000 [00:41<00:10, 19.36it/s, loss=8.08]
80%|████████ | 800/1000 [00:41<00:10, 19.36it/s, loss=8.07]
80%|████████ | 802/1000 [00:41<00:10, 19.34it/s, loss=8.07]
80%|████████ | 802/1000 [00:41<00:10, 19.34it/s, loss=8.06]
80%|████████ | 802/1000 [00:41<00:10, 19.34it/s, loss=8.05]
80%|████████ | 804/1000 [00:41<00:10, 19.32it/s, loss=8.05]
80%|████████ | 804/1000 [00:41<00:10, 19.32it/s, loss=8.04]
80%|████████ | 804/1000 [00:41<00:10, 19.32it/s, loss=8.03]
81%|████████ | 806/1000 [00:41<00:10, 19.32it/s, loss=8.03]
81%|████████ | 806/1000 [00:42<00:10, 19.32it/s, loss=8.02]
81%|████████ | 806/1000 [00:42<00:10, 19.32it/s, loss=8.01]
81%|████████ | 808/1000 [00:42<00:10, 19.18it/s, loss=8.01]
81%|████████ | 808/1000 [00:42<00:10, 19.18it/s, loss=8]
81%|████████ | 808/1000 [00:42<00:10, 19.18it/s, loss=8]
81%|████████ | 810/1000 [00:42<00:09, 19.18it/s, loss=8]
81%|████████ | 810/1000 [00:42<00:09, 19.18it/s, loss=7.99]
81%|████████ | 810/1000 [00:42<00:09, 19.18it/s, loss=7.98]
81%|████████ | 812/1000 [00:42<00:09, 19.25it/s, loss=7.98]
81%|████████ | 812/1000 [00:42<00:09, 19.25it/s, loss=7.97]
81%|████████ | 812/1000 [00:42<00:09, 19.25it/s, loss=7.96]
81%|████████▏ | 814/1000 [00:42<00:09, 19.25it/s, loss=7.96]
81%|████████▏ | 814/1000 [00:42<00:09, 19.25it/s, loss=7.95]
81%|████████▏ | 814/1000 [00:42<00:09, 19.25it/s, loss=7.94]
82%|████████▏ | 816/1000 [00:42<00:09, 19.31it/s, loss=7.94]
82%|████████▏ | 816/1000 [00:42<00:09, 19.31it/s, loss=7.93]
82%|████████▏ | 816/1000 [00:42<00:09, 19.31it/s, loss=7.93]
82%|████████▏ | 818/1000 [00:42<00:09, 19.34it/s, loss=7.93]
82%|████████▏ | 818/1000 [00:42<00:09, 19.34it/s, loss=7.92]
82%|████████▏ | 818/1000 [00:42<00:09, 19.34it/s, loss=7.91]
82%|████████▏ | 820/1000 [00:42<00:09, 19.31it/s, loss=7.91]
82%|████████▏ | 820/1000 [00:42<00:09, 19.31it/s, loss=7.9]
82%|████████▏ | 820/1000 [00:42<00:09, 19.31it/s, loss=7.89]
82%|████████▏ | 822/1000 [00:42<00:09, 19.30it/s, loss=7.89]
82%|████████▏ | 822/1000 [00:42<00:09, 19.30it/s, loss=7.88]
82%|████████▏ | 822/1000 [00:42<00:09, 19.30it/s, loss=7.87]
82%|████████▏ | 824/1000 [00:42<00:09, 19.27it/s, loss=7.87]
82%|████████▏ | 824/1000 [00:42<00:09, 19.27it/s, loss=7.86]
82%|████████▏ | 824/1000 [00:43<00:09, 19.27it/s, loss=7.86]
83%|████████▎ | 826/1000 [00:43<00:09, 19.25it/s, loss=7.86]
83%|████████▎ | 826/1000 [00:43<00:09, 19.25it/s, loss=7.85]
83%|████████▎ | 826/1000 [00:43<00:09, 19.25it/s, loss=7.84]
83%|████████▎ | 828/1000 [00:43<00:08, 19.32it/s, loss=7.84]
83%|████████▎ | 828/1000 [00:43<00:08, 19.32it/s, loss=7.83]
83%|████████▎ | 828/1000 [00:43<00:08, 19.32it/s, loss=7.82]
83%|████████▎ | 830/1000 [00:43<00:08, 19.33it/s, loss=7.82]
83%|████████▎ | 830/1000 [00:43<00:08, 19.33it/s, loss=7.81]
83%|████████▎ | 830/1000 [00:43<00:08, 19.33it/s, loss=7.8]
83%|████████▎ | 832/1000 [00:43<00:08, 19.26it/s, loss=7.8]
83%|████████▎ | 832/1000 [00:43<00:08, 19.26it/s, loss=7.8]
83%|████████▎ | 832/1000 [00:43<00:08, 19.26it/s, loss=7.79]
83%|████████▎ | 834/1000 [00:43<00:08, 19.29it/s, loss=7.79]
83%|████████▎ | 834/1000 [00:43<00:08, 19.29it/s, loss=7.78]
83%|████████▎ | 834/1000 [00:43<00:08, 19.29it/s, loss=7.77]
84%|████████▎ | 836/1000 [00:43<00:08, 19.22it/s, loss=7.77]
84%|████████▎ | 836/1000 [00:43<00:08, 19.22it/s, loss=7.76]
84%|████████▎ | 836/1000 [00:43<00:08, 19.22it/s, loss=7.75]
84%|████████▍ | 838/1000 [00:43<00:08, 19.28it/s, loss=7.75]
84%|████████▍ | 838/1000 [00:43<00:08, 19.28it/s, loss=7.75]
84%|████████▍ | 838/1000 [00:43<00:08, 19.28it/s, loss=7.74]
84%|████████▍ | 840/1000 [00:43<00:08, 19.24it/s, loss=7.74]
84%|████████▍ | 840/1000 [00:43<00:08, 19.24it/s, loss=7.73]
84%|████████▍ | 840/1000 [00:43<00:08, 19.24it/s, loss=7.72]
84%|████████▍ | 842/1000 [00:43<00:08, 19.19it/s, loss=7.72]
84%|████████▍ | 842/1000 [00:43<00:08, 19.19it/s, loss=7.71]
84%|████████▍ | 842/1000 [00:43<00:08, 19.19it/s, loss=7.7]
84%|████████▍ | 844/1000 [00:43<00:08, 19.17it/s, loss=7.7]
84%|████████▍ | 844/1000 [00:44<00:08, 19.17it/s, loss=7.7]
84%|████████▍ | 844/1000 [00:44<00:08, 19.17it/s, loss=7.69]
85%|████████▍ | 846/1000 [00:44<00:08, 19.21it/s, loss=7.69]
85%|████████▍ | 846/1000 [00:44<00:08, 19.21it/s, loss=7.68]
85%|████████▍ | 846/1000 [00:44<00:08, 19.21it/s, loss=7.67]
85%|████████▍ | 848/1000 [00:44<00:07, 19.16it/s, loss=7.67]
85%|████████▍ | 848/1000 [00:44<00:07, 19.16it/s, loss=7.66]
85%|████████▍ | 848/1000 [00:44<00:07, 19.16it/s, loss=7.65]
85%|████████▌ | 850/1000 [00:44<00:07, 19.24it/s, loss=7.65]
85%|████████▌ | 850/1000 [00:44<00:07, 19.24it/s, loss=7.65]
85%|████████▌ | 850/1000 [00:44<00:07, 19.24it/s, loss=7.64]
85%|████████▌ | 852/1000 [00:44<00:07, 19.24it/s, loss=7.64]
85%|████████▌ | 852/1000 [00:44<00:07, 19.24it/s, loss=7.63]
85%|████████▌ | 852/1000 [00:44<00:07, 19.24it/s, loss=7.62]
85%|████████▌ | 854/1000 [00:44<00:07, 19.27it/s, loss=7.62]
85%|████████▌ | 854/1000 [00:44<00:07, 19.27it/s, loss=7.61]
85%|████████▌ | 854/1000 [00:44<00:07, 19.27it/s, loss=7.6]
86%|████████▌ | 856/1000 [00:44<00:07, 19.27it/s, loss=7.6]
86%|████████▌ | 856/1000 [00:44<00:07, 19.27it/s, loss=7.6]
86%|████████▌ | 856/1000 [00:44<00:07, 19.27it/s, loss=7.59]
86%|████████▌ | 858/1000 [00:44<00:07, 19.30it/s, loss=7.59]
86%|████████▌ | 858/1000 [00:44<00:07, 19.30it/s, loss=7.58]
86%|████████▌ | 858/1000 [00:44<00:07, 19.30it/s, loss=7.57]
86%|████████▌ | 860/1000 [00:44<00:07, 19.26it/s, loss=7.57]
86%|████████▌ | 860/1000 [00:44<00:07, 19.26it/s, loss=7.56]
86%|████████▌ | 860/1000 [00:44<00:07, 19.26it/s, loss=7.56]
86%|████████▌ | 862/1000 [00:44<00:07, 19.25it/s, loss=7.56]
86%|████████▌ | 862/1000 [00:44<00:07, 19.25it/s, loss=7.55]
86%|████████▌ | 862/1000 [00:44<00:07, 19.25it/s, loss=7.54]
86%|████████▋ | 864/1000 [00:44<00:07, 19.31it/s, loss=7.54]
86%|████████▋ | 864/1000 [00:45<00:07, 19.31it/s, loss=7.53]
86%|████████▋ | 864/1000 [00:45<00:07, 19.31it/s, loss=7.52]
87%|████████▋ | 866/1000 [00:45<00:06, 19.26it/s, loss=7.52]
87%|████████▋ | 866/1000 [00:45<00:06, 19.26it/s, loss=7.52]
87%|████████▋ | 866/1000 [00:45<00:06, 19.26it/s, loss=7.51]
87%|████████▋ | 868/1000 [00:45<00:06, 19.23it/s, loss=7.51]
87%|████████▋ | 868/1000 [00:45<00:06, 19.23it/s, loss=7.5]
87%|████████▋ | 868/1000 [00:45<00:06, 19.23it/s, loss=7.49]
87%|████████▋ | 870/1000 [00:45<00:06, 19.24it/s, loss=7.49]
87%|████████▋ | 870/1000 [00:45<00:06, 19.24it/s, loss=7.49]
87%|████████▋ | 870/1000 [00:45<00:06, 19.24it/s, loss=7.48]
87%|████████▋ | 872/1000 [00:45<00:06, 19.27it/s, loss=7.48]
87%|████████▋ | 872/1000 [00:45<00:06, 19.27it/s, loss=7.47]
87%|████████▋ | 872/1000 [00:45<00:06, 19.27it/s, loss=7.46]
87%|████████▋ | 874/1000 [00:45<00:06, 19.26it/s, loss=7.46]
87%|████████▋ | 874/1000 [00:45<00:06, 19.26it/s, loss=7.45]
87%|████████▋ | 874/1000 [00:45<00:06, 19.26it/s, loss=7.45]
88%|████████▊ | 876/1000 [00:45<00:06, 19.19it/s, loss=7.45]
88%|████████▊ | 876/1000 [00:45<00:06, 19.19it/s, loss=7.44]
88%|████████▊ | 876/1000 [00:45<00:06, 19.19it/s, loss=7.43]
88%|████████▊ | 878/1000 [00:45<00:06, 19.21it/s, loss=7.43]
88%|████████▊ | 878/1000 [00:45<00:06, 19.21it/s, loss=7.42]
88%|████████▊ | 878/1000 [00:45<00:06, 19.21it/s, loss=7.42]
88%|████████▊ | 880/1000 [00:45<00:06, 19.22it/s, loss=7.42]
88%|████████▊ | 880/1000 [00:45<00:06, 19.22it/s, loss=7.41]
88%|████████▊ | 880/1000 [00:45<00:06, 19.22it/s, loss=7.4]
88%|████████▊ | 882/1000 [00:45<00:06, 19.19it/s, loss=7.4]
88%|████████▊ | 882/1000 [00:45<00:06, 19.19it/s, loss=7.39]
88%|████████▊ | 882/1000 [00:46<00:06, 19.19it/s, loss=7.38]
88%|████████▊ | 884/1000 [00:46<00:06, 19.30it/s, loss=7.38]
88%|████████▊ | 884/1000 [00:46<00:06, 19.30it/s, loss=7.38]
88%|████████▊ | 884/1000 [00:46<00:06, 19.30it/s, loss=7.37]
89%|████████▊ | 886/1000 [00:46<00:05, 19.27it/s, loss=7.37]
89%|████████▊ | 886/1000 [00:46<00:05, 19.27it/s, loss=7.36]
89%|████████▊ | 886/1000 [00:46<00:05, 19.27it/s, loss=7.35]
89%|████████▉ | 888/1000 [00:46<00:05, 19.27it/s, loss=7.35]
89%|████████▉ | 888/1000 [00:46<00:05, 19.27it/s, loss=7.35]
89%|████████▉ | 888/1000 [00:46<00:05, 19.27it/s, loss=7.34]
89%|████████▉ | 890/1000 [00:46<00:05, 19.27it/s, loss=7.34]
89%|████████▉ | 890/1000 [00:46<00:05, 19.27it/s, loss=7.33]
89%|████████▉ | 890/1000 [00:46<00:05, 19.27it/s, loss=7.32]
89%|████████▉ | 892/1000 [00:46<00:05, 19.35it/s, loss=7.32]
89%|████████▉ | 892/1000 [00:46<00:05, 19.35it/s, loss=7.32]
89%|████████▉ | 892/1000 [00:46<00:05, 19.35it/s, loss=7.31]
89%|████████▉ | 894/1000 [00:46<00:05, 19.36it/s, loss=7.31]
89%|████████▉ | 894/1000 [00:46<00:05, 19.36it/s, loss=7.3]
89%|████████▉ | 894/1000 [00:46<00:05, 19.36it/s, loss=7.29]
90%|████████▉ | 896/1000 [00:46<00:05, 19.31it/s, loss=7.29]
90%|████████▉ | 896/1000 [00:46<00:05, 19.31it/s, loss=7.29]
90%|████████▉ | 896/1000 [00:46<00:05, 19.31it/s, loss=7.28]
90%|████████▉ | 898/1000 [00:46<00:05, 19.24it/s, loss=7.28]
90%|████████▉ | 898/1000 [00:46<00:05, 19.24it/s, loss=7.27]
90%|████████▉ | 898/1000 [00:46<00:05, 19.24it/s, loss=7.26]
90%|█████████ | 900/1000 [00:46<00:05, 19.27it/s, loss=7.26]
90%|█████████ | 900/1000 [00:46<00:05, 19.27it/s, loss=7.26]
90%|█████████ | 900/1000 [00:46<00:05, 19.27it/s, loss=7.25]
90%|█████████ | 902/1000 [00:46<00:05, 19.26it/s, loss=7.25]
90%|█████████ | 902/1000 [00:47<00:05, 19.26it/s, loss=7.24]
90%|█████████ | 902/1000 [00:47<00:05, 19.26it/s, loss=7.23]
90%|█████████ | 904/1000 [00:47<00:04, 19.29it/s, loss=7.23]
90%|█████████ | 904/1000 [00:47<00:04, 19.29it/s, loss=7.23]
90%|█████████ | 904/1000 [00:47<00:04, 19.29it/s, loss=7.22]
91%|█████████ | 906/1000 [00:47<00:04, 19.28it/s, loss=7.22]
91%|█████████ | 906/1000 [00:47<00:04, 19.28it/s, loss=7.21]
91%|█████████ | 906/1000 [00:47<00:04, 19.28it/s, loss=7.21]
91%|█████████ | 908/1000 [00:47<00:04, 19.28it/s, loss=7.21]
91%|█████████ | 908/1000 [00:47<00:04, 19.28it/s, loss=7.2]
91%|█████████ | 908/1000 [00:47<00:04, 19.28it/s, loss=7.19]
91%|█████████ | 910/1000 [00:47<00:04, 19.23it/s, loss=7.19]
91%|█████████ | 910/1000 [00:47<00:04, 19.23it/s, loss=7.18]
91%|█████████ | 910/1000 [00:47<00:04, 19.23it/s, loss=7.18]
91%|█████████ | 912/1000 [00:47<00:04, 19.11it/s, loss=7.18]
91%|█████████ | 912/1000 [00:47<00:04, 19.11it/s, loss=7.17]
91%|█████████ | 912/1000 [00:47<00:04, 19.11it/s, loss=7.16]
91%|█████████▏| 914/1000 [00:47<00:04, 19.07it/s, loss=7.16]
91%|█████████▏| 914/1000 [00:47<00:04, 19.07it/s, loss=7.15]
91%|█████████▏| 914/1000 [00:47<00:04, 19.07it/s, loss=7.15]
92%|█████████▏| 916/1000 [00:47<00:04, 19.10it/s, loss=7.15]
92%|█████████▏| 916/1000 [00:47<00:04, 19.10it/s, loss=7.14]
92%|█████████▏| 916/1000 [00:47<00:04, 19.10it/s, loss=7.13]
92%|█████████▏| 918/1000 [00:47<00:04, 19.09it/s, loss=7.13]
92%|█████████▏| 918/1000 [00:47<00:04, 19.09it/s, loss=7.13]
92%|█████████▏| 918/1000 [00:47<00:04, 19.09it/s, loss=7.12]
92%|█████████▏| 920/1000 [00:47<00:04, 19.13it/s, loss=7.12]
92%|█████████▏| 920/1000 [00:47<00:04, 19.13it/s, loss=7.11]
92%|█████████▏| 920/1000 [00:48<00:04, 19.13it/s, loss=7.1]
92%|█████████▏| 922/1000 [00:48<00:04, 19.14it/s, loss=7.1]
92%|█████████▏| 922/1000 [00:48<00:04, 19.14it/s, loss=7.1]
92%|█████████▏| 922/1000 [00:48<00:04, 19.14it/s, loss=7.09]
92%|█████████▏| 924/1000 [00:48<00:03, 19.16it/s, loss=7.09]
92%|█████████▏| 924/1000 [00:48<00:03, 19.16it/s, loss=7.08]
92%|█████████▏| 924/1000 [00:48<00:03, 19.16it/s, loss=7.08]
93%|█████████▎| 926/1000 [00:48<00:03, 19.28it/s, loss=7.08]
93%|█████████▎| 926/1000 [00:48<00:03, 19.28it/s, loss=7.07]
93%|█████████▎| 926/1000 [00:48<00:03, 19.28it/s, loss=7.06]
93%|█████████▎| 928/1000 [00:48<00:03, 19.29it/s, loss=7.06]
93%|█████████▎| 928/1000 [00:48<00:03, 19.29it/s, loss=7.05]
93%|█████████▎| 928/1000 [00:48<00:03, 19.29it/s, loss=7.05]
93%|█████████▎| 930/1000 [00:48<00:03, 19.30it/s, loss=7.05]
93%|█████████▎| 930/1000 [00:48<00:03, 19.30it/s, loss=7.04]
93%|█████████▎| 930/1000 [00:48<00:03, 19.30it/s, loss=7.03]
93%|█████████▎| 932/1000 [00:48<00:03, 19.30it/s, loss=7.03]
93%|█████████▎| 932/1000 [00:48<00:03, 19.30it/s, loss=7.03]
93%|█████████▎| 932/1000 [00:48<00:03, 19.30it/s, loss=7.02]
93%|█████████▎| 934/1000 [00:48<00:03, 19.34it/s, loss=7.02]
93%|█████████▎| 934/1000 [00:48<00:03, 19.34it/s, loss=7.01]
93%|█████████▎| 934/1000 [00:48<00:03, 19.34it/s, loss=7.01]
94%|█████████▎| 936/1000 [00:48<00:03, 19.28it/s, loss=7.01]
94%|█████████▎| 936/1000 [00:48<00:03, 19.28it/s, loss=7]
94%|█████████▎| 936/1000 [00:48<00:03, 19.28it/s, loss=6.99]
94%|█████████▍| 938/1000 [00:48<00:03, 19.31it/s, loss=6.99]
94%|█████████▍| 938/1000 [00:48<00:03, 19.31it/s, loss=6.99]
94%|█████████▍| 938/1000 [00:48<00:03, 19.31it/s, loss=6.98]
94%|█████████▍| 940/1000 [00:48<00:03, 19.28it/s, loss=6.98]
94%|█████████▍| 940/1000 [00:48<00:03, 19.28it/s, loss=6.97]
94%|█████████▍| 940/1000 [00:49<00:03, 19.28it/s, loss=6.96]
94%|█████████▍| 942/1000 [00:49<00:03, 19.29it/s, loss=6.96]
94%|█████████▍| 942/1000 [00:49<00:03, 19.29it/s, loss=6.96]
94%|█████████▍| 942/1000 [00:49<00:03, 19.29it/s, loss=6.95]
94%|█████████▍| 944/1000 [00:49<00:02, 19.31it/s, loss=6.95]
94%|█████████▍| 944/1000 [00:49<00:02, 19.31it/s, loss=6.94]
94%|█████████▍| 944/1000 [00:49<00:02, 19.31it/s, loss=6.94]
95%|█████████▍| 946/1000 [00:49<00:02, 19.32it/s, loss=6.94]
95%|█████████▍| 946/1000 [00:49<00:02, 19.32it/s, loss=6.93]
95%|█████████▍| 946/1000 [00:49<00:02, 19.32it/s, loss=6.92]
95%|█████████▍| 948/1000 [00:49<00:02, 19.28it/s, loss=6.92]
95%|█████████▍| 948/1000 [00:49<00:02, 19.28it/s, loss=6.92]
95%|█████████▍| 948/1000 [00:49<00:02, 19.28it/s, loss=6.91]
95%|█████████▌| 950/1000 [00:49<00:02, 19.28it/s, loss=6.91]
95%|█████████▌| 950/1000 [00:49<00:02, 19.28it/s, loss=6.9]
95%|█████████▌| 950/1000 [00:49<00:02, 19.28it/s, loss=6.9]
95%|█████████▌| 952/1000 [00:49<00:02, 19.32it/s, loss=6.9]
95%|█████████▌| 952/1000 [00:49<00:02, 19.32it/s, loss=6.89]
95%|█████████▌| 952/1000 [00:49<00:02, 19.32it/s, loss=6.88]
95%|█████████▌| 954/1000 [00:49<00:02, 19.34it/s, loss=6.88]
95%|█████████▌| 954/1000 [00:49<00:02, 19.34it/s, loss=6.88]
95%|█████████▌| 954/1000 [00:49<00:02, 19.34it/s, loss=6.87]
96%|█████████▌| 956/1000 [00:49<00:02, 19.29it/s, loss=6.87]
96%|█████████▌| 956/1000 [00:49<00:02, 19.29it/s, loss=6.86]
96%|█████████▌| 956/1000 [00:49<00:02, 19.29it/s, loss=6.86]
96%|█████████▌| 958/1000 [00:49<00:02, 19.28it/s, loss=6.86]
96%|█████████▌| 958/1000 [00:49<00:02, 19.28it/s, loss=6.85]
96%|█████████▌| 958/1000 [00:49<00:02, 19.28it/s, loss=6.84]
96%|█████████▌| 960/1000 [00:49<00:02, 19.31it/s, loss=6.84]
96%|█████████▌| 960/1000 [00:50<00:02, 19.31it/s, loss=6.84]
96%|█████████▌| 960/1000 [00:50<00:02, 19.31it/s, loss=6.83]
96%|█████████▌| 962/1000 [00:50<00:01, 19.32it/s, loss=6.83]
96%|█████████▌| 962/1000 [00:50<00:01, 19.32it/s, loss=6.82]
96%|█████████▌| 962/1000 [00:50<00:01, 19.32it/s, loss=6.82]
96%|█████████▋| 964/1000 [00:50<00:01, 19.31it/s, loss=6.82]
96%|█████████▋| 964/1000 [00:50<00:01, 19.31it/s, loss=6.81]
96%|█████████▋| 964/1000 [00:50<00:01, 19.31it/s, loss=6.8]
97%|█████████▋| 966/1000 [00:50<00:01, 19.32it/s, loss=6.8]
97%|█████████▋| 966/1000 [00:50<00:01, 19.32it/s, loss=6.8]
97%|█████████▋| 966/1000 [00:50<00:01, 19.32it/s, loss=6.79]
97%|█████████▋| 968/1000 [00:50<00:01, 19.28it/s, loss=6.79]
97%|█████████▋| 968/1000 [00:50<00:01, 19.28it/s, loss=6.78]
97%|█████████▋| 968/1000 [00:50<00:01, 19.28it/s, loss=6.78]
97%|█████████▋| 970/1000 [00:50<00:01, 19.18it/s, loss=6.78]
97%|█████████▋| 970/1000 [00:50<00:01, 19.18it/s, loss=6.77]
97%|█████████▋| 970/1000 [00:50<00:01, 19.18it/s, loss=6.77]
97%|█████████▋| 972/1000 [00:50<00:01, 19.24it/s, loss=6.77]
97%|█████████▋| 972/1000 [00:50<00:01, 19.24it/s, loss=6.76]
97%|█████████▋| 972/1000 [00:50<00:01, 19.24it/s, loss=6.75]
97%|█████████▋| 974/1000 [00:50<00:01, 19.21it/s, loss=6.75]
97%|█████████▋| 974/1000 [00:50<00:01, 19.21it/s, loss=6.75]
97%|█████████▋| 974/1000 [00:50<00:01, 19.21it/s, loss=6.74]
98%|█████████▊| 976/1000 [00:50<00:01, 19.29it/s, loss=6.74]
98%|█████████▊| 976/1000 [00:50<00:01, 19.29it/s, loss=6.73]
98%|█████████▊| 976/1000 [00:50<00:01, 19.29it/s, loss=6.73]
98%|█████████▊| 978/1000 [00:50<00:01, 19.34it/s, loss=6.73]
98%|█████████▊| 978/1000 [00:50<00:01, 19.34it/s, loss=6.72]
98%|█████████▊| 978/1000 [00:51<00:01, 19.34it/s, loss=6.71]
98%|█████████▊| 980/1000 [00:51<00:01, 19.37it/s, loss=6.71]
98%|█████████▊| 980/1000 [00:51<00:01, 19.37it/s, loss=6.71]
98%|█████████▊| 980/1000 [00:51<00:01, 19.37it/s, loss=6.7]
98%|█████████▊| 982/1000 [00:51<00:00, 19.34it/s, loss=6.7]
98%|█████████▊| 982/1000 [00:51<00:00, 19.34it/s, loss=6.69]
98%|█████████▊| 982/1000 [00:51<00:00, 19.34it/s, loss=6.69]
98%|█████████▊| 984/1000 [00:51<00:00, 19.37it/s, loss=6.69]
98%|█████████▊| 984/1000 [00:51<00:00, 19.37it/s, loss=6.68]
98%|█████████▊| 984/1000 [00:51<00:00, 19.37it/s, loss=6.68]
99%|█████████▊| 986/1000 [00:51<00:00, 19.38it/s, loss=6.68]
99%|█████████▊| 986/1000 [00:51<00:00, 19.38it/s, loss=6.67]
99%|█████████▊| 986/1000 [00:51<00:00, 19.38it/s, loss=6.66]
99%|█████████▉| 988/1000 [00:51<00:00, 19.32it/s, loss=6.66]
99%|█████████▉| 988/1000 [00:51<00:00, 19.32it/s, loss=6.66]
99%|█████████▉| 988/1000 [00:51<00:00, 19.32it/s, loss=6.65]
99%|█████████▉| 990/1000 [00:51<00:00, 19.29it/s, loss=6.65]
99%|█████████▉| 990/1000 [00:51<00:00, 19.29it/s, loss=6.64]
99%|█████████▉| 990/1000 [00:51<00:00, 19.29it/s, loss=6.64]
99%|█████████▉| 992/1000 [00:51<00:00, 19.26it/s, loss=6.64]
99%|█████████▉| 992/1000 [00:51<00:00, 19.26it/s, loss=6.63]
99%|█████████▉| 992/1000 [00:51<00:00, 19.26it/s, loss=6.63]
99%|█████████▉| 994/1000 [00:51<00:00, 19.22it/s, loss=6.63]
99%|█████████▉| 994/1000 [00:51<00:00, 19.22it/s, loss=6.62]
99%|█████████▉| 994/1000 [00:51<00:00, 19.22it/s, loss=6.61]
100%|█████████▉| 996/1000 [00:51<00:00, 19.29it/s, loss=6.61]
100%|█████████▉| 996/1000 [00:51<00:00, 19.29it/s, loss=6.61]
100%|█████████▉| 996/1000 [00:51<00:00, 19.29it/s, loss=6.6]
100%|█████████▉| 998/1000 [00:51<00:00, 19.31it/s, loss=6.6]
100%|█████████▉| 998/1000 [00:51<00:00, 19.31it/s, loss=6.59]
100%|█████████▉| 998/1000 [00:52<00:00, 19.31it/s, loss=6.59]
100%|██████████| 1000/1000 [00:52<00:00, 19.34it/s, loss=6.59]
100%|██████████| 1000/1000 [00:52<00:00, 19.21it/s, loss=6.59]
sgd_params
| mu_x | mu_y | sigma | shape_1 | rate_1 | shape_2 | rate_2 | weight | baseline | amplitude | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | -2.100923 | 1.451135 | 1.401596 | 6.0 | 0.9 | 12.0 | 0.9 | 0.35 | 9.262899 | 1.165602 |
We can plot the predicted model response and see that it matches the original simulated response almost perfectly.
sgd_pred_response = prf_model(stimulus, sgd_params)
fig, ax = plt.subplots()
ax.plot(simulated_response[0], label="True")
ax.plot(sgd_pred_response[0], "--", label="Predicted (SGD)")
fig.legend();
Conclusion¶
In this tutorial, we showed how to setup a standard Gaussian pRF model for a two-dimensional stimulus. We demonstrated how to fit the model to simulated data (without noise) using a multi-stage workflow: First, we used a grid search to find good starting values, then, we estimated baseline and amplitude using least squares, and finally we finetuned the model fit using stochastic gradient descent. At each stage, we compared the predicted model response against the original simulated response to check how well the model fit the data.
Stay Tuned¶
More tutorials on fitting models to empirical data and creating custom models are in the making.
For questions and issues, please make an issue on GitHub or contact Malte Lüken (m.luken@esciencecenter.nl).
References¶
Dumoulin, S. O., & Wandell, B. A. (2008). Population receptive field estimates in human visual cortex. NeuroImage, 39(2), 647–660. https://doi.org/10.1016/j.neuroimage.2007.09.034