Practical 1: The canonical Gaussian 2D population receptive field model#
This is the notebook for the first practical of the workshop. This practical covers the canonical Gaussian 2D population receptive field (pRF) model, its different computational parts, and how to fit it to empirical data from a visual functional magnetic resonance imaging (fMRI) experiment.
We recommend going through this notebook first to get familiar with the prfmodel package before starting with the notebooks of the other practicals.
The first part of this practical goes through the canonical Gaussian 2D pRF model step-by-step. The model input and output as well as every submodel is motivated and explained with interavtive visualizations. This part only looks at model prediction but not at fitting the model to data. This is what the second part is about: Fitting the model to an empirical dataset obtained with a 7T fMRI scanner. This part covers data inspection and visualization on an inflated surface of the brain. It goes through multiple consecutive fitting stages (grid search, least-squares, and SGD) and concludes with evaluation the model fit and interpreting its parameters.
Contents#
Installation and Setup#
First, we need to install prfmodel from PyPI. We install the package with the TensorFlow backend (for more information on the different backends, see the Backends section in the package documentation).
Please make sure to use a GPU instance when installing prfmodel and running this notebook in Colab.
!pip install "prfmodel[tensorflow]==0.1.0"
We set the backend to TensorFlow and a default for printing.
import os
import pandas as pd
# Set keras backend to 'tensorflow' (this is normally the default)
os.environ["KERAS_BACKEND"] = "tensorflow"
# Print parameter DataFrames with three decimals
pd.set_option("display.precision", 3)
Exercise 1
Import the prfmodel package and check whether the TensorFlow backend has been installed with GPU support (hint: use tensorflow.config.list_physical_devices('GPU') and tensorflow.test.is_built_with_cuda()).
# Your code goes here
The canonical Gaussian 2D pRF model#
A population receptive field (pRF) model maps the recorded activity of neuron populations to an experimental stimulus that is presented during the recording. In this example, we use the canonical Gaussian 2D pRF model adapted from Dumoulin and Wandell (2008) to map the recorded blood oxygenation level-dependent (BOLD) response from an fMRI experiment to a series of bar apertures that move through the visual field of the recorded subject (i.e., the bar moves on a computer screen in front of the subject).
Input: Stimulus design matrix and grid#
A pRF is defined in the feature space of the experimental stimulus. In this example, the stimulus has a two-dimensional visual feature space. How the position of the bar on the screen changes during the experiment is recorded as a stimulus design matrix. The design matrix indicates which cell in feature space is stimulated by the bar at which time frame (i.e., recording sample) of the experiment. The stimulus grid contains the feature space coordinates of every cell in the stimulus design matrix. In this example, the unit of the coordinates is degree of visual angle (but it could also be, e.g., screen pixels).
We can look at the experimental stimulus that is attached to the dataset that we will use in this example. The dataset is
called the Arrow-of-Time (AOT) dataset and the BOLD response was recorded with a 7 Tesla MRI scanner. Therefore, the
dataset has the tag 7t-aot-visual. We also specify that we want to load an inflated surface of the subject that we will
use later for visualization.
from prfmodel.examples import load_dataset
# Downloads on first use and caches in a user data directory (see prfmodel.examples.get_data_dir)
dataset = load_dataset("7t-aot-visual", surface_type="inflated")
The stimulus can be accessed through the dataset.stimulus attribute.
stimulus = dataset.stimulus
Exercise 2
Inspect the stimulus design matrix and grid. What do the dimensions and values refer to? Hint: You can
get an overview of the stimulus with print(stimulus).
# Your code goes here
To get an impression of the experimental stimulus, we can also visualize the stimulus design matrix projected into grid coordinates as a video.
from IPython.display import HTML
from prfmodel.plotting import animate_2d_prf_stimulus
ani = animate_2d_prf_stimulus(stimulus, interval=25) # Pause 25 ms between time frames
HTML(ani.to_html5_video())
Exercise 3
Going back to Exercise 2, does the video confirm your assumptions about what the stimulus design and grid dimensions refer to?
Exercise 4
What do you expect the activity of a neuron population that responds to this stimulus to look like?
The design matrix and grid are square and span about -4 to 4 degrees in both directions. However, the stimulus was shown on a rectangular screen that is wider than it is tall, so the bar never reaches the top and bottom of the design matrix. The cells outside the screen are never stimulated. Therefore, we crop the design matrix and grid to the cells that the bar covers in at least one time frame.
import numpy as np
from prfmodel.stimuli import PRFStimulus
# Cells that the bar covers in at least one time frame lie on the screen
is_on_screen = stimulus.design.max(axis=0) > 0.0
# The screen is a rectangle, so we keep the rows (y) and columns (x) of the design matrix between its edges
rows_on_screen = np.flatnonzero(is_on_screen.any(axis=1))
cols_on_screen = np.flatnonzero(is_on_screen.any(axis=0))
rows = slice(rows_on_screen[0], rows_on_screen[-1] + 1)
cols = slice(cols_on_screen[0], cols_on_screen[-1] + 1)
stimulus = PRFStimulus(
design=stimulus.design[:, rows, cols],
grid=stimulus.grid[rows, cols],
frame_duration=stimulus.frame_duration,
dimension_labels=stimulus.dimension_labels,
)
print(stimulus)
# The grid stores y in grid[..., 0] and x in grid[..., 1]
(y_min, x_min), (y_max, x_max) = stimulus.grid.min(axis=(0, 1)), stimulus.grid.max(axis=(0, 1))
print(f"Screen: x from {x_min:.2f} to {x_max:.2f} degrees, y from {y_min:.2f} to {y_max:.2f} degrees")
Building the model step by step#
The pRF is the part of the stimulus feature space that stimulates activity of the neuron population. The pRF model takes the stimulus design matrix and grid as input and predicts the neuron population activity given a set of parameters. By comparing the predicted against observed activity and optimizing the model parameters, the pRF model can estimate the pRF properties for the neuron population of the observed activity. Typically, pRFs are defined through a location (e.g., a center x- and y-coordinate in visual space) and a size or tuning width.
The canonical Gaussian 2D pRF model gets from input (i.e., stimulus design matrix and grid) to output (i.e., predicted BOLD response) in several steps. Each step is performed by a submodel that does a dedicated computation:
The tuning profile describes which part of the visual field stimulates the neuron population.
The stimulus encoding computes the overlap between the tuning profile and the stimulus at every time frame, which gives the predicted neuron population activity.
The impulse response turns the neuron population activity into a predicted BOLD response.
The scaling adjusts the baseline and amplitude of the predicted BOLD response.
We build up the model one submodel at a time and then combine all submodels into the full model.
To explore how the parameters of each submodel affect its predictions, we use interactive figures. The code that creates the figures is hidden: Run a hidden cell to show its figure and move the sliders to change the parameters. The figure updates when you release a slider. You can expand a hidden cell if you are curious about its code.
The first hidden cell defines helper functions for the interactive figures. Some of them are shared by all practicals and are imported from the module plotting_helpers.py, which lies next to this notebook.
Tuning profiles#
In prfmodel, the core of every pRF model is called tuning profile. In the canonical Gaussian 2D pRF model, the tuning profile is an isotropic multivariate Gaussian density. This means that the Gaussian has the same standard deviation in all dimensions and no correlation between them (i.e., its covariance matrix is the identity matrix multiplied by the variance). It describes the stimulation of the neuron population at every input from the visual field.
To illustrate this, we create an instance of a Gaussian tuning profile and evaluate it on the stimulus grid. The Gaussian 2D tuning profile only has three parameters, namely, the x- and y-coordinate of its center, and the size defined as the standard deviation. Parameters in prfmodel live in pandas.DataFrame objects where each row corresponds to a different unit (e.g., a vertex or voxel in fMRI) and each column to a different model parameter.
from prfmodel.models.prf import Gaussian2DPRFTuning
prf_tuning = Gaussian2DPRFTuning()
tuning_params = pd.DataFrame(
{
"mu_x": [0.0],
"mu_y": [0.0],
"sigma": [1.0],
},
)
# Predict the tuning profile
simulated_tuning_profile = prf_tuning(stimulus, tuning_params)
simulated_tuning_profile.shape # One value for every cell of the stimulus grid
The predicted tuning profile assigns a value to every cell of the stimulus grid. We can visualize it as a heatmap in visual field coordinates: the darkest cells lie at the pRF center (mu_x, mu_y), and the spread of the profile is determined by its size (sigma).
Exercise 5
Use the sliders to change the tuning parameters to the following combinations:
mu_x=2.1,mu_y=1.4,sigma=2.0mu_x=-2.1,mu_y=-1.4,sigma=0.7
How does the predicted tuning profile change and why?
Stimulus encoding#
In pRF models, the tuning profile is typically encoded with the stimulus design matrix through a dot-product operation. This operation computes the overlap between tuning profile and stimulus design matrix at every time frame and returns the predicted neuron population activity.
We illustrate this by creating an instance of the pRF stimulus encoder.
from prfmodel.models.prf import PRFStimulusEncoder
prf_encoder = PRFStimulusEncoder()
# We first predict the tuning profile
simulated_tuning_profile = prf_tuning(stimulus, tuning_params)
# Then we stimulus-encode it
simulated_encoded_profile = prf_encoder(stimulus, simulated_tuning_profile, tuning_params)
simulated_encoded_profile.shape # One value per time frame
The figure below shows the stimulus at a single time frame together with the contours of the tuning profile (left) and the predicted neuron population activity (right). The time_frame slider selects the time frame shown on the left, which is marked with a dot on the right.
Exercise 6
Why does the predicted neuron population activity have this particular shape? How does it relate to the experimental stimulus (compare it to the video and move the time_frame slider)?
Exercise 7
Change the tuning parameters to the same combinations as in Exercise 5:
mu_x=2.1,mu_y=1.4,sigma=2.0mu_x=-2.1,mu_y=-1.4,sigma=0.7
How does the predicted neuron population activity change and why?
Exercise 8
What happens to the predicted neuron population activity when you move the sliders to their extremes (e.g., a very small sigma or a pRF center at the edge of the stimulus)? Why?
Impulse response models for BOLD measurements#
The predicted neuron population activity does not yet account for the characteristic shape of the BOLD recording. By default, the canonical Gaussian 2D pRF model in prfmodel approximates the shape of the BOLD response with the weighted difference of two gamma densities. From this difference, we then also subtract its weighted derivative. While the parameters for the difference weight and the gamma densities are typically fixed at default values, we estimate the weight of the derivative. This is the weight_deriv parameter.
We create an instance of the default impulse response model, define a set of parameters, and predict the impulse response.
from prfmodel.impulse import DerivativeTwoGammaImpulse
impulse_model = DerivativeTwoGammaImpulse()
impulse_params = pd.DataFrame({"weight_deriv": [-0.5]})
simulated_impulse_response = impulse_model(impulse_params)
simulated_impulse_response.shape # One value per time frame of the impulse response
The figure below shows the impulse response for the weight_deriv that you set with the slider. The dashed gray line shows the impulse response without the derivative (i.e., weight_deriv = 0).
To predict the response that accounts for the shape of the BOLD measurement, the canonical Gaussian 2D pRF model convolves the predicted neuron population activity with the impulse response.
from prfmodel.impulse import convolve_prf_impulse_response
convolution_params = pd.DataFrame(
{
"mu_x": [0.0],
"mu_y": [0.0],
"sigma": [1.0],
"weight_deriv": [-0.5],
},
)
# We first predict the tuning profile
simulated_tuning_profile = prf_tuning(stimulus, convolution_params)
# We stimulus-encode the tuning profile
simulated_encoded_profile = prf_encoder(stimulus, simulated_tuning_profile, convolution_params)
# We predict the impulse response
simulated_impulse_response = impulse_model(convolution_params)
# We convolve the encoded tuning profile with the impulse response
convolved_response = convolve_prf_impulse_response(simulated_encoded_profile, simulated_impulse_response)
convolved_response.shape
Exercise 9
Move the weight_deriv slider to change the impulse model parameter. How does the convolved response change and why?
Exercise 10
Imagine a pRF model that is not applied to BOLD, but for example, electrocorticography (ECoG) measurements. Would the model still require a convolution of the predicted neuron population activity with an impulse response?
Scaling models#
The predicted convolved pRF model response has a certain amplitude (i.e., scale) and a baseline of zero (i.e., no activity when the screen is blank). However, observed BOLD responses of different neuron populations often differ in their amplitude and baseline from the predictions of the canonical Gaussian 2D pRF model (e.g., due to differences in the vasculature or in the distance to the receiver coil). To account for these differences, a scaling submodel can modify the amplitude and baseline of the predicted response.
We create an instance of a scaling model and modify the predicted convolved model response.
from prfmodel.scaling import BaselineAmplitude
scaling_model = BaselineAmplitude()
scaling_params = pd.DataFrame({"baseline": [0.0], "amplitude": [1.0]})
scaled_response = scaling_model(convolved_response, scaling_params)
scaled_response.shape
The figure below shows the convolved response before (blue) and after (orange) scaling. The dashed gray line shows the scaled response for the starting parameters.
Exercise 11
Move the baseline and amplitude sliders. How does the predicted response change and why?
Exercise 12
Change both the amplitude and sigma parameter at the same time. How does this affect the scale of the predicted response?
The full canonical model#
Now that we have seen every submodel, we can combine them into the full pRF model. In prfmodel, we call this a canonical model because it performs all necessary steps to get from input (i.e., stimulus design matrix and grid) to output (i.e., predicted BOLD response): tuning profile, stimulus encoding, convolution with the impulse response, and scaling. We create an instance of the canonical Gaussian 2D pRF model, which uses the submodels from above by default, and make a prediction with all parameters at once.
from prfmodel.models.prf import Gaussian2DPRFModel
prf_model = Gaussian2DPRFModel()
start_params = pd.DataFrame(
{
"mu_x": [0.0],
"mu_y": [0.0],
"sigma": [1.0],
"weight_deriv": [-0.5],
"baseline": [0.0],
"amplitude": [1.0],
},
)
# Make prediction with pRF model
simulated_response = prf_model(stimulus, start_params)
simulated_response.shape # One predicted timecourse with one value per time frame
The figure below shows the pRF in the visual field (left) and the predicted BOLD response of the full model (right). The dashed gray line shows the prediction for the starting parameters above. Use the sliders to see how all parameters together shape the predicted BOLD response.
Fitting the model to data#
Inspecting the data#
Before we fit the canonical Gaussian 2D pRF model to empirical data, we first load and inspect the recorded BOLD response timecourses. The BOLD response was recorded in a volume of voxels with a size of 2 mm. The dataset only contains the voxels that
fall inside a (dilated) gray matter mask. We can access the timecourses from the dataset.response attribute. Note that the raw response includes a non-zero baseline to accommodate pRF models that cannot predict a zero baseline (e.g., the divisive normalization pRF model; Aqil et al., 2020).
response_raw = dataset.response
response_raw.shape
Exercise 13
What do the dimensions of response_raw refer to? Visualize one or multiple timecourses with matplotlib or plotly. What is the unit of the BOLD response? Do you see any patterns in the timecourses and if so, how do they relate to the stimulus?
# Your code goes here
We transform the raw BOLD response timecourses by subtracting their baseline. With a baseline parameter of zero, our pRF model predicts a response of zero at blank stimulus frames, so setting the baseline to zero matches this default and makes the numbers a bit easier to work with.
response_no_baseline = response_raw - 100.0
To get a more complete overview of the BOLD timecourses, we can plot them for all voxels at once in a heatmap with prfmodel.plotting.plot_response_heatmap.
from prfmodel.plotting import plot_response_heatmap
# Uses matplotlib because plotly cannot handle this many voxels
plot_response_heatmap(
response_no_baseline,
vmin=-2,
vmax=5,
xlabel="Time frame (in TR)",
ylabel="Voxel index",
colorbar_label="BOLD response",
figsize=(6, 6),
);
Exercise 14
Which patterns do you see in the heatmap and how do they relate to the stimulus?
Exercise 15
Why do BOLD response patterns differ between voxels?
To visualize the response on the cortical surface, we need to project it from the volume onto the surface. The volume and the surfaces of the subject share the same coordinate system, so we can use nilearn.surface.vol_to_surf for the projection. The function samples the volume at several depths between the pial surface and the white matter surface, so we load these surfaces as well. To map a value per voxel back into the volume, we fill the values into the voxels inside the gray matter mask from the dataset.
The projection linearly interpolates between neighboring voxels. To prevent voxels outside the mask or voxels with NaN values from affecting the result, we set them to zero and additionally project a volume that is one for all other (valid) voxels and zero otherwise. Dividing the first projection by the second gives a weighted average of only the valid voxels at each vertex. Vertices without any valid voxels are NaN. The function make_surface_projection from plotting_helpers.py implements these steps and creates the function project_to_surface for our dataset.
from plotting_helpers import make_surface_projection
project_to_surface = make_surface_projection(dataset)
We also create a helper function with make_surface_plot from plotting_helpers.py to plot a per-voxel statistic on the inflated surface. The inflated surfaces of both hemispheres overlap in space, so we first move them apart. The helper shows three views that focus on the visual cortex in the occipital lobe: the inner (medial) side of each hemisphere, where the calcarine sulcus lies, and both hemispheres from behind (posterior).
Each hemisphere of the surface has about 150,000 vertices, which makes plotting slow (especially on Colab). Because the figures are too small to show that much detail, the helper plots a downsampled surface: All vertices that fall into the same cube of a 2 mm grid are merged into a single vertex that gets the average value of the merged vertices. This reduces the surface to about 27,000 vertices per hemisphere.
from plotting_helpers import make_surface_plot
plot_surf_stat_map_helper = make_surface_plot(dataset.mesh)
We can use the helper to plot, for example, the standard deviation of each timecourse on the surface to get an overview of the variability in the BOLD timecourses across the surface.
# Calculate standard deviation of each timecourse
response_sd = response_no_baseline.std(axis=1)
plot_surf_stat_map_helper(project_to_surface(response_sd), vmax=5.0, title="BOLD response standard deviation");
Exercise 16
What can the response standard deviation tell us about neuron activity in different brain regions? Can you find the primary visual cortex (i.e., V1)?
Exercise 17
Plot the average response of each timecourse on the surface. Do you see any interesting patterns?
# Your code goes here
Selecting valid voxels#
Some voxels in the mask do not have valid timecourses, that is, their timecourses are constant and do not contain any signal. We filter out all voxels that have a constant timecourse.
response_is_valid = response_no_baseline.std(axis=1) > 0.0
response_valid = response_no_baseline[response_is_valid]
response_is_valid.mean() # Fraction of valid voxels
Defining the pRF model#
Now that the stimulus and the BOLD response are in place, we can fit the canonical Gaussian 2D pRF model. However, we need to slightly customize the model: The stimulus design matrix and the BOLD response were recorded with a repetition time (TR) of 0.9 seconds. By default, pRF models in prfmodel assume a TR of 1 second. The TR is set through the resolution of the impulse response model, so we replace the default with a custom impulse response model that has a resolution of 0.9 seconds. Matching the resolution of the impulse response model with the TR ensures that the predicted pRF model response has the same sampling rate as the stimulus and the observed BOLD timecourses. For details, see the Important Details section in the online documentation of prfmodel.
# Define repetition time (TR)
tr = 0.9
# Create custom impulse model
impulse_model = DerivativeTwoGammaImpulse(resolution=tr)
# Define pRF model with custom impulse response submodel
prf_model = Gaussian2DPRFModel(
impulse_model=impulse_model,
)
Before we let prfmodel fit the model, you can try to fit it yourself. The figure below shows the observed BOLD response of a voxel with a clear signal (gray) together with the prediction of the pRF model (blue). The R-squared in the title shows how much of the variance in the observed response the prediction explains (1 is a perfect fit). Note that the amplitude of the observed response is much smaller than in the simulations above, so the amplitude slider covers a smaller range.
Exercise 18
Move the sliders until the prediction matches the observed response as well as you can. Which R-squared can you reach? Which parameters are easy to find and which are hard? Why?
Grid search#
We fit the canonical Gaussian 2D pRF model in two stages. In the first stage, we perform a “brute-force” grid search to find good estimates for the parameters of the pRF tuning profile (i.e., mu_x, mu_y, and sigma).
For the grid search, we define ranges of mu_x, mu_y, and sigma that we want to construct a grid of parameter values from. We create them from the stimulus with prfmodel.fitters.grid_values_2d_prf. Here, the values for mu_x and mu_y span twice the visual field covered by the stimulus (mu_extent=2.0). Later, we will filter out voxels for which the pRF center lies outside the stimulus grid because their responses cannot be reliably measured by the experiment. The values for sigma are log-spaced between 0.05 and 4 degrees. We use 21 values for mu_x and mu_y so that the center (0 degrees) and edges of the stimulus (about \(\pm 3.9\) degrees for mu_x and \(\pm 2.8\) degrees for mu_y) land on grid values. If they did not, grid_values_2d_prf would warn us and suggest a number of values that aligns them.
For baseline, amplitude, and weight_deriv, we only provide a single value so that they will stay constant across the entire grid.
from prfmodel.fitters import grid_values_2d_prf
grid_param_ranges = grid_values_2d_prf(
stimulus,
num_mu=21,
mu_extent=2.0, # Span twice the visual field covered by the stimulus
num_sigma=10,
sigma_range=(0.05, 4.0), # Log-spaced by default
) | { # Add constant parameters
"weight_deriv": [-0.5],
"baseline": [0.0],
"amplitude": [1.0],
}
grid_param_ranges
To run the grid search, we construct the prfmodel.fitters.GridFitter. Note that we set batch_size=20 to let the fitter evaluate 20 parameter combinations at the same time which saves us a lot of memory. By default, the loss (i.e., the metric the fitter minimizes between model predictions and data) is the negative correlation coefficient, which ignores differences in baseline and amplitude between model predictions and observed data.
from prfmodel.fitters import GridFitter
# Create grid fitter object
grid_fitter = GridFitter(
model=prf_model,
stimulus=stimulus,
compile_step=True, # Setting 'compile_step=True' speeds up the fitting
)
# Run grid search
grid_history, grid_params = grid_fitter.fit(
data=response_valid,
parameter_values=grid_param_ranges,
batch_size=20,
)
We can print the parameters estimated by the grid search.
grid_params
Because the grid search can only return values that are in the grid, it is worth checking how the estimates are distributed across the grid. We plot the distribution of the pRF centers with prfmodel.plotting.plot_2d_prf_centers and the distribution of the pRF sizes with prfmodel.plotting.plot_grid_parameter_distribution. The rectangle marks the part of the visual field covered by the stimulus, and pRF sizes at the smallest or largest value of the grid are highlighted in red.
from prfmodel.plotting import plot_2d_prf_centers, plot_grid_parameter_distribution
fig, axes = plt.subplots(1, 2, figsize=(10, 4.5), layout="constrained")
# Count estimates for each combination of pRF center coordinates in the grid
plot_2d_prf_centers(grid_params, grid_param_ranges, stimulus=stimulus, ax=axes[0])
axes[0].set(xlabel="mu_x (in degrees)", ylabel="mu_y (in degrees)", title="pRF centers")
# Count estimates for each pRF size in the grid
plot_grid_parameter_distribution(grid_params, grid_param_ranges, "sigma", ax=axes[1]) # Detects log spacing
axes[1].set(xlabel="sigma (in degrees)", title="pRF sizes");
Exercise 19
Compare the parameter estimates to the ranges of the grid search. What do you observe in the distributions of pRF centers and sizes? Change the grid ranges with the arguments of grid_values_2d_prf (e.g., mu_extent, num_mu, sigma_range; you can also create a range for weight_deriv). Can you get to a lower (i.e., better) mean loss across voxels (shown in the progress bar under loss)?
# Your code goes here
Least-squares#
In the grid search, we ignored baseline and amplitude parameters. In prfmodel, we estimate them with least-squares by regressing the observed responses on the predicted pRF model responses for each voxel. The intercept and slope of the least-squares regression become the baseline and amplitude parameter, respectively.
from prfmodel.fitters import LeastSquaresFitter
# Create least-squares fitter
ls_fitter = LeastSquaresFitter(
model=prf_model,
stimulus=stimulus,
)
# Run least squares fit
ls_history, ls_params = ls_fitter.fit(
data=response_valid,
parameters=grid_params,
slope_name="amplitude", # Names of parameters to be optimized with least squares
intercept_name="baseline",
batch_size=200,
)
We can print the parameter estimates from the grid search together with the least-squares parameter estimates.
ls_params
Stochastic gradient descent#
After least-squares estimation, the estimates for the tuning profile parameters are still lying on one of our grid points. Moreover, least-squares estimates baseline and amplitude given the other parameters, but it cannot adjust these other parameters to achieve a better fit. To finetune all parameters (or a subset) at once, we use stochastic gradient descent (SGD). It calculates the gradient of each parameter with respect to a loss function between model predictions and observed data and adjusts the parameters accordingly in small steps to minimize the loss. Importantly, the default mean squared error loss function in SGD is scale-sensitive, meaning that large differences in scale between model predictions and observed data can impair SGD estimation or make it inefficient. Thus, SGD requires good starting values for the parameters, so that it quickly converges to a good minimum of the loss function instead of getting stuck in a poor local minimum. Therefore, we first perform the grid search and least-squares optimization and use their parameter estimates as starting values for SGD.
Without any adjustments, SGD moves parameter estimates in an unconstrained space, which can potentially lead to implausible values for parameters that are strictly positive, such as sigma. Therefore, and to make the estimation more efficient, we create an adapter with a parameter transformation that allows SGD to optimize sigma in an unconstrained log space while converting it to a strictly positive space when making model predictions. This way, sigma always has plausible values when making predictions.
We limit the flexibility of the estimation by fixing weight_deriv to its starting value. This forces SGD to focus on the other parameters.
Note: SGD is computationally very heavy for BOLD fMRI timecourses from an entire brain. Therefore, we only use a small number of steps (num_steps=100) to illustrate the procedure. In practice, more steps should be used (the default is num_steps=1000).
from keras import ops
from prfmodel.fitters import SGDFitter
from prfmodel.fitters.adapter import Adapter, ParameterTransform
adapter = Adapter([ParameterTransform(["sigma"], transform_fun=ops.log, inverse_fun=ops.exp)])
sgd_fitter = SGDFitter(
model=prf_model,
stimulus=stimulus,
adapter=adapter,
compile_step=True, # Setting 'compile_step=True' speeds up the fitting
)
sgd_history, sgd_params = sgd_fitter.fit(
data=response_valid,
init_parameters=ls_params,
fixed_parameters=["weight_deriv"], # Fix weight_deriv to starting value
batch_size=20_000,
num_steps=100,
)
We can print the SGD parameter estimates.
sgd_params
Exercise 20
Compare the SGD parameter estimates against the grid search and least-squares estimates. How do they differ? (hint: you can use plot_2d_prf_centers and plot_grid_parameter_distribution)
# Your code goes here
Evaluating the model fit#
Now that the pRF model parameters are estimated, we can compare the model predictions against the observed responses. Because we want to make predictions for all voxels at once, we wrap our prf_model in the prfmodel.utils.batched function. The modifier changes the behavior of the model to make predictions for batches of voxels sequentially. This saves us a lot of memory at the expense of minimal runtime overhead.
We first evaluate the fit of the pRF model using the grid search and least-squares parameter estimates.
from prfmodel.utils import batched
predict_batched = batched(prf_model)
# Make predictions with the estimated parameters
pred_response = np.asarray(predict_batched(stimulus, ls_params, batch_size=200))
We can quantify how well the predictions align with the observed timecourses using the R-squared metric, which we compute with prfmodel.utils.calculate_r_squared. This metric indicates the proportion of variance in the observed data explained by our model predictions.
from prfmodel.utils import calculate_r_squared
r_squared = calculate_r_squared(response_valid, pred_response) # One score per voxel
r_squared.shape
We can look at the distribution of R-squared scores across voxels with prfmodel.plotting.plot_r_squared_hist.
from prfmodel.plotting import plot_r_squared_hist
# Many voxels have scores close to zero; a log scale also shows the voxels with higher scores
plot_r_squared_hist(r_squared, log=True);
Exercise 21
Plot the R-squared score on the inflated surface. You can use the helper function below. For which brain regions does the model fit the data well?
def fill_valid_voxels(values: np.ndarray) -> np.ndarray:
"""Fill values for the valid voxels into an array with one value per voxel in the mask."""
values_full = np.full((response_no_baseline.shape[0],), fill_value=np.nan)
values_full[response_is_valid] = values
return values_full
# Your code goes here
Exercise 22
In addition to computing model fit with R-squared, we also recommend comparing model predictions against observed timecourses for individual voxels. Plot predictions against observed responses for voxels with different fit quality, for example, by sorting them according to R-squared (e.g., with np.argsort(r_squared); hint: use prfmodel.plotting.plot_observed_predicted). Does the plot confirm the impression you get from the R-squared scores? Why does the model not fit well for some voxels?
# Your code goes here
Exercise 23
Evaluate the model fit based on the SGD parameter estimates and compare it against the grid search and least-squares fit. Does SGD improve the fit and if not, why? (hint: use prfmodel.plotting.plot_r_squared_comparison to compare R-squared scores).
# Your code goes here
Interpreting the model parameters#
To analyze and interpret the pRF parameters, we zoom in on voxels for which we can trust the pRF estimates. We select voxels that fulfill three criteria:
The R-squared is above 0.3 (note that this threshold is somewhat arbitrary).
The pRF center lies inside the visual field covered by the stimulus. We extended the grid for
mu_xandmu_ybeyond the stimulus, but pRF centers outside the stimulus are only weakly constrained by the data.The pRF size is below the largest
sigmavalue in the grid. An estimate at the ceiling of the grid suggests that the best fitting pRF size lies outside the grid.
We select voxels instead of surface vertices because we fitted the pRF model to the timecourse of each voxel. The
projection onto the surface averages neighboring voxels, so selecting vertices would mix the parameters of poorly fitting
voxels into the selected vertices. Instead, we set the parameters of voxels that are not selected to NaN before
the projection, so that they are ignored.
mu_x = ls_params["mu_x"].to_numpy()
mu_y = ls_params["mu_y"].to_numpy()
sigma = ls_params["sigma"].to_numpy()
r_squared_threshold = 0.3
edge_tolerance = 1e-3 # in degrees
grid_min_xy, grid_max_xy = stimulus.grid.min(axis=(0, 1)), stimulus.grid.max(axis=(0, 1))
selection_criteria = pd.DataFrame({
f"r_squared > {r_squared_threshold}": r_squared > r_squared_threshold,
# The tolerance includes the grid values at the stimulus edges, which can be off due to rounding
"center inside stimulus": (
(mu_x >= grid_min_xy[1] - edge_tolerance)
& (mu_x <= grid_max_xy[1] + edge_tolerance)
& (mu_y >= grid_min_xy[0] - edge_tolerance)
& (mu_y <= grid_max_xy[0] + edge_tolerance)
),
# Least squares does not change sigma, so it is always one of the grid values
"sigma below grid ceiling": ~np.isclose(sigma, np.max(grid_param_ranges["sigma"])),
})
is_selected = selection_criteria.all(axis=1).to_numpy()
def project_selected(voxel_values: np.ndarray) -> np.ndarray:
"""Project values of the selected voxels onto the surface, ignoring all other voxels."""
return project_to_surface(fill_valid_voxels(np.where(is_selected, voxel_values, np.nan)))
# Fraction of valid voxels that fulfill each criterion and all criteria together
selection_criteria.assign(all_criteria=is_selected).mean()
First, we look at the pRF size indicated by sigma and plot it on the surface.
size_surf = project_selected(sigma)
plot_surf_stat_map_helper(size_surf, vmin=0.0, vmax=5.0, title="pRF size (sigma)");
Exercise 24
How does the pRF size change across the inflated surface? Change the R-squared threshold (r_squared_threshold) and recreate the plot. Does the pattern change?
# Your code goes here
Exercise 25
Compute the polar angle of the pRF center with prfmodel.utils.calculate_polar_angle. Plot the polar angle of the selected voxels on the inflated surface (hint: use cyclic=True in the plotting helper to get a cyclic colormap). How does polar angle change on the inflated surface? Note that the projection onto the surface averages neighboring voxels, but angles cannot be averaged directly because they wrap around from \(-\pi\) to \(\pi\). Instead, project mu_x and mu_y onto the surface separately and compute the polar angle afterwards.
# Your code goes here
Exercise 26
Compute the eccentricity (i.e., the distance of the pRF center from the center of the screen) with prfmodel.utils.calculate_eccentricity. Plot it for the selected voxels on the inflated surface. How does eccentricity change across the inflated surface?
# Your code goes here
Exercise 27 (optional)
Compare the polar angle and eccentricity patterns you observed to the ones reported in the paper by Dumoulin and Wandell (2008). Do your results agree with theirs?
Exercise 28 (optional, advanced)
Fit the compressive spatial summation (CSS) pRF model (Kay et al., 2013) to the dataset using the same workflow. The CSS pRF model is implemented in the prfmodel.models.prf.Gaussian2DCSSPRFModel class. Look at the API documentation of the model class to get more information about its parameters. You can also take a look at the paper by Kay et al. (2013) for more background information.
# Your code goes here
References#
Aqil, M., Knapen, T., & Dumoulin, S. O. (2021). Divisive normalization unifies disparate response signatures throughout the human visual hierarchy. Proceedings of the National Academy of Sciences, 118(46), e2108713118. https://doi.org/10.1073/pnas.2108713118
Dumoulin, S. O., & Wandell, B. A. (2008). Population receptive field estimates in human visual cortex. NeuroImage, 39(2), 647-660. https://doi.org/10.1016/j.neuroimage.2007.09.034
Kay, K. N., Winawer, J., Mezer, A., & Wandell, B. A. (2013). Compressive spatial summation in human visual cortex. Journal of Neurophysiology, 110(2), 481-494. https://doi.org/10.1152/jn.00105.2013