Practical 7: The neural contrast sensitivity function model#
This is the notebook for the neural contrast sensitivity function practical of the workshop. This practical covers the neural contrast sensitivity function (nCSF) model by Roelofzen et al. (2025), its different computational parts, and how to fit it to empirical data from a visual functional magnetic resonance imaging (fMRI) experiment.
We recommend going through the first practical on the canonical Gaussian population receptive field (pRF) model first to get familiar with the prfmodel package before starting with this practical.
The first part of this practical goes through the nCSF model step-by-step. The model input and output as well as every submodel is motivated and explained with interactive visualizations. The second part is about fitting the model to an empirical dataset obtained with a 7T fMRI scanner. This part covers data inspection and visualization on an inflated surface of the brain. It goes through multiple consecutive fitting stages (grid search, least-squares, and SGD) and concludes with evaluating the model fit and interpreting its parameters.
Contents#
Installation and Setup#
First, we need to install prfmodel from PyPI. We install the package with the TensorFlow backend (for more information on the different backends, see the Backends section in the package documentation).
Please make sure to use a GPU instance when installing prfmodel and running this notebook in Colab.
!pip install "prfmodel[tensorflow]==0.2.0"
We set the backend to TensorFlow and a default for printing.
import os
import numpy as np
import pandas as pd
# Set keras backend to 'tensorflow' (this is normally the default)
os.environ["KERAS_BACKEND"] = "tensorflow"
# Print parameter DataFrames with three decimals
pd.set_option("display.precision", 3)
The neural contrast sensitivity function model#
The contrast sensitivity function (CSF) is a classic measure of visual function from psychophysics. It describes the lowest contrast of a grating that an observer can perceive (the contrast threshold) as a function of the spatial frequency (SF) of the grating, that is, how many pairs of light and dark stripes fit into one degree of visual angle. Instead of the threshold, the CSF is usually expressed as the contrast sensitivity, which is the inverse of the threshold: When the contrast is given in percent, a threshold of 1% corresponds to a sensitivity of 100. Humans are most sensitive to SFs of a few cycles per degree, and their sensitivity falls off towards lower and, more steeply, towards higher SFs. In ophthalmology, the CSF is used, for example, to detect deficits in visual function.
Roelofzen et al. (2025) translated the CSF into a model of the blood oxygenation level-dependent (BOLD) response that is similar to a pRF model. The neural CSF (nCSF) model estimates a CSF for the neuron population of every vertex: To which SFs does the population respond, and how much contrast does it need to respond? Whereas a pRF model describes where in the visual field a neuron population responds, the nCSF model describes which visual patterns it responds to.
Input: Spatial frequency and contrast#
In this practical, we use the BOLD response of a single subject from Roelofzen et al. (2025), which was recorded with a 7 Tesla MRI scanner. The dataset has the tag 7t-ncsf-visual. We can get a description of the dataset with prfmodel.examples.describe_dataset.
from prfmodel.examples import describe_dataset, load_dataset
print(describe_dataset("7t-ncsf-visual"))
The dataset contains two tasks: In the csf task, the subject saw gratings with different SFs and contrasts, which we use to fit the nCSF model. In the prf task, the subject saw bars that moved through the visual field, which we only need for an optional exercise at the end of this practical. We load the csf task for both hemispheres (the default). The response is given for every vertex of a standardized cortical surface from the Human Connectome Project (HCP; Van Essen et al., 2012). Like in the fourth practical, we also load this surface with the tag hcp-999999-surface to visualize the response and the model parameters later.
# Downloads on first use and caches in a user data directory (see prfmodel.examples.get_data_dir)
dataset = load_dataset("7t-ncsf-visual", task="csf")
surface = load_dataset("hcp-999999-surface", surface_type="inflated")
print(dataset)
The stimulus can be accessed through the dataset.stimulus attribute.
stimulus = dataset.stimulus
print(stimulus)
Unlike the stimulus of a pRF model, the stimulus of a CSF model has no design matrix and no grid. Every time frame shows a grating that covers the entire screen, so we do not need to describe which part of the visual field is stimulated. Instead, the stimulus describes the features of the grating at every time frame: its SF (stimulus.sf, in cycles per degree) and its Michelson contrast (stimulus.contrast, in percent). The frame_duration is the duration of a time frame in seconds, which is the repetition time (TR) of 1.5 seconds. During blank frames, the screen does not show a grating. Blank frames have a contrast and an SF of zero.
We plot the contrast of the grating at every time frame with prfmodel.plotting.plot_csf_stimulus_design. The color shows the SF of the grating (in cycles per degree). The function leaves out the blank frames, so they show up as gaps between the lines. Because the contrasts span more than two orders of magnitude, we use a logarithmic y-axis.
import matplotlib.pyplot as plt
from plotting_helpers import style_axes
from prfmodel.plotting import plot_csf_stimulus_design
time_frames = np.arange(stimulus.sf.shape[0])
is_blank = stimulus.contrast == 0.0
# SFs of the gratings (in cycles per degree)
sf_levels = np.unique(stimulus.sf[~is_blank])
fig, ax = plot_csf_stimulus_design(stimulus, figsize=(12, 3.5), layout="constrained")
ax.set_yscale("log")
style_axes(ax, "Time frame (in TR)", "Contrast (in %)")
Exercise 1
How is the experiment structured? How many blocks with gratings are there, and how is each block built up? How often is each SF shown, and at how many contrast levels (hint: use np.unique on stimulus.sf and stimulus.contrast)?
Solution
print("SFs:", np.unique(stimulus.sf[~is_blank]))
print("Contrasts:", np.unique(stimulus.contrast[~is_blank]).round(2))
# Number of time frames and contrast levels per SF
for sf in sf_levels:
is_sf = (stimulus.sf == sf) & ~is_blank
print(f"{sf:g} cycles/degree: {is_sf.sum()} time frames, {len(np.unique(stimulus.contrast[is_sf]))} contrast levels")
# Number of blank time frames
print("Blank time frames:", is_blank.sum())
The experiment has 214 time frames (321 seconds). It contains six blocks with gratings of 24 time frames (36 seconds) each. The blocks are separated by blank periods of 10 time frames (15 seconds), and the experiment starts and ends with a blank period (70 blank time frames in total). Each block consists of two sweeps through the contrasts of 12 time frames each: In the first sweep, the contrast steps down from 80% to 0.25% at one SF. In the second sweep, the contrast steps back up from 0.25% to 80% at another SF. The blocks pair the SFs as follows: 3 and 18, 1 and 6, 0.5 and 12, 6 and 1, 12 and 3, and 18 and 0.5 cycles per degree.
Each of the six SFs (0.5, 1, 3, 6, 12, and 18 cycles per degree) is shown in two blocks, once with decreasing and once with increasing contrast, so 24 time frames per SF. There are 14 contrast levels in total, but each SF is shown at only 12 of them: The low SFs skip the high contrasts of 21% and 51% and instead include the low contrasts of 0.39% and 0.95%, and the high SFs do the opposite. This puts more contrast levels where the threshold is expected to lie: Sensitivity is usually higher for low SFs than for high SFs.
# Your code goes here
Building the model step by step#
The nCSF model gets from input (i.e., the SF and contrast at every time frame) to output (i.e., the predicted BOLD response) in several steps. Each step is performed by a submodel that does a dedicated computation:
The contrast sensitivity function (CSF) describes how sensitive the neuron population is to every SF.
The contrast response function (CRF) turns the contrast of the grating and the sensitivity at its SF into the predicted neuron population activity at every time frame.
The impulse response turns the neuron population activity into a predicted BOLD response.
The scaling adjusts the baseline and amplitude of the predicted BOLD response.
In prfmodel, the first two steps together form the tuning model of the nCSF model. Note that, compared to the canonical Gaussian 2D pRF model from the first practical, the nCSF model has no stimulus encoding step: The tuning profile of a pRF model is defined on the stimulus grid, so it must be combined with the design matrix to predict the activity at every time frame. The CSF tuning model directly predicts the activity at every time frame from the SF and contrast of the grating at that frame.
We build up the model one submodel at a time and then combine all submodels into the full model. To explore how the parameters of each submodel affect its predictions, we use interactive figures. The code that creates the figures is hidden: Run a hidden cell to show its figure and move the sliders to change the parameters. The figure updates when you release a slider. You can expand a hidden cell if you are curious about its code. Note that the sliders of cs_peak, sf_peak, and slope_crf move on a logarithmic scale.
The first hidden cell defines helper functions for the interactive figures. Some of them are shared by all practicals and are imported from the module plotting_helpers.py, which lies next to this notebook.
Contrast sensitivity function#
The nCSF model describes the CSF with an asymmetric log-parabola: With logarithmic axes, the CSF is a downward-opening parabola that can have different curvatures to the left and to the right of its peak. The CSF \(S(f)\) at the SF \(f\) has four parameters: the peak contrast sensitivity \(CS_p\) (cs_peak), the SF at the peak \(f_p\) (sf_peak), and the curvatures of the left branch \(w_L\) (width_l) and of the right branch \(w_R\) (width_r):
Note that despite their names, larger values of width_l and width_r make the CSF narrower. Like Roelofzen et al. (2025), prfmodel fixes width_l at 0.68 by default, so that only cs_peak, sf_peak, and width_r are estimated.
We can predict the contrast sensitivity at the SF of every time frame with prfmodel.models.csf.predict_contrast_sensitivity. Like all model parameters in prfmodel, we store the parameters in a pandas.DataFrame, where each row corresponds to a different unit (e.g., a vertex) and each column to a different parameter. The function expects every parameter as a DataFrame with a single column, so we select the columns with double brackets.
csf_params = pd.DataFrame(
{
"cs_peak": [50.0],
"sf_peak": [2.0],
"width_r": [1.0],
},
)
simulated_sensitivity = predict_contrast_sensitivity(
sf=stimulus.sf,
cs_peak=csf_params[["cs_peak"]],
sf_peak=csf_params[["sf_peak"]],
width_l=0.68,
width_r=csf_params[["width_r"]],
)
simulated_sensitivity.shape # One row per unit with one value for every time frame
To relate the CSF to the stimulus, we plot every time frame with a grating as a point in the space of the CSF with prfmodel.plotting.plot_csf_stimulus_curve: The SF of the grating is on the x-axis, and the contrast sensitivity that corresponds to its contrast (100 / contrast) is on the y-axis. Both axes are logarithmic. The function draws the predicted CSF on top of the points (black line), connecting the sensitivities at the SFs in the stimulus. A neuron population responds to the grating at a time frame if the contrast lies above its threshold, that is, if the point lies below its CSF. Points at the top of the plot have a low contrast and only drive neuron populations with a high sensitivity.
from prfmodel.plotting import plot_csf_stimulus_curve
fig, ax = plot_csf_stimulus_curve(stimulus, np.asarray(simulated_sensitivity), figsize=(5.5, 4.5), layout="constrained")
ax.set_title("Stimulus and CSF in CSF space")
style_axes(ax, "Spatial frequency (in cycles/degree)", "Contrast sensitivity (100 / contrast)")
The interactive figure below draws the CSF as a smooth curve over a wider range of SFs (blue line). Time frames with a contrast above the threshold of the CSF (filled blue circles) lie below the curve, and time frames with a contrast below the threshold (empty gray circles) lie above it.
Exercise 2
Use the sliders to change the CSF parameters to the following combinations:
cs_peak=300,sf_peak=1.0,width_r=0.7cs_peak=20,sf_peak=6.0,width_r=1.5
How does the CSF change? Which time frames have a contrast above the threshold of the neuron population?
Solution
cs_peak moves the whole CSF up or down, sf_peak moves it left or right, and width_r determines how steeply the sensitivity falls off for SFs above the peak (larger values give a narrower CSF). The left branch does not change with width_r because width_l is fixed.
cs_peak=300,sf_peak=1.0,width_r=0.7: A high and broad CSF with its peak at a low SF. The sensitivity at all SFs in the stimulus lies above 50 (i.e., the threshold lies below 2% contrast), so the population responds to 122 of the 144 time frames with a grating. Only the lowest contrasts are below its threshold.cs_peak=20,sf_peak=6.0,width_r=1.5: A low and narrow CSF with its peak at a high SF. Even at the peak, the threshold is 5% contrast, and at 0.5 and 18 cycles per degree it is above 15%. Only 52 time frames have a contrast above the threshold, mostly gratings with a high contrast at intermediate SFs (3 to 12 cycles per degree).
Exercise 3
Why does it make sense to fix width_l instead of estimating it from the data? Think about which SFs the stimulus contains and move sf_peak to values around 1 cycle per degree.
Solution
The left branch of the CSF is only constrained by the SFs in the stimulus that lie below the peak. The lowest SF in the stimulus is 0.5 cycles per degree. With sf_peak around 1 cycle per degree, only the gratings with 0.5 cycles per degree lie on the left branch, and for a peak at or below 0.5 cycles per degree, none of them do. Thus, the data contain little or no information about width_l for many neuron populations. If we estimated it anyway, it would trade off with sf_peak and cs_peak (e.g., a higher peak with a steeper left branch can predict the same sensitivity at 0.5 cycles per degree), and the estimates of all three parameters would become unstable. Fixing width_l removes this trade-off.
Contrast response function#
The CSF gives us the contrast threshold of the neuron population at every SF: \(Q(f) = 100 / S(f)\) (in percent). The contrast response function (CRF) turns the contrast \(C(t)\) of the grating at time frame \(t\) into the predicted activity of the neuron population. The nCSF model uses a Naka-Rushton function with the threshold at the SF of the grating \(f(t)\) as the semi-saturation constant:
The predicted activity is zero when there is no contrast, reaches half of its maximum when the contrast equals the threshold, and approaches one when the contrast lies far above the threshold. The exponent \(q\) (slope_crf) determines how steeply the activity rises around the threshold.
We can predict the contrast response with prfmodel.models.csf.predict_contrast_response. It takes the contrast at every time frame and the contrast sensitivity at the SF of every time frame. We first predict the sensitivity for the SFs of the stimulus and then the response to its contrasts.
from prfmodel.models.csf import predict_contrast_response
crf_params = pd.DataFrame(
{
"cs_peak": [50.0],
"sf_peak": [2.0],
"width_r": [1.0],
"slope_crf": [2.0],
},
)
# Contrast sensitivity at the SF of every time frame
frame_sensitivity = predict_contrast_sensitivity(
sf=stimulus.sf,
cs_peak=crf_params[["cs_peak"]],
sf_peak=crf_params[["sf_peak"]],
width_l=0.68,
width_r=crf_params[["width_r"]],
)
simulated_activity = predict_contrast_response(
contrast=stimulus.contrast,
sensitivity=frame_sensitivity,
slope_crf=crf_params[["slope_crf"]],
)
simulated_activity.shape # One row per unit with one value for every time frame
The figure below shows the CRF for each SF in the stimulus: the predicted activity as a function of the contrast. The dot on each curve marks the contrast threshold of the population at this SF, where the activity reaches half of its maximum.
Exercise 4
Move the slope_crf slider. How does the CRF change? What happens for very large values of slope_crf? What does the neuron population then “compute”?
Solution
slope_crf determines how steeply the predicted activity rises with the contrast around the threshold. It does not change the thresholds: All curves still reach half of their maximum at the same contrast.
With a small
slope_crf(e.g., 0.5), the activity rises slowly over a wide range of contrasts. Even contrasts far below the threshold drive some activity, and contrasts far above it do not drive the maximum activity.With a large
slope_crf(e.g., 10), the CRF becomes a step function: The activity is close to zero for all contrasts below the threshold and close to one for all contrasts above it.
With a large slope_crf, the neuron population acts as a threshold detector: It signals whether a grating is visible to it or not, but not how far its contrast lies above the threshold.
The CSF tuning model#
In prfmodel, prfmodel.models.csf.CSFTuning combines the CSF and the CRF into the tuning model of the nCSF model. It predicts the neuron population activity at every time frame of the stimulus in the same two steps as above.
from prfmodel.models.csf import CSFTuning
csf_tuning = CSFTuning() # Fixes width_l at 0.68 by default
simulated_tuning_response = csf_tuning(stimulus, crf_params)
simulated_tuning_response.shape # One row per unit with one value for every time frame
The figure below shows the CSF (left) and the predicted neuron population activity at every time frame (right). The shaded areas mark the sweeps through the contrasts, and the number above each sweep is its SF (in cycles per degree).
Exercise 5
Why does the predicted neuron population activity have this particular shape? How does it relate to the stimulus? Change sf_peak to 0.5 and to 12. How does the predicted activity change and why?
Solution
The predicted activity is zero during the blank periods because there is no contrast. In the first sweep of a block, the contrast decreases. As long as the contrast lies above the threshold at the SF of the sweep, the activity is high, and it falls when the contrast drops below the threshold. In the second sweep, the contrast increases at another SF and the activity rises again when the contrast passes the threshold at that SF. Thus, every block has a valley in the middle, where the contrast is lowest. The width of the valley depends on the sensitivity at the SFs of the block: For SFs close to the peak of the CSF, the threshold is low, so the activity stays high for more time frames. For SFs far from the peak, the threshold is high, so the activity drops early in the first sweep (or rises late in the second sweep), or the population does not respond at all.
With
sf_peak=0.5, the population is most sensitive to the lowest SFs. The sweeps at 0.5 and 1 cycles per degree drive the most activity, whereas the sweeps at 12 and 18 cycles per degree drive almost none, because the threshold lies above the highest contrast of 80%.With
sf_peak=12, it is the other way around: The sweeps at high SFs drive the most activity, and the sweeps at 0.5 and 1 cycles per degree only drive activity at their highest contrasts.
Exercise 6
The stimulus only contains gratings with six different SFs. Can you find two clearly different CSFs (i.e., combinations of sf_peak and width_r) that predict a similar neuron population activity? What does this mean for estimating the CSF from data?
Solution
# The starting CSF and two CSFs with clearly different peak SFs (sf_peak, width_r)
example_csfs = {"Starting CSF": (2.0, 1.0), "Lower peak SF": (1.25, 0.8), "Higher peak SF": (2.8, 1.2)}
fig, (ax_csf, ax_activity) = plt.subplots(
1, 2, figsize=(12, 3.8), gridspec_kw={"width_ratios": [1, 2.2]}, layout="constrained",
)
example_activity = {}
for label, (example_sf_peak, example_width_r) in example_csfs.items():
params = to_frame(cs_peak=50.0, sf_peak=example_sf_peak, width_r=example_width_r, slope_crf=2.0)
example_activity[label] = np.asarray(csf_tuning(stimulus, params))[0]
ax_csf.plot(sf_values, predict_sensitivity(sf_values, 50.0, example_sf_peak, example_width_r), label=label)
ax_activity.plot(time_frames, example_activity[label], label=label)
print(label, "- sensitivity at the SFs of the stimulus:", predict_sensitivity(sf_levels, 50.0, example_sf_peak, example_width_r).round(1))
for sf in sf_levels:
ax_csf.axvline(sf, color=GRAY, linewidth=0.8, linestyle=":") # SFs in the stimulus
ax_csf.set(xscale="log", yscale="log", ylim=(1.0, 100.0))
ax_csf.legend(frameon=False, fontsize=8)
style_axes(ax_csf, "Spatial frequency (in cycles/degree)", "Contrast sensitivity")
style_axes(ax_activity, "Time frame (in TR)", "Predicted activity")
for label in ["Lower peak SF", "Higher peak SF"]:
correlation = np.corrcoef(example_activity["Starting CSF"], example_activity[label])[0, 1]
print(f"Correlation between the activity of the starting CSF and the CSF with a {label.lower()}: {correlation:.3f}")
For example, the starting CSF (sf_peak=2.0, width_r=1.0), a CSF with a lower peak SF and a broader right branch (sf_peak=1.25, width_r=0.8), and a CSF with a higher peak SF and a narrower right branch (sf_peak=2.8, width_r=1.2) predict almost the same activity (correlations above 0.99), although their peak SFs differ by more than a factor of two. The predicted activity only depends on the sensitivity at the six SFs in the stimulus, and these CSFs have similar sensitivities at these SFs. Between and beyond these SFs, the CSFs can differ without changing the prediction. In addition, the contrast levels are spaced coarsely, and a smooth CRF changes the activity only a little when the threshold shifts slightly.
This means that sf_peak and width_r (and to a lesser degree cs_peak) trade off against each other: Different combinations can fit the data almost equally well, especially when the data are noisy. The estimates of the individual parameters are therefore less certain than the shape of the CSF at the SFs in the stimulus. Measures that summarize the whole CSF can be more robust than individual parameters, and a stimulus with more SFs would constrain the CSF better.
Impulse response and scaling#
The remaining two submodels are the same as in the canonical Gaussian 2D pRF model from the first practical: The predicted neuron population activity is convolved with an impulse response that accounts for the slow and delayed BOLD response (prfmodel.impulse.DerivativeTwoGammaImpulse with the parameter weight_deriv). Then, the convolved response is scaled with the parameters baseline and amplitude (prfmodel.scaling.BaselineAmplitude). See the first practical for interactive figures of both submodels.
The impulse response must be sampled at the same rate as the stimulus. The model takes the sampling rate from the frame_duration of the stimulus, so we do not need to create a custom impulse response model for the TR of 1.5 seconds.
The full canonical model#
Now that we have seen every submodel, we can combine them into the full nCSF model. prfmodel implements it as the canonical model prfmodel.models.csf.CSFModel, which uses the submodels from above by default.
from prfmodel.models.csf import CSFModel
csf_model = CSFModel()
csf_model.parameter_names
The model has more parameters than the submodels that we have seen so far: delay, dispersion, undershoot, u_dispersion, and ratio determine the shape of the two gamma densities of the impulse response. These parameters are optional: If they are missing from the parameter DataFrame, the model uses default values, which correspond to the impulse response from the first practical. We make a prediction with all other parameters at once.
start_params = pd.DataFrame(
{
"cs_peak": [50.0],
"sf_peak": [2.0],
"width_r": [1.0],
"slope_crf": [2.0],
"weight_deriv": [-0.5],
"baseline": [0.0],
"amplitude": [1.0],
},
)
# Make prediction with nCSF model
simulated_response = csf_model(stimulus, start_params)
simulated_response.shape # One predicted timecourse with one value per time frame
The figure below shows the CSF (left) and the predicted BOLD response of the full model (right). The dashed gray line shows the prediction for the starting parameters above. Use the sliders to see how all parameters together shape the predicted BOLD response.
Exercise 7
Compare the predicted BOLD response to the predicted neuron population activity of the CSF tuning model for the same parameters. How does the impulse response change the response to each sweep? Why do you think the first sweep of every block goes from high to low contrast and the second sweep from low to high contrast?
Solution
The impulse response delays the predicted response by several seconds and smooths it. The valley in the middle of each block becomes shallower, so the responses to the two sweeps of a block partly merge, and the response dips below zero after each block (the undershoot of the impulse response).
Because the BOLD response is slow, the response at a time frame also reflects the stimulus at previous time frames. The design deals with this by placing the highest contrasts, which drive the strongest activity, at the start and end of each block and the lowest contrasts in the middle. This way, the responses to the high contrasts of the two SFs in a block are as far apart in time as possible, and the response to the first SF has declined before the response to the second SF rises. If the contrast increased in the first sweep and decreased in the second, the strongest responses to both SFs would follow each other directly and could not be separated. In addition, every SF is shown once with decreasing and once with increasing contrast, which balances effects of the presentation order, such as adaptation.
Fitting the model to data#
Inspecting the data#
Before we fit the nCSF model to empirical data, we inspect the recorded BOLD response timecourses, which we can access from the dataset.response attribute.
response = dataset.response
response.shape # One timecourse for every vertex (left hemisphere first)
The dataset contains a timecourse with 214 time frames for each of the 118,584 vertices of the standardized surface (59,292 per hemisphere). The responses were already detrended, converted to percent signal change (PSC), and averaged over the runs of the experiment. Thus, every timecourse fluctuates around zero, and we do not need to convert it before fitting.
To visualize the response on the inflated surface, we create a helper function with make_surface_plot from plotting_helpers.py like in the first practical. The helper shows three views that focus on the visual cortex in the occipital lobe: the inner (medial) side of each hemisphere, where the calcarine sulcus lies, and both hemispheres from behind (posterior). Like in the fourth practical, the response is already given for every vertex of the surface, so we do not need to project it from a volume onto the surface.
from plotting_helpers import make_surface_plot
plot_surf_stat_map_helper = make_surface_plot(surface.mesh)
We plot the standard deviation of each timecourse on the surface to get an overview of the variability in the BOLD timecourses across the surface.
# Calculate standard deviation of each timecourse
response_sd = response.std(axis=1)
plot_surf_stat_map_helper(response_sd, vmax=2.0, title="BOLD response standard deviation (in PSC)");
The dataset also contains a region of interest (ROI) label for every vertex. The labels come from the template atlas by Benson et al. (2014), which predicts the location of twelve visual areas from the anatomy of the subject’s brain. Vertices outside these areas have the label 0.
roi_index = dataset.roi_index # One ROI label per vertex
dataset.roi_mapping # Which ROI each label refers to
We plot the ROIs on the surface. Note that the helper averages the values of neighboring vertices for plotting, so vertices at the borders between ROIs can get the color of a different ROI.
import matplotlib as mpl
roi_labels = [label for label in dataset.roi_mapping if label > 0]
# Only color vertices that belong to one of the ROIs
fig, _ = plot_surf_stat_map_helper(
np.where(roi_index > 0, roi_index, np.nan),
vmin=0.5,
vmax=len(roi_labels) + 0.5,
cmap=mpl.colors.ListedColormap(plt.get_cmap("Paired").colors[: len(roi_labels)]),
title="Visual areas (Benson et al., 2014)",
)
# Label the colors of the colorbar (the last axes of the figure) with the names of the ROIs
fig.axes[-1].set_yticks(roi_labels, labels=[dataset.roi_mapping[label] for label in roi_labels]);
Exercise 8
Compare the ROIs to the map of the response standard deviation. Do the vertices with the largest variability lie inside the ROIs? Do all parts of V1 have a similarly variable response?
Solution
Many of the vertices with the most variable responses lie in the occipital lobe inside the ROIs, especially at the occipital pole and on the lateral and dorsal side of the posterior view. These are the vertices that respond to the gratings. However, not all parts of the ROIs have a variable response: The front (anterior) part of V1 and V2 along the calcarine sulcus in the medial views varies about as little as the rest of the brain. Some vertices outside the ROIs also have a variable response, either because they belong to visual areas that the atlas does not include or because of noise (e.g., from large veins).
V1 is organized retinotopically: The occipital pole represents the center of the visual field, and the anterior part of the calcarine sulcus represents the periphery. The gratings only covered the central part of the visual field (up to about 5 degrees from the center), so the vertices that represent the periphery do not see the stimulus and do not respond to it (see also Exercise 13).
Selecting valid vertices#
The nCSF model only makes sense for vertices that respond to visual stimulation, and fitting all vertices of the surface would take a long time. Therefore, we only fit the model to vertices that belong to one of the ROIs. We also make sure that their responses vary over time: A vertex with a constant response contains no information about the stimulus.
# Vertices in one of the ROIs with a response that varies over time
response_is_valid = (roi_index > 0) & (response_sd > 0.0)
response_valid = response[response_is_valid]
# Name of the ROI of every valid vertex
roi_valid = pd.Series(roi_index[response_is_valid]).map(dataset.roi_mapping).to_numpy()
def fill_valid_vertices(values: np.ndarray) -> np.ndarray:
"""Fill values for the valid vertices into an array with one value per vertex, using NaN for all other vertices."""
values_full = np.full((response_is_valid.shape[0],), fill_value=np.nan)
values_full[response_is_valid] = values
return values_full
response_valid.shape
Exercise 9
Visualize the BOLD response of the valid vertices (hint: use prfmodel.plotting.plot_response_heatmap). Can you see the blocks of the stimulus? How does the response change within a block? Do all valid vertices respond to the stimulus?
Solution
from prfmodel.plotting import plot_response_heatmap
fig, ax = plot_response_heatmap(
response_valid,
vmin=-2,
vmax=2,
xlabel="Time frame (in TR)",
ylabel="Valid vertex index",
colorbar_label="BOLD response (in PSC)",
figsize=(10, 6),
)
# Mark the start and the end of each block
for first_sweep, second_sweep in zip(sweeps[::2], sweeps[1::2]):
ax.axvline(first_sweep[0] - 0.5, color="white", linestyle="--", linewidth=1)
ax.axvline(second_sweep[1] + 0.5, color="white", linestyle="--", linewidth=1)
# Average response of the 1,000 valid vertices with the most variable responses
most_variable = np.argsort(response_valid.std(axis=1))[-1000:]
fig, ax = plt.subplots(figsize=(12, 3.5), layout="constrained")
mark_sweeps(ax)
ax.plot(time_frames, response_valid[most_variable].mean(axis=0), color=BLUE, linewidth=2)
ax.set_title("Average response of the 1,000 most variable valid vertices", loc="left", pad=14)
style_axes(ax, "Time frame (in TR)", "BOLD response (in PSC)");
Many of the valid vertices respond to the stimulus with the same pattern in every block, which shows up as vertical stripes in the heatmap. The average response of the most variable vertices shows this pattern more clearly:
The response rises shortly after the start of each block, when the first sweep shows the highest contrasts, and falls in the middle of the block, when the contrast is lowest.
The response rises again towards the end of each block, when the second sweep shows the highest contrasts. Because the BOLD response is delayed, this peak lies just after the end of the block.
During the blank periods, the response falls below zero. Because the timecourses are centered around their mean, the baseline lies below zero.
The height of the peaks depends on the SF of the sweep: The peaks for SFs of 1 and 3 cycles per degree are high, and the peaks for 18 cycles per degree are low (e.g., the first peak of the last block).
Not all valid vertices respond, though: Several groups of rows in the heatmap do not show the pattern. Many of these vertices represent parts of the visual field that the screen does not cover (see Exercises 8 and 13).
# Your code goes here
Before we let prfmodel fit the model, you can try to fit it yourself. The figure below shows the observed BOLD response of a V1 vertex with a clear signal (gray) together with the prediction of the nCSF model (blue). The R-squared in the title shows how much of the variance in the observed response the prediction explains (1 is a perfect fit).
Exercise 10
Move the sliders until the prediction matches the observed response as well as you can. Which R-squared can you reach? Which parameters are easy to find and which are hard? Why?
Solution
For vertex 4635 in V1, the fits of prfmodel reach an R-squared of about 0.59 with cs_peak ≈ 70, sf_peak ≈ 2.6 cycles per degree, width_r ≈ 1.9, slope_crf ≈ 1.1, baseline ≈ -0.45, and amplitude ≈ 1.6. You may reach a similar R-squared with quite different values for some of the parameters.
baseline and amplitude are easy to find because they shift and scale the entire prediction. sf_peak is also relatively easy to find because it determines which sweeps drive the largest responses. cs_peak, width_r, and slope_crf are harder to find: They change the prediction in similar ways, for example, both cs_peak and slope_crf determine at which contrast the response within a sweep drops, and changes in one parameter can be compensated by changes in the others (see Exercise 6). In addition, the observed response rises a bit earlier than the prediction, which none of the CSF parameters can fix and which limits the R-squared that we can reach.
Grid search#
We fit the nCSF model in three stages like in the first practical. In the first stage, we perform a grid search to find good starting values for the parameters of the CSF tuning model (i.e., cs_peak, sf_peak, width_r, and slope_crf). prfmodel does not have a function that creates grid values for the nCSF model, so we create them ourselves. All four parameters are positive and act on a logarithmic scale, so we space their values logarithmically with np.geomspace:
cs_peakbetween 2 and 400, which covers the contrast sensitivities in the stimulus (100 / 80 = 1.25 to 100 / 0.25 = 400).sf_peakbetween 0.25 and 10 cycles per degree. Values below the lowest SF in the stimulus (0.5 cycles per degree) allow CSFs that fall off across all SFs in the stimulus.width_rbetween 0.3 and 2.slope_crfbetween 0.5 and 8.
For weight_deriv, baseline, and amplitude, we only provide a single value so that they stay constant across the entire grid. The grid has 12 × 12 × 6 × 8 = 6,912 parameter combinations.
grid_param_ranges = {
"cs_peak": np.geomspace(2.0, 400.0, 12),
"sf_peak": np.geomspace(0.25, 10.0, 12),
"width_r": np.geomspace(0.3, 2.0, 6),
"slope_crf": np.geomspace(0.5, 8.0, 8),
"weight_deriv": [-0.5],
"baseline": [0.0],
"amplitude": [1.0],
}
To run the grid search, we construct the prfmodel.fitters.GridFitter. We set batch_size=100 to let the fitter evaluate 100 parameter combinations at the same time. By default, the loss is the negative correlation coefficient, which ignores differences in baseline and amplitude between model predictions and observed data.
from prfmodel.fitters import GridFitter
# Create grid fitter object
grid_fitter = GridFitter(
model=csf_model,
stimulus=stimulus,
compile_step=True, # Setting 'compile_step=True' speeds up the fitting
)
# Run grid search
grid_history, grid_params = grid_fitter.fit(
data=response_valid,
parameter_values=grid_param_ranges,
batch_size=100,
)
grid_params
Because the grid search can only return values that are in the grid, we check how the estimates are distributed across the grid with prfmodel.plotting.plot_grid_parameter_distribution. Estimates at the smallest or largest value of the grid are highlighted in red.
from prfmodel.plotting import plot_grid_parameter_distribution
csf_param_names = ["cs_peak", "sf_peak", "width_r", "slope_crf"]
fig, axes = plt.subplots(2, 2, figsize=(10, 7), layout="constrained")
for ax, name in zip(axes.flat, csf_param_names):
plot_grid_parameter_distribution(grid_params, grid_param_ranges, name, ax=ax) # Detects log spacing
# Label every other grid value instead of the powers of ten
tick_values = grid_param_ranges[name][::2]
ax.set_xticks(tick_values, labels=[f"{value:.3g}" for value in tick_values])
ax.xaxis.set_minor_formatter(mpl.ticker.NullFormatter())
ax.set(xlabel=name, title=name)
Exercise 11
Which parameters often have estimates at the smallest or largest value of the grid? What could be the reason (hint: think about the vertices in the ROIs that do not respond to the stimulus)?
Solution
# The grid search loss is the negative correlation between the prediction and the response
grid_correlation = -np.asarray(grid_history.history["loss"])
is_responsive = grid_correlation > 0.5
# Fraction of vertices with an estimate at the smallest or largest grid value
edge_fraction = {}
for name in csf_param_names:
estimates = grid_params[name].to_numpy()
grid_values = grid_param_ranges[name]
is_at_edge = np.isclose(estimates, np.min(grid_values)) | np.isclose(estimates, np.max(grid_values))
edge_fraction[name] = {
"correlation > 0.5": is_at_edge[is_responsive].mean(),
"correlation <= 0.5": is_at_edge[~is_responsive].mean(),
}
print(f"{is_responsive.sum()} of {len(is_responsive)} vertices have a correlation above 0.5")
pd.DataFrame(edge_fraction).round(2)
About 5,500 of the 12,388 valid vertices have a correlation above 0.5 between the best grid prediction and their response. For these vertices, the estimates rarely lie at the edge of the grid. The exception is width_r: About a fifth of them have the largest value (2.0), which suggests that their CSF falls off even more steeply at high SFs. SGD can move these estimates beyond the grid, up to the upper bound of 3.
For the other vertices, most estimates of cs_peak, width_r, and slope_crf lie at the edge of the grid: the smallest peak sensitivity, the narrowest CSF, and the steepest CRF. Many vertices in the ROIs do not respond to the stimulus, and their response only contains noise. The combination of a low and narrow CSF with a steep CRF predicts a response to only a few time frames with a high contrast at SFs close to the peak. Such a sparse prediction can match a few random peaks in the noise and therefore reaches the highest correlation with a noisy response. These estimates do not describe a CSF, so we do not interpret them.
Least-squares#
In the grid search, we ignored the baseline and amplitude parameters. Like in the first practical, we estimate them with least-squares by regressing the observed responses on the predicted responses for each vertex.
from prfmodel.fitters import LeastSquaresFitter
# Create least-squares fitter
ls_fitter = LeastSquaresFitter(
model=csf_model,
stimulus=stimulus,
)
# Run least squares fit
ls_history, ls_params = ls_fitter.fit(
data=response_valid,
parameters=grid_params,
slope_name="amplitude", # Names of parameters to be optimized with least squares
intercept_name="baseline",
batch_size=1000,
)
ls_params
Stochastic gradient descent#
Finally, we use SGD to finetune all parameters at once, starting from the least-squares estimates.
We need to be more careful with the nCSF parameters than with the pRF parameters in the first practical. All four CSF parameters must be positive, but that is not enough: The data only constrain the CSF within the range of SFs and contrasts in the stimulus. Beyond this range, SGD can move the parameters of some vertices to implausible values without changing the prediction much. Without upper bounds, some vertices end up with a peak SF of more than 100 cycles per degree, a width_r of more than 10, or a slope_crf of more than 100. Therefore, we keep each CSF parameter between a lower and an upper bound during SGD.
In the first practical, we used a ParameterTransform in an adapter to let SGD optimize sigma in log space. prfmodel also provides prfmodel.fitters.adapter.ParameterConstraint, which keeps a parameter above a lower or below an upper bound. To keep a parameter between a lower and an upper bound, we write our own ParameterTransform: SGD optimizes the parameter on an unbounded scale \(y\), and the model sees the value \(x = \text{lower} + (\text{upper} - \text{lower}) \cdot \text{sigmoid}(y)\), which always lies between the bounds. The transformation from the bounded to the unbounded scale is the inverse of this function: \(y = \log\left((x - \text{lower}) / (\text{upper} - x)\right)\). Starting values must lie strictly between the bounds. The bounds are a bit wider than the grid, so that SGD can move the estimates beyond the grid values.
from keras import ops
from prfmodel.fitters.adapter import Adapter, ParameterTransform
def box_transform(parameter_name: str, lower: float, upper: float) -> ParameterTransform:
"""Keep a parameter strictly between `lower` and `upper` during SGD."""
return ParameterTransform(
[parameter_name],
transform_fun=lambda x: ops.log((x - lower) / (upper - x)), # From the bounded to the unbounded scale
inverse_fun=lambda y: lower + (upper - lower) * ops.sigmoid(y), # From the unbounded to the bounded scale
)
# Lower and upper bound of each CSF parameter
parameter_bounds = {
"cs_peak": (1.0, 1000.0),
"sf_peak": (0.1, 20.0),
"width_r": (0.1, 3.0),
"slope_crf": (0.2, 10.0),
}
adapter = Adapter([box_transform(name, lower, upper) for name, (lower, upper) in parameter_bounds.items()])
The default optimizer of SGD (Adam) changes each parameter on the unbounded scale by at most about the learning rate in each step. With the default learning rate of 0.001, 500 steps can only move the parameters a little, so we increase the learning rate to 0.01. Like in the first practical, we fix weight_deriv to its starting value.
from keras.optimizers import Adam
from prfmodel.fitters import SGDFitter
sgd_fitter = SGDFitter(
model=csf_model,
stimulus=stimulus,
adapter=adapter,
optimizer=Adam(learning_rate=0.01),
compile_step=True, # Setting 'compile_step=True' speeds up the fitting
)
sgd_history, sgd_params = sgd_fitter.fit(
data=response_valid,
init_parameters=ls_params,
fixed_parameters=["weight_deriv"], # Fix weight_deriv to starting value
num_steps=500,
)
sgd_params
Evaluating the model fit#
We make predictions with the SGD estimates and compute the R-squared of each vertex. Like in the first practical, we wrap our csf_model in prfmodel.utils.batched to make predictions for batches of vertices sequentially, and compute the R-squared of each vertex with prfmodel.utils.calculate_r_squared. We look at the distribution of the R-squared scores with prfmodel.plotting.plot_r_squared_hist.
from prfmodel.plotting import plot_r_squared_hist
from prfmodel.utils import batched, calculate_r_squared
predict_batched = batched(csf_model)
pred_response = np.asarray(predict_batched(stimulus, sgd_params, batch_size=1000))
r_squared = calculate_r_squared(response_valid, pred_response) # One score per vertex
plot_r_squared_hist(r_squared);
Exercise 12
Compare the R-squared between the least-squares and the SGD stage (hint: use prfmodel.plotting.plot_r_squared_comparison). Does SGD improve the fit and if so, by how much?
Solution
from prfmodel.plotting import plot_r_squared_comparison
pred_response_ls = np.asarray(predict_batched(stimulus, ls_params, batch_size=1000))
r_squared_ls = calculate_r_squared(response_valid, pred_response_ls)
fig, ax = plot_r_squared_comparison(
r_squared_ls,
r_squared,
value_range=(0.0, 1.0),
xlabel="R-squared (least-squares)",
ylabel="R-squared (SGD)",
)
r_squared_gain = r_squared - r_squared_ls
print(f"Fraction of vertices with a higher R-squared after SGD: {np.mean(r_squared_gain > 0):.2f}")
# Vertices that least-squares already fits reasonably well
is_fitted_ls = r_squared_ls > 0.1
print(
f"Median R-squared of {is_fitted_ls.sum()} vertices with a least-squares R-squared above 0.1: "
f"{np.median(r_squared_ls[is_fitted_ls]):.3f} (least-squares), {np.median(r_squared[is_fitted_ls]):.3f} (SGD); "
f"largest gain: {r_squared_gain[is_fitted_ls].max():.3f}"
)
# Vertices whose R-squared jumps from below 0.1 to above 0.3
is_jump = ~is_fitted_ls & (r_squared > 0.3)
print(
f"{is_jump.sum()} vertices jump from an R-squared below 0.1 to above 0.3, "
f"{np.mean(sgd_params['amplitude'].to_numpy()[is_jump] < 0.0):.0%} of them with a negative amplitude"
)
pd.Series(roi_valid[is_jump]).value_counts()
SGD improves the R-squared of almost all vertices, but in two different ways. For the vertices that least-squares already fits reasonably well (R-squared above 0.1), the improvement is small: Their median R-squared increases from about 0.52 to 0.53, and no vertex gains more than about 0.03. The grid is dense enough to find estimates close to the best fit, and the prediction changes smoothly with the CSF parameters. Thus, least-squares already gets close to the best fit, and SGD mostly finetunes the parameters. Note that some of these small changes in R-squared can still come with noticeable changes in the parameters (see Exercise 6).
Most of the other vertices stay close to an R-squared of zero because their response only contains noise. However, about 230 vertices, mostly in V1 and V2, jump from an R-squared below 0.1 to above 0.3 (up to about 0.75). All of them have a negative amplitude: Their response decreases when the gratings are shown. The grid search looks for the prediction with the highest correlation with the response. Every CSF predicts more activity for gratings with a higher contrast, so all predictions correlate negatively with such a response, and the grid search picks the prediction that correlates the least negatively. Least-squares then finds a negative amplitude for this prediction but cannot change the CSF. SGD changes all parameters at once and finds a CSF that matches the inverted response. Almost all of these vertices represent the visual field beyond the screen (see Exercise 13). We exclude them from the interpretation with the criterion that the amplitude is positive.
# Your code goes here
Exercise 13
Plot the R-squared on the inflated surface (hint: use fill_valid_vertices). In which parts of the visual cortex does the model fit the data well? Why does it fit poorly in large parts of V1 and V2? (hint: Plot the R-squared against the eccentricity that the template atlas predicts for each vertex, which is stored in dataset.units["benson14_eccen"], and remember the size of the screen.)
Solution
from plotting_helpers import ORANGE
plot_surf_stat_map_helper(fill_valid_vertices(r_squared), vmin=0.0, vmax=0.8, title="Variance explained (R-squared)")
eccentricity_template = dataset.units["benson14_eccen"].to_numpy()[response_is_valid]
is_negative = sgd_params["amplitude"].to_numpy() < 0.0
fig, axes = plt.subplots(1, 2, figsize=(12, 4), sharey=True, layout="constrained")
for ax, name in zip(axes, ["V1", "V2"]):
is_roi = roi_valid == name
for is_shown, color, label in [(~is_negative, BLUE, "Positive amplitude"), (is_negative, ORANGE, "Negative amplitude")]:
ax.scatter(
eccentricity_template[is_roi & is_shown], r_squared[is_roi & is_shown], s=4, color=color, alpha=0.3, label=label,
)
ax.axvline(5.1, color=GRAY, linestyle="--", linewidth=1, label="Edge of the screen")
ax.set(xscale="log", title=name)
ax.legend(frameon=False)
style_axes(ax, "Eccentricity of the template atlas (in degrees)", "R-squared")
for name in ["V1", "V2"]:
is_roi = roi_valid == name
is_inside = eccentricity_template[is_roi] < 5.1
print(
f"{name}: {is_inside.mean():.0%} of the vertices lie inside the screen; median R-squared "
f"{np.median(r_squared[is_roi][is_inside]):.2f} inside and {np.median(r_squared[is_roi][~is_inside]):.2f} outside"
)
The model fits the data well at the occipital pole and in the dorsal and lateral visual areas (R-squared up to about 0.8). In large parts of V1 and V2, especially their anterior parts along the calcarine sulcus, the R-squared is close to zero.
The template atlas predicts that only about a quarter of the V1 vertices and about a third of the V2 vertices represent the part of the visual field that the screen covers (up to about 5 degrees from the center). Inside the screen, the median R-squared is about 0.54 in both areas, and outside the screen it is about zero: The vertices that represent the periphery never see a grating, so their response does not contain any information about the CSF. The higher visual areas have larger pRFs and represent the periphery less, so a larger part of them responds to the stimulus. The R-squared is also lower for some vertices with an eccentricity below about 0.5 degrees. Close to the center of the visual field, the visual areas are small and lie close together, so the template atlas is less accurate there.
A few vertices beyond the screen still reach a high R-squared. Many of them have a negative amplitude (see Exercise 12): Their response decreases when the gratings are shown. Such negative responses often occur in the parts of the visual cortex that represent the visual field around a stimulus. The template atlas is not exact either, so some of the vertices that it places beyond the screen may represent a part of the visual field on the screen.
# Your code goes here
We compare the R-squared between ROIs with prfmodel.plotting.plot_parameter_by_roi, which shows the mean and standard deviation across the vertices in each ROI. The function expects a DataFrame, so we wrap the R-squared in one.
from prfmodel.plotting import plot_parameter_by_roi
# Visual areas ordered roughly from early to higher visual cortex
roi_order = ["V1", "V2", "V3", "hV4", "VO1", "VO2", "V3a", "V3b", "LO1", "LO2", "TO1", "TO2"]
fig, ax = plt.subplots(figsize=(9, 4), layout="constrained")
plot_parameter_by_roi(pd.DataFrame({"R-squared": r_squared}), "R-squared", roi_valid, order=roi_order, ax=ax)
ax.set_ylim(-0.1, 1.0);
Exercise 14
Plot the predicted vs. the observed response for vertices with a high, medium, and low R-squared (hint: use prfmodel.plotting.plot_observed_predicted). Can you see any patterns in the observed response that the nCSF model does not capture?
Solution
from prfmodel.plotting import plot_observed_predicted
r_squared_order = np.argsort(r_squared)
fig, axes = plt.subplots(3, 1, figsize=(12, 9), sharex=True, layout="constrained")
# Vertices at the 99th, 80th, and 60th percentile of the R-squared scores
for ax, percentile, label in zip(axes, [99, 80, 60], ["High", "Medium", "Low"]):
vertex = r_squared_order[int(percentile / 100 * (len(r_squared_order) - 1))]
plot_observed_predicted(response_valid[vertex], pred_response[vertex], ax=ax)
mark_sweeps(ax)
ax.set_title(
f"{label}: vertex {vertex} ({roi_valid[vertex]}), R-squared = {r_squared[vertex]:.2f}", loc="left", pad=14,
)
ax.set(xlabel="", ylabel="BOLD response (in PSC)")
axes[-1].set_xlabel("Time frame (in TR)");
For well fitting vertices, the model captures the main pattern of the response: a peak after the start of each block (the high contrasts of the first sweep), a dip in the middle, a peak after the end of each block (the high contrasts of the second sweep), and different peak heights for different SFs. However, there are a few patterns that the model does not capture:
Timing: The observed response often rises and peaks a bit earlier than the prediction. The impulse response has a fixed shape, and only
weight_derivcan shift it in time, which we fixed during SGD. Estimatingweight_derivor the other parameters of the impulse response (e.g.,delay) can improve the fit.Responses at the end of a block: Some vertices, especially in higher visual areas, respond after the end of a block even if the model predicts no response to the SF of the second sweep (e.g., after a sweep at 18 cycles per degree). This could be a response to the disappearance of the grating (an offset response), which the nCSF model does not include.
Responses below baseline: Some vertices respond with a decrease below baseline to some sweeps (e.g., in the block with 0.5 and 12 cycles per degree). The nCSF model can only predict activity that increases with contrast, so it cannot capture such suppression.
Noise: For vertices with a lower R-squared, a large part of the response is noise, which no model should capture.
# Your code goes here
Interpreting the model parameters#
Like in the first practical, we only interpret the parameters of vertices for which we can trust the estimates. We select vertices that fulfill four criteria:
The R-squared is above 0.3 (note that this threshold is somewhat arbitrary).
The amplitude is positive.
The peak SF lies within the range of SFs in the stimulus (0.5 to 18 cycles per degree).
The peak contrast sensitivity lies within the range of contrast sensitivities in the stimulus (100 / 80 = 1.25 to 100 / 0.25 = 400).
The second criterion excludes vertices whose response decreases when the gratings are shown. The model can fit such a response with a negative amplitude, which flips the sign of the predicted response, but the CSF then describes which gratings decrease the response and not to which gratings the neuron population is sensitive. The last two criteria make sure that the peak of the CSF lies inside the stimulus in CSF space, similar to the criterion that the pRF center lies inside the stimulus in the first practical. A peak outside the stimulus is only weakly constrained by the data because we never measured the response at the peak.
cs_peak = sgd_params["cs_peak"].to_numpy()
sf_peak = sgd_params["sf_peak"].to_numpy()
width_r = sgd_params["width_r"].to_numpy()
r_squared_threshold = 0.3
# Range of contrast sensitivities in the stimulus
stimulus_sensitivity_range = 100.0 / stimulus.contrast[~is_blank].max(), 100.0 / stimulus.contrast[~is_blank].min()
selection_criteria = pd.DataFrame({
f"r_squared > {r_squared_threshold}": r_squared > r_squared_threshold,
"amplitude > 0": sgd_params["amplitude"].to_numpy() > 0.0,
"sf_peak inside stimulus": (sf_peak >= sf_levels.min()) & (sf_peak <= sf_levels.max()),
"cs_peak inside stimulus": (cs_peak >= stimulus_sensitivity_range[0]) & (cs_peak <= stimulus_sensitivity_range[1]),
})
is_selected = selection_criteria.all(axis=1).to_numpy()
def fill_selected_vertices(values: np.ndarray) -> np.ndarray:
"""Fill values for the selected vertices into an array with one value per vertex, using NaN for all others."""
return fill_valid_vertices(np.where(is_selected, values, np.nan))
# Number of valid vertices that fulfill each criterion and all criteria together
selection_criteria.assign(all_criteria=is_selected).sum()
First, we look at the peak SF and plot it on the surface. Because the SFs are spaced logarithmically, we plot the logarithm of the peak SF.
plot_surf_stat_map_helper(
fill_selected_vertices(np.log10(sf_peak)),
vmin=np.log10(0.5),
vmax=np.log10(5.0),
cmap="viridis",
title="Peak SF (log10 of cycles/degree)",
);
Exercise 15
How does the peak SF change across the surface? Plot the peak contrast sensitivity (cs_peak) on the surface as well. How does it change across the surface?
Solution
plot_surf_stat_map_helper(
fill_selected_vertices(np.log10(cs_peak)),
vmin=1.0,
vmax=2.6,
cmap="plasma",
title="Peak contrast sensitivity (log10)",
);
The peak SF is highest (above 3 cycles per degree) at the occipital pole, where V1 to V3 represent the center of the visual field. It decreases towards the periphery within these areas and is lower (around 1 cycle per degree) in the dorsal and lateral visual areas.
The peak contrast sensitivity varies less systematically. Half of the selected vertices have a peak sensitivity between about 65 and 170 (log10 between 1.8 and 2.2), which corresponds to a contrast threshold between about 0.6% and 1.5%. Lower values occur in patches, mostly at the borders of the area that responds to the stimulus and in the ventral visual areas, where the fits are often worse.
# Your code goes here
To compare the CSFs between visual areas, we predict the CSF of every selected vertex and plot the median CSF of each ROI with at least 50 selected vertices. Note that the median CSF is not the same as the CSF of the median parameters. We group the ROIs into early visual cortex, ventral visual cortex, and dorsal and lateral visual cortex.
roi_groups = {
"Early": ["V1", "V2", "V3"],
"Ventral": ["hV4", "VO1", "VO2"],
"Dorsal and lateral": ["V3a", "V3b", "LO1", "LO2", "TO1", "TO2"],
}
roi_colors = {name: color for names in roi_groups.values() for name, color in zip(names, plt.get_cmap("tab10").colors)}
# ROIs with at least 50 selected vertices
selected_rois = [name for name in roi_order if np.sum(is_selected & (roi_valid == name)) >= 50]
# Spatial frequencies to predict the contrast sensitivity for (in cycles per degree)
sf_range = np.geomspace(0.25, 40.0, 200)
fig, axes = plt.subplots(1, 3, figsize=(15, 4.5), sharey=True, layout="constrained")
for ax, (group, names) in zip(axes, roi_groups.items()):
for name in [name for name in names if name in selected_rois]:
roi_params = sgd_params[is_selected & (roi_valid == name)]
roi_sensitivity = np.asarray(
predict_contrast_sensitivity(
sf=sf_range,
cs_peak=roi_params[["cs_peak"]],
sf_peak=roi_params[["sf_peak"]],
width_l=0.68,
width_r=roi_params[["width_r"]],
)
)
ax.plot(sf_range, np.median(roi_sensitivity, axis=0), color=roi_colors[name], linewidth=2, label=name)
ax.set(xscale="log", yscale="log", ylim=(1.0, 300.0), title=group)
ax.legend(frameon=False)
style_axes(ax, "Spatial frequency (in cycles/degree)", "Median contrast sensitivity")
Exercise 16
How do the CSFs differ between visual areas? Which areas are most sensitive to high SFs?
Solution
# Median parameters of the selected vertices in each ROI
sgd_params[is_selected].groupby(roi_valid[is_selected])[csf_param_names].median().loc[selected_rois]
The CSFs of the early visual areas peak at higher SFs than those of the higher visual areas: The median peak SF is about 2.7 cycles per degree in V1, about 2 in V2 and V3, and about 1 cycle per degree in the dorsal and lateral areas (V3a, V3b, LO1, LO2, TO1, and TO2). Between about 3 and 10 cycles per degree, V1 has the highest sensitivity of the early visual areas, whereas V2 and V3 are more sensitive to low SFs. Most dorsal and lateral areas have a higher peak sensitivity than the early areas and a larger area under the log CSF, but their sensitivity is highest at low SFs. VO2 has a much lower sensitivity than all other areas, but the model also fits it worse than most other areas.
In this dataset, V3b has the highest sensitivity to SFs above 10 cycles per degree because its CSFs are broad (low width_r). However, only 75 vertices of V3b are selected, and its SF cutoff lies beyond the highest SF in the stimulus (18 cycles per degree), so this part of its CSF is an extrapolation of the model and not a measurement.
The CSF is often summarized with two numbers that describe the whole curve:
The SF cutoff (SFmax) is the highest SF that a neuron population can detect at all. It is the SF at which the sensitivity drops to 1, that is, where even a grating with 100% contrast lies at the threshold.
The area under the log CSF (AULCSF) summarizes the sensitivity across all SFs. It is the area between the CSF and a sensitivity of 1 when both axes are logarithmic.
For the asymmetric log-parabola, both measures follow directly from the parameters (as long as cs_peak is above 1):
We compute both measures and compare them, together with the peak SF, between ROIs. The SF cutoff of broad CSFs can lie far beyond the highest SF in the stimulus, where it is an extrapolation, so we compare the logarithm of the SFs.
width_l = 0.68
log_cs_peak = np.log10(cs_peak)
csf_measures = pd.DataFrame({
"log10_sf_peak": np.log10(sf_peak),
"log10_sf_max": np.log10(sf_peak) + np.sqrt(log_cs_peak) / width_r,
"aulcsf": 2.0 / 3.0 * log_cs_peak**1.5 * (1.0 / width_l + 1.0 / width_r),
})
fig, axes = plt.subplots(1, 3, figsize=(15, 4), layout="constrained")
for ax, name, label in zip(
axes,
csf_measures.columns,
["Peak SF (log10 of cycles/degree)", "SF cutoff (log10 of cycles/degree)", "AULCSF"],
):
plot_parameter_by_roi(csf_measures[is_selected], name, roi_valid[is_selected], order=selected_rois, ax=ax)
style_axes(ax, "ROI", label)
Finally, we look at how the CSF changes across the visual field. In the visual system, acuity is highest at the center of the visual field and falls off towards the periphery. Do the CSFs of the neuron populations follow the same pattern? We do not know the pRFs of the vertices from the csf task, but we can use the eccentricity that the template atlas predicts for each vertex. The gratings covered the screen up to about 5 degrees from its center (the bars in the prf task cover about -5.1 to 5.1 degrees), so we only look at vertices with an eccentricity below 5 degrees. We plot the median peak SF in bins of 1 degree for V1, V2, and V3.
eccentricity = dataset.units["benson14_eccen"].to_numpy()[response_is_valid]
eccentricity_bins = np.arange(0.0, 6.0, 1.0)
bin_centers = (eccentricity_bins[:-1] + eccentricity_bins[1:]) / 2
fig, ax = plt.subplots(figsize=(7, 4.5), layout="constrained")
for name in ["V1", "V2", "V3"]:
is_roi = is_selected & (roi_valid == name) & (eccentricity < eccentricity_bins[-1])
# Bin of every selected vertex in the ROI
bin_index = np.digitize(eccentricity[is_roi], eccentricity_bins) - 1
median_sf_peak = [
np.median(sf_peak[is_roi][bin_index == i]) if np.sum(bin_index == i) >= 20 else np.nan
for i in range(len(bin_centers))
]
ax.plot(bin_centers, median_sf_peak, "o-", color=roi_colors[name], label=f"{name} (n = {is_roi.sum()})")
ax.legend(frameon=False)
ax.set_title("Peak SF across the visual field")
style_axes(ax, "Eccentricity of the template atlas (in degrees)", "Median peak SF (in cycles/degree)")
Exercise 17
How does the peak SF change with eccentricity? Do the estimated CSFs agree with the results reported by Roelofzen et al. (2025)?
Solution
In V1, V2, and V3, the median peak SF is highest close to the center of the visual field (about 2.7 to 3.7 cycles per degree below 1 degree eccentricity) and decreases with eccentricity, especially within the central 2 degrees. Beyond about 2 degrees, the peak SF levels off in our data. The peak SF is also higher in V1 than in V2 and V3 at all eccentricities.
Roelofzen et al. (2025) also report that the peak SF decreases with eccentricity, in line with the decrease of visual acuity from the center to the periphery of the visual field. Our analysis has several limitations, though: The eccentricity comes from a template atlas instead of a measurement in this subject, the stimulus only covers the central 5 degrees, and the selection only includes vertices that fit well. In Exercise 19, we replace the template eccentricity by the eccentricity of pRFs that we estimate from the prf task.
Exercise 18 (optional, advanced)
In this practical, we fitted the model to the response averaged over all runs and evaluated it on the same data. The dataset also contains the average of the odd and of the even runs as separate splits (load_dataset("7t-ncsf-visual", task="csf", split="odd")). Following the second practical, fit the nCSF model to the odd runs and evaluate its predictions on the even runs. Then, fix slope_crf for all vertices at the median of its estimates (i.e., use a single value for slope_crf in the grid and add it to the fixed_parameters of SGD) and repeat. Which of the two models predicts the even runs better?
Solution
We load both splits and select the same vertices as for the full dataset. Both splits were measured with the same stimulus as the full dataset.
dataset_odd = load_dataset("7t-ncsf-visual", task="csf", split="odd")
dataset_even = load_dataset("7t-ncsf-visual", task="csf", split="even")
response_odd = dataset_odd.response[response_is_valid]
response_even = dataset_even.response[response_is_valid]
# Mean of each timecourse
print("Odd runs:", np.percentile(response_odd.mean(axis=1), [5, 50, 95]).round(2))
print("Even runs:", np.percentile(response_even.mean(axis=1), [5, 50, 95]).round(2))
The odd runs and the even runs were processed differently: The mean of every timecourse is zero for the even runs (like for the average of all runs), but differs between vertices for the odd runs. This offset has nothing to do with the CSF, but it reduces the R-squared on held-out data when the training and test data have a different mean. We therefore remove the mean from the observed and the predicted timecourses before we compute the R-squared on the test data.
We wrap all three fitting stages in a function, so that we can fit both models in the same way.
def fit_ncsf(
response_train: np.ndarray, grid_values: dict, fixed_parameters: list[str],
) -> tuple[pd.DataFrame, np.ndarray]:
"""Fit the nCSF model with grid search, least-squares, and SGD; return the estimates and the in-sample R-squared."""
_, params_grid = GridFitter(model=csf_model, stimulus=stimulus, compile_step=True).fit(
data=response_train, parameter_values=grid_values, batch_size=100,
)
_, params_ls = LeastSquaresFitter(model=csf_model, stimulus=stimulus).fit(
data=response_train, parameters=params_grid, slope_name="amplitude", intercept_name="baseline", batch_size=1000,
)
# Keep the bounded parameters that are not fixed between their bounds
fit_adapter = Adapter([
box_transform(name, lower, upper)
for name, (lower, upper) in parameter_bounds.items()
if name not in fixed_parameters
])
_, params_sgd = SGDFitter(
model=csf_model, stimulus=stimulus, adapter=fit_adapter, optimizer=Adam(learning_rate=0.01), compile_step=True,
).fit(
data=response_train, init_parameters=params_ls, fixed_parameters=fixed_parameters, num_steps=500,
)
r_squared_train = calculate_r_squared(response_train, np.asarray(predict_batched(stimulus, params_sgd, batch_size=1000)))
return params_sgd, r_squared_train
def calculate_r_squared_without_mean(observed: np.ndarray, predicted: np.ndarray) -> np.ndarray:
"""Calculate the R-squared after removing the mean of each observed and predicted timecourse."""
return calculate_r_squared(
observed - observed.mean(axis=1, keepdims=True), predicted - predicted.mean(axis=1, keepdims=True),
)
First, we fit the model with a free slope_crf to the odd runs and evaluate it on the even runs.
params_free, r_squared_train_free = fit_ncsf(response_odd, grid_param_ranges, ["weight_deriv"])
r_squared_test_free = calculate_r_squared_without_mean(
response_even, np.asarray(predict_batched(stimulus, params_free, batch_size=1000)),
)
Then, we fix slope_crf at the median of its estimates for the vertices that the first model fits well on the odd runs. We do not use the even runs to choose this value, so that they stay independent from the training data.
slope_crf_fixed = np.median(params_free.loc[r_squared_train_free > 0.3, "slope_crf"])
print(f"slope_crf fixed at {slope_crf_fixed:.2f}")
params_fixed, r_squared_train_fixed = fit_ncsf(
response_odd, grid_param_ranges | {"slope_crf": [slope_crf_fixed]}, ["weight_deriv", "slope_crf"],
)
r_squared_test_fixed = calculate_r_squared_without_mean(
response_even, np.asarray(predict_batched(stimulus, params_fixed, batch_size=1000)),
)
We compare both models on the vertices that at least one of the models fits well on the training data.
is_compared = (r_squared_train_free > 0.3) | (r_squared_train_fixed > 0.3)
fig, axes = plt.subplots(1, 2, figsize=(12, 5), layout="constrained")
plot_r_squared_comparison(
r_squared_train_free[is_compared], r_squared_test_free[is_compared], value_range=(-0.5, 1.0),
xlabel="R-squared (odd runs)", ylabel="R-squared (even runs)", ax=axes[0],
)
axes[0].set_title("Free slope_crf: training vs. test")
plot_r_squared_comparison(
r_squared_test_free[is_compared], r_squared_test_fixed[is_compared], value_range=(-0.5, 1.0),
xlabel="R-squared on even runs (free slope_crf)", ylabel="R-squared on even runs (fixed slope_crf)", ax=axes[1],
)
axes[1].set_title("Free vs. fixed slope_crf on the test data")
pd.DataFrame({
"free slope_crf": pd.Series(r_squared_test_free[is_compared]).groupby(roi_valid[is_compared]).median(),
"fixed slope_crf": pd.Series(r_squared_test_fixed[is_compared]).groupby(roi_valid[is_compared]).median(),
"fraction better with fixed slope_crf": pd.Series(
r_squared_test_fixed[is_compared] > r_squared_test_free[is_compared]
).groupby(roi_valid[is_compared]).mean(),
}).loc[roi_order].round(3)
With a free slope_crf, the R-squared on the even runs is a bit lower than on the odd runs for most vertices, but the two are closely related: The nCSF model captures a pattern in the response that we can reproduce in independent runs, and not just the noise of the training runs.
Fixing slope_crf (here at about 1.2) hardly changes the R-squared on the even runs: In all ROIs, the median R-squared of both models differs by less than 0.01, and the fixed model is better for about half of the vertices in most ROIs (for about two thirds in V1 and TO2, and for about 40% in V3a and V3b). Thus, a single slope_crf for all vertices predicts held-out data as well as a separate slope_crf for every vertex. Following the third practical, we would prefer the simpler model with a fixed slope_crf: It has one parameter less per vertex, and its other estimates do not trade off against slope_crf.
# Your code goes here
Exercise 19 (optional, advanced)
Fit the canonical Gaussian 2D pRF model from the first practical to the prf task of the dataset (load_dataset("7t-ncsf-visual", task="prf")). How does the peak SF of the nCSF model relate to the eccentricity and size of the estimated pRFs?
Solution
from prfmodel.fitters import grid_values_2d_prf
from prfmodel.models.prf import Gaussian2DPRFModel
from prfmodel.utils import calculate_eccentricity
dataset_prf = load_dataset("7t-ncsf-visual", task="prf")
stimulus_prf = dataset_prf.stimulus
response_prf = dataset_prf.response[response_is_valid] # Same vertices as for the nCSF model
print(stimulus_prf)
We fit the canonical Gaussian 2D pRF model with a grid search and least-squares like in the first practical.
prf_model = Gaussian2DPRFModel()
grid_param_ranges_prf = grid_values_2d_prf(
stimulus_prf,
num_mu=21,
mu_extent=2.0,
num_sigma=10,
sigma_range=(0.1, 5.0),
) | {
"weight_deriv": [-0.5],
"baseline": [0.0],
"amplitude": [1.0],
}
_, grid_params_prf = GridFitter(model=prf_model, stimulus=stimulus_prf, compile_step=True).fit(
data=response_prf, parameter_values=grid_param_ranges_prf, batch_size=20,
)
_, ls_params_prf = LeastSquaresFitter(model=prf_model, stimulus=stimulus_prf).fit(
data=response_prf, parameters=grid_params_prf, slope_name="amplitude", intercept_name="baseline", batch_size=1000,
)
r_squared_prf = calculate_r_squared(
response_prf, np.asarray(batched(prf_model)(stimulus_prf, ls_params_prf, batch_size=1000)),
)
We select vertices with a good fit of both models whose pRF center lies inside the stimulus and relate the peak SF to the eccentricity and size of the pRF.
prf_mu_x = ls_params_prf["mu_x"].to_numpy()
prf_mu_y = ls_params_prf["mu_y"].to_numpy()
prf_sigma = ls_params_prf["sigma"].to_numpy()
prf_eccentricity = calculate_eccentricity(prf_mu_x, prf_mu_y)
grid_min_xy, grid_max_xy = stimulus_prf.grid.min(axis=(0, 1)), stimulus_prf.grid.max(axis=(0, 1))
is_prf_selected = (
(r_squared_prf > 0.3)
# Center inside the stimulus (the grid stores y in grid[..., 0] and x in grid[..., 1])
& (prf_mu_x >= grid_min_xy[1]) & (prf_mu_x <= grid_max_xy[1])
& (prf_mu_y >= grid_min_xy[0]) & (prf_mu_y <= grid_max_xy[0])
# Least squares does not change sigma, so it is always one of the grid values
& ~np.isclose(prf_sigma, np.max(grid_param_ranges_prf["sigma"]))
)
is_both_selected = is_selected & is_prf_selected
print(f"{is_both_selected.sum()} vertices are selected for both models")
fig, axes = plt.subplots(1, 2, figsize=(12, 4.5), sharey=True, layout="constrained")
for name in ["V1", "V2", "V3"]:
is_roi = is_both_selected & (roi_valid == name)
axes[0].scatter(prf_eccentricity[is_roi], sf_peak[is_roi], s=6, alpha=0.5, color=roi_colors[name], label=name)
axes[1].scatter(prf_sigma[is_roi], sf_peak[is_roi], s=6, alpha=0.5, color=roi_colors[name], label=name)
axes[0].set_yscale("log")
axes[0].legend(frameon=False)
style_axes(axes[0], "pRF eccentricity (in degrees)", "Peak SF (in cycles/degree)")
axes[1].set_xscale("log")
style_axes(axes[1], "pRF size (sigma, in degrees)", "Peak SF (in cycles/degree)")
# Rank correlation of the peak SF with the pRF eccentricity and size within each ROI
correlations = {}
for name in ["V1", "V2", "V3"]:
is_roi = is_both_selected & (roi_valid == name)
correlations[name] = {
"number of vertices": is_roi.sum(),
"eccentricity": pd.Series(prf_eccentricity[is_roi]).corr(pd.Series(sf_peak[is_roi]), method="spearman"),
"size": pd.Series(prf_sigma[is_roi]).corr(pd.Series(sf_peak[is_roi]), method="spearman"),
}
pd.DataFrame(correlations).round(2)
The prf task has the same vertices as the csf task, but some valid vertices have a constant response in the prf task. The fitters exclude them with a warning and return NaN parameters for them, so they are not selected.
In V1, V2, and V3, the peak SF clearly decreases with the eccentricity of the pRF: Vertices with pRFs close to the center of the visual field prefer high SFs, and vertices with pRFs in the periphery prefer low SFs. The rank correlation between peak SF and eccentricity is about -0.4 in V1 and about -0.7 to -0.8 in V2 and V3. The relationship is clearer than with the eccentricity of the template atlas in Exercise 17 because the pRF estimates come from measurements in this subject.
The peak SF also decreases with the size of the pRF: Neuron populations with small pRFs prefer fine patterns, and populations with large pRFs prefer coarse patterns. Note that pRF size increases with eccentricity, so the two relationships are not independent of each other. Also note that the pRF estimates only take the values of the grid because we did not finetune them with SGD.
# Your code goes here
Summary#
The nCSF model describes to which SFs and contrasts a neuron population responds. Its tuning model combines a CSF (an asymmetric log-parabola with the parameters
cs_peak,sf_peak, andwidth_r) with a contrast response function (a Naka-Rushton function with the exponentslope_crf).Unlike pRF models, the nCSF model does not need a stimulus grid or a stimulus encoding step. The stimulus only describes the SF and contrast of a full-screen grating at every time frame.
We fit the nCSF model with the same stages as pRF models: grid search, least-squares, and SGD. During SGD, we keep the CSF parameters between bounds, because the data hardly constrain them beyond the range of SFs and contrasts in the stimulus.
The estimated CSFs differ between visual areas and across the visual field: The peak SF is highest in early visual cortex and decreases with eccentricity.
References#
Benson, N. C., Butt, O. H., Brainard, D. H., & Aguirre, G. K. (2014). Correction of distortion in flattened representations of the cortical surface allows prediction of V1-V3 functional organization from anatomy. PLoS Computational Biology, 10(3), e1003538. https://doi.org/10.1371/journal.pcbi.1003538
Roelofzen, C., Daghlian, M., van Dijk, J. A., de Jong, M. C., & Dumoulin, S. O. (2025). Modeling neural contrast sensitivity functions in human visual cortex. Imaging Neuroscience, 3, imag_a_00469. https://doi.org/10.1162/imag_a_00469
Van Essen, D. C., Ugurbil, K., Auerbach, E., Barch, D., Behrens, T. E. J., Bucholz, R., Chang, A., Chen, L., Corbetta, M., Curtiss, S. W., Della Penna, S., Feinberg, D., Glasser, M. F., Harel, N., Heath, A. C., Larson-Prior, L., Marcus, D., Michalareas, G., Moeller, S., … WU-Minn HCP Consortium. (2012). The Human Connectome Project: A data acquisition perspective. NeuroImage, 62(4), 2222-2231. https://doi.org/10.1016/j.neuroimage.2012.02.018