The intake-esgf paradigm is meant to somewhat hide from you the notion of files. You search for datasets and the related files are loaded and aggregated into a dictionary of xarray Datasets. However, many times your intended use of any given dataset is only part of the whole time series. While there is not a consistent convention throughout ESGF, many centers will save the variables in separate files spanning portions of the total time span. Thus you may end up downloading many files that you will never touch.
In order to avoid downloading all the files, when searching you may specify
file_start and/or file_end. Internally, we will save these time stamps and
when obtaining file information for loading/downloading, only touch files that
fall within some part of that span. If, for example, we only wish to analyze
data from the 60’s through the 90’s, we can add the following to our search:
from intake_esgf import ESGFCatalog
cat = ESGFCatalog().search(
experiment_id="historical",
variable_id="msftmz",
source_id="NorESM2-LM",
variant_label="r2i1p1f1",
file_start="1960-01",
file_end="1999-12"
)
dsd = cat.to_dataset_dict()The NorESM2-LM model tends to heavily split up their model output, in this case by decades. However, notice the time span listed when we print the resulting dataset:
dsd["msftmz"]Loading...
It is only that span which contains our desired date range. You do not need to provide both timestamps. If you want to check what intake-esgf filtered, we write these out in the session log:
print(cat.session_log())2026-05-26 19:05:28 search begin experiment_id=['historical'], variable_id=['msftmz'], source_id=['NorESM2-LM'], variant_label=['r2i1p1f1'], type=['Dataset'], project=['CMIP6'], latest=[True], retracted=[False]
2026-05-26 19:05:28 └─GlobusESGFIndex('ESGF2-US-1.5-Catalog') results=2 response_time=0.15
2026-05-26 19:05:28 combine_time=0.01
2026-05-26 19:05:28 search end total_time=0.17
2026-05-26 19:05:28 file info begin
2026-05-26 19:05:28 └─GlobusESGFIndex('ESGF2-US-1.5-Catalog') results=34 response_time=0.11
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_185001-185912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_185001-185912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_186001-186912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_186001-186912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_187001-187912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_187001-187912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_188001-188912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_188001-188912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_189001-189912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_189001-189912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_190001-190912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_190001-190912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_191001-191912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_191001-191912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_192001-192912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_192001-192912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_193001-193912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_193001-193912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_194001-194912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_194001-194912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_195001-195912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_195001-195912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_200001-200912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_200001-200912.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_201001-201412.nc
2026-05-26 19:05:28 Filtered for out of time range: CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_201001-201412.nc
2026-05-26 19:05:28 combine_time=0.00
2026-05-26 19:05:28 file info end total_time=0.12
2026-05-26 19:05:29 transfer_time=0.42 [s] at 25.00 [Mb s-1] https://g-52ba3.fd635.8443.data.globus.org/css03_data/CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_197001-197912.nc
2026-05-26 19:05:29 transfer_time=0.47 [s] at 22.55 [Mb s-1] https://g-52ba3.fd635.8443.data.globus.org/css03_data/CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_199001-199912.nc
2026-05-26 19:05:29 transfer_time=0.47 [s] at 22.65 [Mb s-1] https://g-52ba3.fd635.8443.data.globus.org/css03_data/CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_196001-196912.nc
2026-05-26 19:05:29 transfer_time=0.47 [s] at 22.54 [Mb s-1] https://g-52ba3.fd635.8443.data.globus.org/css03_data/CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_198001-198912.nc
2026-05-26 19:05:29 accessed /home/docs/.esgf/CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_196001-196912.nc
2026-05-26 19:05:29 accessed /home/docs/.esgf/CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_197001-197912.nc
2026-05-26 19:05:29 accessed /home/docs/.esgf/CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_198001-198912.nc
2026-05-26 19:05:29 accessed /home/docs/.esgf/CMIP6/CMIP/NCC/NorESM2-LM/historical/r2i1p1f1/Omon/msftmz/grz/v20190920/msftmz_Omon_NorESM2-LM_historical_r2i1p1f1_grz_199001-199912.nc