Error when running analysis on ACCESS-ESM1-5 data

[jumin94]

Hello! I am not 100% new to ESMValTool, but I think that I was just lucky with my first recipes and didn't encounter the problems I am encountering now.

I had previously run a recipe using ACCESS-ESM1-5 data from CMIP6 with no problems. Now, I find the problem that you can see below. I am wondering if there's a general way to address these issues (so that I don't have to start a discussion every time I have an error like this)

>`2024-08-06 13:35:57,161 UTC [129573] ERROR   Failed to run preprocessor function 'add_supplementary_variables' on the data
[<iris 'Cube' of air_temperature / (K) (time: 1800; air_pressure: 19; latitude: 144; longitude: 192)>]
loaded from original input file(s)
[LocalFile('/climca/data/CMIP6/CMIP/CSIRO/ACCESS-ESM1-5/historical/r1i1p1f1/Amon/ta/gn/v20191115/ta_Amon_ACCESS-ESM1-5_historical_r1i1p1f1_gn_195001-201412.nc'),
 LocalFile('/climca/data/CMIP6/ScenarioMIP/CSIRO/ACCESS-ESM1-5/ssp585/r1i1p1f1/Amon/ta/gn/v20210318/ta_Amon_ACCESS-ESM1-5_ssp585_r1i1p1f1_gn_201501-210012.nc'),
 LocalFile('/climca/data/CMIP6/CMIP/CSIRO/ACCESS-ESM1-5/historical/r1i1p1f1/fx/areacella/gn/v20191115/areacella_fx_ACCESS-ESM1-5_historical_r1i1p1f1_gn.nc')]
with function argument(s)
supplementary_cubes = [<iris 'Cube' of cell_area / (m2) (latitude: 145; longitude: 192)>]
2024-08-06 13:35:57,897 UTC [129378] INFO    Maximum memory used (estimate): 1.1 GB
2024-08-06 13:35:57,899 UTC [129378] INFO    Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
2024-08-06 13:35:57,901 UTC [129378] ERROR   Program terminated abnormally, see stack trace below for more information:
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/_task.py", line 816, in _run_task
    output_files = task.run()
                   ^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/_task.py", line 264, in run
    self.output_files = self._run(input_files)
                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/preprocessor/__init__.py", line 661, in _run
    product.apply(step, self.debug)
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/preprocessor/__init__.py", line 469, in apply
    self.cubes = preprocess(self.cubes, step,
                            ^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/preprocessor/__init__.py", line 479, in cubes
    self._cubes = [ds.load() for ds in self.datasets]
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/preprocessor/__init__.py", line 479, in <listcomp>
    self._cubes = [ds.load() for ds in self.datasets]
                   ^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/dataset.py", line 692, in load
    cubes = preprocess(
            ^^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/preprocessor/__init__.py", line 378, in preprocess
    result.append(_run_preproc_function(function, item, settings,
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/preprocessor/__init__.py", line 323, in _run_preproc_function
    return function(items, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/preprocessor/_supplementary_vars.py", line 134, in add_supplementary_variables
    add_cell_measure(cube, supplementary_cube, measure_name)
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/preprocessor/_supplementary_vars.py", line 76, in add_cell_measure
    cube.add_cell_measure(measure, range(start_dim, cube.ndim))
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/iris/cube.py", line 1640, in add_cell_measure
    data_dims = self._check_multi_dim_metadata(cell_measure, data_dims)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/iris/cube.py", line 1523, in _check_multi_dim_metadata
    raise iris.exceptions.CannotAddError(
iris.exceptions.CannotAddError: Unequal lengths. Cube dimension 2 => 144; metadata 'cell_area' dimension 0 => 145.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/_main.py", line 518, in run
    fire.Fire(ESMValTool())
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/fire/core.py", line 143, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/fire/core.py", line 477, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/_main.py", line 405, in run
    self._run(recipe, session)
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/_main.py", line 447, in _run
    process_recipe(recipe_file=recipe, session=session)
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/_main.py", line 130, in process_recipe
    recipe.run()
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/_recipe/recipe.py", line 1075, in run
    self.tasks.run(max_parallel_tasks=self.session['max_parallel_tasks'])
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/_task.py", line 738, in run
    self._run_parallel(address, max_parallel_tasks)
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/_task.py", line 782, in _run_parallel
    _copy_results(task, running[task])
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/site-packages/esmvalcore/_task.py", line 805, in _copy_results
    task.output_files, task.products = future.get()
                                       ^^^^^^^^^^^^
  File "/opt/miniforge3/envs/esmvaltool/lib/python3.11/multiprocessing/pool.py", line 774, in get
    raise self._value
iris.exceptions.CannotAddError: Unequal lengths. Cube dimension 2 => 144; metadata 'cell_area' dimension 0 => 145.
2024-08-06 13:35:57,907 UTC [129378] INFO    `

[bouweandela]

@rbeucher Would you or one of your team be able to help out here? It looks like the areacella file is on a different grid from the ta file.

[rbeucher]

@flicj191 ? Do you think you can have a look?

[jumin94]

I was able to run the recipe with the following instruction to avoid using the areacello file. I imagine that this is not the best way to go here, but it allowed me to run the recipe. I still haven't looked into it, but I am aware that I have to check if the regridding went well.

supplementary_variables: - short_name: areacella skip: true

[flicj191]

I'll have a look once our systems is up and accessible