uproot4 error with coffea:local backend

[asnaylor]

I tried to use the coffea local backend which has worked for me previously:

$ fast_carpenter dataset.yml local_cfg.yml --no-bookkeeping --mode coffea:local

But today I got this error message about uproot4 not having MultithreadedFileSource.

Trace:

Preprocessing: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:02<00:00,  2.00s/file]
Processing:   0%|                                                                                                                                       | 0/1 [00:00<?, ?chunk/s]
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 129, in __call__
    out = self.function(*args, **kwargs)
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 872, in _work_function
    raise e
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 790, in _work_function
    file_handler=uproot4.MemmapSource if mmap else uproot4.MultithreadedFileSource,
AttributeError: module 'uproot4' has no attribute 'MultithreadedFileSource'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/bin/fast_carpenter", line 8, in <module>
    sys.exit(main())
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/fast_carpenter/__main__.py", line 72, in main
    results, _ = backend.execute(sequence, datasets, args)
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/fast_carpenter/backends/coffea.py", line 175, in execute
    out = cop.run_uproot_job(coffea_datasets, 'events', fp,
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 1123, in run_uproot_job
    executor(chunks, closure, wrapped_out, **exe_args)
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 557, in futures_executor
    _futures_handler(futures, accumulator, status, unit, desc, add_fn, tailtimeout)
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/site-packages/coffea/processor/executor.py", line 187, in _futures_handler
    add_fn(output, finished.pop().result())
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/home/anaylor/.pyenv/versions/miniconda3-4.3.30/envs/notebook_env/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
AttributeError: module 'uproot4' has no attribute 'MultithreadedFileSource'

Versions:

python_version=3.8.5
fast-carpenter==0.19.0
coffea==0.6.45

[asnaylor]

The coffea team introduced uproot4 in version 0.6.43 so I tested the above command with 0.6.42 and it worked fine. Until they have a fix, you can just pin fast_carpenter to use coffea==0.6.42.

[kreczko]

uproot 4.0.0rc2 is now available, together with awkward 1.0.0rc2.

I've now released scikit-validate 0.4.0rc1 against the two above, so I can start looking at this here next.

@benkrikler What are your plans w.r.t. the new uproot and awkward versions?
They have now moved into a stable territory, so we can try to move.
BTW: This might certainly spell the end of Python 2 support (FINALLY!)

[benkrikler]

I want to move over to that as soon as possible, and fully migrate from python 2 at the same time (we're already no longer testing support there, but I think we might still have some six things and from __future__ stuff going on). If you want to tackle moving us to uproot 4 already then that would be great!

[kreczko]

The current main issue with coffea: There is no simple way anymore to extract the tree.

I will try today to use the df directly, but I need to check other code dependencies first.

[kreczko]

This will be fixed in #144