Refactors

Refactoring ExeTera

Out of date page!!!

This page is gradually becoming out of date, and the reader should refer to Roadmap instead

A number of high-level, big impact changes need to be made to the ExeTera codebase. These are required to make ExeTera API more generalisable and easier to use, but allow to allow for upcoming changes to be carried out with less impact, both in terms of code changes and in terms of changes to the API.

The following major refactors are planned:

• Remove all covid-specific code from ExeTera into ExeTeraCovid
• Move all data access inside the Session object
• Provide rich functionality to groups and fields
• Move ExeTera away from HDF5 as a storage mechanism
  • Provide file-system-based datastore
  • Provide server-based datastore

Provide rich functionality to groups and fields

Problem examples with hypothetical improvements

Apply filter to field

From

s = Session()
src = s.open("my/dataset", "src")
flt = # calculate a filter
f = s.get(src["table/field"])
s.apply_filter(flt, f)

To

s = Session()
src = s.open("my/dataset", "src")
flt = # calculate a filter
f = src["table/field"]
f.apply_filter(flt)

or

s = Session()
src = s.open("my/dataset", "src")
flt = # calculate a filter
src["table/field"].apply_filter(flt)

Apply filter to table/group

From

s = Session()
src = s.open("my/dataset", "src")
flt = # calculate a filter
for k, v in src['table'].items():
    s.apply_filter(flt, s.get(v))

To

s = Session()
src = s.open("my/dataset", "src")
flt = # calculate a filter
tab = src['table']
tab.apply_filter(flt)

or

s = Session()
src = s.open("my/dataset", "src")
flt = # calculate a filter
src['table'].apply_filter(flt)