Software Architecture¶
The main goal of HDMF is to enable users and developers to efficiently interact with the hierarchical object data. The following figures provide an overview of the high-level architecture of HDMF and functionality of the various components.
Main Concepts¶
Container¶
In memory objects
Interface for (most) applications
Similar to a table row
HDMF does not provide these. They are left for standards developers to define how users interact with data.
There are two Container base classes:
Main Module:
hdmf.container
Builder¶
Intermediary objects for I/O
Interface for I/O
Backend readers and writers must return and accept these
There are different kinds of builders for different base types:
GroupBuilder
- represents a collection of objectsDatasetBuilder
- represents dataLinkBuilder
- represents soft-linksRegionBuilder
- represents a slice into data (Subclass ofDatasetBuilder
)
Main Module:
hdmf.build.builders
Spec¶
Interact with format specifications
Data structures to specify data types and what said types consist of
Python representation for YAML specifications
Interface for writing extensions or custom specification
There are several main specification classes:
AttributeSpec
- specification for metadataGroupSpec
- specification for a collection of objects (i.e. subgroups, datasets, link)DatasetSpec
- specification for dataset (like and n-dimensional array). Specifies data type, dimensions, etc.LinkSpec
- specification for link (like a POSIX soft link)RefSpec
- specification for references (References are like links, but stored as data)DtypeSpec
- specification for compound data types. Used to build complex data type specification, e.g., to define tables (used only inDatasetSpec
and correspondinglyDatasetSpec
)
Main Modules:
hdmf.spec
Note
A data_type
defines a reusable type in a format specification that can be
referenced and used elsewhere in other specifications. The specification of
the standard is basically a collection of data_types
,
data_type_inc
is used to include an existing type anddata_type_def
is used to define a new type
i.e, if both keys are defined then we create a new type that uses/inherits an existing type as a base.
ObjectMapper¶
Maintains the mapping between Container attributes and Spec components
Provides a way of converting between Container and Builder, while leaving standards developers with the flexibility of presenting data to users in a user-friendly manner, while storing data in an efficient manner
ObjectMappers are constructed using a Spec
Ideally, one ObjectMapper for each data type
Things an ObjectMapper should do:
Main Module:
hdmf.build.objectmapper
Additional Concepts¶
Namespace, NamespaceCatalog, NamespaceBuilder¶
Namespace
A namespace for specifications
Necessary for making standards extensions and standard core specification
Contains basic info about who created extensions
NamespaceCatalog
– A class for managing namespacesNamespaceBuilder
– A utility for building extensions
TypeMap¶
Map between data types, Container classes (i.e. a Python class object) and corresponding ObjectMapper classes
Constructed from a NamespaceCatalog
Things a TypeMap does:
Given a data_type, return the associated Container class
Given a Container class, return the associated ObjectMapper
HDMF has one of these classes:
the base class (i.e.
TypeMap
)
TypeMaps can be merged, which is useful when combining extensions
BuildManager¶
Constructed from a TypeMap
HDMF only has one of these:
hdmf.build.manager.BuildManager
HDMFIO¶
Abstract base class for I/O
HDMFIO
has two key abstract methods:write_builder
– given a builder, write data to storage formatread_builder
– given a handle to storage format, return builder representation
Constructed with a BuildManager
Extend this for creating a new I/O backend
HDMF has one concrete form of this:
HDF5IO
- reading and writing HDF5