Software Architecture

The main goal of HDMF is to enable users and developers to efficiently interact with the hierarchical object data. The following figures provide an overview of the high-level architecture of HDMF and functionality of the various components.

HDMF Software Architecture

Overview of the high-level software architecture of HDMF (click to enlarge).

HDMF Software Architecture Functions

We choose a modular design for HDMF to enable flexibility and separate the various levels of standardizing hierarchical data (click to enlarge).

Main Concepts

HDMF Software Architecture Concepts

Overview of the main concepts/classes in HDMF and their location in the overall software architecture (click to enlarge).

Container

  • In memory objects

  • Interface for (most) applications

  • Similar to a table row

  • HDMF does not provide these. They are left for standards developers to define how users interact with data.

  • There are two Container base classes:

    • Container - represents a collection of objects
    • Data - represents data
  • Main Module: hdmf.container

Builder

Spec

  • Interact with format specifications

  • Data structures to specify data types and what said types consist of

  • Python representation for YAML specifications

  • Interface for writing extensions or custom specification

  • There are several main specification classes:

    • AttributeSpec - specification for metadata
    • GroupSpec - specification for a collection of objects (i.e. subgroups, datasets, link)
    • DatasetSpec - specification for dataset (like and n-dimensional array). Specifies data type, dimensions, etc.
    • LinkSpec - specification for link (like a POSIX soft link)
    • RefSpec - specification for references (References are like links, but stored as data)
    • DtypeSpec - specification for compound data types. Used to build complex data type specification, e.g., to define tables (used only in DatasetSpec and correspondingly DatasetSpec)
  • Main Modules: hdmf.spec

Note

A data_type defines a reusable type in a format specification that can be referenced and used elsewhere in other specifications. The specification of the standard is basically a collection of data_types,

  • data_type_inc is used to include an existing type and
  • data_type_def is used to define a new type

i.e, if both keys are defined then we create a new type that uses/inherits an existing type as a base.

ObjectMapper

  • Maintains the mapping between Container attributes and Spec components

  • Provides a way of converting between Container and Builder, while leaving standards developers with the flexibility of presenting data to users in a user-friendly manner, while storing data in an efficient manner

  • ObjectMappers are constructed using a Spec

  • Ideally, one ObjectMapper for each data type

  • Things an ObjectMapper should do:

    • Given a Builder, return a Container representation
    • Given a Container, return a Builder representation
  • Main Module: hdmf.build.objectmapper

HDMF Software Architecture Main Concepts

Relationship between Container, Builder, ObjectMapper, and Spec

Additional Concepts

Namespace, NamespaceCatalog, NamespaceBuilder

  • Namespace

    • A namespace for specifications
    • Necessary for making standards extensions and standard core specification
    • Contains basic info about who created extensions
  • NamespaceCatalog – A class for managing namespaces

  • NamespaceBuilder – A utility for building extensions

TypeMap

  • Map between data types, Container classes (i.e. a Python class object) and corresponding ObjectMapper classes

  • Constructed from a NamespaceCatalog

  • Things a TypeMap does:

    • Given a data_type, return the associated Container class
    • Given a Container class, return the associated ObjectMapper
  • HDMF has one of these classes:

  • TypeMaps can be merged, which is useful when combining extensions

BuildManager

HDMF Software Architecture BuildManager and TypeMap

Overview of BuildManager (and TypeMap) (click to enlarge).

HDMFIO

  • Abstract base class for I/O

  • HDMFIO has two key abstract methods:

  • Constructed with a BuildManager

  • Extend this for creating a new I/O backend

  • HDMF has one concrete form of this:

    • HDF5IO - reading and writing HDF5
HDMF Software Architecture FormIO

Overview of HDMFIO (click to enlarge).