hdmf.common.resources module

class hdmf.common.resources.KeyTable(name='keys', data=[])

Bases: Table

A table for storing keys used to reference external resources.

Parameters:
add_row(key)
Parameters:

key (str) – The user key that maps to the resource term / registry symbol.

class hdmf.common.resources.Key(key, table=None, idx=None)

Bases: Row

A Row class for representing rows in the KeyTable.

Parameters:

  • key (str) – The user key that maps to the resource term / registry symbol.

  • table (Table) – None

  • idx (int) – None

todict()
class hdmf.common.resources.EntityTable(name='entities', data=[])

Bases: Table

A table for storing the external resources a key refers to.

Parameters:
add_row(entity_id, entity_uri)
Parameters:

  • entity_id (str) – The unique ID for the resource term / registry symbol.

  • entity_uri (str) – The URI for the resource term / registry symbol.

class hdmf.common.resources.Entity(entity_id, entity_uri, table=None, idx=None)

Bases: Row

A Row class for representing rows in the EntityTable.

Parameters:

  • entity_id (str) – The unique ID for the resource term / registry symbol.

  • entity_uri (str) – The URI for the resource term / registry symbol.

  • table (Table) – None

  • idx (int) – None

todict()
class hdmf.common.resources.FileTable(name='files', data=[])

Bases: Table

A table for storing file ids used in external resources.

Parameters:
add_row(file_object_id)
Parameters:

file_object_id (str) – The file id of the file that contains the object

class hdmf.common.resources.File(file_object_id, table=None, idx=None)

Bases: Row

A Row class for representing rows in the FileTable.

Parameters:

  • file_object_id (str) – The file id of the file that contains the object

  • table (Table) – None

  • idx (int) – None

todict()
class hdmf.common.resources.ObjectTable(name='objects', data=[])

Bases: Table

A table for storing objects (i.e. Containers) that contain keys that refer to external resources.

Parameters:
add_row(files_idx, object_id, object_type, relative_path, field)
Parameters:

  • files_idx (int) – The row idx for the file_object_id in FileTable containing the object.

  • object_id (str) – The object ID for the Container/Data.

  • object_type (str) – The type of the object. This is also the parent in relative_path.

  • relative_path (str) – (‘The relative_path of the attribute of the object that uses ‘, ‘an external resource reference key. Use an empty string if not applicable.’)

  • field (str) – The field of the compound data type using an external resource. Use an empty string if not applicable.

class hdmf.common.resources.Object(files_idx, object_id, object_type, relative_path, field, table=None, idx=None)

Bases: Row

A Row class for representing rows in the ObjectTable.

Parameters:

  • files_idx (int) – The row idx for the file_object_id in FileTable containing the object.

  • object_id (str) – The object ID for the Container/Data.

  • object_type (str) – The type of the object. This is also the parent in relative_path.

  • relative_path (str) – (‘The relative_path of the attribute of the object that uses ‘, ‘an external resource reference key. Use an empty string if not applicable.’)

  • field (str) – The field of the compound data type using an external resource. Use an empty string if not applicable.

  • table (Table) – None

  • idx (int) – None

todict()
class hdmf.common.resources.ObjectKeyTable(name='object_keys', data=[])

Bases: Table

A table for identifying which keys are used by which objects for referring to external resources.

Parameters:
add_row(objects_idx, keys_idx)
Parameters:

  • objects_idx (int or Object) – The index into the objects table for the Object that uses the Key.

  • keys_idx (int or Key) – The index into the keys table that is used to make an external resource reference.

class hdmf.common.resources.EntityKeyTable(name='entity_keys', data=[])

Bases: Table

A table for identifying which entities are used by which keys for referring to external resources.

Parameters:
add_row(entities_idx, keys_idx)
Parameters:

  • entities_idx (int or Entity) – The index into the EntityTable for the Entity that associated with the Key.

  • keys_idx (int or Key) – The index into the KeyTable that is used to make an external resource reference.

class hdmf.common.resources.EntityKey(entities_idx, keys_idx, table=None, idx=None)

Bases: Row

A Row class for representing rows in the EntityKeyTable.

Parameters:

  • entities_idx (int or Entity) – The index into the EntityTable for the Entity that associated with the Key.

  • keys_idx (int or Key) – The index into the KeyTable that is used to make an external resource reference.

  • table (Table) – None

  • idx (int) – None

todict()
class hdmf.common.resources.ObjectKey(objects_idx, keys_idx, table=None, idx=None)

Bases: Row

A Row class for representing rows in the ObjectKeyTable.

Parameters:

  • objects_idx (int or Object) – The index into the objects table for the Object that uses the Key.

  • keys_idx (int or Key) – The index into the keys table that is used to make an external resource reference.

  • table (Table) – None

  • idx (int) – None

todict()
class hdmf.common.resources.HERD(keys=None, files=None, entities=None, objects=None, object_keys=None, entity_keys=None, type_map=None)

Bases: Container

HDMF External Resources Data Structure. A table for mapping user terms (i.e. keys) to resource entities.

Parameters:

  • keys (KeyTable) – The table storing user keys for referencing resources.

  • files (FileTable) – The table for storing file ids used in external resources.

  • entities (EntityTable) – The table storing entity information.

  • objects (ObjectTable) – The table storing object information.

  • object_keys (ObjectKeyTable) – The table storing object-key relationships.

  • entity_keys (EntityKeyTable) – The table storing entity-key relationships.

  • type_map (TypeMap) – The type map. If None is provided, the HDMF-common type map will be used.

property keys

The table storing user keys for referencing resources.

property files

The table for storing file ids used in external resources.

property entities

The table storing entity information.

property objects

The table storing object information.

property object_keys

The table storing object-key relationships.

property entity_keys

The table storing entity-key relationships.

static assert_external_resources_equal(left, right, check_dtype=True)

Compare that the keys, resources, entities, objects, and object_keys tables match

Parameters:

  • left – HERD object to compare with right

  • right – HERD object to compare with left

  • check_dtype – Enforce strict checking of dtypes. Dtypes may be different for example for ids, where depending on how the data was saved ids may change from int64 to int32. (Default: True)

Returns:

The function returns True if all values match. If mismatches are found, AssertionError will be raised.

Raises:

AssertionError – Raised if any differences are found. The function collects all differences into a single error so that the assertion will indicate all found differences.

add_ref_container(root_container)
Method to search through the root_container for all instances of TermSet.

Currently, only datasets are supported. By using a TermSet, the data comes validated and can use the permissible values within the set to populate HERD.

Parameters:

root_container (HERDManager) – The root container or file containing objects with a TermSet.

add_ref_termset(termset, file=None, container=None, attribute=None, field='', key=None)
This method allows users to take advantage of using the TermSet class to provide the entity information

for add_ref, while also validating the data. This method supports adding a single key or an entire dataset to the HERD tables. For both cases, the term, i.e., key, will be validated against the permissible values in the TermSet. If valid, it will proceed to call add_ref. Otherwise, the method will return a dict of missing terms (terms not found in the TermSet).

Parameters:

  • termset (TermSet) – The TermSet to be used if the container/attribute does not have one.

  • file (HERDManager) – The file associated with the container.

  • container (str or AbstractContainer) – The Container/Data object that uses the key or the object_id for the Container/Data object that uses the key.

  • attribute (str) – The attribute of the container for the external reference.

  • field (str) – The field of the compound data type using an external resource.

  • key (str or Key) – The name of the key or the Key object from the KeyTable for the key to add a resource for.

add_ref(entity_id, container=None, attribute=None, field='', key=None, entity_uri=None, file=None)

Add information about an external reference used in this file.

It is possible to use the same name of the key to refer to different resources so long as the name of the key is not used within the same object, relative_path, and field combination. This method does not support such functionality by default.

Parameters:

  • entity_id (str) – The identifier for the entity at the resource.

  • container (str or AbstractContainer) – The Container/Data object that uses the key or the object_id for the Container/Data object that uses the key.

  • attribute (str) – The attribute of the container for the external reference.

  • field (str) – The field of the compound data type using an external resource.

  • key (str or Key) – The name of the key or the Key object from the KeyTable for the key to add a resource for.

  • entity_uri (str) – The URI for the identifier at the resource.

  • file (HERDManager) – The file associated with the container.

get_key(key_name, file=None, container=None, relative_path='', field='')

Return a Key.

If container, relative_path, and field are provided, the Key that corresponds to the given name of the key for the given container, relative_path, and field is returned.

If there are multiple matches, a list of all matching keys will be returned.

Parameters:

  • key_name (str) – The name of the Key to get.

  • file (HERDManager) – The file associated with the container.

  • container (str or AbstractContainer) – The Container/Data object that uses the key or the object id for the Container/Data object that uses the key.

  • relative_path (str) – (‘The relative_path of the attribute of the object that uses ‘, ‘an external resource reference key. Use an empty string if not applicable.’)

  • field (str) – The field of the compound data type using an external resource.

get_entity(entity_id)
Parameters:

entity_id (str) – The ID for the identifier at the resource.

get_object_type(object_type, relative_path='', field='', all_instances=False)

Get all entities/resources associated with an object_type.

Parameters:

  • object_type (str) – The type of the object. This is also the parent in relative_path.

  • relative_path (str) – (‘The relative_path of the attribute of the object that uses ‘, ‘an external resource reference key. Use an empty string if not applicable.’)

  • field (str) – The field of the compound data type using an external resource.

  • all_instances (bool) – (‘The bool to return a dataframe with all instances of the object_type.’, ‘If True, relative_path and field inputs will be ignored.’)

get_object_entities(container, file=None, attribute=None, relative_path='', field='')

Get all entities/resources associated with an object.

Parameters:

  • container (str or AbstractContainer) – The Container/data object that is linked to resources/entities.

  • file (HERDManager) – The file.

  • attribute (str) – The attribute of the container for the external reference.

  • relative_path (str) – (‘The relative_path of the attribute of the object that uses ‘, ‘an external resource reference key. Use an empty string if not applicable.’)

  • field (str) – The field of the compound data type using an external resource.

to_dataframe(use_categories=False)
Convert the data from the keys, resources, entities, objects, and object_keys tables

to a single joint dataframe. I.e., here data is being denormalized, e.g., keys that are used across multiple entities or objects will duplicated across the corresponding rows.

Returns: DataFrame with all data merged into a single, flat, denormalized table.

Parameters:

use_categories (bool) – Use a multi-index on the columns to indicate which category each column belongs to.

Returns:

A DataFrame with all data merged into a flat, denormalized table.

Return type:

DataFrame

to_zip(path)

Write the tables in HERD to zipped tsv files.

Parameters:

path (str) – The path to the zip file.

classmethod get_zip_directory(path)

Return the directory of the file given.

Parameters:

path (str) – The path to the zip file.

classmethod from_zip(path)

Method to read in zipped tsv files to populate HERD.

Parameters:

path (str) – The path to the zip file.

data_type = 'HERD'
namespace = 'hdmf-experimental'