datakiln.metadata.location module

File system location metadata.

class datakiln.metadata.location.SSHLocation(host, folder, regex, match, default_values=None, match_map=None, **kwargs)[source]

Bases: ColumnMeta

Column with local file locations.

find(conn, table)[source]

Find file names in directory and store them in a staging table.

Parameters:
  • con (sqlalchemy.Engine) – SQLAlchemy connection/session.

  • table (TableMeta) – Table metadata

Returns:

None

create_table(conn, rows, table)[source]

Create LFS Location table.

Parameters:
  • conn (sqlalchemy.Engine) – SQLAlchemy connection/session.

  • rows (list) – List of formatted rows to be inserted as values.

  • table (TableMeta) – Table metadata

Returns:

None

add_column(conn, table)[source]

Add local location column to table.

Parameters:
  • conn (sqlalchemy.Engine) – SQLAlchemy connection/session.

  • table (TableMeta) – Table metadata

Returns:

None

insert_file_locations(conn, match_columns, table)[source]

Add staged local file system column to loaded table.

Returns:

str