datakiln.extraction.exporters module¶
Data and metadata export logic for the extraction pipeline.
- class datakiln.extraction.exporters.MetadataExtractor[source]¶
Bases:
objectClass to handle metadata extraction and generation.
- static get_metadata(session: Session, schema: str, tables: List[str]) MetaData[source]¶
Generate column metadata from DBMS.
- class datakiln.extraction.exporters.HashingFileWriter(fp)[source]¶
Bases:
objectA file-like writer that calculates the MD5 hash on-the-fly.
- class datakiln.extraction.exporters.DataExporter[source]¶
Bases:
objectClass to handle data export logic.
- static stream_to_csv(session: Session, query: Select, export_path: Path, table_name: str = 'unknown', insert_to: Table | None = None, chunk_size: int = 10000, abort: Event | None = None) str[source]¶
Streams a single SQL query into a compressed CSV.
Returns the MD5 checksum of the generated compressed CSV file.