Skip to content

Supported Instruments

AeroViz provides comprehensive support for reading and processing data from a wide range of aerosol measurement instruments. Each instrument has a dedicated reader that handles its specific data format, file structure, and measurement characteristics.

Instrument Support

You name the instrument; AeroViz does not guess it. instrument= is required and is validated against the supported list — a wrong or unknown name raises KeyError listing what is valid. The reader then handles that instrument's file format, header layout and status codes for you, including the dialect differences between host-software versions.

Instrument Categories

Aethalometers (Black Carbon Measurement)

Instruments for measuring black carbon and light absorption:

  • AE33 - Magee Scientific AE33 (7-wavelength aethalometer)
  • AE43 - Magee Scientific AE43 (real-time BC measurements)
  • BC1054 - MetOne BC1054 (high-resolution absorption)
  • MA350 - AethLabs MA350 (multi-angle absorption photometer)

Nephelometers (Light Scattering)

Instruments for measuring aerosol light scattering:

  • Aurora - Ecotech Aurora (3-wavelength nephelometer)
  • NEPH - TSI Nephelometer (standard scattering measurements)

Particle Sizers

Instruments for measuring particle size distributions:

  • SMPS - Scanning Mobility Particle Sizer (10-600 nm)
  • APS - Aerodynamic Particle Sizer (0.5-20 μm)
  • GRIMM - GRIMM Aerosol Spectrometer (optical sizing)

Chemical Analysis

Instruments for chemical composition analysis:

  • IGAC - Ion chromatography (water-soluble ions)
  • OCEC - Organic/Elemental Carbon Analyzer
  • Xact - Xact 625i XRF Analyzer (elemental analysis)

Mass Concentration

Instruments for PM mass concentration measurement:

  • TEOM - Tapered Element Oscillating Microbalance
  • BAM1020 - Beta Attenuation Monitor (PM2.5)

External / pre-aggregated sources

Not instruments, but read through the same factory:

  • EPA - Taiwan EPA hourly air-quality export (big5-encoded 測項 / 直式 CSV)

Not readable

  • Q-ACSM - a real instrument, but its reader is not written yet: calling it raises NotImplementedError explaining what to contribute. See Instrument Formats & QC.
  • VOC, Minion - pre-aggregated, second-hand data with no raw log to parse. The readers were removed; calling them raises KeyError carrying the migration advice. See VOC Data.

Usage Example

from AeroViz import RawDataReader

# Name the instrument and the folder holding its raw files.
df = RawDataReader('AE33', '/data/NZ_AE33', start='2024-01-01', end='2024-06-30')

# What was actually read is reported in df.attrs
print(df.attrs['raw_freq'])         # native resolution detected from the files
print(df.attrs['coverage_start'])   # first timestamp carrying real data
print(df.attrs['total_rate'])       # % of expected periods that passed QC

Technical Specifications

Instrument Time Resolution File Type QC Rules
AE33 1 min .dat Status Error, Invalid BC, Invalid AAE, Insufficient
AE43 1 min .dat Status Error, Invalid BC, Invalid AAE, Insufficient
BC1054 1 min .csv Status Error, Invalid BC, Invalid AAE, Insufficient
MA350 1 min .csv Status Error, Invalid BC, Invalid AAE, Insufficient
NEPH 5 min .dat Status Error, No Data, Invalid Scat Value, Invalid Scat Rel, Insufficient
Aurora 1 min .csv Status Error, No Data, Invalid Scat Value, Invalid Scat Rel, Insufficient
SMPS 6 min .txt, .csv Status Error, Invalid Number Conc, DMA Water, Insufficient
APS 6 min .txt Status Error, Invalid Number Conc, Insufficient
GRIMM 6 min .dat No Data, Negative Conc, Insufficient
TEOM 6 min .csv Status Error, High Noise, Non-positive, NV > Total, Spike, Insufficient
BAM1020 1 h .csv Invalid Conc, Spike
OCEC 1 h *LCRes.csv Invalid Carbon, Below MDL, Spike, Missing OC
IGAC 1 h .csv Mass Closure, Missing Main, Above MR, Ion Balance
Xact 1 h .csv Calibration Mode, Instrument Error, Upscale Warning, Invalid Value, Internal Std Drift
EPA 1 h .csv Negative Value

Quality Control System

All instruments use the declarative QCFlagBuilder system:

  • Declarative Rules - Each instrument defines QC rules as QCRule dataclass instances
  • Consistent Processing - All instruments use QC_Flag internally for quality control
  • Clean Output - Final output has invalid data set to NaN, QC_Flag column removed

QC Flag Processing

The QC_Flag column is used internally during processing:

  • "Valid" - All QC rules passed
  • "Rule1, Rule2" - Comma-separated list of failed rule names

Output Files:

File QC_Flag Description
_read_{inst}_raw.pkl/csv ❌ No Raw data before QC
_read_{inst}_qc.pkl/csv ✅ Yes QC'd data with flag
output_{inst}.csv ❌ No Final output (invalid → NaN)

Adding New Instruments

To add support for a new instrument, you need to:

  1. Create a new reader class inheriting from AbstractReader
  2. Implement the required methods for data parsing
  3. Add instrument detection logic
  4. Include appropriate quality control methods

Example Reader Implementation

from AeroViz.rawDataReader.core.AbstractReader import AbstractReader

class MyInstrumentReader(AbstractReader):
    def __init__(self, file_path, **kwargs):
        super().__init__(file_path, **kwargs)

    def read_data(self):
        # Implement your data reading logic
        pass

    def apply_qc(self):
        # Implement quality control
        pass