Skip to content

AE33 Aethalometer

The AE33 is a multi-wavelength aethalometer used for measuring black carbon aerosol concentrations in the atmosphere.

AeroViz.rawDataReader.script.AE33.Reader 

Reader(path: Path | str, reset: bool | str = False, qc: bool | str = True, **kwargs)

Bases: AbstractReader

AE33 Aethalometer Data Reader.

A specialized reader for AE33 Aethalometer data files, which measure black carbon concentrations at seven wavelengths.

See docs/api/instruments/aethalometers/AE33.md for usage and docs/guide/reader-reference.md for the file layout, status codes and QC rules.

Attributes

nam class-attribute instance-attribute 

nam = 'AE33'

BC_COLUMNS class-attribute instance-attribute 

BC_COLUMNS = ['BC1', 'BC2', 'BC3', 'BC4', 'BC5', 'BC6', 'BC7']

ABS_COLUMNS class-attribute instance-attribute 

ABS_COLUMNS = ['abs_370', 'abs_470', 'abs_520', 'abs_590', 'abs_660', 'abs_880', 'abs_950']

CAL_COLUMNS class-attribute instance-attribute 

CAL_COLUMNS = ['abs_550', 'AAE', 'eBC', 'Delta-C']

BB_COLUMN class-attribute instance-attribute 

BB_COLUMN = 'BB(%)'

DELTA_C_BANDS class-attribute instance-attribute 

DELTA_C_BANDS = ('BC1', 'BC6')

K_COLUMNS class-attribute instance-attribute 

K_COLUMNS = ['K1', 'K2', 'K3', 'K4', 'K5', 'K6', 'K7']

LATE_QC_FLAGS class-attribute instance-attribute 

LATE_QC_FLAGS = ('Invalid AAE',)

MIN_BC class-attribute instance-attribute 

MIN_BC = 0

MAX_BC class-attribute instance-attribute 

MAX_BC = 20000

MIN_AAE class-attribute instance-attribute 

MIN_AAE = 0.7

MAX_AAE class-attribute instance-attribute 

MAX_AAE = 3.0

ERROR_STATES class-attribute instance-attribute 

ERROR_STATES = [1, 2, 3, 4, 16, 32, 1024, 2048, 4096]

STATUS_BITS class-attribute instance-attribute 

STATUS_BITS = {1: 'Tape advance / fast calibration / warm-up', 2: 'First measurement (obtaining ATN0)', 4: 'Flow off by more than 0.5 LPM', 16: 'Calibrating LED', 32: 'Calibration error (at least one channel OK)', 1024: 'Stability test', 2048: 'Clean air test', 4096: 'Optical test'}

Methods:

_raw_reader 

_raw_reader(file)

Read and parse raw AE33 Aethalometer data files.

Returns all columns from the raw file. Column selection is deferred to _QC() and _process() stages.

_QC 

_QC(_df)

Perform quality control on AE33 Aethalometer raw data.

QC Rules Applied (raw data only)
  1. Status Error : Invalid instrument status codes
  2. Invalid BC : BC concentration outside 0-20000 ng/m³
  3. Insufficient : Less than 50% hourly data completeness

Note: AAE validation is done in _process() after calculation.

_process 

_process(_df)

Calculate absorption coefficients and validate derived parameters.

Processing Steps
  1. Calculate absorption coefficients at each wavelength
  2. Calculate AAE (Absorption Ångström Exponent)
  3. Calculate eBC (equivalent Black Carbon)
  4. Calculate Delta-C = BC1 − BC6 (370 nm − 880 nm)
  5. Validate AAE range and update QC_Flag

Parameters:

Name Type Description Default
_df DataFrame

Quality-controlled DataFrame with BC columns and QC_Flag

 required

Returns:

Type Description
DataFrame

DataFrame with absorption coefficients, AAE, eBC, Delta-C, and updated QC_Flag

Raw format

  • File pattern: [!ST|!CT|!FV]*[!log]_AE33*.dat — whitespace-delimited text (.dat). Log files are not included, e.g. AE33_log_AE33-S05-00494_20220401.dat.
  • Native frequency: 1 minute (the meta['freq'] fallback; the grid actually found in each file is reported as df.attrs['raw_freq']).
  • Header: 5 rows of header information; the column-name line is line 6 and its names carry a trailing semicolon.
  • Data columns: 67 columns including timestamp and measurements.

Parse recipe

  • read_table(delimiter=r'\s+', skiprows=5, usecols=range(67)); the trailing semicolons are stripped from the column names.
  • The index is built from columns 0 + 1 (Date(yyyy/MM/dd) + Time(hh:mm:ss)), those two source columns are dropped, and rows with an unparseable or duplicated timestamp are removed.
  • Every other source column (ATN, flows, Status, K1–K7, BB(%), …) is kept at L1; the frame is narrowed only in _process (see Output).

Quirks

  • Files smaller than 550 KB log "may not be a whole daily data" — a warning, not a rejection.
  • The pattern's [!ST|!CT|!FV] is a single-character negated class (it excludes S T C F V | ! as the first character), not an alternation of ST/CT/FV prefixes. It happens to filter the ST/CT/FV log files, but not for the reason it looks like.

Measurement Parameters

The AE33 provides black carbon concentration measurements at seven wavelengths:

Column Wavelength Description
BC1 370 nm UV channel
BC2 470 nm Blue channel
BC3 520 nm Green channel
BC4 590 nm Yellow channel
BC5 660 nm Red channel
BC6 880 nm Near-IR channel (reference)
BC7 950 nm IR channel

Status & error codes

The Status column is tested in bitwise mode — a row is an error when any non-whitelisted code in ERROR_STATES matches it bitwise; see status modes. The raw file also carries ContStatus, DetectStatus, LedStatus and ValveStatus, which are not checked. A missing Status column is logged as a warning and the rule then passes every row.

Codes treated as errors:

  • 1 — Tape advance, fast calibration, warm-up
  • 2 — First measurement – obtaining ATN0
  • 3 — Stopped (= 1 | 2; see the note under the register)
  • 4 — Flow low/high by more than 0.5 LPM
  • 16 — Calibrating LED
  • 32 — Calibration error (at least one channel OK)
  • 1024 — Stability test
  • 2048 — Clean air test
  • 4096 — Optical test

Not errors: 128 and 256 are tape low warnings — the data is still valid — so neither they nor their sum 384 (the instrument's "Tape error: tape not moving, end of tape" code) is in ERROR_STATES. Rows carrying them are not flagged.

Status Condition Register

The Status column is a bitfield: the instrument OR-sums every active condition and reports the sum, so one value can mean several things at once. Reader.STATUS_BITS carries this table, and it is what turns a raw status into df.attrs['status_conditions'] — a named condition instead of a number.

Bit Decimal Condition
12 4096 Optical test
11 2048 Clean air test
10 1024 Stability test
5 32 Calibration error (at least one channel OK)
4 16 Calibrating LED
2 4 Flow off by more than 0.5 LPM
1 2 First measurement (obtaining ATN0)
0 1 Tape advance / fast calibration / warm-up

ERROR_STATES also lists 3 ("Stopped"), which has no row here: 3 is 1 | 2, so under bitwise testing it fires whenever tape advance or first-measurement does. It still counts as an error — it simply has no unambiguous name to report.

To stop treating one condition as an error, whitelist its decimal value:

RawDataReader('AE33', path, ignored_status_errors=[1])  # ignore Tape advance / fast calibration / warm-up

QC rules

Rule Condition Severity
Status Error any non-whitelisted ERROR_STATES code set in Status error
Invalid BC any of BC1–BC7 ≤ 0 or > 20 000 ng/m³ error
Insufficient an hour holds < 50 % of the BC points it could have held, given how much of that hour the read covers advisory
Invalid AAE AAE < 0.7 or AAE > 3.0 — added in _process, after AAE has been computed error

Invalid AAE cannot be a _QC rule because AAE does not exist until _process has derived it; it is raised there through update_qc_flag and counted into the same QC summary. The upper bound of 3.0 is deliberately loose so that BrC-rich biomass-burning and dust episodes, where bulk AAE runs 2–3, survive. Insufficient is advisory because a sparse hour's readings are fine in themselves — only an hourly average would misrepresent it; promote it with flag_severity={'Insufficient': 'error'}.

Output

The final output (output_ae33.csv) contains the following columns:

Column Unit Description
BC1-BC7 ng/m³ Black carbon at 7 wavelengths
abs_370-950 Mm⁻¹ Absorption coefficients (abs_370, abs_470, abs_520, abs_590, abs_660, abs_880, abs_950)
abs_550 Mm⁻¹ Interpolated absorption at 550nm
AAE - Absorption Angstrom Exponent (positive by convention)
eBC ng/m³ Equivalent black carbon (= BC6, 880 nm)
Delta-C ng/m³ BC1 − BC6 (370 nm − 880 nm): UV excess over the IR reference, a brown-carbon / biomass-burning tracer (Wang et al. 2011). May be negative when AAE < 1
BB(%) % Biomass-burning fraction from the instrument's built-in source apportionment (passed through when present in the raw file)
K1–K7 – Per-wavelength loading-compensation parameter k of the dual-spot algorithm (BC = BC_spot / (1 − k·ATN), Drinovec et al. 2015), passed through untouched. A loading-effect diagnostic: typical |k| ≈ 10⁻³, ≈ 0 for aged/coated aerosol; negative values occur

How the derived columns are made (_absCoe in core/pre_process.py):

  • abs_λ = BC_λ × MAE_λ × 10⁻³, with MAE = 18.47, 14.54, 13.14, 11.58, 10.35, 7.77, 7.19 m²/g at 370–950 nm.
  • AAE is the slope of a log-log fit over all seven bands, sign-flipped so it is positive.
  • abs_550 is not measured: it is extrapolated from the nearest measured band as abs(λ₂) = abs(λ₁)·(λ₂/λ₁)^(−AAE).
  • A row whose BC channels are all 0, or where any channel is NaN, gets NaN in every derived column.

AE33 is the only aethalometer that narrows its L2 frame: _process keeps exactly the columns above (plus K1–K7 and BB(%) only when the raw file has them). Status is used for QC but is not carried into the output.

Files written per read are listed in RawDataReader Reference §1.

Units

BC1–BC7, eBC and Delta-C are ng/m³ exactly as the instrument reports them — AeroViz does not rescale them. The absorption coefficients are derived as abs = BC[ng/m³] × MAE[m²/g] × 10⁻³, which is what yields Mm⁻¹.

Notes

  • The AE33 provides real-time black carbon concentration measurements
  • Useful for distinguishing between biomass burning and fossil fuel combustion
  • Files smaller than 550KB may indicate incomplete data