Skip to content

AE43 Aethalometer

The AE43 is a multi-wavelength aethalometer used for measuring black carbon aerosol concentrations in the atmosphere.

AeroViz.rawDataReader.script.AE43.Reader 

Reader(path: Path | str, reset: bool | str = False, qc: bool | str = True, **kwargs)

Bases: AbstractReader

AE43 Aethalometer Data Reader

A specialized reader for AE43 Aethalometer data files, which measure black carbon concentrations at seven wavelengths.

See docs/api/instruments/aethalometers/AE43.md for usage and docs/guide/reader-reference.md for the file layout, status codes and QC rules.

Attributes

nam class-attribute instance-attribute 

nam = 'AE43'

BC_COLUMNS class-attribute instance-attribute 

BC_COLUMNS = ['BC1', 'BC2', 'BC3', 'BC4', 'BC5', 'BC6', 'BC7']

ABS_COLUMNS class-attribute instance-attribute 

ABS_COLUMNS = ['abs_370', 'abs_470', 'abs_520', 'abs_590', 'abs_660', 'abs_880', 'abs_950']

CAL_COLUMNS class-attribute instance-attribute 

CAL_COLUMNS = ['abs_550', 'AAE', 'eBC']

LATE_QC_FLAGS class-attribute instance-attribute 

LATE_QC_FLAGS = ('Invalid AAE',)

MIN_BC class-attribute instance-attribute 

MIN_BC = 0

MAX_BC class-attribute instance-attribute 

MAX_BC = 20000

MIN_AAE class-attribute instance-attribute 

MIN_AAE = 0.7

MAX_AAE class-attribute instance-attribute 

MAX_AAE = 3.0

ERROR_STATES class-attribute instance-attribute 

ERROR_STATES = [1, 2, 3, 4, 16, 32, 1024, 2048, 4096]

STATUS_BITS class-attribute instance-attribute 

STATUS_BITS = {1: 'Tape advance / fast calibration / warm-up', 2: 'First measurement (obtaining ATN0)', 4: 'Flow off by more than 0.5 LPM', 16: 'Calibrating LED', 32: 'Calibration error (at least one channel OK)', 1024: 'Stability test', 2048: 'Clean air test', 4096: 'Optical test'}

Methods:

_raw_reader 

_raw_reader(file)

Read and parse raw AE43 Aethalometer data files.

Returns all columns from the raw file. Column selection is deferred to _QC() and _process() stages.

_QC 

_QC(_df)

Perform quality control on AE43 Aethalometer raw data.

QC Rules Applied (raw data only)
  1. Status Error : Invalid instrument status codes
  2. Invalid BC : BC concentration outside 0-20000 ng/m³
  3. Insufficient : Less than 50% hourly data completeness

Note: AAE validation is done in _process() after calculation.

_process 

_process(_df)

Calculate absorption coefficients and validate derived parameters.

Processing Steps
  1. Calculate absorption coefficients at each wavelength
  2. Calculate AAE (Absorption Ångström Exponent)
  3. Calculate eBC (equivalent Black Carbon)
  4. Validate AAE range and update QC_Flag

Note: AE43 uses AE33 coefficients for absorption calculation.

Raw format

  • File pattern: [!ST|!CT|!FV]*[!log]_AE43*.dat — a comma-separated (CSV) file despite the .dat extension. Log files are not included.
  • Native frequency: 1 minute (the meta['freq'] fallback; the grid actually found in each file is reported as df.attrs['raw_freq']).
  • Timestamp column: StartTime.
  • Configuration tracking: uses SetupID for configuration management.

Parse recipe

  • read_csv(parse_dates=['StartTime'], index_col='StartTime').
  • Keeps only the rows of the last SetupID in the file — a setup change mid-file discards the earlier segment. This is how the reader "selects the latest configuration data": it groups by SetupID and takes the group of the final row.
  • Rows with an unparseable or duplicated timestamp are removed.
  • Every other source column (SetupID, Status, flows, …) is kept at L1 and, unlike the AE33, all the way to the output.

Quirks

  • The pattern's [!ST|!CT|!FV] is a single-character negated class (it excludes S T C F V | ! as the first character), not an alternation of ST/CT/FV prefixes. It happens to filter the ST/CT/FV log files, but not for the reason it looks like.

Measurement Parameters

The AE43 provides black carbon concentration measurements at seven wavelengths:

Column Wavelength Description
BC1 370 nm UV channel
BC2 470 nm Blue channel
BC3 520 nm Green channel
BC4 590 nm Yellow channel
BC5 660 nm Red channel
BC6 880 nm Near-IR channel (reference)
BC7 950 nm IR channel

Status & error codes

The Status column is tested in bitwise mode — a row is an error when any non-whitelisted code in ERROR_STATES matches it bitwise; see status modes. A missing Status column is logged as a warning and the rule then passes every row.

Same instrument family, same status register, same manual as the AE33: ERROR_STATES is identical to AE33's and a test pins the two lists equal. It used to keep 384, which the AE33 had already dropped as a tape-low warning.

Codes treated as errors:

  • 1 — Tape advance, fast calibration, warm-up
  • 2 — First measurement – obtaining ATN0
  • 3 — Stopped (= 1 | 2; see the note under the register)
  • 4 — Flow low/high by more than 0.5 LPM
  • 16 — Calibrating LED
  • 32 — Calibration error (at least one channel OK)
  • 1024 — Stability test
  • 2048 — Clean air test
  • 4096 — Optical test

Not errors: 128 and 256 are tape low warnings — the data is still valid — so neither they nor their sum 384 (the instrument's "Tape error: tape not moving, end of tape" code) is in ERROR_STATES. Rows carrying them are not flagged.

Status Condition Register

The Status column is a bitfield: the instrument OR-sums every active condition and reports the sum, so one value can mean several things at once. Reader.STATUS_BITS carries this table, and it is what turns a raw status into df.attrs['status_conditions'] — a named condition instead of a number.

Bit Decimal Condition
12 4096 Optical test
11 2048 Clean air test
10 1024 Stability test
5 32 Calibration error (at least one channel OK)
4 16 Calibrating LED
2 4 Flow off by more than 0.5 LPM
1 2 First measurement (obtaining ATN0)
0 1 Tape advance / fast calibration / warm-up

ERROR_STATES also lists 3 ("Stopped"), which has no row here: 3 is 1 | 2, so under bitwise testing it fires whenever tape advance or first-measurement does. It still counts as an error — it simply has no unambiguous name to report.

To stop treating one condition as an error, whitelist its decimal value:

RawDataReader('AE43', path, ignored_status_errors=[1])  # ignore Tape advance / fast calibration / warm-up

QC rules

Rule Condition Severity
Status Error any non-whitelisted ERROR_STATES code set in Status error
Invalid BC any of BC1–BC7 ≤ 0 or > 20 000 ng/m³ error
Insufficient an hour holds < 50 % of the BC points it could have held, given how much of that hour the read covers advisory
Invalid AAE AAE < 0.7 or AAE > 3.0 — added in _process, after AAE has been computed error

Invalid AAE cannot be a _QC rule because AAE does not exist until _process has derived it; it is raised there through update_qc_flag and counted into the same QC summary. The upper bound of 3.0 is deliberately loose so that BrC-rich biomass-burning and dust episodes, where bulk AAE runs 2–3, survive. Insufficient is advisory because a sparse hour's readings are fine in themselves — only an hourly average would misrepresent it; promote it with flag_severity={'Insufficient': 'error'}.

Output

The processed data contains the following columns:

Column Unit Description
BC1-BC7 ng/m³ Black carbon at 7 wavelengths
abs_370-950 Mm⁻¹ Absorption coefficients (abs_370, abs_470, abs_520, abs_590, abs_660, abs_880, abs_950)
abs_550 Mm⁻¹ Interpolated absorption at 550nm
AAE - Absorption Angstrom Exponent (positive by convention)
eBC ng/m³ Equivalent black carbon (= BC6, 880 nm)
every other source column as in the raw file SetupID, Status, flows, … are kept alongside the derived columns, like BC1054 / MA350

How the derived columns are made (_absCoe in core/pre_process.py):

  • The AE43 has no coefficient set of its own: it is computed with the AE33 coefficients (_absCoe(..., instru='AE33')), i.e. abs_λ = BC_λ × MAE_λ × 10⁻³ with MAE = 18.47, 14.54, 13.14, 11.58, 10.35, 7.77, 7.19 m²/g at 370–950 nm, and eBC = BC6.
  • AAE is the slope of a log-log fit over all seven bands, sign-flipped so it is positive.
  • abs_550 is not measured: it is extrapolated from the nearest measured band as abs(λ₂) = abs(λ₁)·(λ₂/λ₁)^(−AAE).
  • A row whose BC channels are all 0, or where any channel is NaN, gets NaN in every derived column.

Files written per read are listed in RawDataReader Reference §1.

Notes

  • The AE43 provides real-time black carbon concentration measurements
  • Uses AE33 absorption coefficients for calculations
  • Automatically handles configuration changes and duplicate timestamps
  • Maintains data integrity across configuration changes using SetupID