# Temple (TUH EEG): full counts / hours / patients tables

Hours are full recordings (no cap). **TOTAL on disk, unfiltered: 26,977 files, 6,703 hours, 5,678 patients (420 GB).** A recording is counted once per corpus it appears in; TUEV files carry no patient ids. Converted shards are local at `/home/dlinsley/eeg_shards_tuh`.

## 1 Totals and funnel

| Stage | Files / recordings | Hours | Patients | GB |
|---|---:|---:|---:|---:|
| TOTAL on disk: raw EDF files (a recording is counted once per corpus it is in) | 26,977 | 6,702.8 | 5,678 | 420.2 |
| Unique recordings (identical copies counted once) | 23,350 | 5,839.5 | 5,678 |  |
|   minus recordings under 30 s (the shard format cannot hold them) | -1,764 | -4.5 |  |  |
|   minus flat or saturated recordings (unit gate) | -164 | -37.2 |  |  |
| CONVERTED to shards (usable) | 21,422 | 5,797.7 | 5,669 | 158.1 |
|   train | 16,497 | 4,595.0 | 5,023 |  |
|   val | 713 | 220.4 | 251 |  |
|   test (held out for evaluation) | 4,212 | 982.4 | 395 |  |
| Trainable (train + val) | 17,210 | 4,815.3 | 5,274 |  |
| TOTAL when the transfer finishes (65,329 files = selection 51,522 TUEG + the six full corpora) | 65,329 | 23,208.9 | 11,642 |  |
|   still to arrive (TUEG, 1,066 GB) | 38,352 | 16,506.1 | 7,025 | 1,066.0 |

## 2 Raw files by corpus

| Corpus | Split | Files | Hours | Patients | GB | Mean min / file | Longest (h) |
|---|---|---:|---:|---:|---:|---:|---:|
| TUEG | all | 13,170 | 3,588.8 | 4,105 | 241.7 | 16.3 | 24.6 |
| TUSZ | all | 8,140 | 1,474.8 | 675 | 84.7 | 10.9 | 0.5 |
| TUSZ | dev | 2,203 | 435.5 | 53 | 22.2 | 11.9 | 0.5 |
| TUSZ | eval | 880 | 128.6 | 43 | 7.2 | 8.8 | 0.5 |
| TUSZ | train | 5,057 | 910.7 | 579 | 55.3 | 10.8 | 0.5 |
| TUAB | all | 1,919 | 736.1 | 1,384 | 40.8 | 23.0 | 5.5 |
| TUAB | eval | 276 | 102.9 | 253 | 5.7 | 22.4 | 1.0 |
| TUAB | train | 1,643 | 633.2 | 1,131 | 35.0 | 23.1 | 5.5 |
| TUEP | all | 2,808 | 626.8 | 200 | 37.7 | 13.4 | 0.5 |
| TUEV | all | 518 | 148.7 |  | 8.0 | 17.2 | 1.5 |
| TUEV | eval | 159 | 48.3 |  | 2.9 | 18.2 | 0.9 |
| TUEV | train | 359 | 100.5 |  | 5.2 | 16.8 | 1.5 |
| TUAR | all | 310 | 100.0 | 213 | 5.8 | 19.4 | 2.9 |
| TUSL | all | 112 | 27.6 | 38 | 1.6 | 14.8 | 0.7 |
| ALL (copies counted) |  | 26,977 | 6,702.8 | 5,678 | 420.2 | 14.9 | 24.6 |

## 3 Unique recordings

| Source corpus (highest priority copy) | Unique recordings | of which >= 30 s | Hours | Patients |
|---|---:|---:|---:|---:|
| TUSZ | 8,104 | 7,559 | 1,475.0 | 675 |
| TUAB | 1,727 | 1,727 | 658.2 | 1,316 |
| TUEV | 322 | 320 | 87.0 | 28 |
| TUEP | 2,052 | 1,855 | 460.7 | 155 |
| TUAR | 47 | 47 | 13.7 | 36 |
| TUEG | 11,098 | 10,078 | 3,145.0 | 3,987 |
| ALL | 23,350 | 21,586 | 5,839.5 | 5,678 |

## 4 Converted shard datasets

| Shard dataset | Recordings | Hours | Patients | GB | Train rec | Train h | Val rec | Val h | Test rec | Test h | Test % of hours | 19-channel rec | 17-channel rec | Failed (unit gate) | Skipped < 30 s | Reference id |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---|
| TUH_CURATED_AR | 10,772 | 2,500.2 | 1,767 | 67.8 | 6,735 | 1,526.3 | 230 | 76.6 | 3,807 | 897.4 | 35.9 | 10,315 | 457 | 81 | 728 | Cz (5) |
| TUH_CURATED_LE | 649 | 166.4 | 325 | 4.6 | 536 | 136.2 | 31 | 7.9 | 82 | 22.3 | 13.4 | 649 | 0 | 6 | 16 | A1-A2 (0) |
| TUH_BULK_AR | 3,130 | 1,023.6 | 515 | 28.0 | 2,733 | 961.3 | 138 | 21.6 | 259 | 40.7 | 4.0 | 3,130 | 0 | 51 | 556 | Cz (5) |
| TUH_BULK_LE | 6,871 | 2,107.5 | 3,911 | 57.7 | 6,493 | 1,971.2 | 314 | 114.3 | 64 | 22.0 | 1.0 | 6,871 | 0 | 26 | 464 | A1-A2 (0) |
| ALL | 21,422 | 5,797.7 | 5,669 | 158.1 | 16,497 | 4,595.0 | 713 | 220.4 | 4,212 | 982.4 | 16.9 | 20,965 | 457 | 164 | 1,764 |  |

## 5 Converted by corpus x reference

| Source corpus | Reference family | Recordings | Hours | Patients | Test rec | Test h |
|---|---|---:|---:|---:|---:|---:|
| TUSZ | AR | 6,947 | 1,320.5 | 452 | 3,028 | 610.7 |
| TUSZ | LE | 558 | 140.0 | 277 | 77 | 20.7 |
| TUAB | AR | 1,720 | 654.5 | 1,312 | 346 | 128.7 |
| TUEV | AR | 318 | 86.8 | 28 | 85 | 22.2 |
| TUEP | AR | 1,740 | 424.9 | 122 | 343 | 134.6 |
| TUEP | LE | 91 | 26.4 | 49 | 5 | 1.7 |
| TUAR | AR | 47 | 13.7 | 36 | 5 | 1.1 |
| TUEG | AR | 3,130 | 1,023.6 | 515 | 259 | 40.7 |
| TUEG | LE | 6,871 | 2,107.5 | 3,911 | 64 | 22.0 |

## 6 Recording properties

| Set | Recordings | Min length (min) | Median (min) | Mean (min) | p90 (min) | Longest (h) | Recordings > 1 h | Hours in those | Recordings > 6 h | Median age (y) | Under 18 y | Under 18 y (% of set) | Age missing | Start year range | Median start year |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---|---:|
| All converted | 21,422 | 0.5 | 11.3 | 16.2 | 26.0 | 24.6 | 261 | 837.6 | 33 | 53 | 848 | 4.0 | 292 | 2000-2019 | 2011 |
| AR family | 13,902 | 0.5 | 10.0 | 15.2 | 30.0 | 24.6 | 127 | 602.0 | 30 | 53 | 189 | 1.4 | 109 | 2000-2019 | 2013 |
| LE family | 7,520 | 0.5 | 20.1 | 18.1 | 24.7 | 9.1 | 134 | 235.6 | 3 | 52 | 659 | 8.8 | 183 | 2000-2011 | 2006 |
| TUH_CURATED_AR | 10,772 | 0.5 | 10.0 | 13.9 | 30.0 | 5.5 | 12 | 30.6 | 0 | 53 | 168 | 1.6 | 100 | 2000-2019 | 2013 |
| TUH_CURATED_LE | 649 | 0.5 | 20.0 | 15.4 | 24.5 | 0.5 | 0 | 0.0 | 0 | 51 | 60 | 9.2 | 8 | 2000-2011 | 2006 |
| TUH_BULK_AR | 3,130 | 0.5 | 5.6 | 19.6 | 22.7 | 24.6 | 115 | 571.4 | 30 | 50 | 21 | 0.7 | 9 | 2000-2017 | 2012 |
| TUH_BULK_LE | 6,871 | 0.5 | 20.1 | 18.4 | 24.7 | 9.1 | 134 | 235.6 | 3 | 52 | 599 | 8.7 | 175 | 2000-2011 | 2006 |

## 7 Sampling rates

| Set | 250 Hz | 256 Hz | 400 Hz | 512 Hz | 1000 Hz | Other |
|---|---:|---:|---:|---:|---:|---:|
| AR family | 3,258 | 8,143 | 1,836 | 421 | 244 | 0 |
| LE family | 7,520 | 0 | 0 | 0 | 0 | 0 |
| All converted | 10,778 | 8,143 | 1,836 | 421 | 244 | 0 |

## 8 Recorders (header tag)

| Reference family | Recorder tag | Recordings | Hours | Patients |
|---|---|---:|---:|---:|
| LE | Intution/Ultrasom | 4,407 | 1,349.3 | 2,541 |
| AR | X | 4,015 | 1,250.7 | 1,813 |
| LE | Alliance | 3,113 | 924.7 | 1,881 |
| AR | Compact_32ch | 2,711 | 705.4 | 145 |
| AR | ##### | 1,918 | 390.8 | 157 |
| AR | Compact_64ch | 1,733 | 471.3 | 89 |
| AR | Bravo | 1,620 | 179.6 | 217 |
| AR | # | 1,374 | 380.8 | 190 |
| AR | B545 | 222 | 14.5 | 37 |
| AR | ######### | 82 | 32.5 | 9 |
| AR | #B####### | 37 | 18.1 | 1 |
| AR | #t####### | 19 | 3.1 | 1 |
| AR | #}####### | 16 | 1.6 | 1 |
| AR | #`#####Y# | 14 | 1.2 | 1 |

## 9 Hold-out

| Evaluation set | Held-out patients | Recordings | Hours |
|---|---:|---:|---:|
| TUAB:eval | 253 |  |  |
| TUSZ:eval | 43 |  |  |
| TUSZ:dev | 53 |  |  |
| TUEV:eval | 61 |  |  |
| UNION (held-out patients) | 395 |  |  |
| recordings flagged by TUSZ:eval |  | 1,915 |  |
| recordings flagged by TUSZ:dev |  | 3,694 |  |
| recordings flagged by TUEV:eval |  | 1,039 |  |
| recordings flagged by TUEV:eval(content link) |  | 71 |  |
| recordings flagged by TUAB:eval |  | 1,057 |  |
| test recordings in an evaluation set themselves |  | 3,112 |  |
| test recordings only through their patient |  | 1,100 |  |
| TOTAL test (held out) | 395 | 4,212 | 982.4 |

## 10 Not converted

| Shard dataset | Rejected by the unit gate | flat (header sigma < 1 uV) | saturated (> 500 uV) | Hours of rejected | Skipped < 30 s (recordings) | Hours of skipped |
|---|---:|---:|---:|---:|---:|---:|
| TUH_CURATED_AR | 81 | 43 | 38 | 24.9 | 728 | 1.5 |
| TUH_CURATED_LE | 6 | 5 | 1 | 1.5 | 16 | 0.1 |
| TUH_BULK_AR | 51 | 47 | 4 | 5.4 | 556 | 1.0 |
| TUH_BULK_LE | 26 | 21 | 5 | 5.4 | 464 | 2.0 |

## 11 Screen arms

| Arm | Temple hours added | Temple items per pass | Share of training draws | Total training hours | Multiple of control | d10 passes in 3,000 steps |
|---|---:|---:|---:|---:|---:|---:|
| control (d10 only) | 0 | 0 | 0.0 | 2,525 | 1.00 | 1.37 |
| curated | 1,650.3 | 49,243 | 41.3 | 4,175.3 | 1.65 | 0.80 |
| curated_bulk | 4,561.3 | 136,891 | 66.1 | 7,086.3 | 2.81 | 0.46 |

## 12 All datasets (hours)

| Bucket | Hours |
|---|---:|
| d10 training corpus (17 datasets) | 3,261.0 |
| Processed, not in a training run (I-CARE 899, neonatal 111, CHBMP 106, ...) | 1,213.0 |
| Converted to shards, not trained on (CinC sleep 15,261, OSA 1,120, CAP 1,004, NMT 489, ...) | 18,506.0 |
| Temple (TUH) converted shards | 5,797.7 |
| TOTAL we hold today | 28,777.7 |
| + Temple still to arrive (~16.5k raw h minus ~3% unusable) | 16,000.0 |
| TOTAL after the Temple transfer (approx.) | 44,777.7 |
| + PEERS (ds004395, not started, 9.6 TB) | 10,500.0 |
| TOTAL with PEERS (approx.) | 55,277.7 |

---
Notes: (8) `#` characters are non-printable bytes in the EDF header recorder tag (garbled tags are small groups); (9) a recording can be flagged by several reasons, so the reasons add up to more than the 4,212 test recordings; (6) age is the EDF header age (0-110 only; missing or sentinel values excluded), sex is never stored.