Paddy
d418f156fa
- Date pickers (entry form + edit dialog) now display TT.MM.JJJJ instead of
ISO (DATE_DISPLAY_FORMAT). Storage stays YYYY-MM-DD; parse_date() already
accepted both formats, so existing data and the self-updater are unaffected.
- New optional "Straße" field (street + house number) in the entry form and
edit dialog, backed by a new `street` CSV column. geocode_city() and
GeocoderWorker.enqueue() gained a street parameter: when set, a full-address
query is tried first for a much more precise map point, falling back
automatically to the existing city/PLZ search if it doesn't resolve.
- Tab 2 and the entry queue show a Straße column; Tab 2 search now also
matches on street.
- Fix: pandas turns a blank CSV cell into NaN even for a dtype=str column, so
every existing (blank-street) row would have shown literal "nan" in Tab 2.
DataStore._load now does street.fillna("") after every read.
Verified with a non-GUI test suite (date parsing, query construction, CSV
round-trip incl. the NaN case) and a full GUI build/drive test on Python
3.12/Tk 9. Docs updated (changelog, overview, architecture, data-model,
dev-notes, improvements).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
82 lines
4.2 KiB
Markdown
82 lines
4.2 KiB
Markdown
# Data model
|
||
|
||
## `data/orte.csv`
|
||
|
||
Plain CSV, comma-separated, UTF-8, with a header row. Created automatically with
|
||
just the header if it does not exist.
|
||
|
||
| Column | Type on disk | Meaning | Notes |
|
||
|--------|--------------|---------|-------|
|
||
| `date` | string | Assignment date | Always **stored as `YYYY-MM-DD`**, regardless of display. Input is validated and normalized by `parse_date()` (accepts `YYYY-MM-DD`, `DD.MM.YYYY`, `DD.MM.YY`, `YYYY/MM/DD`); invalid input is rejected before saving. The date pickers *display* `TT.MM.JJJJ` (`DATE_DISPLAY_FORMAT`) since the 2026-09-12 change, but that's cosmetic — parsing/storage is unchanged, so old and new rows are identical on disk. |
|
||
| `city` | string | Place name | Free text. Used for autocomplete and duplicate detection (trimmed, case-insensitive). |
|
||
| `postal_code` | string | German PLZ | Optional. Kept as a string so leading zeros survive. Regex elsewhere accepts 4–5 digits. |
|
||
| `street` | string | Street + house number | Optional (e.g. `"Hauptstraße 12"`). Added 2026-09-12 for more precise geocoding — see [overview.md](overview.md#geocoding-behaviour). Rows written before that date have it blank. |
|
||
| `lat` | string | Latitude | Blank until geocoded. Rounded to 5 dp. Parsed with `pd.to_numeric(errors="coerce")` when building the map. |
|
||
| `lon` | string | Longitude | As above. |
|
||
|
||
Every column is read as a string (`dtype={"postal_code": str, "lat": str, "lon": str}`
|
||
plus `city`/`date` default object) — **except this doesn't stop a blank cell from
|
||
coming back as `NaN`** (a float), even for a column forced to `dtype=str`. This
|
||
bit `street` immediately (all pre-existing rows have it blank): `_load()` now
|
||
does `self.df["street"] = self.df["street"].fillna("")` right after reading.
|
||
See [dev-notes.md](dev-notes.md#pandas-turns-blank-csv-cells-into-nan-even-with-dtypestr).
|
||
|
||
### Row identity
|
||
|
||
There is **no ID column**. Rows are identified by their pandas `DataFrame`
|
||
index, which is reset to `0..n-1` after every delete. UI tables use that index
|
||
as the Treeview `iid`.
|
||
|
||
Implication: a geocode callback that is in flight while the user deletes a
|
||
different row can land on the wrong row, because indices shift. `update_row` /
|
||
`_apply_edit_geocode` guard against a *missing* index but not against a
|
||
*reused* one. Still open — see [improvements.md](improvements.md#7-stable-row-identity-).
|
||
The manual lat/lon fields in the edit dialog give a way to correct any row
|
||
that ends up wrong.
|
||
|
||
### Duplicate detection
|
||
|
||
`find_duplicates` flags a queued row when an existing row matches on all three
|
||
of: `date` (string-equal), `city` (trimmed, lower-cased), `postal_code`
|
||
(trimmed). `street` is **not** part of the match (unchanged by the 2026-09-12
|
||
address feature) — two visits to the same city/PLZ on the same day are flagged
|
||
as possible duplicates even with different streets. A modal now asks whether to
|
||
save anyway (see [changelog.md](changelog.md)).
|
||
|
||
### Map aggregation
|
||
|
||
`get_map_data` groups by `(city, lat, lon)` and counts rows. Two entries for the
|
||
same city with different coordinates (e.g. Karlsruhe geocoded once to `76133`
|
||
and once to `76185`) produce **two separate markers**.
|
||
|
||
## `data/karte.html`
|
||
|
||
Regenerated from scratch every time the user opens the map. Safe to delete; it
|
||
is a build artefact, not data. Should be git-ignored if the project is ever put
|
||
under version control.
|
||
|
||
## `data/backups/`
|
||
|
||
Automatic timestamped copies of `orte.csv`, named `orte-YYYYMMDD-HHMMSS.csv`.
|
||
One is written at startup and one before every change (skipped when nothing
|
||
changed since the last backup). The newest 20 are kept; older ones are pruned.
|
||
Git-ignored. To restore, copy a backup over `data/orte.csv` while the app is
|
||
closed.
|
||
|
||
## `data/app.log`
|
||
|
||
Rotating log file (512 KB × 3 generations). Git-ignored. Records saves, map
|
||
generation, geocoding failures, and uncaught exceptions.
|
||
|
||
## `data/window.json`
|
||
|
||
`{"geometry": "<w>x<h>+<x>+<y>"}` – the window size and position, saved on close
|
||
and restored on start. Git-ignored, per-machine. Safe to delete (window opens at
|
||
its default size).
|
||
|
||
## `icon.png` (repo root) and `data/icon.png`
|
||
|
||
`icon.png` in the repo root is the **bundled** launcher/window icon (committed).
|
||
`data/icon.png` is only a runtime-drawn fallback the app writes if the bundled
|
||
one is missing — git-ignored.
|