Paddy
d418f156fa
- Date pickers (entry form + edit dialog) now display TT.MM.JJJJ instead of
ISO (DATE_DISPLAY_FORMAT). Storage stays YYYY-MM-DD; parse_date() already
accepted both formats, so existing data and the self-updater are unaffected.
- New optional "Straße" field (street + house number) in the entry form and
edit dialog, backed by a new `street` CSV column. geocode_city() and
GeocoderWorker.enqueue() gained a street parameter: when set, a full-address
query is tried first for a much more precise map point, falling back
automatically to the existing city/PLZ search if it doesn't resolve.
- Tab 2 and the entry queue show a Straße column; Tab 2 search now also
matches on street.
- Fix: pandas turns a blank CSV cell into NaN even for a dtype=str column, so
every existing (blank-street) row would have shown literal "nan" in Tab 2.
DataStore._load now does street.fillna("") after every read.
Verified with a non-GUI test suite (date parsing, query construction, CSV
round-trip incl. the NaN case) and a full GUI build/drive test on Python
3.12/Tk 9. Docs updated (changelog, overview, architecture, data-model,
dev-notes, improvements).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
118 lines
6.9 KiB
Markdown
118 lines
6.9 KiB
Markdown
# Architecture
|
||
|
||
Everything lives in [`app.py`](../app.py). There is no package structure.
|
||
|
||
## Components
|
||
|
||
| Class / function | Responsibility |
|
||
|------------------|----------------|
|
||
| `DataStore` | Owns `orte.csv`. Loads it into a pandas `DataFrame`, exposes read helpers (`get_map_data`, `get_autocomplete_strings`, `total_count`, `find_duplicates`) and write helpers (`append_rows`, `update_row`, `delete_rows`). All writes go through `_save_csv` → rotating backup + atomic write. All columns are read as strings. Falls back to an empty frame (or the newest backup) if `orte.csv` is missing/empty/corrupt. |
|
||
| `geocode_city()` | Pure function: `(city, plz, street="") -> (lat, lon) | None`. Tries several Nominatim queries with a configurable regional bias (`GEOCODE_REGIONS`); if `street` is given, a full-address query is tried first and falls back to the city/PLZ-only queries on no match. Raises `GeocodingUnavailable` if *every* query failed on a service/network error (vs. simply not finding the place). |
|
||
| `GeocoderWorker` | A `threading.Thread` (daemon). Holds a FIFO list of `(row_id, city, plz, street, callback)` items guarded by a lock, woken by an `Event`. Processes one item per second, calls `geocode_city`, then invokes `callback` **from the worker thread**. Each item is wrapped in `try/except` so one failure never kills the thread; after `SERVICE_ERROR_THRESHOLD` consecutive service errors it fires the optional `on_service_error` callback. |
|
||
| `parse_date()` / `parse_coord()` / helpers | Module-level pure functions for normalizing user input (date → ISO, coordinate strings → float, `"City (PLZ)"` splitting). Shared by the entry tab and `EditDialog`. |
|
||
| `generate_map()` | Builds `karte.html` from `DataStore.get_map_data()` with Folium. |
|
||
| `EditDialog` | `tk.Toplevel` modal dialog for editing one row. Returns its result via `self.result`. |
|
||
| `Updater` | Wraps `git` for self-update. `available` is true only from a git clone with `git` on `PATH`. `check()` → new-commit count (`fetch` + `rev-list`); `update()` → `fetch` + `merge --ff-only` + conditional `pip install`. All git calls are `subprocess.run` with timeouts. Raises `UpdateError` (user-facing German text) on any failure. |
|
||
| `App` | `ttkbootstrap.Window`. Builds the UI + menubar, owns the `DataStore`, the `GeocoderWorker`, the `Updater`, and the in-memory `_queue` list for Tab 1. |
|
||
|
||
## Data flow
|
||
|
||
### Adding entries
|
||
```
|
||
user types -> _on_add_row()
|
||
-> append dict to self._queue
|
||
-> insert row into Tab-1 Treeview (coords = "⏳ wird gesucht…")
|
||
-> geocoder.enqueue(row_id, city, plz, street, self._geocode_done)
|
||
|
||
GeocoderWorker thread (1/sec):
|
||
coords = geocode_city(...)
|
||
-> self._geocode_done(row_id, coords) [worker thread]
|
||
-> self.after(0, self._apply_geocode_result, ...) [hop to UI thread]
|
||
-> mutate self._queue entry + update Treeview cell
|
||
|
||
user clicks "Alle speichern" -> _on_save_all()
|
||
-> DataStore.find_duplicates() (status-bar hint only)
|
||
-> DataStore.append_rows() -> CSV append + in-memory concat
|
||
-> clear queue, refresh Tab 2, update autocomplete
|
||
```
|
||
|
||
### Editing / deleting
|
||
`EditDialog` -> `DataStore.update_row(df_idx, values)` or
|
||
`DataStore.delete_rows(indices)` -> full `df.to_csv()` rewrite -> refresh.
|
||
|
||
## Threading model
|
||
|
||
- Tkinter is single-threaded; all widget access must happen on the main thread.
|
||
- The geocoder runs off-thread so the UI never blocks for the 1-second-per-item
|
||
rate limit.
|
||
- The **callback is invoked on the worker thread**, and every callback in
|
||
`App` immediately does `self.after(0, ...)` to marshal back onto the UI
|
||
thread. This is the load-bearing convention — any new callback must follow it.
|
||
(`on_service_error` does the same.)
|
||
- The worker catches every exception per item, so a bug in a callback or a
|
||
network error logs a traceback but never stops the queue.
|
||
- On window close, `_on_close()` calls `geocoder.stop()` (sets a flag + wakes
|
||
the event) and then `destroy()`. The thread is a daemon, so a missed stop
|
||
won't hang the process.
|
||
|
||
## Persistence model
|
||
|
||
- `orte.csv` is the single source of truth. It is read once at startup and then
|
||
kept in sync in memory.
|
||
- **Every** write (append, update, delete) rebuilds the full `DataFrame` and
|
||
goes through `DataStore._save_csv`:
|
||
1. `_make_backup()` – copy the current file to
|
||
`data/backups/orte-YYYYMMDD-HHMMSS.csv` (skipped if byte-identical to the
|
||
newest backup); prune to `MAX_BACKUPS` (20).
|
||
2. `_write_atomic()` – write to `orte.csv.tmp`, `fsync`, then `os.replace()`
|
||
onto `orte.csv` (atomic on POSIX). A crash mid-write leaves the old file
|
||
intact.
|
||
- A backup is also taken once at startup.
|
||
- Still **no file locking**: if the CSV is edited in another program while the
|
||
app is open, the next in-app save overwrites those changes (but the pre-write
|
||
backup captures them).
|
||
|
||
## Self-update flow
|
||
|
||
```
|
||
App start ─► daemon thread: sleep 2s ─► Updater.check()
|
||
(fetch + count) └─ on error: log only, stay quiet
|
||
│
|
||
N > 0 ─► self.after(0, …) ─► status hint + menu label
|
||
+ "Update verfügbar?" dialog
|
||
|
||
Hilfe ▸ Nach Update suchen ─► worker thread ─► Updater.check()
|
||
│
|
||
error ─► warning dialog (offline?)
|
||
N = 0 ─► "aktuell" dialog
|
||
N > 0 ─► "jetzt installieren?" dialog
|
||
|
||
install ─► worker thread ─► Updater.update() (fetch, ff-only merge, pip)
|
||
success + changed ─► info dialog ─► _restart()
|
||
save window state,
|
||
stop geocoder,
|
||
os.chdir(BASE_DIR),
|
||
os.execv(python, [python, app.py])
|
||
ff-only fails / pip fails ─► error dialog, no restart
|
||
```
|
||
|
||
Everything network- or subprocess-bound runs off the UI thread; results are
|
||
marshalled back with `self.after(0, …)` (same rule as the geocoder).
|
||
|
||
## Logging
|
||
|
||
`_setup_logging()` (called from `__main__`) attaches a `RotatingFileHandler` to
|
||
`data/app.log` (512 KB × 3). Logger name `arbeitsorte`, module-level `log`.
|
||
Records saves, map generation, geocoding failures, and any uncaught exception.
|
||
This is the file to ask for when diagnosing a problem on the user's laptop.
|
||
|
||
## External dependencies at runtime
|
||
|
||
| Dependency | Needed for | Offline behaviour |
|
||
|------------|-----------|-------------------|
|
||
| Nominatim (OSM) | geocoding new/edited entries | entries save with blank coords, shown as `⚠ nicht gefunden` |
|
||
| CDN (jsdelivr, jquery, cloudflare) | rendering `karte.html` | map page loads blank / unstyled |
|
||
| Esri tiles (`server.arcgisonline.com`) | map background | blank/grey tiles |
|
||
|
||
Existing entries that already have coordinates do **not** need the network.
|