Compare commits

..
Author SHA1 Message Date
PaddyandClaude Opus 5.5 adf723419d Match a submission's client token only within its event
Test and deploy / test-and-deploy (pull_request) Successful in 27s
The token is per form visit, and one visit can move on to another
concert's form in the same tab. With a global match, the second event's
submission was answered with the first event's id and its feedback was
silently dropped. The lookup now joins on the event slug, and the unique
index in 003 is (event_id, client_token) instead of client_token alone.

003 is edited in place: it has not been applied anywhere yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 18:27:27 +01:00
PaddyandClaude Opus 5.5 f3f4ca2c9f Answer a retried feedback submission with the one already saved
Test and deploy / test-and-deploy (pull_request) Successful in 46s
When a submission saved but its response never reached the phone, the
form showed an error, the visitor tapped "Feedback absenden" again, and a
second identical row was stored (seen twice in production: 82/83, 87/88).

The form now sends a clientToken, one UUID per visit, on every attempt.
A token that is already stored returns the original submissionId and
saves nothing; the lookup runs before the eligibility check, so a retry
still succeeds if the feedback window closed in between. Two concurrent
retries are settled by a UNIQUE index on submissions.client_token: the
loser gets ER_DUP_ENTRY, rolls back and returns the winner's id. A missing
or non-UUID token behaves as before, so cached copies of the old form
still submit.

Migration 003 must be applied before this is deployed - the INSERT now
writes client_token.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 17:48:33 +01:00
Paddy c9c40af382 Merge pull request 'Run tests and deploy in one Gitea Actions job' (#23) from feature/ci-single-job into master
Test and deploy / test-and-deploy (push) Successful in 55s
Reviewed-on: #23
2026-11-08 21:53:58 +00:00
PaddyandClaude Opus 5.5 ae4f153482 Run tests and deploy in one Gitea Actions job
Test and deploy / test-and-deploy (pull_request) Successful in 33s
Two jobs meant checking out, setting up Node and running npm ci (and in the
frontend the build) twice, about two minutes each, because every job starts
in a fresh container. Now one job tests and, for pushes to master only,
deploys. The push-to-master condition is spelled out on every deploy step
instead of an env flag compared to 'true', which would skip the deploy
silently and stay green if the boolean were rendered differently. A failed
test still stops the deploy (an if: without a status function implies
success()).

Docs: the one-job layout, and that Jenkins deployed as root - files it left
in an app folder must be chowned to the subscription user before the first
Gitea deploy (hit on the API).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 22:51:30 +01:00
7 changed files with 268 additions and 36 deletions

No files matched your search

+23 -21
View File
@@ -13,23 +13,12 @@ on:
pull_request:
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: .nvmrc
- run: npm ci
- run: npx tsc --noEmit
# Unit tests only. The integration tests (npm run test:integration) start their own MariaDB
# container and aren't run here.
# FEEDBACK_IP_SALT (which the Jenkins job exported) isn't needed: vitest.config.ts sets one.
- run: npm test
deploy:
needs: test
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
# One job, not test + deploy: every job starts in a fresh container, so two jobs meant checking out,
# setting up Node and running `npm ci` twice (~2 min each, 2026-11-08). The deploy steps only run
# for pushes to master; on pull requests the job ends after the tests. The condition is spelled out
# on every deploy step rather than stored in an env flag: a flag compared as a string ('true') would
# skip the deploy silently - and green - if the runner ever rendered the boolean differently.
test-and-deploy:
runs-on: ubuntu-latest
env:
DEPLOY_HOST: ${{ vars.DEPLOY_HOST }}
@@ -45,11 +34,22 @@ jobs:
with:
node-version-file: .nvmrc
- run: npm ci
- run: npm run build
- run: npx tsc --noEmit
# Unit tests only. The integration tests (npm run test:integration) start their own MariaDB
# container and aren't run here.
# FEEDBACK_IP_SALT (which the Jenkins job exported) isn't needed: vitest.config.ts sets one.
- run: npm test
- name: Build
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
run: npm run build
# Only runtime dependencies go to the server (no typescript, vitest, ...).
- run: npm prune --omit=dev
- name: Prune dev dependencies
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
run: npm prune --omit=dev
- name: SSH setup
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
env:
DEPLOY_SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
DEPLOY_KNOWN_HOSTS: ${{ secrets.DEPLOY_KNOWN_HOSTS }}
@@ -65,6 +65,7 @@ jobs:
printf '%s\n' "$DEPLOY_KNOWN_HOSTS" > "$SSH_DIR/known_hosts"
- name: Upload
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
run: |
rsh="ssh -p ${DEPLOY_PORT:-22} -i $SSH_DIR/key -o UserKnownHostsFile=$SSH_DIR/known_hosts -o IdentitiesOnly=yes -o BatchMode=yes"
target="$DEPLOY_USER@$DEPLOY_HOST:$DEPLOY_PATH"
@@ -78,6 +79,7 @@ jobs:
rsync -az -e "$rsh" package.json package-lock.json "$target/"
- name: Restart
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
run: |
# Passenger restarts the app on its next request when tmp/restart.txt is touched.
ssh -p "${DEPLOY_PORT:-22}" -i "$SSH_DIR/key" -o "UserKnownHostsFile=$SSH_DIR/known_hosts" \
@@ -87,7 +89,7 @@ jobs:
# Set DEPLOY_SMOKE_URL to an endpoint that must answer 200 without a session, e.g.
# https://api.nachklang.art/calendar/events/public/json (what nachklang.art itself reads).
- name: Smoke test
if: vars.DEPLOY_SMOKE_URL != ''
if: github.event_name == 'push' && github.ref == 'refs/heads/master' && vars.DEPLOY_SMOKE_URL != ''
env:
DEPLOY_SMOKE_URL: ${{ vars.DEPLOY_SMOKE_URL }}
run: |
@@ -100,5 +102,5 @@ jobs:
exit 1
- name: Remove key
if: always()
if: always() && github.event_name == 'push' && github.ref == 'refs/heads/master'
run: rm -f "$SSH_DIR/key"
+1
View File
@@ -1,3 +1,4 @@
USE nachklang_feedback;
SOURCE /migrations/feedback/001_init.sql;
SOURCE /migrations/feedback/002_add_poster_image_url.sql;
SOURCE /migrations/feedback/003_add_submission_client_token.sql;
+18 -4
View File
@@ -8,12 +8,13 @@ Node.js App").
```
push to master ──▶ Gitea ──▶ runner (container on the server)
├─ test job: npm ci, tsc --noEmit, npm test (unit tests)
└─ deploy job: npm ci, build, prune dev deps ─ rsync over SSH ─▶
└─ one job: npm ci, tsc --noEmit, npm test (unit tests)
└─ (master only) build, prune dev deps ─ rsync over SSH ─▶
plesk-nachklang@nachklang.art:<app root> + tmp/restart.txt
```
Pull requests only run the test job.
Pull requests run the same job up to the tests; the deploy steps only run for pushes to master. It
used to be two jobs, which ran `npm ci` twice (~2 min each); merged into one on 2026-11-08.
## Plesk (as configured, 2026-11-08)
@@ -69,8 +70,14 @@ is the one nachklang.art itself depends on.
## Switchover (done once)
1. Pause the Jenkins API job (done 2026-11-08).
1a. **Fix file ownership** (as root): Jenkins deployed as **root**, so `dist/`, `src/` and
`node_modules/` were `root:root`, and the first Gitea deploy failed with rsync "Operation not
permitted" / "Permission denied" - the subscription user can't change root's files. Fixed with
`chown -R plesk-nachklang:psacln dist src node_modules` in the app root (the group of the files
the app creates itself, e.g. `.env`, `logs/`); then re-run the failed job.
Check: `find . ! -user plesk-nachklang` prints nothing.
2. Set `DEPLOY_PATH` and `DEPLOY_SMOKE_URL`.
3. Open a PR → green test job → merge → watch the deploy job. Unlike the calendar there is no gap:
3. Open a PR → green run → merge → watch the deploy steps. Unlike the calendar there is no gap:
the folder keeps working while files are replaced, and Passenger restarts on the next request.
4. Check `https://api.nachklang.art/docs` and the calendar/feedback/tickets apps.
@@ -93,6 +100,13 @@ Not checked: the workflow on the real runner.
## Notes
- There was **no `package.json` on the server** before the first rsync deploy (Jenkins never copied
it); `dist/app.js` still ran as ES modules, probably because Node ≥ 22 detects ES module syntax on
its own [inference, not checked]. The deploy now ships `package.json`, so it doesn't depend on that.
- A failed upload leaves new and old files mixed in `dist/`, but the running app is unaffected:
the restart step is skipped, so Passenger keeps the old code in memory until the next deploy (or an
unplanned restart - fix and re-run soon).
- `bcrypt` was removed on 2026-11-08: nothing imported it any more (passwords moved to better-auth,
which hashes with its own scrypt). It was the only production dependency with a native build step.
- `FEEDBACK_IP_SALT`, which the Jenkins test step exported, isn't needed in CI: `vitest.config.ts`
@@ -0,0 +1,17 @@
-- Nachklang e.V. Feedback module — makes submitting idempotent per form visit.
-- Apply manually against the FEEDBACK_DB database, after 002_add_poster_image_url.sql:
-- mysql -h <DB_HOST> -u <DB_USER> -p <FEEDBACK_DB> < 003_add_submission_client_token.sql
--
-- The form generates one UUID per visit and resends it on every retry. When
-- a submission saved but its response never reached the phone, the visitor
-- sees an error and taps "absenden" again; the API now finds the stored token
-- and answers with the original submission instead of saving a copy.
--
-- Unique per event, not globally: one visit can switch to another concert's
-- form in the same tab, and that event's submission must still save.
--
-- NULL for every row written before this migration and for any client that
-- sends no token; a UNIQUE index allows any number of NULLs.
ALTER TABLE submissions
ADD COLUMN client_token CHAR(36) NULL AFTER ip_hash,
ADD UNIQUE KEY uq_sub_event_client_token (event_id, client_token);
@@ -56,6 +56,10 @@
* website:
* type: string
* description: Honeypot field. Must stay empty; a real visitor never fills it in.
* clientToken:
* type: string
* format: uuid
* description: Generated once per form and resent on every retry. A retry whose token is already stored for this event returns the original submissionId instead of saving a duplicate. Optional; a value that isn't a UUID is ignored.
* SubmissionResponse:
* type: object
* properties:
@@ -95,4 +99,5 @@ export interface SubmissionRequestBody {
guestBook?: GuestBookInput;
newsletter?: NewsletterInput;
website?: string;
clientToken?: string;
}
@@ -11,6 +11,8 @@ const CONSENT_TEXT_VERSION = '2026-08-02';
const EMAIL_RE = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
const CLIENT_TOKEN_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
// A real setlist tops out around a few dozen songs and a handful of
// questions, so a legitimate submission never comes close to this. Caps
// total generated rows regardless of how large the client's answers/ratings
@@ -136,6 +138,33 @@ export const validateNewsletter = (input?: NewsletterInput): ValidatedNewsletter
return {firstName, lastName, email};
};
/**
* The form generates one UUID per visit and resends it on every retry, so a
* request whose response was lost (the row saved, but the phone never heard
* back) can be told apart from a genuinely new submission. Anything that
* isn't a UUID is treated as "no token" rather than rejected - an old cached
* copy of the form must still be able to submit.
*/
export const normalizeClientToken = (token: unknown): string | null => {
return typeof token === 'string' && CLIENT_TOKEN_RE.test(token) ? token.toLowerCase() : null;
};
// Scoped to the event: the token is per form visit, and one visit can move
// on to another concert's form, so a match for a different event is not a
// retry of this one.
const findSubmissionIdByClientToken = async (clientToken: string, slug: string): Promise<number | null> => {
const conn = await NachklangFeedbackDB.getConnection();
try {
const rows = await conn.query(
'SELECT s.submission_id FROM submissions s JOIN events e ON e.event_id = s.event_id WHERE s.client_token = ? AND e.slug = ?',
[clientToken, slug]
);
return rows.length > 0 ? rows[0].submission_id : null;
} finally {
await conn.end();
}
};
export type SubmitResult =
| { status: 'OK'; submissionId: number; newsletterDropped: boolean }
| { status: 'NOT_FOUND' }
@@ -146,9 +175,29 @@ export type SubmitResult =
* Validates and persists one feedback submission. Re-checks event
* eligibility (the window may have closed between page load and submit),
* validates every answer against the event's live questions/songs, then
* inserts everything in a single transaction.
* inserts everything in a single transaction. A retry carrying an already
* stored client token gets the original submission back instead of a copy.
*/
export const submitFeedback = async (slug: string, body: SubmissionRequestBody, ipHash: string | null): Promise<SubmitResult> => {
const clientToken = normalizeClientToken(body.clientToken);
const newsletter = validateNewsletter(body.newsletter);
// body.newsletter is only sent at all when the visitor had the opt-in
// checkbox on (see FeedbackForm.tsx), so a present-but-invalid block
// (e.g. a mistyped email) is distinguishable from "didn't opt in" - the
// rest of the submission still saves, but the client can tell the
// visitor their newsletter signup specifically didn't go through.
const newsletterDropped = !!body.newsletter && !newsletter;
// Checked before eligibility, so a retry still succeeds if the feedback
// window closed in the seconds after the original saved.
if (clientToken) {
const existingId = await findSubmissionIdByClientToken(clientToken, slug);
if (existingId !== null) {
logger.info('Feedback submission retry answered with the stored submission', {submissionId: existingId});
return {status: 'OK', submissionId: existingId, newsletterDropped};
}
}
const lookup = await getEventConfigBySlug(slug);
if (lookup.status === 'NOT_FOUND') return {status: 'NOT_FOUND'};
if (lookup.status === 'CLOSED') return {status: 'CLOSED'};
@@ -159,13 +208,6 @@ export const submitFeedback = async (slug: string, body: SubmissionRequestBody,
const answerRows = validateAnswers(body.answers || [], questionsById, songTitleById);
const guestBook = validateGuestBook(body.guestBook);
const newsletter = validateNewsletter(body.newsletter);
// body.newsletter is only sent at all when the visitor had the opt-in
// checkbox on (see FeedbackForm.tsx), so a present-but-invalid block
// (e.g. a mistyped email) is distinguishable from "didn't opt in" - the
// rest of the submission still saves, but the client can tell the
// visitor their newsletter signup specifically didn't go through.
const newsletterDropped = !!body.newsletter && !newsletter;
if (answerRows.length === 0 && !guestBook && !newsletter) {
return {status: 'EMPTY'};
@@ -175,8 +217,8 @@ export const submitFeedback = async (slug: string, body: SubmissionRequestBody,
try {
await conn.beginTransaction();
const subQuery = 'INSERT INTO submissions (event_id, ip_hash, has_guestbook, has_newsletter) VALUES (?,?,?,?) RETURNING submission_id';
const subRes = await conn.query(subQuery, [eventId, ipHash, guestBook ? 1 : 0, newsletter ? 1 : 0]);
const subQuery = 'INSERT INTO submissions (event_id, ip_hash, has_guestbook, has_newsletter, client_token) VALUES (?,?,?,?,?) RETURNING submission_id';
const subRes = await conn.query(subQuery, [eventId, ipHash, guestBook ? 1 : 0, newsletter ? 1 : 0, clientToken]);
const submissionId = subRes[0].submission_id;
for (const row of answerRows) {
@@ -220,8 +262,16 @@ export const submitFeedback = async (slug: string, body: SubmissionRequestBody,
}
return {status: 'OK', submissionId, newsletterDropped};
} catch (err) {
} catch (err: any) {
await conn.rollback();
// Two copies of the same retry both got past the lookup above; the
// unique index on (event_id, client_token) let only one of them commit.
if (clientToken && err?.code === 'ER_DUP_ENTRY') {
const existingId = await findSubmissionIdByClientToken(clientToken, slug);
if (existingId !== null) {
return {status: 'OK', submissionId: existingId, newsletterDropped};
}
}
throw err;
} finally {
await conn.end();
@@ -0,0 +1,143 @@
import {vi, describe, it, expect, beforeEach, type Mock} from 'vitest';
vi.mock('../../src/models/feedback/Feedback.db.js', () => ({
NachklangFeedbackDB: {getConnection: vi.fn()}
}));
vi.mock('../../src/models/feedback/public/events.public.service.js', () => ({
getEventConfigBySlug: vi.fn()
}));
vi.mock('../../src/models/feedback/integrations/salesforce.service.js', () => ({
syncNewsletterSignup: vi.fn().mockResolvedValue(undefined)
}));
vi.mock('../../src/middleware/logger.js', () => ({
__esModule: true,
default: {info: vi.fn(), error: vi.fn()}
}));
import {NachklangFeedbackDB} from '../../src/models/feedback/Feedback.db.js';
import {getEventConfigBySlug} from '../../src/models/feedback/public/events.public.service.js';
import {submitFeedback} from '../../src/models/feedback/public/submissions.service.js';
const mockGetConnection = NachklangFeedbackDB.getConnection as Mock;
const mockGetEventConfigBySlug = getEventConfigBySlug as Mock;
const TOKEN = '3f2b8c1e-4d5a-4b6c-9e7f-0a1b2c3d4e5f';
const BODY = {answers: [{eventQuestionId: 26, text: 'Danke!'}], clientToken: TOKEN};
const SLUG = 'sommerkonzert-2026';
const LOOKUP_SQL = 'SELECT s.submission_id FROM submissions s';
// One fake connection serves every query; `existingId` is what a lookup by
// client_token finds when it was stored for `existingSlug`, and
// `insertError` makes the submissions INSERT throw.
const makeConn = ({existingId = null, existingSlug = SLUG, insertError = null}: {existingId?: number | null; existingSlug?: string; insertError?: any} = {}) => ({
query: vi.fn(async (sql: string, params: any[] = []) => {
if (sql.startsWith(LOOKUP_SQL)) {
return existingId === null || params[1] !== existingSlug ? [] : [{submission_id: existingId}];
}
if (sql.startsWith('INSERT INTO submissions')) {
if (insertError) throw insertError;
return [{submission_id: 90}];
}
return [];
}),
beginTransaction: vi.fn().mockResolvedValue(undefined),
commit: vi.fn().mockResolvedValue(undefined),
rollback: vi.fn().mockResolvedValue(undefined),
end: vi.fn().mockResolvedValue(undefined)
});
const allQueries = (conn: ReturnType<typeof makeConn>): [string, any[]][] => conn.query.mock.calls as any;
describe('submitFeedback - client token idempotency', () => {
beforeEach(() => {
mockGetConnection.mockReset();
mockGetEventConfigBySlug.mockReset();
mockGetEventConfigBySlug.mockResolvedValue({
status: 'OK',
eventId: 22,
event: {
questions: [{eventQuestionId: 26, questionId: 4, type: 'FREE_TEXT', label: 'Sonstiges', position: 0}],
songs: []
}
});
});
it('returns the already-saved submission for a repeated token without inserting again', async () => {
const conn = makeConn({existingId: 82});
mockGetConnection.mockResolvedValue(conn);
const result = await submitFeedback('sommerkonzert-2026', BODY, 'hash');
expect(result).toEqual({status: 'OK', submissionId: 82, newsletterDropped: false});
expect(allQueries(conn).some(([sql]) => sql.startsWith('INSERT'))).toBe(false);
});
it('still answers a retry with the saved submission after the feedback window closed', async () => {
mockGetConnection.mockResolvedValue(makeConn({existingId: 82}));
mockGetEventConfigBySlug.mockResolvedValue({status: 'CLOSED'});
const result = await submitFeedback('sommerkonzert-2026', BODY, 'hash');
expect(result).toEqual({status: 'OK', submissionId: 82, newsletterDropped: false});
});
it('stores a new token, lowercased, on the submissions row', async () => {
const conn = makeConn();
mockGetConnection.mockResolvedValue(conn);
const result = await submitFeedback('sommerkonzert-2026', {...BODY, clientToken: TOKEN.toUpperCase()}, 'hash');
expect(result).toEqual({status: 'OK', submissionId: 90, newsletterDropped: false});
const insert = allQueries(conn).find(([sql]) => sql.startsWith('INSERT INTO submissions'))!;
expect(insert[1]).toContain(TOKEN);
});
it('saves a new submission when the token was only ever used for a different event', async () => {
// Same browser tab, two concerts: the token must not make the second
// event's feedback come back as the first event's submission.
const conn = makeConn({existingId: 82, existingSlug: 'winterkonzert-2026'});
mockGetConnection.mockResolvedValue(conn);
const result = await submitFeedback(SLUG, BODY, 'hash');
expect(result).toEqual({status: 'OK', submissionId: 90, newsletterDropped: false});
expect(allQueries(conn).find(([sql]) => sql.startsWith(LOOKUP_SQL))![1]).toEqual([TOKEN, SLUG]);
});
it('ignores a malformed token: no lookup, and the row is stored without one', async () => {
const conn = makeConn();
mockGetConnection.mockResolvedValue(conn);
await submitFeedback('sommerkonzert-2026', {...BODY, clientToken: 'not-a-uuid'}, 'hash');
expect(allQueries(conn).some(([sql]) => sql.startsWith(LOOKUP_SQL))).toBe(false);
const insert = allQueries(conn).find(([sql]) => sql.startsWith('INSERT INTO submissions'))!;
expect(insert[1]).not.toContain('not-a-uuid');
expect(insert[1][insert[1].length - 1]).toBeNull();
});
it('resolves a concurrent duplicate that loses the race on the unique index to the winning submission', async () => {
const dupError = Object.assign(new Error('Duplicate entry'), {code: 'ER_DUP_ENTRY', errno: 1062});
const racingConn = makeConn({insertError: dupError});
// First lookup (before the INSERT) finds nothing; the one after the
// duplicate-key error finds the row the other request committed.
const lookupAfterRace = makeConn({existingId: 83});
mockGetConnection
.mockResolvedValueOnce(makeConn())
.mockResolvedValueOnce(racingConn)
.mockResolvedValueOnce(lookupAfterRace);
const result = await submitFeedback('sommerkonzert-2026', BODY, 'hash');
expect(result).toEqual({status: 'OK', submissionId: 83, newsletterDropped: false});
expect(racingConn.rollback).toHaveBeenCalled();
expect(racingConn.commit).not.toHaveBeenCalled();
});
it('rethrows a duplicate-key error when the request carried no token', async () => {
const dupError = Object.assign(new Error('Duplicate entry'), {code: 'ER_DUP_ENTRY', errno: 1062});
mockGetConnection.mockResolvedValue(makeConn({insertError: dupError}));
await expect(submitFeedback('sommerkonzert-2026', {answers: BODY.answers}, 'hash')).rejects.toThrow('Duplicate entry');
});
});