Teams Pipeline

Downloads team rosters from Sportlink, creates team posts in Rondo Club, and links members to teams via dated work history.

Schedule

Runs weekly on Sunday at 6:00 AM (Amsterdam time).

scripts/sync.sh teams           # Production (with locking + email report)
node pipelines/sync-teams.js --verbose    # Direct execution (verbose)

Pipeline Flow

pipelines/sync-teams.js
├── Step 1: steps/download-teams-from-sportlink.js   → data/rondo-sync.sqlite
├── Step 2: steps/submit-rondo-club-teams.js          → Rondo Club API (teams)
├── Step 3: steps/submit-rondo-club-work-history.js   → Rondo Club API (quick work_history)
└── Step 4: steps/submit-rondo-club-player-history.js → Sportlink details + Rondo Club API (dated work_history)

Step-by-Step Details

Step 1: Download Teams from Sportlink

Script: steps/download-teams-from-sportlink.js Function: runTeamDownload({ logger, verbose })

1. Launches headless Chromium via Playwright
2. Logs into Sportlink Club
3. Calls Sportlink API to fetch team data:
   • UnionTeams (KNVB-assigned teams, preferred source)
   • ClubTeams (club-assigned teams, fallback)
4. For each team, fetches the team roster:
   • Players with their roles (Speler, Keeper, etc.)
   • Staff members with their roles (Trainer, Leider, etc.)
5. Stores team metadata in data/rondo-sync.sqlite → rondo_club_teams:
   • team_name, sportlink_id, game_activity, gender, player_count, staff_count
6. Stores team membership in data/rondo-sync.sqlite → sportlink_team_members:
   • sportlink_team_id, sportlink_person_id, role_description

Output: { success, teamCount, memberCount, currentSportlinkIds }

currentSportlinkIds comes directly from validated UnionTeams and ClubTeams responses. Both pipelines pass this snapshot to the team sync; the accumulated tracking database must never be used as the current source list.

Rate limiting: 500ms-1.5s random jitter between member scrapes.

Step 2: Sync Teams to Rondo Club

Script: steps/submit-rondo-club-teams.js Function: runSync({ logger, verbose, force, currentSportlinkIds })

1. Reads all teams from data/rondo-sync.sqlite → rondo_club_teams
2. For each team where source_hash != last_synced_hash:
   • No rondo_club_id: POST /wp/v2/teams (create new team)
   • Has rondo_club_id: PUT /wp/v2/teams/{rondo_club_id} (update existing)
3. Stores returned WordPress post ID as rondo_club_id
4. Updates last_synced_hash on success
5. Detects tracked teams missing from the fresh, complete, non-empty Sportlink snapshot. Missing teams are excluded from create/update, including force runs.
6. Verifies each missing team’s WordPress ID, post type, title and publicteamid. Untracked teams are never automatically removed.
7. Reads all accessible non-deleted persons, including former members and unpublished records. Trashed persons remain outside REST access; their original team references stay recoverable because archived team posts are retained. A missing team with an unended role is deferred and reported for source-history reconciliation; disappearance alone never invents an end date.
8. Converts historical references to team_id: null, team_name_text and entity_type: external_team, retaining roles, dates and other row values. Each save is independently read back, followed by a complete scan of accessible references. Verification also accounts for the existing former-member lifecycle: saving history can close other still-current roles at the membership end date (or today when that date is unavailable). Any other difference stops archival.
9. Moves only verified, unreferenced teams to draft, verifies their status and then removes their local sync mapping. The team post is retained; normal published-team lists no longer include it.

Output: { total, synced, created, updated, skipped, archived, errors }

Team renames: Uses sportlink_id as the conflict key, so renamed teams update the existing WordPress post instead of creating duplicates.

Preview or apply missing-team cleanup

Run on the production sync host as the service account, under the teams lock:

cd /home/rondo
sudo -u rondo flock -n /home/rondo/.sync-teams.lock node tools/retire-missing-teams.js
sudo -u rondo flock -n /home/rondo/.sync-teams.lock node tools/retire-missing-teams.js --apply

Both commands fetch only the fresh Sportlink team lists, without downloading rosters or changing the local source cache. Preview reports candidates and historical reference counts without writing to WordPress; --apply preserves history and archives the verified teams. Failed, malformed and empty snapshots cannot trigger cleanup. Individual roster failures do not make an existing team disappear from the source list.

Step 3: Sync Work History

Script: steps/submit-rondo-club-work-history.js Function: runSync({ logger, verbose, force })

1. Reads team membership from sportlink_team_members joined with rondo_club_teams and rondo_club_members
2. Compares current team assignments against rondo_club_work_history table
3. For each member with changes:
   • Fetches current work_history ACF repeater from Rondo Club
   • Adds new team assignments (creates new rows in the repeater)
   • Ends removed assignments (sets is_current: false, end_date: today)
   • Only modifies sync-created entries (manual entries are preserved)
4. Sends PUT /wp/v2/people/{rondo_club_id} with updated work_history repeater
5. Skips members without a rondo_club_id

Output: { total, synced, created, ended, skipped, errors }

Important: The work history sync only touches entries it previously created (tracked via rondo_club_work_history table). Manually added work history entries in Rondo Club are left untouched.

Same-named teams (for example Saturday and Sunday AWC 4) never overwrite one another in lookup maps. Both names and codes must be unique, or the quick sync resolves the member against exactly one current Sportlink roster. Multiple matches are left to the detailed membership sync. Detailed membership records with PublicTeamId use that ID exclusively; an unknown ID remains external history rather than falling back to a current namesake.

Step 4: Enrich Work History with Sportlink Dates

Script: steps/submit-rondo-club-player-history.js Function: runSync({ logger, verbose, force })

The fast team-roster response used in step 1 does not expose the start and end dates of a team relation. This step immediately follows the quick work-history sync and:

1. Reads tracked people’s Rondo work history in batches of 100 and compares current team/role pairs against the downloaded Sportlink rosters, using stable team IDs
2. Compares each member’s current team signature with the last successful detail sync; an unmatched active Rondo role bypasses the unchanged-signature shortcut, even when the member has no current teams or work-history tracking rows
3. Fetches changed or unmatched memberships from Sportlink’s member-details endpoint
4. Maps RelationStart and RelationEnd to the corresponding work-history row
5. Reconciles the existing row instead of creating a dated duplicate

Before mapping, lib/team-membership-periods.js interprets an empty RelationEnd using SeasonDescription. A recognized closed season such as 2025/'26 gets June 30, 2026 as its inferred end date. The cutoff follows Europe/Amsterdam. Explicit source end dates remain authoritative; current/future seasons, unrecognized seasons, and contradictory start dates are left unchanged. A continuing current-season copy with the same team, role, and start date takes precedence over an inferred historical end. Multiple historical copies retain the latest season. The normal reconciliation closes the existing current row and keeps unrelated and already ended history.

Sportlink’s Status: INACTIVE takes precedence over an empty RelationEnd: the importer writes is_current: false and preserves the unknown end date. A status-only change is reconciled in place, retaining the team link and start date. Rondo excludes explicitly inactive undated roles from current team membership, counts, fee matching and staff views. Replaying the same source does not write again.

A missing membership panel, failed request, or malformed Sportlink response is an error, never an authoritative empty history. If an unmatched Rondo role cannot be found in verified Sportlink history, the sync preserves it and reports it for review. It never invents a termination date from absence alone. Committee roles and external history are outside this audit. Once an official end date is reconciled, the next run skips the unchanged member again. The audit runs as part of the weekly teams pipeline and every standalone history run.

The standalone monthly player-history run remains a safety net, but new team assignments no longer wait for it before their dates appear in Rondo Club.

Output: { total, downloaded, synced, created, reconciled, skippedUnchanged, skippedQuarantined, errors }

Field Mappings

Sportlink → Rondo Club Teams

Rondo Club Field	Sportlink Source	Notes
title	TeamName / Name	Post title
acf.publicteamid	PublicTeamId	Sportlink team identifier
acf.activiteit	GameActivityDescription	”Veld” or “Zaal”
acf.gender	Gender	Mannen→male, Vrouwen→female, Gemengd→skipped

Sportlink → Rondo Club Work History

The ACF work_history is a repeater field on person posts:

Repeater Field	Source	Notes
team	rondo_club_teams.rondo_club_id	WordPress post ID of the team
job_title	role_description or fallback	”Speler”, “Keeper”, “Trainer”, “Staflid”
is_current	Computed	true if currently on team
start_date	RelationStart from member details	The quick roster step may temporarily use today; the detail pass immediately replaces it with Sportlink’s date
end_date	RelationEnd from member details	Empty while current

Database Tables Used

Database	Table	Usage
rondo-sync.sqlite	rondo_club_teams	Team → WordPress ID mapping + metadata
rondo-sync.sqlite	sportlink_team_members	Raw team roster data from Sportlink
rondo-sync.sqlite	rondo_club_work_history	Tracks which work_history entries sync created
rondo-sync.sqlite	rondo_club_members	KNVB ID → Rondo Club ID lookup (for work history)

CLI Flags

Flag	Effect
--verbose	Detailed per-team/per-member logging
--force	Skip change detection, sync all teams and fetch player history for every member

Error Handling

• Team download failure allows cached team updates but never cleanup; standalone submission without fresh source IDs also skips cleanup
• Individual team sync failures don’t stop the pipeline
• Work history sync skips members not yet in Rondo Club (counted as skipped)
• Player-history detail sync skips unchanged members and reports quarantined or failed Sportlink detail records
• All errors collected in summary report

Source Files

File	Purpose
pipelines/sync-teams.js	Pipeline orchestrator
steps/download-teams-from-sportlink.js	Sportlink team scraping (Playwright)
steps/submit-rondo-club-teams.js	Rondo Club team API sync
steps/submit-rondo-club-work-history.js	Rondo Club work history API sync
steps/submit-rondo-club-player-history.js	Sportlink relation-date enrichment and reconciliation
steps/prepare-rondo-club-teams.js	Team data preparation
lib/rondo-club-db.js	SQLite operations
lib/rondo-club-client.js	Rondo Club HTTP client
lib/sportlink-login.js	Sportlink authentication