Skip to main content

FCM Phase 4 Readiness Runbook

๐ŸŸข CLOSED 2026-05-19 โ€” Phase 4 final GO confirmed, 24h soak passed tanpa rollback. Service renamed fcm_service โ†’ firebase_service di repo dan merchant_fcm โ†’ merchant_firebase (binary firebase-service, systemd unit firebase-service.service) di VM. Runbook ini kept untuk reference historical โ€” referensi fcm_service / fcm-service di bawah harus dibaca sebagai nama lama service. Untuk runbook current state lihat services/firebase_service/README.md (di repo, di luar Docusaurus).

Tujuan (historical): Memvalidasi services/fcm_service (sekarang services/firebase_service) stabil sebelum Phase 4 cleanup (hapus merchant_core_api/internal/push/ + fallback path di dispatchPush).

Prasyarat aktif:

  • Phase 1 (token store ke service): aktif 2026-05-11. Flag FCM_TOKEN_STORE sudah dihapus per Phase 4 step 4 (2026-05-14) โ€” single path tanpa flag.
  • Phase 2 (send transport ke service): aktif sebelum 2026-05-11. Flag FCM_TRANSPORT sudah dihapus per Phase 4 step 4 (2026-05-14).
  • Phase 4 step 1+2+4+5+6+7 DONE (2026-05-14). Folder merchant_core_api/internal/push/ dihapus seluruhnya โ€” sumber FCM Firebase SDK + copy resolver tunggal di services/firebase_service/. Sisa hanya step 3 (MarkTokenInvalid di service postgres.go).
  • Mirror obligation feedback_fcm_service_mirror.md masih aktif โ€” EXPIRED per Phase 4 step 7 (2026-05-14). Memory di-rewrite ke single-source mode.

Observation window: 14 hari berturut tanpa red flag.

Owner: Backend Lead Plan terkait:

1. Quick health snapshotโ€‹

Jalankan setiap kali mulai sesi monitoring:

# 1. Service alive + Firebase + Postgres OK
curl -sS http://127.0.0.1:8093/health | jq

# expected (post Phase 4 final rename 2026-05-19):
# {
# "service": "firebase-service",
# "status": "ok",
# "firebase_configured": true,
# "postgres_configured": true,
# "app_env": "production"
# }

# 2. Tidak ada fallback warning di 24 jam terakhir
journalctl -u core-api --since="24 hours ago" 2>/dev/null \
| grep -cE "fcm_service send failed; falling back to internal|fcm_service register failed; falling back to db"
# expected: 0

2. Monitoring categoriesโ€‹

2.1 Fallback rate (CRITICAL)โ€‹

Wrapper dispatchPush fallback ke pushClient internal kalau HTTP service gagal. Kalau fallback sering jalan = service belum stabil. Phase 4 wajib 0 fallback selama 48 jam terakhir.

Daily check:

journalctl -u core-api --since="24 hours ago" 2>/dev/null \
| grep -E "falling back" \
| awk '{print $1, $2, $3}' \
| wc -l

Weekly aggregate:

journalctl -u core-api --since="7 days ago" 2>/dev/null \
| grep "falling back" \
| awk '{print $1, $2, $3}' | cut -d: -f1 | sort | uniq -c

Threshold:

Daily countStatus
0โœ… Healthy
1โ€“10โš ๏ธ Edge case (network blip) โ€” catat penyebab
>10โŒ Service tidak stabil โ€” investigate sebelum lanjut

2.2 Latency send-sideโ€‹

HTTP loopback (127.0.0.1:8093) target add < 5ms vs internal call. Plan ยง4 Phase 2 acceptance: P99 < 500ms (termasuk hop HTTP).

Benchmark manual (mingguan):

KEY=$(grep ^INTERNAL_NOTIFICATION_API_KEY services/firebase_service/.env.production | cut -d= -f2-)
for i in {1..20}; do
curl -sS -o /dev/null -w "%{time_total}\n" \
-X POST -H "X-Internal-API-Key: $KEY" -H "Content-Type: application/json" \
-d '{"push_token":"dummy_for_bench","type":"news","title":"bench","body":"bench"}' \
http://127.0.0.1:8093/internal/fcm/send
done | awk '{s+=$1; n++; if($1>max)max=$1} END {print "avg=" s/n "s, max=" max "s"}'

Note: token "dummy_for_bench" akan return 410 invalidated โ€” itu OK, ini hanya untuk ukur roundtrip HTTP + Firebase auth latency.

Threshold:

MetricTargetInvestigate if
Avg< 200ms> 500ms
Max< 500ms> 2s
P99 (rough)< 500ms> 1s

Penyebab umum P99 lonjak: Firebase JWT cache expired bersamaan (re-auth ke Google OAuth), DNS resolution slow, GC pause di Go process.


2.3 Token invalidation rateโ€‹

FCM API return UNREGISTERED โ†’ token auto-deactivate. Rate baru harus konsisten dengan baseline pre-Phase-2.

Cek harian:

PGPASSWORD='<password>' psql -h 10.8.0.1 -U kesles -d db_kesles_merchant -c "
SELECT date_trunc('day', updated_at) AS day,
COUNT(*) FILTER (WHERE is_active = FALSE) AS deactivated,
COUNT(*) FILTER (WHERE is_active = TRUE) AS active_touched
FROM notification.fcm_push_tokens
WHERE updated_at >= now() - interval '14 days'
GROUP BY day
ORDER BY day DESC;
"

Red flag:

  • Mendadak deactivate 100% token dalam 1 hari โ†’ bug error translation di dispatchPush (false-positive IsInvalidTokenError)
  • 0 deactivate >7 hari padahal merchant aktif โ†’ bug, tidak ada turnover natural dari reinstall/uninstall

2.4 Service uptime + resourceโ€‹

firebase_service tidak boleh crash atau memory leak. Plan ยง9 Phase 3 target: CPU < 5%, RAM < 100MB idle.

Cek harian:

# Status systemd (kalau dideploy via systemd)
systemctl status firebase-service
journalctl -u firebase-service --since="7 days ago" 2>/dev/null \
| grep -iE "panic|fatal|crash|killed|oom" | head

# Resource (mac dev / linux server)
ps aux | grep firebase-service | grep -v grep
# RSS column = memory KB. Target < 100MB = 102400 KB.

# Restart count
systemctl show firebase-service -p NRestarts
# expected: 0 (untuk window observation)

Red flag: restart count > 0, RSS naik linear (memory leak >50MB/hari).


2.5 End-to-end delivery + multi-staff (Phase B.1 verify)โ€‹

Verifikasi FCM benar-benar sampai device + fan-out ke owner/admin/staff.

Mingguan manual test:

  1. Login mobile app sebagai 2-3 user dari merchant sama (mis. Makan Kini MRC-0174105647947242) dengan role berbeda: owner, admin, staff.
  2. Dashboard PSP tester โ†’ klik "Push transaction event โ†’ FCM + TTS (Voice Announcement)" 1x.
  3. Verify dalam 5 detik:
    • FCM tray muncul di semua 3 device.
    • TTS announce "Transaksi sukses senilai Rp 19.590" di semua device.
    • Inbox in-app menerima entry.
  4. Tap notif โ†’ app open ke tab Transactions, transaksi tampil di list.

Cek event log untuk audit kuantitatif:

PGPASSWORD='<password>' psql -h 10.8.0.1 -U kesles -d db_kesles_merchant -c "
SELECT processing_status, COUNT(*) AS events_24h
FROM psp.event_log
WHERE created_at >= now() - interval '24 hours'
GROUP BY processing_status
ORDER BY events_24h DESC;
"

Expected: 'success' jadi mayoritas. 'logged_only_*' boleh ada (merchant inactive / no active device / not found) untuk traffic tester, bukan transaksi production real.


3. Daily log templateโ€‹

Salin baris baru tiap hari selama observation window:

2026-05-__ โ”‚ fallback=__ โ”‚ latency_avg=__ms โ”‚ deactivate=__ โ”‚ uptime=OK/CRASH โ”‚ smoke=PASS/FAIL โ”‚ note=__

Log entries (isi setiap hari)โ€‹

DateFallback (24h)Latency avgToken deactivated (24h)UptimeSmoke testNote
2026-05-11TBDTBDTBDOK(start of obs)observation kickoff, FCM_TRANSPORT=service + FCM_TOKEN_STORE=service di production
2026-05-12TBD~300msTBDOKPASSsmoke test post-deploy: sent_count: 1 lewat /internal/notifications/push/test
2026-05-130TBD1OKPASSregresi 8 guard pushClient.IsConfigured() di-fix via canDispatchPush() helper. Re-smoke 3 event (login_alert / order_paid / merchant_status_active) all sent_count:1. 1 token stale auto-deactivated
2026-05-140TBDTBDOKPASSPhase 4 step 1+2+4+5+6+7 DONE (code) + Step 3 deploy + smoke verified malam: (1) delete internal/push/fcm.go + migrate 6 caller ke fcmservice.SendInput, dispatchPush jadi single-path. (2) Drop FirebaseProjectID/ClientEmail/PrivateKey field + loader dari internal/config/config.go. (4) Drop FCMTokenStore/FCMTransport flag dari config + 5 env file. (5) Drop DB fallback di registerPushTokenViaConfiguredStore. (6) Migrate LoginNewDeviceCopy ke services/firebase_service/internal/push/copy.go::ResolveCopy + auto-resolve di handleSend saat caller pass Type=login_alert + DeviceName. HAPUS folder internal/push/ di core_api seluruhnya. (7) Mirror obligation memory di-rewrite ke single-source. go build/vet/test clean kedua repo. Rollback path berubah dari flag-flip ke git revert+ rebuild. Phase 4 Step 3 (MarkTokenInvalid single-writer) โ€” service v2 deploy 19:53 WIB + smoke verified 21:28 WIB (fake-token push returns {invalidated:true, deactivated_at:"2026-05-14T14:28:40Z", status:"invalidated"}). Production incident: gateway 192.168.62.253 lost internet upstream ~18:09-21:28 WIB (~3 jam, full FCM push delivery DOWN). Network self-recovered (atau operator DC fix). Action item filed: fix static default route metric supaya kernel auto-fallback ke dhcp default. Soak window aktif T+0 = 21:28:40 WIB โ†’ T+24h decision 2026-05-15 21:28 WIB
2026-05-150TBD10OKPASST+24h backend soak PASSED. fcm-service.service uptime 16h post manual restart 06:34:57 WIB (clean stop+start systemd, no signal/crash, 56s ops gap). Step 3 (MarkTokenInvalid single-writer auto-deactivate) verified pada production traffic: 10 fcm_send: invalidated events (FCM HTTP v1 return errorCode: UNREGISTERED), rate ~1.6/h. Error-level log 24h = 1 entry (very clean). Token masking benar (dIUtxpuH...), tidak ada deactivate failed log โ†’ SQL UPDATE sukses tiap kali. Opsi C smart fallback di core_api 4 callsite tidak perlu trigger backup path (DB sudah konsisten via service-side write). Next: monitor AAB v10 mobile soak Day 1-7 (T+0 = 15 Mei dini hari โ†’ T+7 promote decision = 22 Mei)
2026-05-160223ms (smoke)0 (prod) + 1 (smoke)OKPASSDay 1 soak PASSED. Service uptime 29h32m sejak restart 06:34:57 WIB Jum, no new restart. Journal 13.5h window (22:36โ†’11:56) total 1 entry (persistent error baseline kemarin, no growth, 0 new invalidated production). Day 1 smoke test 12:09:49 WIB return {invalidated:true, deactivated_at:"2026-05-16T05:09:49Z", status:"invalidated"}, HTTP 410, latency 223ms โ†’ Step 3 path live + DB UPDATE sukses. Interpretasi 10โ†’0 invalidated Day 0โ†’Day 1: pool stale token dibersihkan via Step 3 Day 0; Day 1 tokens yang is_active=true semuanya valid โ†’ no UNREGISTERED triggers. FCM error class shift di smoke: fake token format invalid โ†’ INVALID_ARGUMENT (vs prod stale โ†’ UNREGISTERED); keduanya match push.IsInvalidTokenError() โ†’ path Step 3 identik
2026-05-170250ms (smoke)10 (prod) + 1 (smoke)OKPASSDay 2-3 soak PASSED. Service uptime 48h+ tanpa restart sejak 15 Mei 06:34:57 WIB. Memory 4.5 MB (stabil, no leak vs 4.3 MB Day 0), CPU 7.336s total 48h (~0.0043% avg). Production window 16 Mei 12:09 โ†’ 17 Mei 19:45 (~31.5h): 10 invalidated events (rate ~0.32/h โ‰ˆ 7.7/hari = natural turnover steady-state). Error-level log tetap 1 entry baseline (no growth Day 0โ†’1โ†’2). Total journal 139 lines / 31.5h = ~4.4 lines/h (activity normal). Day 2 smoke 17 Mei 19:52 WIB: {invalidated:true, deactivated_at:"2026-05-17T12:52:27Z", status:"invalidated"} HTTP 410, latency 250ms โ†’ Step 3 path live + DB UPDATE sukses. Steady-state pattern Day 0 surge (10 cleanup) โ†’ Day 1 quiet (0 new) โ†’ Day 2-3 natural turnover (10 in 31.5h โ‰ˆ ekspektasi long-term). Cross-stack: merchant-whatsapp.service active 3+ hari (since 2026-05-13 21:08) memory 5.1 MB โ€” stack overall sehat untuk publish-readiness
2026-05-180n/a (no smoke)1 (prod)OKDEFERREDDay 3 soak PASSED (dengan catatan deploy). Manual ops 18 Mei 00:30-00:37 WIB: operator stop fcm-service.service 00:30:53, binary rebuild/replace 00:33:31 (mtime 2026-05-18 00:33), start kembali 00:37:35 (clean systemd: dotenv loaded โ†’ postgres connected โ†’ listening :8093, no Failed, no Main process exited). PID baru 144961, uptime 20h tanpa restart. Memory 4.6 MB (stable vs 4.5 MB Day 2). Production traffic 15:35:38 WIB: 1 fcm_send: invalidated token=ebFcQnwx... UNREGISTERED event โ†’ Step 3 logic verified-by-traffic di binary baru (rate ~0.05/h, pool sudah cleaned out, natural turnover decay). Caveat: source diff untuk binary baru tidak bisa di-verify (folder deploy flat, no .git, source repo path tidak terverifikasi). Behavior Step 3 identik pattern Day 0-2 (auto-deactivate via FCM v1 UNREGISTERED), tapi continuous-uptime metric formal di-reset 00:37:35. Smoke test endpoint deferred (endpoint path /fcm/send 404, perlu discovery `strings
2026-05-190n/a (no smoke)0 (prod, 4.6h window)OKDEFERREDDay 4 soak PASSED (behavioral) โ€” redeploy ke-2 berturut. Manual ops 19 Mei 05:05-05:08 WIB: operator stop fcm-service.service 05:05:53, binary rebuild/replace 05:06:37 (mtime 2026-05-19 05:06:37, size 7,770,274 bytes โ€” identik size dengan deploy 18 Mei tapi mtime beda), start kembali 05:08:07 (clean systemd: dotenv loaded โ†’ postgres connected โ†’ listening :8093, no Failed, no Main process exited). PID baru 183422 (โ‰  144961 kemarin), uptime 4h39min saat check 09:47 WIB. Memory 1.8 MB (fresh boot, vs 4.6 MB Day 3 yang lebih lama running). CPU 91ms (idle). Total downtime gap 2 menit 14 detik. Production events: 0 UNREGISTERED dalam window 05:08 โ†’ 09:47 (~4.6h) โ€” bisa karena pool sudah cleaned out (mirror Day 1 quiet pattern) atau natural Poisson variance (steady-state Day 2-3 ~0.32/h ekspektasi ~1.5 events di 4.6h, observasi 0 masih dalam range). Caveat strict soak: 2 redeploy berturut-turut (18 Mei + 19 Mei) breaks "7-day continuous binary" claim untuk 22 Mei promote decision. Behavioral soak masih โœ… karena: Step 3 logic verified-by-traffic di binary 18 Mei (event 15:35:38), no crash, no auto-recover systemd, restart selalu clean operator action. Size binary identik 18โ†’19 (7770274) suggests same source rebuild (Go embed timestamp di output โ†’ mtime beda walau source sama) atau trivial tweak yang tidak mengubah size. Action item operator: freeze fcm-service redeploy dari 19 Mei sampai 22 Mei (โ‰ฅ3 hari) untuk lock soak window โ€” perubahan code urgent defer ke post-launch (25 Mei+). Kalau redeploy harian terus berlanjut, decision 22 Mei wajib pakai behavioral verdict (Step 3 verified + 0 crash) bukan strict uptime
2026-05-200409ms (smoke)1 (smoke, post-recovery)๐ŸŸข RECOVEREDPASSDay 5 โ€” SERVICE RENAMED (Phase 4 final closeout) + VPN OUTAGE โ†’ RECOVERED + Step 3 VERIFIED. Cut-over fcm-service.service โ†’ firebase-service.service di 19 Mei 20:46:05 WIB. Description: "Merchant Firebase Service (FCM+Phone Auth)" โ€” Phone Auth belum di-set / belum implementasi (confirmed operator 20 Mei): description aspirational, scope sebenarnya saat ini masih FCM-only. Binary effectively fcm-service yang di-rename. Path baru: /home/enalfarid/kesles_merchant/merchant_firebase/firebase-service (sebelumnya merchant_fcm/fcm-service). Repo rename: services/fcm_service โ†’ services/firebase_service + merchant_fcm โ†’ merchant_firebase (source confirmed di binary strings: git.kesles.com/merchant/services/firebase_service/internal/push/copy.go + fcm.go). Binary mtime 2026-05-19 10:41:58, size identik 7,770,274 bytes dengan fcm-service binary 18-19 Mei โ†’ confirmed same FCM code base (Phone Auth pending). Service started 19 Mei 20:46:05 WIB, uptime ~27h saat check Day 5, memory 8.6 MB, CPU 105ms (idle), Tasks 5, PID 686. Endpoints inventory (via `strings
2026-05-21
2026-05-22
2026-05-23
2026-05-24
2026-05-25(end of observation)

4. Decision matrix (Phase 4 GO / NO-GO)โ€‹

Saat hari ke-14 (2026-05-25), evaluasi:

KriteriaThresholdHasil
Fallback warning 48 jam terakhir0__
Service crash 14 hari0__
Latency P99< 500ms__
Token deactivation ratenormal__
Multi-staff smoke test (mingguan)pass ร— 2 minggu__
event_log processing_status='success' mayoritas> 90% non-test traffic__

Decision:

  • โœ… Semua kriteria pass โ†’ GO Phase 4 cleanup
  • โš ๏ธ 1-2 kriteria warning (recoverable) โ†’ WAIT, fix dulu, reset counter 48 jam
  • โŒ 1+ kriteria fail (uncoverable) โ†’ NO-GO, investigate akar masalah, pertimbangkan rollback flag ke internal/db

5. Phase 4 cleanup actions (kalau GO)โ€‹

Update 2026-05-14: step 1-3 + 7 dieksekusi lebih awal (sebelum Phase 3 14-hari observation selesai) atas keputusan team. Phase 3 day 3 zero red flag + scope step 1-3 self-contained tanpa butuh data observasi. Step 4-6 ditahan sampai 2026-05-25 GO confirmed.

  1. [DONE 2026-05-14] Hapus code embedded di merchant_core_api/:
    • internal/push/fcm.go โ€” DELETED (305 baris)
    • internal/push/message_copy.go โ€” DIPERTAHANKAN (masih dipakai auth_mobile_handlers.go::LoginNewDeviceCopy). Akan dihapus kalau copy helper di-migrate ke service.
  2. [DONE 2026-05-14] Hapus field & init:
    • pushClient *push.Client dari httpapi.Server struct โ€” REMOVED
    • push.NewClient(cfg) di NewServer โ€” REMOVED
  3. [DONE 2026-05-14] Sederhanakan dispatchPush (push_dispatcher.go):
    • Fallback ke s.pushClient.Send โ€” REMOVED
    • Translation hack "fcm api error via service: token unregistered" โ€” REMOVED
    • Signature jadi (*fcmservice.SendResult, error)
    • 6 caller migrate: push.SendInput โ†’ fcmservice.SendInput, push.IsInvalidTokenError(err) โ†’ result.Invalidated
  4. [DONE 2026-05-14] Hapus feature flag:
    • FCMTransport field + loader getEnv("FCM_TRANSPORT", ...) di config.go โ€” REMOVED
    • FCMTokenStore field + loader getEnv("FCM_TOKEN_STORE", ...) di config.go โ€” REMOVED
    • FCM_TRANSPORT + FCM_TOKEN_STORE baris dari semua env files (.env, .env.production, .env.example, .env.development.example, .env.production.example) โ€” REMOVED
    • Bonus Step 2: field FirebaseProjectID, FirebaseClientEmail, FirebasePrivateKey + loader os.Getenv("FIREBASE_*") di config.go juga REMOVED โ€” merchant_core_api benar-benar tidak tahu apa-apa tentang Firebase. Kredensial hanya di services/firebase_service/.env.
  5. [DONE 2026-05-14] Hapus fallback DB di auth_profile_handlers.go::registerPushTokenViaConfiguredStore:
    • Cuma path service, error = real failure. Drop DB fallback path.
    • Drop import log/slog (warning log "falling back to db" tidak ada lagi).
  6. [DONE 2026-05-14] Migrate copy resolution + delete internal/push/:
    • Decision: Option A (migrate ke service).
    • LoginNewDeviceCopy dipindah dari merchant_core_api/internal/push/message_copy.go โ†’ services/firebase_service/internal/push/copy.go::ResolveCopy + loginNewDeviceCopy.
    • Tambah field DeviceName di SendInput (core_api fcmservice + firebase_service push).
    • Service auto-resolve title/body saat Type=login_alert + caller pass DeviceName tanpa Title/Body (wire di handleSend).
    • Caller auth_mobile_handlers.go drop pre-resolve, kirim Type + DeviceName saja.
    • 4 dead code fn (EmailVerifiedCopy/WelcomeUserCopy/MerchantRegistrationPending/MerchantActivated) ikut hapus.
    • HAPUS folder merchant_core_api/internal/push/ (fcm.go + message_copy.go).
    • Mirror di repo_exports/.../internal/push/ ikut HAPUS.
    • PENDING separately: profileAuth.RegisterPushToken (DB direct write) tidak dipanggil dari auth_profile_handlers.go lagi. Cek apakah ada caller lain; kalau tidak ada โ†’ deprecate atau hapus method. Not blocking Phase 4 GO.
  7. [DONE 2026-05-14] Mirror obligation berakhir:
    • services/firebase_service/internal/push/fcm.go jadi sumber tunggal
    • Memory feedback_fcm_service_mirror.md di-rewrite untuk single source
  8. [PARTIAL] Lainnya:
    • PENDING: Implementasi MarkTokenInvalid di services/firebase_service/internal/store/postgres.go (auto-deactivate token saat FCM return UNREGISTERED โ€” saat ini caller core_api yang s.postgresStore.DeactivatePushToken)
    • DONE 2026-05-14: Update service-topology.md reflect firebase-service sebagai standalone node
    • DONE 2026-05-14: Update architecture.md ยง10 caller table + ยง3.7 /ready endpoint + ยง6 schema reconciliation (migration 125)
    • PENDING: Decision retention internal/push/message_copy.go (masih dipakai auth_mobile_handlers.go::LoginNewDeviceCopy โ€” opsi: tinggalkan permanen di core_api, atau migrate ke service supaya package internal/push/ bisa benar-benar dihapus)
  9. Update plan status:
    • extraction-plan.md Phase 4 header โ†’ "Step 1+2+4+5+7 DONE"
    • services/firebase_service/README.md status table โ€” belum di-update

6. Rollback procedure (kalau NO-GO)โ€‹

โš ๏ธ Post 2026-05-14 (Phase 4 step 1): rollback flag-flip ke FCM_TRANSPORT=internal sudah tidak applicable karena internal/push/fcm.go + pushClient field sudah dihapus dari code base. Rollback satu-satunya = git revert <Phase 4 step 1 commit> + rebuild.

Rollback emergency post-Phase-4 (recommended path)โ€‹

ETA ~5-10 menit:

# 1. Identifikasi commit Phase 4 step 1
git log --oneline --all | grep -i "phase 4\|fcm cleanup\|delete fcm.go" | head -5

# 2. Revert commit (multi-file: push_dispatcher, server.go, 6 caller, push/fcm.go)
git revert <commit-sha>

# 3. Restore FIREBASE_* di merchant_core_api/.env.production
# (sebelumnya di-comment out di Phase 2 โ€” uncomment baris berikut)
# FIREBASE_PROJECT_ID=kesles-merchant
# FIREBASE_CLIENT_EMAIL=firebase-adminsdk-fbsvc@kesles-merchant.iam.gserviceaccount.com
# FIREBASE_PRIVATE_KEY=<PEM>

# 4. Set FCM_TRANSPORT=internal (force pakai pushClient lokal)
echo "FCM_TRANSPORT=internal" >> merchant_core_api/.env.production

# 5. Rebuild + redeploy core_api
make build && scp dist/merchant_core_api kesles-merchant:/home/.../bin/
ssh kesles-merchant 'sudo systemctl restart merchant_core_api'

# 6. Smoke test
curl -X POST .../internal/notifications/push/test ...

Rollback FCM_TOKEN_STORE saja (PRE-Phase-4 era, sudah tidak applicable)โ€‹

# HISTORICAL โ€” pre-2026-05-14 valid path:
# Di merchant_core_api/.env.production:
FCM_TOKEN_STORE=db # ganti dari "service"
# Restart core_api โ†’ register pakai DB langsung lagi

Catatan: FCM_TOKEN_STORE flag masih ada di config (no-op since Phase 4 step 1 cleanup belum hapus flag handler). Tapi semantik post-cleanup adalah db-only fallback path tidak ada lagi kalau code di auth_profile_handlers.go::registerPushTokenViaConfiguredStore ikut dibersihkan di step 5 (pending).

Yang tidak perlu rollback:

  • DB migration 125 (app_version, heartbeat_at) โ€” additive, non-breaking
  • Code di services/firebase_service/ โ€” tetap kompil & jalan idle
  • Token data di notification.fcm_push_tokens โ€” tetap valid

7. Signoffโ€‹

FieldValue
Observation start2026-05-11
Observation end2026-05-25
Decision date__
DecisionGO / WAIT / NO-GO
Decided by__
Notes__

8. Referensi terkaitโ€‹

  • extraction-plan.md (di working notes)
  • architecture.md
  • shadow-mode-plan.md โ€” Phase B (Polqo migration; di working notes)
  • Memory feedback_fcm_service_mirror.md โ€” mirror obligation core_api โ†” firebase_service