Kiến trúc hệ thốngSystem architecture

The system has four blocks:

• MT5 terminals on Windows VPSes (Proxmox on OVH Dedicated Servers 02…N, behind OpenWRT routers, see Server infrastructure): the trading EA GoldenRobber, the monitoring EA AccountStatusUpdater, the tick publisher TickStreamerPub, and vps-agent (HeartbeatAgent + CmdAgent + stunnel) managing the machine.
• Docker host (OVH Dedicated Server 01) running the fxconsole stack (Laravel + React, nginx, Redis, TimescaleDB, scheduler, claude-cli, Guacamole) and four realtime Node.js brokers (Tick, AccountStatus, VpsStream, EaLink), all on the Docker network fxconsole_default.
• Remote MySQL on the aaPanel host (database trade_copier): source of truth for users, accounts, EA config, TMap and the settings table.
• Network edge: openresty (aaPanel, Let's Encrypt certificate) for HTTPS; OVH Network Firewall + IPFire forward the brokers' TLS ports straight to the Docker host; each VPS server has a WireGuard CT (wg-easy), and the fxconsole server is a client of all of them.

One rule runs through everything: sensitive values (config, MT5 passwords, install packages) only travel over HTTPS with a real certificate; TLS sockets to the brokers use self-signed certificates, so they carry only signals, telemetry or pointers. If a broker dies, every component still has an HTTPS fallback path.

flowchart LR
  subgraph VPS["Windows VPS (MT5)"]
    GR["GoldenRobber EA"]
    ASU["AccountStatusUpdater EA"]
    TSP["TickStreamerPub EA"]
    VA["HeartbeatAgent + CmdAgent"]
    ST["stunnel"]
    VNC["TightVNC"]
  end
  subgraph EDGE["Network edge"]
    OR["openresty :443 (aaPanel, Let's Encrypt)"]
    FW["OVH firewall + IPFire (port forward)"]
  end
  subgraph HOST["Docker host · network fxconsole_default"]
    WEB["fxconsole-web nginx :8080"]
    APP["fxconsole-app php-fpm"]
    SCH["fxconsole-scheduler"]
    RED[("fxconsole-redis")]
    TS[("fxconsole-timescale")]
    CC["fxconsole-claude-cli :8788"]
    GUA["guacamole :8282 + guacd"]
    TB["tick-broker :9600 / :9601"]
    AB["account-status-broker :9602 / :9603"]
    VB["vps-stream-broker :9605 / :9606"]
    EB["ea-link-broker :9607"]
  end
  MY[("MySQL trade_copier (aaPanel host)")]
  CF["Cloudflare Worker + D1 (optional)"]
  ADM(["Admin / user browser"])
  GR -- "HTTPS heartbeat, TMap" --> OR
  ASU -- "HTTPS /api/account-status" --> OR
  TSP -- "HTTPS /api/sync-*" --> OR
  VA -- "HTTPS fallback + payload" --> OR
  ADM -- "HTTPS + WSS" --> OR
  OR --> WEB
  GR -- "TLS 9607 signal" --> FW
  ASU -- "TLS 9602" --> FW
  TSP -- "TLS 9600" --> FW
  VA --> ST
  ST -- "TLS 9605" --> FW
  FW --> EB
  FW --> AB
  FW --> TB
  FW --> VB
  WEB --> APP
  WEB -- "/ws /status-ws /vps-ws" --> TB
  WEB --> AB
  WEB --> VB
  APP --> RED
  SCH --> RED
  APP --> TS
  SCH --> TS
  APP --> MY
  SCH --> MY
  SCH --> CC
  APP --> CC
  TB --> RED
  AB --> RED
  VB --> RED
  EB --> RED
  EB -- "settings" --> MY
  VB -- "vpses, settings" --> MY
  AB -- "settings" --> MY
  EB -- "POST /api/internal/ea-link/key" --> WEB
  VB -- "POST /api/internal/broker/command-ack" --> WEB
  GR -. "tmap (empty in production)" .-> CF
  ADM -- "?data=token" --> GUA
  GUA -- "VNC" --> VNC

The whole picture: solid lines are flows in use, dashed lines are optional. Each flow is drawn in detail under Data flows.

The system runs on several OVH Dedicated Servers with two roles: server 01 runs fxconsole; servers 02, 03 … N host VPSes, and each VPS runs MT5 accounts.

Server 01 is a WireGuard client with one tunnel per VPS server. The tunnel reaches the VPS server's main WAN IP, and the first LAN's OpenWRT forwards UDP 51820 to the WireGuard CT; the CT has a leg on every LAN, so guacd can VNC into any VPS. The other OpenWRTs use additional IPs, so each LAN goes out through its own IP.

A VPS reaches the Internet, including its MT5 broker connection, through the WAN IP of the OpenWRT it sits behind. Splitting VPSes across many LANs spreads the MT5 accounts over many different IPs, so a broker never sees a batch of accounts coming from one IP and grows suspicious. VPSes in the same LAN share one WAN IP.

• Each VPS server has one WireGuard CT (wg-easy) with one NIC on the LAN bridge of every OpenWRT. The CT MASQUERADEs client traffic into the target LAN.
• Server 01 is a client of the WireGuard CT on each server 02 … N, one tunnel per server. The tunnel endpoint is the WAN IP of the OpenWRT that port-forwards UDP 51820 to the CT. On node fxbot (server 02) that is the server's main IP (fxbot.sv02.gamelau.uk): the Proxmox host hands the main IP to the OpenWRT of LAN 101, which forwards UDP 51820 to the CT at 192.168.101.211. That server's OVH Network Firewall must let UDP 51820 through.
• The tunnels on server 01 must be split-tunnel: AllowedIPs lists only that server's LAN subnets. The client file wg-easy generates contains 0.0.0.0/0 by default; left as is, all of server 01's traffic, fxconsole included, would be pulled through the VPS server.
• Result: server 01 can ping LAN IPs in every range, i.e. every VPS on every server. It is a star with server 01 at the centre; the VPS servers are not connected to each other through this path.
• Guacamole relies on this path: guacd (a container on server 01) opens VNC to the lan_ip:5900 that vps-agent reports in its heartbeat (Guacamole); packets follow the host's routing table into the right tunnel. Because of MASQUERADE in the CT, the VPS sees the connection coming from the CT's IP in its own LAN, so no return route to server 01 is needed.
• The other direction does not use the VPN: EAs and vps-agent on the VPS call fxconsole over the public Internet (fxconsole.gamelau.uk: HTTPS 443 and the broker TLS ports), leaving through the OpenWRT's WAN IP.
• Node fxbot example: CT 107, 3 NICs vmbr101/102/103 (IP .211 in 192.168.101/102/103, only the first NIC has a gateway), clients 10.8.2.0/24. After every client create or delete, restart Docker inside the CT.

• Public HTTPS enters through openresty on aaPanel (Let's Encrypt certificate), then reaches fxconsole-web on port 8080. Laravel sits behind a proxy, so it needs an https APP_URL and TRUSTED_PROXIES so it trusts the X-Forwarded-Proto header.
• Broker TLS ports (9600, 9602, 9605, 9607) are opened on the OVH Network Firewall and port-forwarded on IPFire directly to the Docker host, bypassing openresty.
• Browser WebSockets need no extra port: the browser speaks wss://fxconsole.gamelau.uk/ws, /status-ws, /vps-ws with openresty's real certificate; nginx inside fxconsole-web proxies on to fxconsole-tick-broker:9601, fxconsole-account-status-broker:9603, fxconsole-vps-stream-broker:9606 with proxy_ssl_verify off (the self-signed certificate is only crossed inside the Docker network), 3600 s read/send timeouts.
• Docker network: the fxconsole stack creates fxconsole_default (project name fxconsole). The four broker stacks declare it external, so fxconsole must be up first for the brokers to resolve fxconsole-redis and fxconsole-web. Guacamole and guacd also use a private network guac_internal.
• VPN: each VPS server has a WireGuard CT (wg-easy) attached to every OpenWRT LAN on that server; the fxconsole server connects to all of them as a client so Guacamole can reach every VPS. Details: Server infrastructure. After every client create or delete, restart Docker inside the CT.

The eight flows below cover all communication between components. Each one states who initiates, how it authenticates, where the data lands, and the fallback path when a link in the chain dies.

• The EA does network calls in OnTimer, timed with GetTickCount64(), every InpCloudSyncIntervalSec seconds (stable config: 10; the standard requires ≤ 60 because the server marks an account offline after 120 s without a heartbeat). Backtests make no network calls.
• URL = InpCloudMyApiDomain + /api/tmap/heartbeat, i.e. https://fxconsole.gamelau.uk/easupported/goldenrobber/api/tmap/heartbeat. Header Authorization: Bearer <InpCloudBearerToken>, compared with hash_equals against the product's easupported.secret_token: wrong token → 401, unknown slug → 404 Unknown EA. WebRequest returning -1 means a network error or the host is missing from Allow WebRequest.
• The request carries magic, account, composite_id (slug(company)::slug(server)::login), node_id, priority, timeout_sec, server, account_name, config_schema (the EA declares its own tunable keys) and payload (the whole Info Panel).
• The server stores the account in ea_account (unique (ea_id, composite_id, magic)) and the node in tmap_nodes, then returns is_leader, current_leader_id, alive_count, alive_nodes, hot_keys (CSV of overridden keys) and the override values from ea_config. A key absent from hot_keys reverts to its input value. Production b21 hot-reloads 14 keys.
• Leader election: leader = the alive node with the lowest priority (ties broken by node_id alphabetically); alive = heartbeat within timeout_sec (InpCloudHeartbeatTimeoutSec = 90). Only the leader opens/closes orders and runs TMap save/clear. Heartbeats failing for more than 2 × timeout → the EA claims leadership; never had a successful heartbeat → leader by default.
• Admins view and edit at /admin/easupported/goldenrobber: an account is Online when its heartbeat is < 120 s old; select several accounts, then Apply Hot config / Clear config. Without EA Link, the EA picks up new values on its next heartbeat.

• From GoldenRobber 1.42 (Include/EaLink.mqh, class CEaLink), each chart keeps one TLS TCP connection to InpLinkHost:InpLinkPort (stable config: fxconsole.gamelau.uk:9607, InpLinkMode=1 = HYBRID). NDJSON protocol, 8 KiB max per line. Link is enabled only with InpCloudSyncEnable, a slug in InpCloudMyApiDomain and an InpCloudBearerToken; never in the tester.
• Why only a signal: MQL5 SocketTlsHandshake() does not verify the server certificate, and the broker uses a self-signed one. The socket therefore never carries config values, tokens or trade commands; on cfg the EA pulls the values over its HTTPS heartbeat (WebRequest verifies the real certificate).
• Challenge-HMAC auth: the broker sends challenge (32-hex nonce) right after TLS; the EA answers hello within 5 s (HELLO_GRACE_MS) with a signature. The broker fetches the slug's link key with POST http://fxconsole-web/api/internal/ea-link/key and header X-Broker-Secret; Laravel returns sha256(secret_token), never the raw token. 200 is cached for KEY_CACHE_SEC (60 s), 404 → unknown_ea (negative cache 10 s), anything else → auth_unavailable (not cached).
• On success → hello_ok (hb_sec 20); the broker writes presence laravel-database-ea_link:presence:{slug}:{composite_id}:{magic}, field {node_id}#{inst} = unix seconds, EXPIRE 90. The EA sends hb every 20 s; 60 s of silence (HB_TIMEOUT_SEC) closes the socket. The web treats presence < 60 s as live and shows the Link badge.
• Signal: admin Apply/Clear → EaLinkNotifier PUBLISH ea:link:cfg (the Redis facade prefixes it to laravel-database-ea:link:cfg) with payload {"slug","targets":[{"composite_id","magic"}],"rev":<ms>} → broker PSUBSCRIBE *ea:link:cfg → frame {"t":"cfg","rev"} to every socket in topic ea/{slug}/{composite_id}/{magic} → the EA coalesces signals and pulls the heartbeat at most once per second → applies config → sends cfg_ack. Latency ≈ ≤ 1 s. Broker log: cfg rev=… slug=… targets=N delivered=M.
• Reconnecting with the same {node_id}#{inst} replaces the old socket (bye superseded); a different inst (several charts, leader/follower) coexists and every one receives cfg. The EA backs off 2 → 60 s ±20% jitter; bad_sig/unknown_ea waits 5 minutes. If the broker is dead, the panel shows Link: DOWN and the EA keeps running like b21.1 (periodic HTTP).

• The EA has three InpTransport modes: HTTP (default), TCP, HYBRID.
• HTTP: POST InpApiUrl (/api/account-status) every InpPostIntervalSec (5 s), Bearer header = InpApiBearer, compared with the ea_api_bearer setting (env EA_API_BEARER); a mismatch returns 401 Unauthorized bearer token. Laravel AccountStatusStore writes Redis (48 h TTL), writes lot_history, runs EquityAlertService; the response may carry pending_report.
• TCP / HYBRID: TLS to InpBrokerHost:InpBrokerPort (default fxconsole.gamelau.uk:9602), first line HELLO|statuspub|v2|<composite_id>|<secret>|<broker_name>, secret = InpBrokerSecret = the broker's BROKER_SECRET (fxconsole shows it read-only as BROKER_STREAM_SECRET). Legacy (pre-v2) HELLOs are rejected. The broker writes the same keys as AccountStatusStore but skips lot_history and EquityAlertService, so HYBRID still calls the HTTP control plane every InpHybridHttpIntervalSec (30 s, timed with GetTickCount64 since 1.14 so it also runs while the market is closed).
• On-demand stream: right after HELLO the broker sends stream_on (cadence_ms) if a browser is subscribed, otherwise stream_off (hb_interval_secs). While off, the EA only sends tcp:hb_min (status, balance, equity, lot totals) every hb_interval_secs (EA clamps to 5–120 s, default 15), skipping positions[], experts[], today_history[]. While on, it sends the full payload every cadence_ms (default 900; 0 = every tick, floored by InpMinPushIntervalMs 100). The broker only fires stream_on/stream_off on 0 ↔ N subscriber edges. Both values are read in the order settings table → Redis app_config:* → env.
• The dashboard in "Stream (socket)" mode opens wss://…/status-ws → :9603 and sends {"action":"subscribe","secret_key"} (only that key's accounts) or {"action":"subscribe_all","admin_token"} (= broker_admin_secret, which only admins get via Inertia props). The broker sends a snapshot, then forwards every push, auto-including accounts that come online later. Polling mode reads the same Redis keys through Laravel.
• Reports: a user clicks Generate report on /account/{id} → ReportService stores the request and PUBLISH report:request {composite_id, from_ts, to_ts} (unprefixed channel) → the broker routes a report_request frame to that EA's socket → the EA sets g_forceHttpNow → the next HTTP push picks up pending_report → weekly chunks, each next chunk on the following 1 s tick (a 3-month report ≈ 13 chunks ≈ 13 s). Without a socket the EA still finds pending_report on its next HTTP heartbeat. The scheduler's report:generate-all requests a 3-month report for every active account daily.

• TickStreamerPub 1.04 opens TLS to InpBrokerHost:InpBrokerPort (fxconsole.gamelau.uk:9600), first line HELLO|tickpub|<account_id>|<secret>|<broker_name> (secret = InpSharedSecret = TICK_BROKER_SECRET; broker name = InpForexBroker or ACCOUNT_COMPANY). The EA polls SymbolInfoTick every InpTickPollMs (100 ms) and sends one JSON line per fresh tick {"type":"tick","s","b","a","t"}, a symbols list, and a NOOP every InpHeartbeatSec (15 s). An empty InpSymbolWhitelist = every Market Watch symbol.
• Per tick, the broker runs XADD ticks:<broker>:<symbol> MAXLEN ~ 100000, SADD brokers, SADD broker:<broker>:symbols, HSET broker_latest:<broker>, then broadcasts to WS clients subscribed to that (broker, symbol). These keys have no Laravel prefix; Laravel reads them through the tickstream Redis connection (empty prefix).
• Economic calendar: a full snapshot every InpCalendarIntervalMinutes (15 min; 7 past days, 14 future days) plus a ±3 h "near now" window re-pushed every 30 s, sent as several {"type":"calendar","seq","total","events"} frames that the broker reassembles into calendar:<broker> + calendar:<broker>:rev.
• OHLC backfill on connect: HTTPS GET /api/sync-state + POST /api/sync-bars to InpFxconsoleUrl, Bearer InpFxconsoleBearer (= EA_API_BEARER; empty = skipped), timeframes M1,M5,M15,M30,H1,H4,D1,W1,MN → the ohlc_data table in TimescaleDB.
• Scheduler: ohlc:aggregate reads the streams with XRANGE, builds M1 bars in ohlc_data, then trims processed ticks; brokers:sync syncs broker_profiles (MySQL) from TimescaleDB + Redis; calendar:sync copies calendar snapshots into the economic_events table.
• The browser on /charts/{slug}/{symbol}: bar history from OhlcApiController (TimescaleDB); live prices over wss://…/ws → :9601, sending {"action":"subscribe","broker","symbol"}, receiving the last tick (from XREVRANGE) then {"br","s","b","a","t"} frames.

• On the VPS: HeartbeatAgent.exe (1.3.2-phase9) is the long-lived process; it supervises CmdAgent.exe and talks to it over NDJSON IPC on 127.0.0.1:9606 (the VPS loopback, not the broker's port 9606). HeartbeatAgent connects to 127.0.0.1:9605; stunnel wraps TLS and connects to vps_broker_host:vps_broker_port (fxconsole.gamelau.uk:9605), verifying with the CA fetched from GET /api/agent/broker-config (header X-Vps-Secret). vps-agent.ini only holds server_url and secret_key.
• Auth: the hello frame carries secret; the broker looks up vpses.secret_key in MySQL (60 s cache) → hello_ok (with vps_id) or hello_err bad_secret. The broker pings every PING_INTERVAL 30 s and closes the socket if no pong arrives within PING_TIMEOUT 10 s.
• Heartbeat: an hb frame every heartbeat_interval. The broker runs SETEX laravel-database-vps:agent:{id}:heartbeat 15 s (HEARTBEAT_TTL), updates vpses.account_logins + account_count, and returns hb_ok with interval when vps:agent:{id}:interval is set (Laravel writes it when an admin edits the interval → the agent changes cadence and rewrites its ini, no restart). On-demand streaming like AccountStatusBroker: while an admin has the WS open the broker sends stream_on (the agent sends full CPU/RAM/disk/net + logins[]); with nobody watching, stream_off (a minimal "alive" frame, no WMI); the cadence stays heartbeat_interval.
• Command push: an admin clicks Deploy / Remove / Upgrade → INSERT vps_commands → VpsCommand::booted PUBLISH laravel-database-vps:cmd {vps_id, cmd_id, kind} → the broker sends {"t":"cmd","cmd_id","kind"} → HeartbeatAgent forwards do_cmd_pull over IPC → CmdAgent calls GET /api/vps/cmd/{id}/payload over HTTPS (encrypted MT5 passwords, package URLs… only travel this way) → executes → ack over IPC → an event frame to the broker.
• ACK: the broker runs UPDATE vps_commands directly, then POST http://fxconsole-web/api/internal/broker/command-ack with X-Broker-Secret so Laravel runs the full state cascade (mt5_accounts transition, observers → PUBLISH vps:state, deploy-status emails, migration stage 2). Without this callback deploys get "stuck at pending".
• Live UI: /vps and /deployments open wss://…/vps-ws → :9606, send {"action":"subscribe_all","admin_token"}, and receive vps_hb, vps_cmd, vps_state.
• Fallback: broker dead or HELLO failed → HeartbeatAgent silently falls back to POST /api/vps/heartbeat (switchover within ~60 s); commands arrive in the heartbeat response and ACKs queue into the next HTTPS heartbeat. A 256-entry dedup ring drops a repeated cmd_id. If CmdAgent dies, the supervisor respawns it within ~5 s; the in-flight command's ACK is lost and the server's 5-minute reaper handles it.
• Other: zero-touch provisioning via POST /api/agent/init-config (MAC + init_token bearer); agent self-upgrade through a TYPE_UPGRADE_AGENT command → upgrade.ps1 downloads GET /api/agent/release/{id}/download, checks SHA-256 (X-Release-Hash), extracts, reboots.

• Why: GoldenRobber uses the Ticket Map to know which round a position belongs to (a partial close on a hedging account changes the ticket and loses the original comment). If MT5 crashes and restarts on another machine, the map is gone and Tia / SingleRoundMode logic goes wrong.
• Key = magic + account. Three stores: the local file MQL5/Files/Common/GR_TicketMap_<magic>.csv (always written, backtests included); fxconsole POST {InpCloudMyApiDomain}/api/tmap/save → MySQL tmap_entries (live only, leader only); Cloudflare Worker /tmap/save → D1 (optional, InpCloudCfDomain empty in production).
• Save: on the InpCloudSyncIntervalSec cadence (new Init, DCA add, Tia close, map cleanup) and immediately after PositionClosePartial or in OnDeinit. The server does REPLACE ALL (delete by magic+account, then insert). Since fxconsole 2026.09.04.1 the server stores a sha1 fingerprint of the payload in Redis (1 h TTL): an identical payload returns "skipped":true without touching the DB; a Redis error still writes (fail open); clear forgets the fingerprint. Map back to 0 → DELETE /api/tmap/clear on every backend.
• Load on OnInit, in priority order: local file → Cloudflare → fxconsole → TicketMapRebuild() from the comments of open positions (older partial-close data is lost but current positions stay managed). Followers read from the cloud but never write.
• Binlog incident: before the fix, the EA re-pushed the whole map every 5–10 s even when unchanged; delete-all + insert-all produced ~10 writes/s (~873k/day) and ~3 GB of binlog/day on a 179 MB database, filling the MySQL disk.

• Trigger: the scheduler runs analysis:run --scope=global on its schedule and analysis:run --scope=weekly when due; admins can also run one pair or all of them from /admin/analysis.
• Inputs: bars from TimescaleDB via OhlcSeries (merges native bars with M1 for the tail) → TechnicalLevels; the economic calendar from Redis calendar:<broker> (tickstream connection, pushed by TickStreamerPub); news from NewsService (news:sync mirrors RSS into news_items) and Finnhub when a key is set.
• Model call: ClaudeService has two engines. Default cli: POST http://claude-cli:8788/analyze {model, system, prompt} → the sidecar runs claude -p --output-format json on the OAuth subscription, at most ANALYZE_CONCURRENCY=1 call at a time, compose timeout ANALYZE_TIMEOUT_MS=900000. The api engine (calls https://api.anthropic.com/v1) is only used when the setting analysis_engine=api and a claude_api_key is set; currently dormant.
• Output: Analysis::create into MySQL, shown on /analysis to every logged-in user, plus notifications/email digest (AnalysisReadyNotification, AnalysisDigestMail). The sidecar's GET /health returns an authed flag; when the login is lost, /admin/analysis shows a red banner and a Re-login button (/relogin/start, /relogin/complete).
• The sidecar publishes no port and only talks inside the Docker network. Driving a subscription from a backend is a ToS grey area: keep few pairs and a relaxed cadence.

• An admin clicks Connect on /vps/{id} → POST /admin/vps/{id}/remote-session. The server decrypts the VPS's VNC password, builds a JSON connection descriptor (protocol=vnc), signs it with HMAC-SHA256 and encrypts it with AES-128-CBC using GUAC_JSON_SECRET_KEY (guacamole-auth-json extension), writes a vps_remote_sessions audit row, and returns <GUAC_PUBLIC_URL>/?data=<token>, which expires after 60 s.
• The browser opens a new tab → reverse proxy (TLS at the edge) → host port 8282 → fxconsole-guacamole (Tomcat, WEBAPP_CONTEXT=ROOT, no DB) → guacd:4822 over the guac_internal network → VNC to the VPS (TightVNC installed by prepare-vm.ps1).
• Why VNC, not RDP (the original plan document says RDP): attaching RDP to the console session locks the desktop on disconnect, which breaks CmdAgent's GUI automation (MT5 WebRequest URL injection uses SendInput). VNC mirrors the framebuffer without creating or taking over a session.
• File drag-and-drop goes through the guac_drive volume (/var/lib/guacamole/drive, virtual drive fxconsole). The scheduler's fxconsole:close-stale-remote-sessions closes abandoned audit sessions.

Laravel uses the laravel-database- prefix (the brokers' REDIS_PREFIX must match). The tick data group uses the tickstream connection with no prefix. Redis has no persistence: everything is lost on container restart, and everything below regenerates itself.

fxconsole stack named volumes: vendor, node_modules, public_build, timescale_data, guac_drive, claude_creds, claude_config. Broker certs live on the bind mount monitor/<Broker>/certs.

• Deployment guide: set up, update, verify, roll back.
• Realtime brokers, fxconsole, VPS agent.
• EA integration standard, Builds & releases, Release note 1.42 b2.