The listening sockets had "unix://" written into Sac_a_push::create, so
the daemon and the frontend had to share a filesystem. USOCK_PATH_PFX can
now also be an address, "tcp://<host>:<port>", and then the pool is the
ports <port>..<port>+USOCK_POOL_N-1 with <port>+USOCK_POOL_N for the
control channel, mirroring the unix names one for one. A prefix without a
scheme keeps meaning exactly what it meant, so nothing changes for an
installation on a single machine.
spu_listen_url() and spu_admin_url() in spush/brisk-spush.phh do the
translation, and they live there because usermgmt.php includes that file
for USOCK_PATH_PFX and reaches the control channel through it. The
unlink of stale socket files is now skipped when there is no file to
unlink, and a failed bind says which address it failed on instead of
returning FALSE in silence.
What made this small is that the frontend and the daemon share no state
but the database. The System V shared memory - challenges, hardbans, the
room - is only ever touched from the nine endpoints the daemon serves;
usermgmt.php, mailmgr.php, error.php and the briskin5 statistics pages
include auth.phh for its database classes and never call
Challenges::load_data(). Under the old descriptor handover that bridge
did exist; moving to direct http had already dissolved it.
Two things the address form changes, both written out in brisk-spush.phh
and in the nginx template: the daemon still believes X-Real-Ip, which is
what the bans and the blacklist are built on, so a port anybody can reach
is a way around every ban; and it still speaks plain http, so the session
cookies cross the network in clear.
Checked end to end: with -U tcp://127.0.0.1:8090 the daemon listens on
8090..8100, answers on several ports of the pool, and 60 players through
nginx over that upstream complete with no error and clean stream
integrity, gzip included. Back on a unix prefix everything is as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
server unix:/home/nastasi/brisk-priv/brisk9.sock max_fails=0;
}
+# The daemon can also run on a machine of its own. Give INSTALL.sh an address
+# instead of a path, "-U tcp://<host>:<port>", install that machine with
+# "-m server" and this one with "-m front", and list the pool as ports:
+#
+# upstream brisk_daemon {
+# server 10.0.0.5:8090 max_fails=0;
+# server 10.0.0.5:8091 max_fails=0;
+# ... # <port>+N-1, N = USOCK_POOL_N
+# }
+#
+# Read the warning in spush/brisk-spush.phh before doing it. In short: over
+# tcp the daemon still believes X-Real-Ip, which is what the bans and the
+# blacklist are built on, and it still speaks plain http, so the session
+# cookies cross the network in clear. The port has to be reachable by this
+# machine and by nothing else. And the comet streams sit idle for as long as
+# proxy_read_timeout allows, an hour below: anything stateful in between -
+# a firewall, a NAT - will drop them long before that unless it is told not
+# to, and the only symptom is a client that reconnects for no visible reason.
+
# Needed to propagate the websocket upgrade. If another configuration in the
# same nginx already defines it, this block has to go (nginx refuses a
# duplicate "map" on the same name).
public $provider_proxy; // list of provider/browser that offer proxy service
public $file_socket_pfx;
- public $unix_socket_pfx;
+ public $is_inet; // TRUE when the pool is tcp instead of unix sockets
public $direct_socket; // socket where read direct commands
public $socks;
public $s2u; // user associated with input socket
$thiz->provider_proxy = ProviderProxy::create();
+ /* The scheme is not written in here any more: spu_listen_url() in
+ spush/brisk-spush.phh turns the configured prefix into the url of
+ each socket of the pool, a unix path or a tcp address alike, so the
+ daemon can sit on a machine of its own. See the comment there for
+ what that costs. */
$thiz->file_socket_pfx = $sockname_pfx;
- $thiz->unix_socket_pfx = "unix://$sockname_pfx";
- $thiz->direct_socket = "unix://{$sockname_pfx}_admin.sock";
+ $thiz->is_inet = spu_is_inet($sockname_pfx);
+ $thiz->direct_socket = spu_admin_url($sockname_pfx);
$thiz->debug = $debug;
$thiz->list_web = array();
$thiz->socks = array();
$thiz->rndstr .= chr(mt_rand(65, 90));
}
- for ($i = 0 ; $i < USOCK_POOL_N ; $i++) {
- $file_socket = $thiz->file_socket_pfx . sprintf("%d.sock", $i);
- if (file_exists($file_socket)) {
- unlink($file_socket);
+ /* A unix socket left behind by a previous run would make the bind
+ fail; a tcp port has no file to remove. */
+ if (!$thiz->is_inet) {
+ for ($i = 0 ; $i < USOCK_POOL_N ; $i++) {
+ $file_socket = $thiz->file_socket_pfx . sprintf("%d.sock", $i);
+ if (file_exists($file_socket)) {
+ unlink($file_socket);
+ }
+ }
+ $file_socket_admin = $thiz->file_socket_pfx . "_admin.sock";
+ if (file_exists($file_socket_admin)) {
+ unlink($file_socket_admin);
}
- }
- $file_socket_admin = $thiz->file_socket_pfx . "_admin.sock";
- if (file_exists($file_socket_admin)) {
- unlink($file_socket_admin);
}
/* The default listen queue (128) is short for a full room: if many
$old_umask = umask(0);
for ($i = 0 ; $i < USOCK_POOL_N ; $i++) {
- $unix_socket = sprintf("%s%d.sock", $thiz->unix_socket_pfx, $i);
- if (($list_sock = stream_socket_server($unix_socket, $err, $errs,
+ $listen_url = spu_listen_url($sockname_pfx, $i);
+ if (($list_sock = stream_socket_server($listen_url, $err, $errs,
STREAM_SERVER_BIND | STREAM_SERVER_LISTEN,
$ctx)) === FALSE) {
+ fprintf(STDERR, "listen on %s failed: %s\n", $listen_url, $err);
return (FALSE);
}
array_push($thiz->list_web, $list_sock);
if (($thiz->list_cmd = stream_socket_server($thiz->direct_socket, $err, $errs,
STREAM_SERVER_BIND | STREAM_SERVER_LISTEN,
$ctx)) === FALSE) {
+ fprintf(STDERR, "listen on %s failed: %s\n", $thiz->direct_socket, $err);
return (FALSE);
}
umask($old_umask);
default (128) a burst of simultaneous entries is partly refused */
define('USOCK_BACKLOG', 511);
+/* Where the daemon listens, and where whoever wants to reach it connects.
+ *
+ * USOCK_PATH_PFX is a path by default, and the pool is the set of unix
+ * sockets <pfx>0.sock .. <pfx>(USOCK_POOL_N-1).sock, plus <pfx>_admin.sock
+ * for the control channel. That is what it has always been, and it requires
+ * the frontend and the daemon to share a filesystem.
+ *
+ * It can also be an address, "tcp://<host>:<port>", and then the pool is the
+ * ports <port> .. <port>+USOCK_POOL_N-1 on that host, with <port>+USOCK_POOL_N
+ * for the control channel. The frontend can then sit on another machine. The
+ * shape is deliberately the same: the daemon opens USOCK_POOL_N listening
+ * sockets to spread the accepts over the pool, and nginx lists them as that
+ * many upstream servers either way.
+ *
+ * Two things change with the address form, and neither is optional:
+ *
+ * - the daemon believes X-Real-Ip (see Sac_a_push::run), and that address is
+ * what feeds the bans, the blacklist and the cloud check. Over a unix
+ * socket only a local process could set it; over tcp anybody who reaches
+ * the port can claim to be anybody, so a reachable port is a way around
+ * every ban.
+ * - the daemon speaks plain http: it is nginx that terminates the tls. Between
+ * the two machines the session cookies travel in clear.
+ *
+ * So bind it to an address only the frontend can reach, and keep the two ends
+ * on a segment you trust or inside a tunnel.
+ */
+function spu_is_inet($pfx)
+{
+ return (strpos($pfx, "://") !== FALSE);
+}
+
+function spu_listen_url($pfx, $idx)
+{
+ if (!spu_is_inet($pfx)) {
+ return (sprintf("unix://%s%d.sock", $pfx, $idx));
+ }
+
+ /* "tcp://host:port" -> "tcp://host:port+idx". The port is after the last
+ colon, which is also right for a bracketed ipv6 address. */
+ $i = strrpos($pfx, ":");
+ return (substr($pfx, 0, $i + 1) . (intval(substr($pfx, $i + 1)) + $idx));
+}
+
+function spu_admin_url($pfx)
+{
+ if (!spu_is_inet($pfx)) {
+ return ("unix://".$pfx."_admin.sock");
+ }
+
+ return (spu_listen_url($pfx, USOCK_POOL_N));
+}
+
/* How the daemon gets the player connection.
TRUE (direct mode, the default): the frontend opens an ordinary http
$cmd_len = mb_strlen($cmd_ser, "ASCII");
do {
- if (($socket = stream_socket_client('unix://'.USOCK_PATH_PFX.'_admin.sock')) == FALSE)
+ /* the control channel follows the daemon: a unix socket beside the
+ pool, or a port on the machine it runs on. See brisk-spush.phh. */
+ if (($socket = stream_socket_client(spu_admin_url(USOCK_PATH_PFX))) == FALSE)
break;
$stp = 1;
if (($rwr = fwrite($socket, $cmd_ser, $cmd_len)) == FALSE
projects / brisk.git / commitdiff