Appendix B: Reproducing the Defensive Backend Lab

This appendix reproduces the constrained research environment used for the book. It starts the original RCS database and worker code in containers, provides the MongoDB topology they expect, and serves the newly written web console on loopback. It does not reproduce an operational surveillance deployment.

The procedure never builds or runs an implant, exploit, collector, anonymizer, Network Injector, delivery package, or original AIR console. Those repositories remain static evidence. The web console renders historical build and installer controls but is designed and tested not to submit build tasks.

The commands assume the project is located at /path/to/hackingteam. Substitute the checkout path on another system; do not reproduce absolute paths inside configuration or scripts.

Safety boundary

Run the lab only on a host or virtual machine dedicated to defensive research. It contains deliberately old software and a backend with severe validated security defects. Treat every service, container, generated credential, and named volume as untrusted.

The current Compose file publishes all browser-facing interfaces on loopback:

MongoDB and the worker have no host-published ports. Inside the private Compose network, Mongo uses 27017 for mongos, 27018 for the shard server, and 27019 for the configuration server. The worker listens internally on 442.

Loopback binding is necessary but not a complete sandbox. Container escapes, browser interactions, host compromise, copied credentials, and accidental configuration changes remain possible. A defensible setup also uses host firewall policy, no port forwarding, no public cloud security-group exposure, and no route to sensitive production networks. Do not add an external reverse proxy or change 127.0.0.1 to 0.0.0.0 for convenience.

Never point the lab at a historical or current RCS address. The Compose names resolve only among local containers. The synthetic demo contains fictional operations, people, locations, devices, communications, and infrastructure.

What the four services do

The resolved stack has four services:

browser -> console-web -> rcs-db -> mongo
                            |
                            +---- rcs-worker -> mongo

mongo runs three MongoDB 2.6 roles in one container. This is a compatibility topology, not a recommended database deployment. rcs-db is the original Ruby EventMachine application with compatibility and commissioning changes in the lab copy. rcs-worker uses the same image but selects the worker entry point. console-web is an nginx container serving plain HTML, CSS, and ES modules and proxying REST and WSS to the backend.

The browser never connects directly to the backend’s host ports. The original REST server has no CORS support, so the web port sends /api/ and /wss/ requests to same-origin nginx. Nginx accepts the lab’s private backend certificate without verifying it. That is an isolated-lab accommodation, not a secure production pattern or an original AIR behavior.

Prerequisites and reproducibility limits

The host needs a working Docker Engine with the Compose v2 plugin. The console tests additionally need Node.js 18 or later and a Chromium build installed by Playwright. Exact minimum Docker versions have not been frozen, so record the versions used for a publication-quality replay:

docker version
docker compose version
node --version
npm --version

The first image build requires network access. It retrieves an archived Debian Jessie package set, Ruby gems, and HackingTeam’s minitar fork. The stack also pulls mongo:2.6 and nginx:1.27-alpine if those images are absent. These are reproducibility risks: archive availability can change, a branch reference is not immutable, and image tags do not identify one manifest forever. After a successful controlled build, record image IDs and repository digests and retain an authorized internal copy where policy permits:

docker image inspect rcs-db-lab:9.2 \
  --format '{{.Id}} {{json .RepoDigests}}'
docker image inspect mongo:2.6 \
  --format '{{.Id}} {{json .RepoDigests}}'
docker image inspect nginx:1.27-alpine \
  --format '{{.Id}} {{json .RepoDigests}}'

Do not “solve” a failed build by upgrading Ruby, MongoDB, gems, or Debian. The point is to preserve the historical runtime contract. Modern replacements are likely to change behavior and may fail in misleading ways.

Verify the source boundary before building

The leak material under data/ is the read-only reference. The build context must use copies under docker-lab/rcs-db and docker-lab/rcs-common. Confirm the expected project structure:

cd /path/to/hackingteam
test -d data/HackingTeam/unpacked/hackedteam/rcs-db
test -d docker-lab/rcs-db
test -d docker-lab/rcs-common
test -f docker-lab/docker-compose.yml
test -f docker-lab/Dockerfile
test -f docker-lab/entrypoint.sh

No build command in this guide uses data/ as a Docker context or volume. Before starting, review the resolved service and port configuration:

cd docker-lab
docker compose config --services
docker compose config --images
docker compose config

The service list should contain mongo, rcs-db, rcs-worker, and console-web. Inspect the resolved port entries and stop if any host binding is not loopback. Also stop if MongoDB or worker port 442 is published to the host.

Build the compatibility image

From docker-lab/:

docker compose build

The Dockerfile deliberately starts from debian:jessie, points APT at the Debian archive, installs distro Ruby 2.1.5, and uses Bundler 1.17.3. It installs libssl-dev before compiling EventMachine. That ordering is essential: without OpenSSL support, the service can appear to start and then crash on the first TLS connection.

The lab Gemfile exactly pins several important dependencies, including EventMachine 1.0.3, em-websocket 0.3.8, Rubyzip 1.0.0, and RestClient 1.6.7, but constrains rather than exactly freezes others. Its ~> 3.1.6 Mongoid constraint currently resolves Mongoid 3.1.7 and Moped 1.5.3. It uses the HackingTeam minitar fork because the backend expects Minitar.pack_stream; the current generated lock resolves commit 56af5840…. These are not the historical lockfile’s Mongoid 3.1.6/Moped 1.5.2 graph. Substituting arbitrary new versions would create a new port rather than reproduce the tested one.

A successful build produces rcs-db-lab:9.2. Both rcs-db and rcs-worker use that image. No core, vector, collector, or injector repository is copied into it.

First boot

Start the stack from docker-lab/:

docker compose up -d
docker compose ps
docker compose logs rcs-db
docker compose logs rcs-worker

The entry point waits up to approximately two minutes for the internal mongos listener. On the first boot only, it then creates default backend configuration, generates a local certificate authority and service certificates, and writes a lab licence. It finally starts either the database or worker according to the service command.

First-boot artifacts persist in named volumes:

• mongo-config and mongo-shard hold database state;
• rcs-config holds generated configuration, certificates, and licence;
• rcs-data holds application data outside MongoDB; and
• rcs-log holds application logs shared by database and worker.

The licence generator extracts verifier material byte-for-byte from the copied backend source and creates broad lab entitlements. One historical key contains a visually fragile private-use Unicode character, which is why the script does not duplicate the literal. Broad entitlement makes original backend features and UI states inspectable; it is not permission to execute offensive components. Archive mode and scout capability remain disabled, and the web console separately blocks build task creation.

A healthy database boot ends with messages equivalent to:

Listening for https on port 443...
Listening for wss on port 444...

The worker should report that its shard worker is ready. Review both container stdout and the rcs-log volume because the old application writes some traces to files more reliably than to standard output.

The Worker does not mount the database service’s rcs-config volume. On each Worker container recreation, the shared entry point therefore generates an ephemeral local configuration, certificate set, and lab licence before the Worker loads its operative licence state from the database. During a clean boot it may report that database licence information is unavailable once and then become ready five seconds later. Treat the final readiness message—not the transient retry—as the gate.

Verify health without disturbing an operator session

Prefer logs and the web console for the first check. An API login is not a passive health probe: logging in as a username invalidates the previous session and queues a user-addressed logout push. A curl login as admin can therefore force a human browser session out moments later.

When no browser user is active, the original login contract can be checked with the documented lab account. Substitute the current disposable password from docker-lab/README.md; the book does not repeat the source-embedded historical default:

curl -sk -X POST https://127.0.0.1:44443/auth/login \
  -H 'Content-Type: application/json' \
  -d '{"user":"admin","pass":"<lab-only-password>","version":"9.6.0"}'

The expected result is HTTP 200 and a JSON session document. If an authenticated request is necessary, copy the returned session value into a temporary shell variable and query /status; do not paste live cookies into notes, source files, or command histories retained for publication.

After a scripted login, explicitly log that session out with its returned cookie before handing the same account to a human. Live suites use unique temporary users and should clean them up through their harness. Do not rely on a backend restart as a session purge: it drops WebSocket connections and in-process caches but preserves human session documents in MongoDB. Startup clears only server-component sessions.

The web console is available at http://127.0.0.1:8080/. Its login form uses the same default administrator account on a fresh database. Do not expose this page through remote desktop gateways, tunnels, or shared development proxies.

Validate the web port

The mock suite needs no running Docker stack. In a fresh checkout, install its test dependencies once:

cd /path/to/hackingteam/docker-lab/console-web/tests
npm install
npx playwright install chromium
npm run test:mock

The last recorded complete mock gate is 135 passing tests. The suite starts a static server on port 8081 and intercepts API calls. It validates the web port, not the original backend.

The live suite requires the Compose stack:

cd /path/to/hackingteam/docker-lab/console-web/tests
npm run test:live

The last recorded complete live gate is 44 passing tests. Each test creates a fresh user and uses a separate request context so helper calls do not overwrite the browser session. Fixtures use unique names. Cleanup destroys operations before groups to avoid the original deferred access-control race.

The final live test is the named synthetic Worker replay. It uses the copied backend serializer to submit a DEVICE record on the internal Compose network and checks decode/store, alert logging, and WebSocket push. It does not run an implant or collector. See research/hackingteam-rcs/WORKER-VALIDATION.md for the sanitized trace and limitations.

The live suite changes the lab database. Run it only against disposable lab state, never against preserved incident evidence. Do not run it while a human is using the console. Restart rcs-db afterward to clear queued session kicks.

The suites include two critical compliance checks: the factory Build form and the anonymizer Download Installer control are render-only, and no task with type build is posted. A green suite does not mean every original feature is safe; it means the defined web-port contract passed.

Optional synthetic demonstration

console-web/tests/seed-demo.js creates the fictional Operation Nightjar scenario used in Chapter 8. It produces operations, targets, synthetic deployed agent records, directly inserted evidence-like rows, entities, alerts, dashboard state, and inert System-section records. It does not execute an implant or make a collector, anonymizer, injector, or connector operational.

The seed is optional and mutates the database. Use it only on a disposable lab after reading its cleanup and session notes. Most demo evidence is inserted directly through Moped to exercise storage and UI paths. A separate narrow live test validates the original worker pipeline with a synthetic encrypted blob. Do not cite the demo rows as collected evidence.

Screenshot scripts also mutate state and can exceed the server’s session timeout. They are publication tooling, not a health check.

Stop, restart, or erase

These operations have different consequences:

# Stop and remove containers and the private network; preserve named volumes.
docker compose down

# Start again using the existing database, configuration, licence, and certs.
docker compose up -d

# Restart only the backend process; preserve volumes and human session documents,
# but drop live sockets, process-local caches, and server-component sessions.
docker compose restart rcs-db

# DESTRUCTIVE: delete containers, database, configuration, certs, licence, logs.
docker compose down -v

Do not run down -v merely to fix a login or stale browser. It is the clean- replay control and destroys material that cannot be reconstructed exactly, including synthetic research state and logs. Confirm that the target is the disposable docker-lab Compose project before authorizing it.

The 2026-09-12 publication replay used a separate Compose project, preserved the normal research volumes, commissioned five fresh volumes, checked all four services, removed only the disposable namespace, and restored the original stack. Its sanitized evidence bundle records input and image hashes, first-boot sequencing, generated-artifact hashes, public certificate metadata, listeners, HTTP outcomes, and limitations in research/hackingteam-rcs/COMMISSIONING-VALIDATION.md. It reused locally built images. A later separate-tag no-cache build produced identical dependency and application-tree inventories; DEPENDENCY-REBUILD-VALIDATION.md records that result. Authorized upstream-artifact retention and a deliberate lockfile policy remain separate publication work.

The companion bounded reliability replay used another disposable namespace. Eight distinct analysts completed 640 authenticated reads over 5,000 synthetic metadata rows before and after individual database, Worker, and Mongo restarts. Counts and target sharding metadata were preserved, and Mongo recovered without a coordinated application restart. The reproducible script, timings, exact boundary, and cleanup record are in RELIABILITY-VALIDATION.md. This is a single-host functional scale point, not a production capacity claim.

Common failure modes

What a successful reproduction proves

A successful replay establishes that the copied RCS database and worker source can run under the documented compatibility capsule; that MongoDB accepts the expected sharded layout; that first boot creates usable configuration, certificates, licence, and administrator state; that the reconstructed console can exercise the tested REST and WebSocket contracts; and that synthetic data can traverse the specifically validated backend paths.

A successful run still leaves the original release pairing, AIR behavior, historical customer topologies, and any real-world monitoring unknown. It does not validate an implant-to-collector session, an anonymizer chain, Network Injector behavior, exploit delivery, OCR, translation, or a real connector export. The backend remains unsafe for exposure.

Reproduction is evidence only when its boundary travels with the result.

Sources and evidence

• docker-lab/docker-compose.yml, Dockerfile, entrypoint.sh, and the copied rcs-db and rcs-common Gemfiles define the runtime capsule.
• docker-lab/README.md, AGENTS.md, and console-web/MILESTONES.md define current operation, test gates, and known failure modes.
• research/hackingteam-rcs/COMMISSIONING-VALIDATION.md records the sanitized disposable empty-volume replay and its remaining dependency-build limits.
• research/hackingteam-rcs/DEPENDENCY-REBUILD-VALIDATION.md records the separate-tag no-cache build, exact comparison hashes, and the still-open artifact-retention/lockfile decisions.
• research/hackingteam-rcs/RELIABILITY-VALIDATION.md and validate-bounded-reliability.js record the bounded synthetic load, sharding, and component-restart replay.
• Chapters 3–6 document source provenance, compatibility decisions, commissioning, and the web-port boundary.
• Chapters 7–10 document worker provenance, synthetic fixtures, inactive edge components, and why the lab must remain isolated.
• The source/version/fidelity appendix records the exact examined source commits and known cross-version pairings.